Announcing POSIX-UEFI

[Octocontrabass]

I noticed it in fopen line 215.

Do you have a smaller example to reproduce the bug? (Or at least the flags you pass to GCC when compiling this file?)

I have also run into a very similar problem where GCC optimizes malloc() + memset() into calloc(). The problem was that I was providing my own implementation of calloc() that was basically doing malloc() + memset(), resulting in another infinite loop.

I'm pretty sure you're supposed to build standard library functions with -fno-builtin.

And another, if you have a struct at address 0, then accessing any of its fields (at non-zero address) would generate an UD2 instruction instead of a MOV, how lovely, isn't it?

That's not a bug, using null pointers is undefined behavior (and using 0 to represent the null pointer is implementation-defined behavior).

[bzt]

I noticed it in fopen line 215.

Do you have a smaller example to reproduce the bug? (Or at least the flags you pass to GCC when compiling this file?)

No, I have no smaller example unfortunately. You have all the flags in the repo. To reproduce the error, simply replace the ifdef in uefi line 1344 with

Code: Select all

#define MEMFNC(a) a

This will avoid the name prefix workaround, and should trigger the issue in stdio.o's fopen. Just compile one of the examples, like 05_file with USE_GCC=1, and then check out "objdump -d uefi/stdio.o".

I'm pretty sure you're supposed to build standard library functions with -fno-builtin.

Yes, but it didn't worked.

And another, if you have a struct at address 0, then accessing any of its fields (at non-zero address) would generate an UD2 instruction instead of a MOV, how lovely, isn't it?

That's not a bug, using null pointers is undefined behavior

You didn't listen, it is a bug because you're not using NULL pointers. With a struct, only the first member's address would be 0, not the rest, yet ALL member access will be replaced by UD2, not just the first, that's the bug. So for example

Code: Select all

struct {
  unsigned char ivt[1024];
  unsigned char bda[256];
  ...
} memorymap_t;

Only ivt's address equals to 0, yet accessing bda (address 1024) would generate UD2 as well. (Let's not get into the part that in freestanding mode accessing address 0 (dereferencing NULL if and only if NULL is defined and it's defined as 0) is not just perfectly valid, but even required on some architectures. It can only be UB in hosted mode, simply because libc defines NULL, not the compiler.)

Cheers,
bzt

[kzinti]

You didn't listen, it is a bug because you're not using NULL pointers. With a struct, only the first member's address would be 0, not the rest, yet ALL member access will be replaced by UD2, not just the first, that's the bug. So for example

You are the one not listening. Accessing any member of a structure through a NULL pointer is undefined behaviour.

Let's not get into the part that in freestanding mode accessing address 0 (...) is not just perfectly valid, but even required on some architectures.

Where did you get that from? Freestanding or not, it is undefined behaviour to access anything through a NULL pointer. You are confusing the C standard with implementation details.

Null pointer and address 0x0 are not the same. It sounds like you think they are.

From the C99 standard section 6.3.2.3:

An integer constant expression with the value 0, or such an expression cast to type void *, is called a null pointer constant. If a null pointer constant is converted to a pointer type, the resulting pointer, called a null pointer, is guaranteed to compare unequal to a pointer to any object or function.

Regarding integer to pointer conversion:

An integer may be converted to any pointer type. Except as previously specified [i.e. for the case of null pointer constant], the result is implementation-defined, might not be correctly aligned, might not point to an entity of the referenced type, and might be a trap representation†.

So GCC decided on x86 that accessing something through the NULL pointer results in UD2 opcodes. That is implementation-defined behaviour and they are allowed to do this.

[bzt]

You are the one not listening. Accessing any member of a structure through a NULL pointer is undefined behaviour. It doesn't matter whether or not happens to be defined as "address 0".

Then again: you are in freestanding mode, there's no such thing as NULL pointer. As a consequence, you cannot dereference something that doesn't even exists. And memorymap_t.bda isn't address 0 either (but even if it were, since NULL isn't defined, you can't dereference NULL).

Where did you get that from?

Architecture specifications. There are MANY architectures where address 0 is used for something. For example: ARM32 vector table, x86 real mode IVT, C64 CPU configuration etc. etc. etc.

Null pointer and address 0x0 are not the same. It sounds like you think they are.

Quite the contrary. You seem to think like that. I'm saying that NULL isn't defined at all (neither 0 nor otherwise), and offsetof(bda) != 0 either, yet you got UD2 instruction, so that's a bug.

Cheers,
bzt

[kzinti]

Dear bzt,

(...) in freestanding mode, there's no such thing as NULL pointer.

This is non-sense.

Architecture specifications. There are MANY architectures where address 0 is used for something. For example: ARM32 vector table, x86 real mode IVT, C64 CPU configuration, Amiga Chip memory etc. etc. etc.

Again you are confusing a null pointer with address 0x0. They are not the same. The fact that some compilers allow you to access address 0x0 through a NULL pointer is implementation specific behaviour which falls under the category of "undefined behaviour". This is permitted by the standard but not required.

I'm saying that NULL isn't defined at all (neither 0 nor otherwise), and offsetof(bda) != 0 either, yet you got UD2 instruction, so that's a bug.

NULL is defined by the standard whether or not you use freestanding mode. offsetof(type, member) has nothing to do with any of this. Applying the resulting offset to a NULL pointer is undefined behaviour. Accessing any member of a structure through the NULL pointer is undefined behaviour.

There is no bug going on here, just undefined behaviour that doesn't happen to align with what you want it to be.

[bzt]

(...) in freestanding mode, there's no such thing as NULL pointer.

This is non-sense.

No, not at all. NULL is defined by libc not by the compiler, take a look at musl stdlib.h for example.

Architecture specifications. There are MANY architectures where address 0 is used for something. For example: ARM32 vector table, x86 real mode IVT, C64 CPU configuration, Amiga Chip memory etc. etc. etc.

Again you are confusing a null pointer with address 0x0. They are not the same.

Why do you think that? I was speaking about computer memory addresses, which has nothing to do with any languages, let alone a C specific definition. Strictly about computer memory addresses, nothing else.

NULL is defined by the standard whether or not you use freestanding mode.

Nope, see the musl stdlib.h link above. NULL is defined in a libc header. Therefore in freestanding mode, when you don't have libc neither its headers, you simply don't have the NULL define. Simple as that. But try it out if you don't believe me!

offsetof(type, member) has nothing to do with any of this.

Yes it does. You just don't understand what I'm saying. Look:

Code: Select all

var = memmap.ivt; -> mov eax, 0; mov eax, [eax]; mov [var], eax
var = memmap.bda; -> mov eax, 1024; mov eax, [eax]; mov [var], eax         <-- no NULL in the C code, and no 0 address in the Assembly

So it doesn't matter if NULL is defined or if it's 0 or not, because neither the C code, nor the (correctly) compiled Assembly contains any of those.

Cheers,
bzt

[PeterX]

Hey, guys. It's nice that you keep polite. But you are not convincing each other. So this discussion can go on forever and ever and not leading anywhere...

Greetings
Peter

[nullplan]

No, not at all. NULL is defined by libc not by the compiler

It is defined by the language standard, actually. But you are mixing up concepts, again. NULL is the name of a macro that is defined in stdlib.h in hosted implementations, that expands to a null pointer constant. A null pointer constant is a constant integer expression with value 0 or such an expression cast to a pointer type. Due to this ambiguity, the NULL macro is pretty much useless, by the way, because any place where the type can be inferred based on context, NULL and 0 are equivalent, and any place where the type cannot be inferred, you need an explicit conversion anyway.

Back to the topic: While the NULL macro is only defined to exist in hosted implementations, null pointers are not so constrained, and are any pointer that compare equal to zero. How your implementation chooses to compare pointers and the integer zero is implementation-defined. Nonetheless, the null pointer is guaranteed to be unequal to the address of any object. Therefore, having an object with a null address is not even undefined, it is breaking the standard.

Why do you think that? I was speaking about computer memory addresses, which has nothing to do with any languages, let alone a C specific definition. Strictly about computer memory addresses, nothing else.

And you are free to manipulate such addresses as you see fit, in a language and implementation that supports that. However, a C implementation that implements null pointers as pointers to address zero is no such thing. You will have to either change the implementation (pointer to address negative one used to be a popular choice, once upon a time), or use a different language, such as assembly.

C is a more abstract language than people like to think. You are not programming your CPU, you are programming some abstract machine that will at some point be mapped onto your CPU. This indirection means that concepts that have a definite meaning in your world and in the world of your CPU might still be utterly meaningless in the middle step, and be implemented in a manner you did not foresee. In this case, having an object at address zero is to the C machine as a four-sided triangle is to you. It is an utterly alien concept.

Hey, guys. It's nice that you keep polite. But you are not convincing each other. So this discussion can go on forever and ever and not leading anywhere...

Agreed, but... hope dies last, alright?

[kzinti]

Hey, guys. It's nice that you keep polite. But you are not convincing each other. So this discussion can go on forever and ever and not leading anywhere...

Someone on the Internet is wrong. We have to correct this.

[PeterX]

Hey, guys. It's nice that you keep polite. But you are not convincing each other. So this discussion can go on forever and ever and not leading anywhere...

Someone on the Internet is wrong. We have to correct this.

Haha, nice motto! XD

[Octocontrabass]

No, I have no smaller example unfortunately. You have all the flags in the repo. To reproduce the error, simply replace the ifdef in uefi line 1344 with

Code: Select all

#define MEMFNC(a) a

This will avoid the name prefix workaround, and should trigger the issue in stdio.o's fopen. Just compile one of the examples, like 05_file with USE_GCC=1, and then check out "objdump -d uefi/stdio.o".

I tried it and was not able to reproduce the bug. Am I supposed to pass any additional flags or use a particular version of GCC?

NULL is defined in a libc header. Therefore in freestanding mode, when you don't have libc neither its headers, you simply don't have the NULL define.

NULL is defined in stddef.h, which is a freestanding header.

[bzt]

Someone on the Internet is wrong. We have to correct this.

Someone on a technical forum is talking against facts and empirical evidence. Any real scientist would want to correct that.

By your reasoning, there's no point in correcting tutorials either, right? Those are just someone on the internet being wrong, right? And since you were so consistent, I'd like to kindly ask you to show us one implementation that defines NULL other than 0. Just one implementation! In a new topic, DO NOT derail this topic any more!

NULL is defined in stddef.h, which is a freestanding header.

How is this different to what I am saying??? First, stddef.h is provided by libc too (see musl, don't make a mistake that gcc has some built-ins, that's implementation specific and not language standard, use -nostdinc). Second, as I've shown, musl defines it in stdlib.h too (and in many others like time.h too for some reason). Third, the point is, it is defined in a header included by the programmer, and not by the compiler. Without a header, there's no NULL, and without NULL there's no NULL dereference, period.

Rant: why do I have a feeling that my topic is run over by paid trolls trying to derail it just because they are jealous that my POSIX-UEFI is actually simpler and better than the bloated EDK2? I don't want to offend anyone, but this is exactly how it looks like.

Is it really too hard to focus on POSIX-UEFI? Please keep on topic!

Cheers,
bzt

[bzt]

I tried it and was not able to reproduce the bug.

Good for you! Which version are you using? I'll try to update to the latest gcc and retry.

Am I supposed to pass any additional flags or use a particular version of GCC?

No. You just have to set USE_GCC in 05_file/Makefile. On test, 05_file/file.efi would stuck in an infinite loop (at memcmp in fopen function). By renaming memcmp to __uefi_memcmp everything works with gcc as expected.

Cheers,
bzt

[bzt]

Hm, this is interesting. If I use the distro provided gcc, then with and without -fno-builtin the problem exists, file.efi freeze on fopen when memcmp called. However if I use the gcc that I've compiled from the source, then with and without -fno-builtin the problem doesn't exists and file.efi works. Both gcc being the same version 10.2.0.

Code: Select all

gcc (GCC) 10.2.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I've upgraded my distro. Gcc was upgraded to 10.2.0-6, and now that works too (gcc still reports the same 10.2.0 version, but the package has an additional "-6" which increased since the last upgrade). I guess it's okay to remove the workaround, it's not needed any more. At least, let's hope so.

Cheers,
bzt

[nullplan]

Someone on a technical forum is talking against facts and empirical evidence. Any real scientist would want to correct that.

Any real scientist would be to busy with their science to care about one Hungarian professor misinterpreting the C standard. Not when on the internet, people misinterpreting the C standard are a dime a dozen.

How is this different to what I am saying???

stddef.h is required to be there in a freestanding implementation. You said the NULL macro does not exist in a freestanding implementation.

Rant: why do I have a feeling that my topic is run over by paid trolls trying to derail it just because they are jealous that my POSIX-UEFI is actually simpler and better than the bloated EDK2? I don't want to offend anyone, but this is exactly how it looks like.

Because obviously nobody could ever disagree with you without being paid to do so, right? You are correct that the topic has strayed from your invention, though, maybe we ought to remedy that.

Oh, and saying something offensive and then claiming that offence was not your intent really only works at most once.