Announcing POSIX-UEFI

[bzt]

No GOP wrapper.

Neither in GNU-EFI nor in the official EDK2. They only provide typedefs and GUID define, same as POSIX-UEFI.

No memory wrapper.

What do you mean? There are plenty of memory wrappers: memcmp, memcpy, memset, memmem, etc. Even functions like mbtowc and wctomb. And what wrapper would you suggest for a single "BS->GetMemoryMap()" call which doesn't have a wrapper function in the EDK2 either? Do you miss "LibMemoryMap" because you can't do "NoEntries = BufferSize / DescriptorSize" on your own? That's all the extra what GNU-EFI's "memory wrapper" provides (in addition to using uefi_call_wrapper, that's why this libefi function exists in the first place).

No SMP wrapper.

UEFI does not support SMP in the first place! That ridiculous MP Services are not covered with wrappers nor in GNU-EFI and neither in EDK2, and that's a totally unusable protocol anyway (must be terminated by the time you call ExitBootServices according to the UEFI spec).

No device detection.

You speak nonsense. Why would an API wrapper detect devices at all? This isn't a kernel nor a firmware! And BTW all UEFI detected devices which are accessible using any other library are also accessible under POSIX-UEFI too. Absolutely no difference!

Cheers,
bzt

[vvaltchev]

Last attempt. Most of the UEFI features do not have a wrapper in GNU-EFI either. But I've already pointed this out to you. It is unreasonable to expect something from this library that neither GNU-EFI, nor the official EDK2 library can't provide.

Bzt, I've understood from the beginning what your library is. The whole time I tried to discuss about what your library could be instead, if you wanted. GNU-EFI and EDK2 are not libraries on the top of UEFI: they just expose UEFI as it is, nothing more. (Well, they offer the Print() function, but it's used in the spec as well.) Your library instead, provides a layer on the top of UEFI. It's a completely different thing or, at least, that's how many people will perceive it. Probably you perceive it as something like GNU-EFI + some bonus POSIX-ish functions, instead. That difference of perception, matters. I hope we can agree at least on that. Now, when a given software component offers a layer on the top of another, there's always the discussion: should it cover everything? if not, what should it cover? Well, that's subjective. Purists insists that it has to cover 100% or nothing. Other people, like myself, are fine with covering just "enough". The problem is that "enough" is subjective as well. For me a fair definition of "enough" is: take a real-world bootloader like GRUB, study it's source code and see which UEFI features uses (when runs in UEFI mode, of course). Try to cover most of them. Or least, aim to do that. In your case, if I've understood you correctly, you just don't care: you're interested in your minimal POSIX-like interface and that's it, no matter what other people say. That's FINE. Just, accept the consequence that people might not use your library because of that. At end, does it matter to you if many people use your software, or not?

[bzt]

GNU-EFI and EDK2 are not libraries on the top of UEFI

Okay, I see your confusion now. Yes, they are exactly that, libraries on top of UEFI protocols to hide all that ugly GUID interfaces.

For GNU-EFI, the function prototypes are included with efilib.h, and they are implemented here. Typically functions like Print, and AllocatePool for example:

Code: Select all

VOID *
AllocatePool (
    IN UINTN                Size
    )
{
    EFI_STATUS              Status;
    VOID                    *p;

    Status = uefi_call_wrapper(BS->AllocatePool, 3, PoolAllocationType, Size, &p);
    if (EFI_ERROR(Status)) {
        DEBUG((D_ERROR, "AllocatePool: out of pool  %x\n", Status));
        p = NULL;
    }
    return p;
}

As you can see, the one and only purpose of having an "AllocatePool" function is to hide uefi_call_wrapper() and to return the allocated address instead of the EFI status code. This "AllocatePool" is not the same as the UEFI firmware provided "BS->AllocatePool", it is a library function, implemented in the GNU-EFI's libefi.a. This is exactly the same what POSIX-UEFI does, only POSIX-UEFI calls this function "malloc" to provide the POSIX standard API for it.

Code: Select all

void *malloc (size_t __size)
{
    void *ret = NULL;
    efi_status_t status = BS->AllocatePool(LIP ? LIP->ImageDataType : EfiLoaderData, __size, &ret);
    if(EFI_ERROR(status) || !ret) { errno = ENOMEM; ret = NULL; }
    return ret;
}

For EDK2 this is a little bit more complicated, as it's using a non C standard naming convention, something like a Pascal renegade would use, and calls these libraries "Pkg"s. For example, my favourite is this FreePool (one of the many), which actually is an empty function. Sure EDK2 won't leak memory... Plus it is extremely bloated, for example "InternalMemSetMem16" is defined here and here and here too. Why? Exactly the same function, with exactly the same prototype and implementation... And the worst of all: EDK2 keeps renaming these libraries all the time, so forget about compatibility! If you can compile your code today, doesn't mean you'll be able to compile it a month later.

And yeah, Print is also implemented as a user library, just like with GNU-EFI and POSIX-UEFI. The only difference is, POSIX-UEFI calls this "printf", and not "Print", again only to provide the usual POSIX API for it.

they just expose UEFI as it is, nothing more.

Not quite. They provide a layer on top of the UEFI. They also do provide a lot of header files with the typedefs and the GUID defines, but so does POSIX-UEFI. Plus POSIX-UEFI was carefully written in a way so that you can include those protocol definition headers from GNU-EFI and EDK2 headers without naming conflicts (tricky, mostly because of the enums. Typedef structs are OK, because POSIX-UEFI uses the standard C naming convention, like efi_system_table_t instead of that horrible and unreadable everything is uppercase naming EFI_SYSTEM_TABLE, where you don't know what that's supposed to be: a typedef struct? Or a GUID? Or a return code scalar?).

A little quiz: what are these? Do they have methods or maybe arguments instead? Or are they just enums or defines perhaps?
EFI_LBAL
EFI_EVENT
EFI_EVENT_TIMER
EFI_TIME_CAPABILITIES
EFI_PROTOCOLS_PER_HANDLE

For me a fair definition of "enough" is: take a real-world bootloader like GRUB, study it's source code and see which UEFI features uses

That's exactly what I did. That's why I've said everything is covered that a bootloader needs (all boot time services, all run time services plus all the required protocols: Simple File System, Serial IO, Block IO etc.). If there's no library function for something, then there's at least a typedef struct and a GUID define for the interface (like GOP) so that you don't have to rely on 3rd party header files. You can't expect more from EDK2 nor from GNU-EFI either.

(Just for the records when I said "what a basic bootloader needs", I meant "basic" as in a bootloader is a basic UEFI application, so not like a device driver or a permanent run-time service and such.)

Or least, aim to do that. In your case, if I've understood you correctly, you just don't care

No, you did not understood at all. I specifically asked for protocols that you need besides of the ones already covered, because I do care. I can't add "I don't know" and "not enough", but I could add EFI_RNG_PROTOCOL for example if it weren't covered already. (But again, if you miss a protocol typedef or a GUID, you can include GNU-EFI's or EDK2's headers to get them without a problem. No need for their libraries, just the header files. And you can include those 3rd part headers because I do care).

Cheers,
bzt

[vvaltchev]

Okay, I see your confusion now. Yes, they are exactly that, libraries on top of UEFI protocols to hide all that ugly GUID interfaces.

Print[/url] is also implemented as a user library, just like with GNU-EFI and POSIX-UEFI. The only difference is, POSIX-UEFI calls this "printf", and not "Print", again only to provide the usual POSIX API for it.

No confusion, I mentioned Print() myself. Sure, there are also AllocatePool() and a bunch of other functions. But all of them are mentioned in the UEFI Spec (including Print). For example, The AllocatePool() wrapper is mentioned in this example:

//
// Create a new child
//
PrivateChild = EfiLibAllocatePool (sizeof (ARP_PRIVATE_DATA));
if (PrivateChild == NULL) {
return EFI_OUT_OF_RESOURCES;
}

UEFI Spec 2.8, Errata B, page 395.

It doesn't really matter that they are not technically in the UEFI firmware. They're part of "UEFI".

they just expose UEFI as it is, nothing more.

Not quite. They provide a layer on top of the UEFI.

A few functions, mentioned in the UEFI spec do not really provide a "layer" on the top of UEFI. They're really minor utils. A software layer is another thing.

They also do provide a lot of header files with the typedefs and the GUID defines, but so does POSIX-UEFI.

OK, this is an important point. The typedefs and GUIDs provided by GNU-EFI, EDK2 and your POSIX-UEFI do NOT count as a software layer. They're not code and cannot be considered as part your library's interface. It's like defining all the syscall numbers, expecting users to use syscall(2) + your defines and claiming that you cover the whole interface of the kernel. Again, that's very useful, but it's not a software layer.

That's exactly what I did. That's why I've said everything is covered that a bootloader needs. If there's no library function for something, then there's at least a typedef struct and a GUID define for the interface so that you don't have to rely on 3rd party header files. You can't expect more from EDK2 nor from GNU-EFI either.

OK, do you realize that we're looking at the things from a completely different angle? Obviously, you "cover" everything with typedefs and GUIDs, like GNU-EFI does. I never even thought of questioning that. But that does not mean "covering" to me, as I just explained. For me, your library covers only the UEFI features that are exposed through your POSIX-like interface. The rest is a fall-back to the native UEFI interface.

Or least, aim to do that. In your case, if I've understood you correctly, you just don't care

No, you did not understood at all. I specifically asked for protocols that you need besides of the ones already covered, because I care.

If you care, do you realize that implementing a bunch of new interfaces will inevitably make POSIX-UEFI to be a different library? Either you add new functions or support "special" files like /dev/diskNp0 etc, it would be a ton of work. Of course, if you show some genuine interest in that, I will start mentioning the interfaces that I believe should be wrapped. But, again, that will change your library significantly. I'd be super surprised if you moved from your original (hard) position to that, but that's how software improves and people collaborate. There's nothing bad in changing your mind. Just, allow me to be skeptic.

Still, to you show some goodwill from my side, I'll mention the first one. The graphics output protocol. It's essential for a bootloader. Through multiboot, the bootloader communicates to the kernel the current video mode with the base pointer and bunch of other info (width, height, bpp, etc.). You need to query the current video mode with GOP. The kernel itself might also have a preferred video mode, which is set in it's ELF binary (according to the multiboot spec). So, you'll have to switch to that video mode, if it's available. Finally, on some laptops with "retina" displays the default video mode (= native display resolution) is NOT good enough, even for the bootloader itself, because the default font is too small and it does not get scaled. Better to select a "human" (smaller) resolution before having any interaction with the user. Note: I'm talking about an interactive bootloader. That case is important.

So, you'll have to figure out a nice way to expose 100% of the graphics output protocol in your library. You might create a bunch of special files like /sys/gop/0, /sys/gop/1, /sys/gop/count plus a bunch of IOCTLs or, in alternative, you can design a whole new interface. People will need to query the available video modes, set them, handle errors, and, write to video framebuffer (there are graphical UEFI bootloaders as well). Note: if the new interface is in C, the risk is ending up with something too similar to the native UEFI interface, maybe with the only difference that structs and typedefs have better (shorter) names. If the new interface is file-based, the risk is that it might be inconvenient to use, it depends. While it's easy for me to enumerate the interfaces that should be wrapped, designing them is a complicated task. So, if you get frustrated with that, I'd totally understand. That's why I proposed C++, actually. I have a love-hate relationship with that language, but at least I can say that you could offer simpler, high-level interfaces with it. Of course, that's at the price of making yours a C++ library. I'd understand if you didn't like the idea. The problem is that it is NOT simple to design good interfaces.

So, what do you think ?

[bzt]

Sure, there are also AllocatePool() and a bunch of other functions. But all of them are mentioned in the UEFI Spec (including Print).
...
It doesn't really matter that they are not technically in the UEFI firmware. They're part of "UEFI".
...
A few functions, mentioned in the UEFI spec do not really provide a "layer" on the top of UEFI.

If they are not part of the firmware, then they must be library functions, no third options exists.

They're really minor utils. A software layer is another thing.

Erhm. First, "util" is not a precise definition and tells nothing about the implementation (dd is an util and DaemonTools is an util too), second, nobody ever called libc or libpcap for example an util before, and third, what is a function library if not a software layer?

They're not code and cannot be considered as part your library's interface.

I strongly disagree. Header files and the definitions in them are very essential parts of any library! Plus this is the way how UEFI defines it's interfaces in the first place! You cannot do a GOP SetMode if you don't have the typedefs.

Obviously, you "cover" everything with typedefs and GUIDs, like GNU-EFI does.

FOR CRYING OUT LOUD. That's how UEFI defines its interfaces. And POSIX-UEFI also covers them with functions (except GOP), as I've said many, many, many, many, many, many, many, many times.

For me, your library covers only the UEFI features that are exposed through your POSIX-like interface.

Che? That's the whole point of the POSIX-UEFI library!!!! It does exactly that, instead of that terrible GUID interfaces and that horrible library in the UEFI spec with millions of everchanging function names, it provides a friendly stable API which looks like POSIX. Is it really that hard to understand this?

The rest is a fall-back to the native UEFI interface.

And this is the same for EDK2 and GNU-EFI libraries too! For the rest protocols they don't provide library functions for, it is a fall-back to native UEFI interface. What else did you expect, seriously?

The graphics output protocol. It's essential for a bootloader.

If typedefs doesn't count, then don't use GOP with EDK2 nor with GNU-EFI either, because that's all they provide for GOP. They do not provide functions for setting the video mode, just typedefs and defines.

So, you'll have to figure out a nice way to expose 100% of the graphics output protocol in your library.

No, I don't. I prefer if the POSIX-UEFI users communicate with GOP the same way as they would with EDK2 and GNU-EFI. There's absolutely no reason to introduce a bloated interface on top of that. Don't try to fix what's not broken!

So, what do you think ?

That you surely haven't read the README, you don't understand the purpose of POSIX-UEFI, and you have problems understanding basic concepts such as software libraries and C header files in general. No offense, you asked, I answered.

Cheers,
bzt

[vvaltchev]

Guys, there's nothing more to add. Bzt is hopeless.
Being a gentlemen, I'd avoid to insult him.
To anyone who still think he is right: please, go find a job, learn, and come back in 5 years.

Goodbye!

[PeterX]

People, this is leading nowhere. I understand vvaltech and nexos. Regarind the topic I'm on bzt's "side", which doesn't matter since this is NOT about who is right. And yes, bzt could work on his temper and manners. ("Speaking nonsense" was not nice.)

What vvaltchev and nexos suggest is a kind of (more or less) complete abstraction layer following POSIX. What bzt intends with his project is quite different: A small and simple interface (mimicying small parts of POSIX) to avoid many of the gory details of UEFI. Sounds similar but is quite different.

If vvaltchev or nexos think their idea is so good that it should be implemented they probably have to fork it. Otherwise let it be bzt's project.

I hope the emotions have not gone up the ceiling on either side and that we can go on in this forum with respect.

Greetings
Peter

[bzt]

Guys, there's nothing more to add. Bzt is hopeless.
Being a gentlemen, I'd avoid to insult him.
To anyone who still think he is right: please, go find a job, learn, and come back in 5 years.

Goodbye!

Typical troll. When someone points out the facts, attacks by calling "hopeless". Goodbye!

I understand that big money corp and their brainwashed developers don't want to hear that what they do stinks. But putting your head in the sand instead of throwing out of the garbage doesn't make it less smelly. UEFI is a garbage, both the firmware and the supporting library built on top of it. We can't throw out the firmware sadly, but we surely can and should throw out the rest! I'm not saying you're a paid M$ developer, all I'm saying is, brainwashing by chanting "Developers, developers, developers" might raise the endorphin level, but it surely doesn't make them write good programs and design good interfaces.

And yes, bzt could work on his temper and manners. ("Speaking nonsense" was not nice.)

I was calm all along. Why do you think I wasn't? I'm sorry if "nonsense" seemed offending to someone, but how else do you say in English if something is pointless and well, just does not make sense? I know no other phrase, sorry. No offense intended, I assure you.

What vvaltchev and nexos suggest is a kind of (more or less) complete abstraction layer following POSIX. What bzt intends with his project is quite different: A small and simple interface (mimicying small parts of POSIX) to avoid many of the gory details of UEFI. Sounds similar but is quite different.

Absolutely that's the case.

If vvaltchev or nexos think their idea is so good that it should be implemented they probably have to fork it. Otherwise let it be bzt's project.

I agree. POSIX-UEFI is Open Source and licensed under the terms of MIT license for that reason. Anybody can fork it and expand it to a fully blown POSIX system if they want to.

Cheers,
bzt

[vvaltchev]

Typical troll. When someone points out the facts, attacks by calling "hopeless". Goodbye!

I understand that big money corp and their brainwashed developers don't want to hear that what they do stinks. But putting your head in the sand instead of throwing out of the garbage doesn't make it less smelly. UEFI is a garbage, both the firmware and the supporting library built on top of it. We can't throw out the firmware sadly, but we surely can and should throw out the rest! I'm not saying you're a paid M$ developer, all I'm saying is, brainwashing by chanting "Developers, developers, developers" might raise the endorphin level, but it surely doesn't make them write good programs and design good interfaces.

That's a plain straight insult. Totally not acceptable.
But that's not even the saddest thing here. The saddest thing is that I tried to suggest a "bloated" interface for an already bloated library and got insulted for that. You have no idea how much I like the UNIX + C philosophy, the KISS principle and how much I hate the bloated software too. Because of that, I'd never try to implement a wrapper on the top of UEFI. Whatever you add on the top of something that's already usable is "bloat", for me. That's why I use the thinnest and lightest library I can for UEFI, which is GNU-EFI. And, again, most of the time, that "library" is just a bunch of typedefs and GUIDs, no code. On the other side, there are people that would prefer to have some amount of "bloat", in favor of a different interface. Let's not judge them. Sometimes there are reasons to do that. I though you where one of them because you were willing to wrap UEFI functions with a not-so-thin layer, just because you didn't like that interface. That's a purely cosmetic thing. And, if you're willing to do that, it's logical to talk about what else can be done. Obviously, all the things I suggested were "bloat", but that's natural. If you don't want bloat, you don't write a wrapper library like that. Of course, it was clear to me after the 2nd post that you considered your product a perfect masterpiece and you where just looking for advertising it, not any kind of feedback. So, if you or nullplan didn't provoked me, I'd never EVER insisted. It's your project, and your project alone.

And yes, bzt could work on his temper and manners. ("Speaking nonsense" was not nice.)

I was calm all along. Why do you think I wasn't? I'm sorry if "nonsense" seemed offending to someone, but how else do you say in English if something is pointless and well, just does not make sense? I know no other phrase, sorry. No offense intended, I assure you.

You have no idea how rough and aggressive you are. And the sad thing is that you're not even a mega-guru like Linus Torvalds who can afford to do that. You're so affected by the Dunning–Kruger effect, you have no idea.

If vvaltchev or nexos think their idea is so good that it should be implemented they probably have to fork it. Otherwise let it be bzt's project.

I hope the emotions have not gone up the ceiling on either side and that we can go on in this forum with respect.

I totally agree. I'm so tired of this BS discussion. If bzt wants, we can ask moderators to just drop the whole thread. Bzt will re-post his project announce and we'll live happily ever after.

[eekee]

Um... smol question: Why is this called POSIX-UEFI?

For two reasons:

First, it provides a very small, easy to use build environment on POSIX systems, like Linux. You just include its Makefile and that detects the toolchain and sets it up correctly (GNU / LLVM with all the nasty flags) to produce UEFI PE executable and provides a suitable crt0 for them. (Unlike GNU-EFI, which does not compile with CLang for example, and unlike EDK2 which is a huge bloated build environment, lot more difficult to use than including a Makefile.)

Second, it provides a POSIXish library (and a single header with ANSI C standard definitions) for the application, nearly identical to the POSIX standard, with only consistent differences. This ensures a much much much shorter learning curve than GNU-EFI or EDK2. Most notable difference is, that it must use wchar_t, not char, otherwise all the function names and arguments are the same, so for example you don't have to think about what opening a file is called, you can be sure it's "fopen", and that it has two arguments, the first being the file's name, the second the opening mode. Similarly, if you want to copy memory, you don't have to think about what funky name EDK2 gave to that function, or what's the ordering of the parameters, you can just use memcpy() as usual. This makes development for UEFI under Linux (and other POSIX systems) easier than ever before. Yes, it is not 100% POSIX, that never was a goal, making it similar so that developers can shorten the learning curve was.

Cheers,
bzt

Oh I see. Thanks bzt. I would be surprised to find a Unix with fopen and not open, so perhaps POSIX is the better name.

[thewrongchristian]

Um... smol question: Why is this called POSIX-UEFI?

For two reasons:

First, it provides a very small, easy to use build environment on POSIX systems, like Linux. You just include its Makefile and that detects the toolchain and sets it up correctly (GNU / LLVM with all the nasty flags) to produce UEFI PE executable and provides a suitable crt0 for them. (Unlike GNU-EFI, which does not compile with CLang for example, and unlike EDK2 which is a huge bloated build environment, lot more difficult to use than including a Makefile.)

Second, it provides a POSIXish library (and a single header with ANSI C standard definitions) for the application, nearly identical to the POSIX standard, with only consistent differences. This ensures a much much much shorter learning curve than GNU-EFI or EDK2. Most notable difference is, that it must use wchar_t, not char, otherwise all the function names and arguments are the same, so for example you don't have to think about what opening a file is called, you can be sure it's "fopen", and that it has two arguments, the first being the file's name, the second the opening mode. Similarly, if you want to copy memory, you don't have to think about what funky name EDK2 gave to that function, or what's the ordering of the parameters, you can just use memcpy() as usual. This makes development for UEFI under Linux (and other POSIX systems) easier than ever before. Yes, it is not 100% POSIX, that never was a goal, making it similar so that developers can shorten the learning curve was.

Cheers,
bzt

Oh I see. Thanks bzt. I would be surprised to find a Unix with fopen and not open, so perhaps POSIX is the better name.

Perhaps it's more of a replacement for libc rather than POSIX?

To me, what makes a POSIX system over and above libc is stuff like the process model (fork/exec/pids/signals). POSIX/UNIX everything is a file/stream of bytes is already perfectly encapsulated by stdio.h, given a suitable file system namespace.

Whereas fopen is definitely a libc thing rather than just a POSIX thing.

Disclaimer: I'm coming into this from a position of ignorance, as I've not looked at all that POSIX-UEFI provides, and I'm making an assumption it doesn't provide a process model.

[nexos]

I could easily defend myself here, but there is no point doing so. PeterX summed the whole thing up quite well. We just need to add this to the list of flame wars that got way out of hand. A simple misunderstanding got this far. But, it is over now. Time to let it go, and let it rest.

[bzt]

Perhaps it's more of a replacement for libc rather than POSIX?

But libc is mostly defined by POSIX To be precise by the "ISO/IEC 9945-1, the POSIX System Application Program Interface". ISO C does not define things like dirent.h for example, POSIX does. But there's no clear cut margin here, ISO C and POSIX are quite intermixed when it comes to libc.

Disclaimer: I'm coming into this from a position of ignorance, as I've not looked at all that POSIX-UEFI provides, and I'm making an assumption it doesn't provide a process model.

Of course, neither does UEFI provide processes, so how could a library which only differs in the API? As @PeterX pointed out, POSIX-UEFI is "A small and simple interface (mimicying small parts of POSIX) to avoid many of the gory details of UEFI."

A simple misunderstanding got this far. But, it is over now. Time to let it go, and let it rest.

Interestingly someone always starts to demand irrational things from my projects, and then accuse me for being "abusive" and such. For example, read the SSFN thread where someone demanded text-shaping abilities from a font rasterizer library. And now here someone demanding POSIX kernel capabilities from an API wrapper library... Deja vu!

Cheers,
bzt

[nexos]

But libc is mostly defined by POSIX To be precise by the "ISO/IEC 9945-1, the POSIX System Application Program Interface". ISO C does not define things like dirent.h for example, POSIX does. But there's no clear cut margin here, ISO C and POSIX are quite intermixed when it comes to libc.

That is a source of confusion in case ISO C and POSIX conflict. From the POSIX spec:

The functionality described on this reference page is aligned with the ISO C standard. Any conflict between the requirements described here and the ISO C standard is unintentional. This volume of POSIX.1-2017 defers to the ISO C standard

Obviously, this may have happened once before .

[eekee]

Perhaps it's more of a replacement for libc rather than POSIX?

To me, what makes a POSIX system over and above libc is stuff like the process model (fork/exec/pids/signals). POSIX/UNIX everything is a file/stream of bytes is already perfectly encapsulated by stdio.h, given a suitable file system namespace.

There's a surprisingly deep philosophical question in there, one I'm not prepared to answer. It goes back a long way: I once read what was probably the very first book on C, introducing the language to the world. (That was normal in those days.) That book stated implementations of C on non-Unix systems should provide a Unix-like file interface via the libc.