Usefulness of UEFI

[bzt]

This in no way invalidates my point.

Yes, because what you're excepting is something that AllocPool can't provide. And you haven't answered the question at all, how could ExitBootServices detect where's the kernel code segment loaded into memory. Let's say it's loaded in an allocated memory region with type EfiLoaderData. It still only contains the raw segment, without any magic bytes, and you don't know it's actual VMA where it's going to be mapped.

How is it a personal insult if its true? You have aptly demonstrated on this topic alone that you are unable or unwilling to acknowledge that you are wrong.

No sir, do not even try to blame this on me. You're the one who is unwilling to acknowledge Microsoft's official statements and CVE tickets and working Proof of Concepts. And refusing to answer my questions. Think about it.

Cheers,
bzt

[rdos]

It's not allocated because there's absolutely no guarantee that UEFI allocation would return the address where the executable is linked at (should I say yet another design failure...? Why AllocPool not reading the pointer passed to it?) And I didn't wanted to overcomplicate this simple tutorial by mapping the load address to the link address, so it was easier to use an address that's surely above the wasted UEFI memory.

I agree that this is really a poor design decision. Some things (like switching to real mode or protected mode from long mode) requires fixed physical addresses to work, but there is no guarantee that you can get a physical address below 1M (or below 4G either), and so you need to guess which physical address might be non-used & present in most UEFI implementations. It wouldn't have been a problem to reserve some range for boot/OS loaders.

[kzinti]

I agree that this is really a poor design decision. Some things (like switching to real mode or protected mode from long mode) requires fixed physical addresses to work, but there is no guarantee that you can get a physical address below 1M (or below 4G either), and so you need to guess which physical address might be non-used & present in most UEFI implementations. It wouldn't have been a problem to reserve some range for boot/OS loaders.

I am not sure why you would need or want to switch to real mode or protected mode from long mode, but let's address the myth that you have to guess anything to do it.

UEFI uses identity mapping. You don't have to guess what the physical memory address is: it's the same as its virtual address. Using AllocatePages() with AllocateMaxAddress, you can allocate memory below any address you want. This is in the design of the UEFI API, it wasn't missed.

Example:

Code: Select all

physaddr_t AllocatePages(int pageCount, physaddr_t maxAddress)
{
    EFI_PHYSICAL_ADDRESS memory = maxAddress;
    EFI_STATUS status = g_efiBootServices->AllocatePages(AllocateMaxAddress, EfiLoaderData, pageCount, &memory);
    if (EFI_ERROR(status))
    {
        Fatal("Out of memory");
    }

    return memory;
}

...

// Allocate 10 page of memory below 1MB
auto memory = AllocatePages(10, 0x100000);

So if you want to allocate memory below 1MB, 4GB or any other address, you can easily do it.

If you are guessing anything, you are doing it wrong.

[rdos]

UEFI uses identity mapping. You don't have to guess what the physical memory address is: it's the same as its virtual address. Using AllocatePages() with AllocateMaxAddress, you can allocate memory below any address you want. This is in the design of the UEFI API, it wasn't missed.

I don't want just any address below 1M or 4G. I want some specific address that I can use as a trampolline when I disable paging. I need this address when the scheduler switches a core between protected mode and long mode (and back) too. Just any address will not do. It must be a fixed address.

There is a way to suggest an address, but this will fail if it is not free, and so it's virtually impossible to reserve some physical address for this with UEFI, while it works fine with BIOS booting.

What I do is to fail to boot if the fixed address cannot be allocated.

[kzinti]

In that case you would have to either design your trampoline to work from any address and/or change your design. I agree this is extra work that you didn't need to do with the BIOS, but that's doesn't make UEFI a broken system.

Different software will have different constraints. UEFI provides memory management, the BIOS doesn't. Expecting UEFI to be compatible with every quirks of the BIOS is unreasonable.

Although this is not guaranteed by the spec, my experience shows that in practice UEFI firmwares will not allocate memory below 1MB on PCs unless you ask for it. That memory is obviously precious. Consider that mainstream OSes like Windows and Linux both need to be able to allocate memory below 1MB for SMP trampolines. How likely is it that UEFI gets in the way and prevent that?

What you can do is ExitBootServices() and then install the trampoline where you want it. But really, make it position-independent to be extra safe (yes, some work required).

Here is an example of a SMP AP startup trampoline (real mode) that can run at any address below 1MB:
https://github.com/kiznit/rainbow-os/bl ... ia32/smp.S

[rdos]

In that case you would have to either design your trampoline to work from any address and/or change your design. I agree this is extra work that you didn't need to do with the BIOS, but that's doesn't make UEFI a broken system.

Different software will have different constraints. UEFI provides memory management, the BIOS doesn't. Expecting UEFI to be compatible with every quirks of the BIOS is unreasonable.

Although this is not guaranteed by the spec, my experience shows that in practice UEFI firmwares will not allocate memory below 1MB on PCs unless you ask for it. That memory is obviously precious. What you can do is ExitBootServices() and then install the trampoline where you want it. But really, make it position-independant to be extra safe (yes, some work required).

I don't need it in the boot-process since I can switch from long mode to protected mode by setting the code selector base before I switch mode. I can also start the kernel basically in the same way (by setting the code selector base to the start of the kernel module). I had this with BIOS booting too so I could load the OS image at any position. The problem is with switching from protected mode to long mode. I've not been able to figure out a way to do that without a trampoline at a fixed physical address. The main problem is that the code segment always atarts at 0 in long mode and that my assembler (JWasm) has some limitations when it comes to mixing 32-bit and 64-bit code. It might be possible to solve this though.

Here is an example of a SMP AP startup trampoline (real mode) that can run at any address below 1MB:
https://github.com/kiznit/rainbow-os/bl ... ia32/smp.S

The trampoline must be at a fixed physical address since it's called from the BIOS / UEFI after it's internal initialization (mine is at 100:0). You might call this another problem with UEFI. There should have been an API for setting up the AP entry-points, and they should have run in the same mode as UEFI itself. That would have required this to have been part of the run-time services.

Actually, my OS also requires that a few pages at the bottom of physical memory are free for compatibility reasons with older BIOS loaders. I cannot change this since the loaders are written to disc on some older systems that still run.

So, basically, I assume that the first page is the real mode IDT (and BIOS settings) and that the second page is used for the AP trampoline. After that, a few more pages will be used in the early initialization of the kernel before the physical memory map is constructed.

[kzinti]

The problem is with switching from protected mode to long mode. I've not been able to figure out a way to do that without a trampoline at a fixed physical address. The main problem is that the code segment always atarts at 0 in long mode and that my assembler (JWasm) has some limitations when it comes to mixing 32-bit and 64-bit code. It might be possible to solve this though.

Here is another SMP trampoline to go from real mode -> protected mode -> long mode. It also works from any address below 1MB:
https://github.com/kiznit/rainbow-os/bl ... 6_64/smp.S

Hopefully you can adapt the technic to work with your assembler.

The trampoline must be at a fixed physical address since it's called from the BIOS / UEFI after it's internal initialization (mine is at 100:0). You might call this another problem with UEFI. There should have been an API for setting up the AP entry-points, and they should have run in the same mode as UEFI itself. That would have required this to have been part of the run-time services.

I must say that you are losing me here. In my mind the firmware has nothing to do with trampolines. What do you mean by the BIOS or UEFI calling your trampolines? Neither firmware has support to help your kernel start APs... Why is it a flaw that UEFI doesn't provide such runtime services? The BIOS doesn't either... isn't that also a flaw? Why not start the APs from your kernel (or after exiting boot services) instead of starting them earlier?

Actually, my OS also requires that a few pages at the bottom of physical memory are free for compatibility reasons with older BIOS loaders. I cannot change this since the loaders are written to disc on some older systems that still run.

I don't know the specifics of what you are dealing with and I don't know that there is a solution to your problem here. But I will say that the BIOS and UEFI firmware are effectively two different platforms. You shouldn't expect one to be compatible with the other. Do you also expect the Raspberry Pi to be compatible with older BIOS loaders? Surely not. UEFI was not designed to be compatible with the BIOS or any software built on it.

Surely your older systems that are still running are using the BIOS. So I am not sure why UEFI would be a problem for them since they are not using it.

[rdos]

The problem is with switching from protected mode to long mode. I've not been able to figure out a way to do that without a trampoline at a fixed physical address. The main problem is that the code segment always atarts at 0 in long mode and that my assembler (JWasm) has some limitations when it comes to mixing 32-bit and 64-bit code. It might be possible to solve this though.

Here is another SMP trampoline to go from real mode -> protected mode -> long mode. It also works from any address below 1MB:
https://github.com/kiznit/rainbow-os/bl ... 6_64/smp.S

Hopefully you can adapt the technic to work with your assembler.

No difference. You still need to locate the code where UEFI jumps to after the initial code is done. When you reset an AP it will start executing in real mode in the UEFI/BIOS area, and you cannot place any code there. So, the way it is solved is that you write a value to the CMOS where you want it to start, but this cannot be anywhere below 1MB. A convenient place is at 0x100:0.

I must say that you are losing me here. In my mind the firmware has nothing to do with trampolines. What do you mean by the BIOS or UEFI calling your trampolines?

That the CPU reset-code is located in the UEFI/BIOS, and so it is always UEFI code that is executed first. When it has finished its own initialization of the core, it will jump to an address that is indicated in the CMOS, and then your code must be here.

Neither firmware has support to help your kernel start APs... Why is it a flaw that UEFI doesn't provide such runtime services? The BIOS doesn't either... isn't that also a flaw? Why not start the APs from your kernel (or after exiting boot services) instead of starting them earlier?

The logic is identical between BIOS and UEFI. The AP core will start executing at the RESET vector in BIOS address space, and then your code will be called based on CMOS settings.

I think this is a flaw in UEFI. First, because this makes it impossible for Intel and AMD to remove real mode, and second because you still need to have code to switch to protected mode or long mode, which UEFI does for you for the BSP.

[kzinti]

No difference. You still need to locate the code where UEFI jumps to after the initial code is done. When you reset an AP it will start executing in real mode in the UEFI/BIOS area, and you cannot place any code there. So, the way it is solved is that you write a value to the CMOS where you want it to start, but this cannot be anywhere below 1MB. A convenient place is at 0x100:0.

What do you mean by resetting the AP and why do you need to do that? Are you talking about sending startup IPIs or something else? With startup IPIs you can start your AP at any address that is page-aligned. There is no BIOS or UEFI code/vector involved here. It works just fine and I've never had to do anything with the CMOS.

That the CPU reset-code is located in the UEFI/BIOS, and so it is always UEFI code that is executed first. When it has finished its own initialization of the core, it will jump to an address that is indicated in the CMOS, and then your code must be here.

Sounds like you are not talking about startup IPIs. What are these resets and what are they for?

The logic is identical between BIOS and UEFI. The AP core will start executing at the RESET vector in BIOS address space, and then your code will be called based on CMOS settings.

Saying that the UEFI will use the reset vector in BIOS address space doesn't seem to make much sense to me. If you are using UEFI, there is no BIOS. So what exactly is the "BIOS address space" in this case?

I think this is a flaw in UEFI. First, because this makes it impossible for Intel and AMD to remove real mode

Yet they are doing it. It's happening.

and second because you still need to have code to switch to protected mode or long mode, which UEFI does for you for the BSP.

That means UEFI does more for you than the BIOS does. UEFI switches the BSP to protected or long mode for you where the BIOS doesn't. So UEFI being helpful here is a design flaw?

[rdos]

You're right. It's the SIPI message that contains the base address, and so at least theoretically, you don't need to have a fixed address of the AP initialization code. OTOH, this is not done in the loader, rather as part of initializing the APIC in the OS kernel, and so if you don't use a fixed address, this will be another thing that must be propagated from the EFI system through the loader down to the OS kernel, and not only for UEFI boots but also for BIOS (even though this could be constant).

I think I will continue to assume that the first page is real mode IDT (it actually is even with UEFI), the second page is the AP initialization, and a few pages after that must also be free for use by the OS initialization code. This far, no UEFI claims those are used for anything, and so I don't see the problem with requiring these to always be free.

The hand-off between UEFI/BIOS through loader to the OS kernel is already complex enough and shouldn't need to cope with this kind of strangeness from UEFI. Part of the problem is that initially, the BIOS loader would search for available memory by itself without asking BIOS, something that was motivated by a desire to create an embedded platform without BIOS. While this is no longer an issue, it's still reflected in the interface between the loader and the OS kernel which supplies available memory areas above 1M only, and always requires the lower 1M area to be present & free.

Provided the OS had a physical memory allocator that could allocate memory below 1M, it might be possible to use this too, but I cannot see any other use of that function, so it would mostly be bloat to the physical memory handler. I also don't use flat selectors in the AP initialization code, and so not using a fixed address would require patching the base. The GDT is dynamic and per core and so cannot be created statically in the trampoline code.

[quirck]

and second because you still need to have code to switch to protected mode or long mode, which UEFI does for you for the BSP.

That means UEFI does more for you than the BIOS does. UEFI switches the BSP to protected or long mode for you where the BIOS doesn't. So UEFI being helpful here is a design flaw?

I wish UEFI was more clear about details of paging mode it sets up. Will it use 5-level paging if available? Will CR4.PCIDE be set? If an OS loader isn't prepared for all that, it's probably going to crash. Who knows what future control bits will be added that need to be accouted for.

[bzt]

That means UEFI does more for you than the BIOS does. UEFI switches the BSP to protected or long mode for you where the BIOS doesn't. So UEFI being helpful here is a design flaw?

I've implemented SMP for RPi, coreboot, BIOS, UEFI etc. And yes, UEFI was the most problematic of all because you must comply with an unknown environment (like GDT and paging), plus jump to a relocated address after set up you don't know at compilation time (and for which you can't generate a relocation record, since the jump is in protmode code, but your PE is going to be compiled for long mode, plus CS in GDT is not specified and can vary from firmware to firmware). Sometimes doing less is actually more.

But the most hilarious design flaw in UEFI is, that it has MP Services in theory to hide all of these problems, yet you can't use that in practice because MP Services is a boot service, so must be stopped by the time you call ExitBootServices... Therefore a completely unusuable service, a pure waste of firmware resources!

Cheers,
bzt

[Gigasoft]

And can you prove that UEFI will allocate the memory area where the ELF is linked at? Nope, of course you can't. UEFI or not, you must use that address because that's where the segment is linked at.

That's just silly. The very first thing you'll likely do is to set up yourself an initial paging structure, are you saying that you can't live with having this simple and tiny piece of code be position independent?

I've not been able to figure out a way to do that without a trampoline at a fixed physical address.

Likewise, this should be a straightforward piece of code. Disable NMI, disabling paging, load new IDTR, CR3 and TR, enable long mode, enable paging, enable NMI. Why does this function need to reside at a fixed address?

[kzinti]

The hand-off between UEFI/BIOS through loader to the OS kernel is already complex enough and shouldn't need to cope with this kind of strangeness from UEFI.

Sounds like your design and abstractions haven't aged well. This happens all the time in software development. I assure you that you can start APs from your kernel without any special handoff between your bootloader and kernel.

There is absolutely no relation whatsoever between starting up APs and the firmware. In fact I use the same code on both BIOS and UEFI and it works just fine.

I wish UEFI was more clear about details of paging mode it sets up.

The UEFI spec is pretty clear that all available memory will be identity-mapping (effectively it's like there is no mapping, so you don't need to do anything here).

Paging mode is enabled and any memory space defined by the UEFI memory map is identity
mapped (virtual address equals physical address), although the attributes of certain regions
may not have all read, write, and execute attributes or be unmarked for purposes of platform
protection. The mappings to other regions are undefined and may vary from implementation
to implementation.

Will it use 5-level paging if available? Will CR4.PCIDE be set?

Why would it matter? You can read CR4.PCIDE if you really want to know. But it shouldn't matter to any bootloader out there if UEFI is using 4 or 5 levels for paging.

If an OS loader isn't prepared for all that, it's probably going to crash. Who knows what future control bits will be added that need to be accounted for.

I see no reason for the UEFI to fail in this way in the future. New processors and motherboards are released all the time and bootloaders continue to work. You seem to imply that if an UEFI implementation used 5-level paging it would crash existing bootloader. I don't see why that would be the case. Can you elaborate what your thinking is here?

I've implemented SMP for RPi, coreboot, BIOS, UEFI etc. And yes, UEFI was the most problematic of all because you must comply with an unknown environment (like GDT and paging), plus jump to a relocated address after set up you don't know at compilation time

That's because you are not supposed to start APs while UEFI boot services are active. Until you call ExitBootServices(), the firmware owns your hardware devices (including the APs) and you can't do as you want with them.

I understand you wanted to get the APs up and running in your bootloader and ready to be used by the kernel and I understand that the approach you took was difficult / impossible with UEFI. But you can always start them after ExitBootServices() if that's really what you want to do. Sometimes it's better to work with the constraints of the system you are dealing with rather than to try to swim against the current.

[quirck]

Paging mode is enabled and any memory space defined by the UEFI memory map is identity
mapped (virtual address equals physical address), although the attributes of certain regions
may not have all read, write, and execute attributes or be unmarked for purposes of platform
protection. The mappings to other regions are undefined and may vary from implementation
to implementation.

<...>

You seem to imply that if an UEFI implementation used 5-level paging it would crash existing bootloader. I don't see why that would be the case. Can you elaborate what your thinking is here?

That quote effectively allows UEFI to make any mappings of EfiConventionalMemory (readonly, no-execute, etc), it may even not map it at all. So the first thing to do after ExitBootServices is to switch to known good page tables.

Consider loaders written prior to introduction of 5-level paging. They might just construct PML4 and point CR3 to that, unaware that CPU can treat it as PML5. This isn't going to work, that code needs to be rewritten. Even if an OS isn't going to support 57-bit linear addresses, it still needs to be aware of it.

Consider another approach when we just want to disable paging altogether. "MOV to CR0 causes a general-protection exception if it would clear CR0.PG to 0 while CR4.PCIDE = 1", for example. Again, if we know about this bit, we can clear it in advance and disable paging afterwards. But how should we handle bits that are reserved now? Is there a safe path if at some time in future our code reads them as 1?