Debugging UEFI Issues on real hardware (POSIX-UEFI)

[kzinti]

You still need relocations on x86_64 because some UEFI firmwares will not load your UEFI app if it doesn't contain any relocation.

[davmac314]

You still need relocations on x86_64 because some UEFI firmwares will not load your UEFI app if it doesn't contain any relocation.

I'm curious about this. Do you know which particular firmwares exhibit this? Is it really "if it doesn't contain any relocation" or is it "if it is marked as having relocations stripped"? I've definitely seen the latter behaviour in the Ovmf firmware, but not the former.

GNU EFI does include a hack to make sure there is at least one PE-style relocation in the output PE image, but AFAICT that was to prevent objcopy from (wrongly) setting the "relocations stripped" flagged, not because the firmware actually requires a relocation to be present. I've definitely been able to execute an EFI image which had no relocations on that firmware.

In any case, GNU EFI / POSIX-UEFI could use -fpie rather than -fpic and still produce a PE executable with a "relocation" (via the hack they already use). The advantage would be there'd be no ELF relocations to process. (GNU EFI and POSIX-UEFI both include a stub which processes the relocations from the original ELF output).

[Octocontrabass]

Is it really "if it doesn't contain any relocation" or is it "if it is marked as having relocations stripped"? I've definitely seen the latter behaviour in the Ovmf firmware, but not the former.

Funny, I've seen OVMF load and execute a binary where I had accidentally stripped the relocations and it ran perfectly fine. I only caught the issue when I saw Mingw-w64 objdump wasn't showing HAS_RELOC. Unless that's a different flag?

[kzinti]

I'm curious about this. Do you know which particular firmwares exhibit this? Is it really "if it doesn't contain any relocation" or is it "if it is marked as having relocations stripped"? I've definitely seen the latter behaviour in the Ovmf firmware, but not the former.

GNU EFI does include a hack to make sure there is at least one PE-style relocation in the output PE image, but AFAICT that was to prevent objcopy from (wrongly) setting the "relocations stripped" flagged, not because the firmware actually requires a relocation to be present. I've definitely been able to execute an EFI image which had no relocations on that firmware.

I took this information from GNU-EFI and you might be right. I have not actually tested without relocations on any firmware.

[zaval]

if there are no references to absolute addresses (in both code and data), there could not be base relocations. so not having base relocations is valid and the only possible right situation if an image doesn't have references to absolute addresses. could it be possible, that it wouldn't? of course! my loader doesn't on x64 and arm64. that is, if there is a firmware refusing to load such an image, it's a wrong firmware. or, gnu-efi is.

[davmac314]

Is it really "if it doesn't contain any relocation" or is it "if it is marked as having relocations stripped"? I've definitely seen the latter behaviour in the Ovmf firmware, but not the former.

Funny, I've seen OVMF load and execute a binary where I had accidentally stripped the relocations and it ran perfectly fine. I only caught the issue when I saw Mingw-w64 objdump wasn't showing HAS_RELOC. Unless that's a different flag?

I think HAS_RELOC might be a BFD-specific flag (i.e. a flag internal to binutils), The flag in the actual PE header is called IMAGE_FILE_RELOCS_STRIPPED (and it has the inverse meaning of course), see: http://docwiki.embarcadero.com/RADStudi ... s_(Delphi)

My guess would be objdump reports "HAS_RELOC" if IMAGE_FILE_RELOCS_STRIPPED isn't set, but it might also (or instead) depend on the presence of a relocation section (.reloc).

It's possible that OVMF will run an EFI with relocs stripped *if* it can load it at its nominal address (i.e. without needing to perform any relocations); I'm not sure. I was doing a lot of experiments a little while back and it seemed like the IMAGE_FILE_RELOCS_STRIPPED flag being present or not was a key difference that could cause an EFI to either run or be refused, but it's hard to be certain. The GNU EFI hack to add a relocation section also prevents objcopy from setting IMAGE_FILE_RELOCS_STRIPPED, and that flag seemed to be the deciding factor, rather than the actual presence or absence of relocations. (I used a utility called "readpe" to check actual PE header flags rather than rely on what objdump told me).

[nexos]

Whats suspicious to me is the use of strings here. What I am wondering is if it might have something to do with .rodata, as the OP said other boot services work. This isn't going to get much further however, as bzt closed the issue on Gitlab as an issue with @charco's hardware. I have my doubts...

Also, I personally wouldn't use ConOut, but rather use the GOP to set a video mode, and then write my own print string function, as I have noticed some firmware (such as OVMF) have some rather atrocious looking fonts.

[Octocontrabass]

My guess would be objdump reports "HAS_RELOC" if IMAGE_FILE_RELOCS_STRIPPED isn't set, but it might also (or instead) depend on the presence of a relocation section (.reloc).

The only difference between my intended 64-bit binary and the accidentally stripped one is IMAGE_FILE_RELOCS_STRIPPED. (Also timestamps and checksums, but those don't matter here.) I'm using a cross-compiler, not the GNU-EFI hack.

It's possible that OVMF will run an EFI with relocs stripped *if* it can load it at its nominal address (i.e. without needing to perform any relocations); I'm not sure.

That would make sense. The binaries I was testing had ImageBase set to 0x400000, so they would easily fit into available memory in the default QEMU+OVMF configuration.

(I used a utility called "readpe" to check actual PE header flags rather than rely on what objdump told me).

The version of Mingw-w64 objdump I'm using does report the PE header flags, but it doesn't use the names from the PE documentation.

[zaval]

the PE specifications states, when this flag (IMAGE_FILE_RELOCS_STRIPPED) is set, then the image must be loaded at its preferred base address. this is why a normal FW would refuse loading such a file if (and only if) the address is not available.

Image only, Windows CE, and
Windows NT® and later. This indicates
that the file does not contain base
relocations and must therefore be
loaded at its preferred base address. If
the base address is not available, the
loader reports an error. The default
behavior of the linker is to strip base
relocations from executable (EXE) files.

that is, this flag is 1) not to indicate, that an image is naturally position independent (because of PC/IP-relative accesses), but that this file must go where it's linked to. and 2) it's default for EXEs, which are main modules for programs and there is no reason to not place it where it wants. Note, however, that UEFI OS Loaders are being linked as DLLs (by sane compilers) and thus relocations are never get stripped, it's just there might be none of them. but the latter case cannot cause FW to refuse loading such an image.

[Octocontrabass]

and 2) it's default for EXEs, which are main modules for programs and there is no reason to not place it where it wants.

You've got some old information there. Neither MSVC nor Clang strip relocations when they generate EXEs. (GCC still does, for some reason.) Relocations are required for ASLR.

Note, however, that UEFI OS Loaders are being linked as DLLs (by sane compilers) and thus relocations are never get stripped, it's just there might be none of them.

No sane compiler links UEFI applications as DLLs. With MSVC and Clang, you don't need to do anything special since relocations are not stripped by default. GCC has the "-pie" flag.

I've seen a few guides that recommend linking UEFI applications as DLLs, but I'm not sure why.

[zaval]

No sane compiler links UEFI applications as DLLs. With MSVC and Clang, you don't need to do anything special since relocations are not stripped by default. GCC has the "-pie" flag.

I've seen a few guides that recommend linking UEFI applications as DLLs, but I'm not sure why.

I don't think, MSVC linker v14.20 is old and I didn't do anything special, except setting /subsystem:EFI_APPLICATION and here is what it produces:

E:\Valera>dumpbin /headers out\antosldr\aarch64\antload.efi
Microsoft (R) COFF/PE Dumper Version 14.20.27508.1
Copyright (C) Microsoft Corporation. All rights reserved.


Dump of file out\antosldr\aarch64\antload.efi

PE signature found

File Type: DLL

FILE HEADER VALUES
AA64 machine (ARM64)
4 number of sections
60D14129 time date stamp Tue Jun 22 04:47:21 2021
0 file pointer to symbol table
0 number of symbols
F0 size of optional header
2022 characteristics
Executable
Application can handle large (>2GB) addresses
DLL

as of not stripping relocations by default for EXEs, I see, it's true, but the quoted was from the current PE spec.
I am not sure what effect the gcc's -pie flag would have on PE targets, if any. in the ELF context, it's a total opposite to the PE approach, and it hardly would work for the latter, because PE loader does apply base relocations, that is fixes up addresses in the code and data itself, while -pie would require fixing up the contents of a special indirection table (GOT), which any PE loader (and UEFI's one as well) has no idea about. so that -pie thing would only work only if there is an ELF "interpreter" embedded into a resulting image, that would care about doing these crazy things on taking control over at the very start. such things are for bzt, for some ELF-UEFI-POSIX.

I can see these recommendations given specifically because of the mentioned reason - to not get base relocations stripped by the linker that thinks they aren't needed for an EXE (which at least was the case in past as seen in the spec).

[davmac314]

No sane compiler links UEFI applications as DLLs. With MSVC and Clang, you don't need to do anything special since relocations are not stripped by default. GCC has the "-pie" flag.

Note that If compiling with GCC and linking with GNU ld (to `pei-XXX` format eg pei-x86-64) you can use `--emit-reloc-section` at link time (possibly requires `-mi386pep` linker emulation setting) rather than `-fpie` at compile time, if you want to preserve relocations (rather than eliminate the need for them, which is what `-fpie` achieves).

[davmac314]

I am not sure what effect the gcc's -pie flag would have on PE targets, if any. in the ELF context, it's a total opposite to the PE approach, and it hardly would work for the latter, because PE loader does apply base relocations, that is fixes up addresses in the code and data itself, while -pie would require fixing up the contents of a special indirection table (GOT)

Only if there are dynamically linked libraries, which in the case of an EFI application there are not. The GOT isn't needed, since every variable is local to the application itself. In this case using -fpie produces an executable which neither has nor requires any relocations. (In contrast, -fpic will create a GOT regardless).

Edit: in fact -fpie won't ever produce a GOT. Any variables will always be located within the executable image.
Edit 2: you can possibly get a GOT with -fpie if you refer to a function address where the function is from a shared library, without calling it (calls should normally go via the PLT instead). But still, with an EFI application that won't happen.

[Octocontrabass]

File Type: DLL

Interesting! It's been a while since I looked, so maybe I remembered wrong. (Or maybe MSVC is not a sane compiler.) Doesn't the PE spec say DLLs aren't supposed to be executed directly?

I am not sure what effect the gcc's -pie flag would have on PE targets, if any.

It tells Mingw-w64 GCC to use linker options that preserve relocations. I don't know if this applies to all PE targets or just Mingw-w64; I don't use any of GCC's other PE targets.

Note that If compiling with GCC and linking with GNU ld (to `pei-XXX` format eg pei-x86-64) you can use `--emit-reloc-section` at link time (possibly requires `-mi386pep` linker emulation setting) rather than `-fpie` at compile time, if you want to preserve relocations (rather than eliminate the need for them, which is what `-fpie` achieves).

Hold on, there's a difference between "-fpie" and "-pie". Mingw-w64 GCC ignores options like "-fpie"; it unconditionally generates relocatable code. Only link options like "-pie" have any effect. I haven't checked exactly how "-pie" changes the options passed to the linker when linking with GCC.

[zaval]

Interesting! It's been a while since I looked, so maybe I remembered wrong. (Or maybe MSVC is not a sane compiler.) Doesn't the PE spec say DLLs aren't supposed to be executed directly?

It says this about that flag:

The image file is a dynamic-link library (DLL). Such files are considered executable files for almost all purposes, although they cannot be directly run.

what is true for a usual OS environment. UEFI environment differs from the former, obviously, but again, the flag is used here for practical reasons (preserving relocations), but if to think about technicalties, then an UEFI application is not a standalone program, running in its own address space and thus - its image doesn't constitute the main module for that process, that after starting will possess its own address space, the image gets loaded into the existing address space of the firmware process, so, it's DLL in the sense, that it's not the main image, but loadable, secondary one. or, it's more like a driver - an OS loading driver. and drivers are a kind of DLLs.