Page 1 of 1
UEFI GetMemoryMap success with odd results
Posted: Mon May 21, 2018 11:56 pm
by spencerb
Hi. I've been working on a UEFI bootloader and have been having some issues with the memory map. I'm sure its just a stupid error on my part but I'd gone over this several times and would appreciate if someone might be able to point out what I'm doing wrong.
Here's the relevant code:
It succeeds with no errors but the memory map 'seems' to contain random data. At least, that's what I assume for a few reasons.
For one, the UEFI spec states that the EFI_MEMORY_DESCRIPTOR.PhysicalStart will be 4KiB aligned - I get values of 0, 7, 0, 1857182, - you get the idea.
Same situation for the VirtualStart field. I assumed since the paging is identity-mapped, the VirtualStart would either be equal to the PhysicalStart or 0. But it appears to be random.
Finally, the thing that struck me as most odd was that the EFI_MEMORY_DESCRIPTOR.Type values are large. The Type value is from an the EFI_MEMORY_TYPE enum with only a handful of values, yet I'm getting extremely high values for this field.
Again, I'm probably missing something simple, in which case I'm sorry for wasting anyone's time, but I've pored over this as well as the spec and I can't seem to find the problem.
Maybe my assumptions as the specified values are wrong? Either way, thanks.
Code: Select all
EFI_STATUS
EFIAPI
efi_main (EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable) {
InitializeLib(ImageHandle, SystemTable);
EFI_STATUS status = EFI_SUCCESS;
EFI_MEMORY_DESCRIPTOR *memory_map = NULL;
UINT32 version = 0;
UINTN map_key = 0;
UINTN descriptor_size = 0;
UINTN memory_map_size = 0;
uefi_call_wrapper(ST->ConOut->ClearScreen, 1, ST->ConOut);
status = uefi_call_wrapper(BS->GetMemoryMap, 5, &memory_map_size, memory_map, &map_key, &descriptor_size, &version);
// descriptor size and version should be correct values. memory_map_size should be needed size.
if (status == EFI_BUFFER_TOO_SMALL) {
UINTN encompassing_size = memory_map_size + (2 * descriptor_size);
void *buffer = NULL;
status = uefi_call_wrapper(BS->AllocatePool, 3, EfiLoaderData, encompassing_size, &buffer);
if (status == EFI_SUCCESS) {
memory_map = (EFI_MEMORY_DESCRIPTOR*) buffer;
memory_map_size = encompassing_size;
status = uefi_call_wrapper(BS->GetMemoryMap, 5, &memory_map_size, memory_map, &map_key, &descriptor_size, &version);
if (status != EFI_SUCCESS) {
Print(L"Second call to GetMemoryMap failed for some reason.");
} else {
Print(L"Second call to GetMemoryMap succeeded.\n");
for (int i = 0; i < (memory_map_size / descriptor_size); ++i) {
Print(L"Physical Address of i-th memory descriptor:\t%x\n", memory_map[i].PhysicalStart);
Print(L"Virtual Address of i-th memory descriptor:\t%x\n", memory_map[i].VirtualStart);
Print(L"Memory Type of i-th memory descriptor:\t%d\n", memory_map[i].Type);
uefi_call_wrapper(BS->Stall, 1, 4000000);
}
}
} else {
Print(L"AllocatePool failure.");
}
} else if (status == EFI_SUCCESS) { // shouldn't happen.
Print(L"First call to GetMemoryMap should never succeed... ???");
} else {
Print(L"(First) GetMemoryMap usage failure.");
}
return EFI_SUCCESS;
}
Re: UEFI GetMemoryMap success with odd results
Posted: Tue May 22, 2018 12:10 am
by justinian
spencerb wrote:
Code: Select all
for (int i = 0; i < (memory_map_size / descriptor_size); ++i) {
Print(L"Physical Address of i-th memory descriptor:\t%x\n", memory_map[i].PhysicalStart);
Print(L"Virtual Address of i-th memory descriptor:\t%x\n", memory_map[i].VirtualStart);
Print(L"Memory Type of i-th memory descriptor:\t%d\n", memory_map[i].Type);
uefi_call_wrapper(BS->Stall, 1, 4000000);
}
In my experience,
sizeof(EFI_MEMORY_DESCRIPTOR) is never the same as the returned
descriptor_size. So instead of indexing like an array, you'll need to do pointer arithmetic instead.
Re: UEFI GetMemoryMap success with odd results
Posted: Tue May 22, 2018 7:05 am
by spencerb
That worked, thanks!
I was really banging my head on a brick wall.
Seems odd that it would behave like that. Is it padding between descriptors in the map?
Regarding forum etiquette, do I need to put a [solved] tag or anything like that?
Re: UEFI GetMemoryMap success with odd results
Posted: Tue May 22, 2018 10:56 am
by justinian
Mostly I think it's for alignment? The values I was seeing looked like they were 16-byte aligning the descriptors, even though most compilers would give them smaller alignment restrictions.
I've been posting [solved] tags on my threads when they're solved, but I haven't seen it explicitly mentioned in the rules. Helps for people searching, though.
Re: UEFI GetMemoryMap success with odd results
Posted: Tue May 22, 2018 11:48 am
by zaval
seriously, do you really think that
is any different from
?
Of course not. It's the same. Your compiler relies on the typedef of EFI_MEMORY_DESCRIPTOR struct anyway. So if it's not as in the UEFI implementation you are working with, it will be broken in both cases.
So you did fix something, but what exactly - nobody knows.
As of alignment. EFI_MEMORY_DESCRIPTOR structure really is defined in a braindamaged way in the spec, namely:
Code: Select all
typedef struct {
UINT32 Type;
EFI_PHYSICAL_ADDRESS PhysicalStart;
EFI_VIRTUAL_ADDRESS VirtualStart;
UINT64 NumberOfPages;
UINT64 Attribute;
} EFI_MEMORY_DESCRIPTOR;
As we see, the Type is defined as UINT32 and all other members following are UINT64, so, basically compiler needs to insert a padding after the Type member. But it can't be a problem you've experienced.
And also:
The Type value is from an the EFI_MEMORY_TYPE enum with only a handful of values, yet I'm getting extremely high values for this field.
It's not exactly that, because:
UEFI spec wrote:
MemoryType
The type of memory to allocate. The type EFI_MEMORY_TYPE is defined in “Related Definitions” below. These memory types are also described in more detail in Table 25 and Table 26. Normal allocations (that is, allocations by any UEFI application) are of type EfiLoaderData. MemoryType values in the range 0x80000000..0xFFFFFFFF are reserved for use by UEFI OS loaders that are provided by operating system vendors. The only illegal memory type values are those in the range EfiMaxMemoryType..0x7FFFFFFF.
Yes, it says that it's for "OS loaders", but who knows what the implementation you work with is "thinking", maybe it also wants to be an "OS Loader" a little.
Show the fixed code.
Re: UEFI GetMemoryMap success with odd results
Posted: Tue May 22, 2018 12:35 pm
by justinian
zaval wrote:seriously, do you really think that
is any different from
?
No. But
Code: Select all
(EFI_MEMORY_DESCRIPTOR*)(((uint8_t*)memory_map) + descriptor_size)
is. It's easy enough to verify this for yourself - it's even true on OVMF.
Re: UEFI GetMemoryMap success with odd results
Posted: Tue May 22, 2018 1:15 pm
by zaval
^ ah, you mean casting to byte pointers. when you said "using pointer arithmetics", I didn't get that you meant this. Yes, it's "easy enough". But it's ugly AF as well. And shouldn't be needed. In the ideal world.
Maybe checking for sizeof(EFI_MEMORY_DESCRIPTOR) == MemoryDescriptor (as returned by GetMemoryMap()) and/or version match are needed.
Anyway, it's not OK to f&ck up the structure without indicating this in the specification. There is no "vendor" extension allowed, only "future standard extensions".
Interesting, what numbers are in your case, OP, for:
sizeof(EFI_MEMORY_DESCRIPTOR)
MemoryDescriptor (as returned by GetMemoryMap())
EFI_MEMORY_DESCRIPTOR_VERSION as defined in headers you use
MemoryDescriptorVersion as returned by GetMemoryMap()
Re: UEFI GetMemoryMap success with odd results
Posted: Tue May 22, 2018 1:44 pm
by justinian
I agree that it's ugly (I hide it behind macros, GNU-EFI does something similar), but it's definitely needed. One of the major goals of UEFI is to allow for future extensibility while retaining backward compatibility.
This is how they allow for software written against older specs to run on firmware implementing newer specs - If later versions add fields, the initial fields must remain the same, but the structure will be larger. GetMemoryMap() returns the descriptor size, so the old software can still find the start of the next structure because it knows it at run time instead of using a version hard-coded at compile time.. Also note that the specification explicitly states that it covers only the interface between firmware and software, not the implementation or memory layout of either - vendors are free to implement it how they like.
Re: UEFI GetMemoryMap success with odd results
Posted: Tue May 22, 2018 2:22 pm
by zaval
EFI_MEMORY_DESCRIPTOR is the interface between FW and its clients, defined by the specification. It is not allowed to have been changed arbitrarily by implementers, only by future expansions by the specification.
I just wanted to say, that it would be better to check for version match and take advantage of this (reading new fields, bringing even more mess to your code, yay). And only as a resort, when you face something you are unaware of, you do that trickery.
But in fact all those "expansions" look not on time, there is nothing to expand, that's why it more looks like a screwing up things. Maybe the poster shows us numbers I asked for.
Re: UEFI GetMemoryMap success with odd results
Posted: Tue May 22, 2018 7:54 pm
by spencerb
I'm using gnu-efi for lack of a need (atm) of anything more advanced.
sizeof(EFI_MEMORY_DESCRIPTOR) is 40 bytes,
but the descriptor size returned by GetMemoryMap is 48.
The returned descriptor version and the definition in the header are both 1.
Here's the fixed code. As mentioned above, I just cast to char* and increased by
descriptor_size bytes before casting back to EFI_MEMORY_DESCRIPTOR for the result.
The output makes
much more sense now. The physical addresses are finally 4KiB aligned as per spec, and virtual addresses are 0.
Code: Select all
EFI_STATUS
EFIAPI
efi_main (EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable) {
InitializeLib(ImageHandle, SystemTable);
EFI_STATUS status = EFI_SUCCESS;
EFI_MEMORY_DESCRIPTOR *memory_map = NULL;
UINT32 version = 0;
UINTN map_key = 0;
UINTN descriptor_size = 0;
UINTN memory_map_size = 0;
uefi_call_wrapper(ST->ConOut->ClearScreen, 1, ST->ConOut);
status = uefi_call_wrapper(BS->GetMemoryMap, 5, &memory_map_size, memory_map, &map_key, &descriptor_size, &version);
// descriptor size and version should be correct values. memory_map_size should be needed size.
if (status == EFI_BUFFER_TOO_SMALL) {
UINTN encompassing_size = memory_map_size + (2 * descriptor_size);
void *buffer = NULL;
status = uefi_call_wrapper(BS->AllocatePool, 3, EfiLoaderData, encompassing_size, &buffer);
if (status == EFI_SUCCESS) {
memory_map = (EFI_MEMORY_DESCRIPTOR*) buffer;
memory_map_size = encompassing_size;
status = uefi_call_wrapper(BS->GetMemoryMap, 5, &memory_map_size, memory_map, &map_key, &descriptor_size, &version);
if (status != EFI_SUCCESS) {
Print(L"Second call to GetMemoryMap failed for some reason.");
} else {
Print(L"Second call to GetMemoryMap succeeded.\n");
char *mem = (char*) memory_map;
Print(L"sizeof(EFI_MEMORY_DESCRIPTOR):\t%d\n", sizeof(EFI_MEMORY_DESCRIPTOR));
Print(L"descriptor_size:\t%d\n", descriptor_size);
Print(L"EFI_MEMORY_DESCRIPTOR_VERSION = %d", version);
for (int i = 0; i < (memory_map_size / descriptor_size); ++i) {
EFI_MEMORY_DESCRIPTOR desc = *( (EFI_MEMORY_DESCRIPTOR*) mem);
Print(L"\n");
Print(L"Physical Address of i-th memory descriptor:\t%x\n", desc.PhysicalStart);
Print(L"Virtual Address of i-th memory descriptor:\t%x\n", desc.VirtualStart);
Print(L"Memory Type of i-th memory descriptor:\t%d\n", desc.Type);
uefi_call_wrapper(BS->Stall, 1, 4000000);
mem += descriptor_size;
}
}
} else {
Print(L"AllocatePool failure.");
}
} else if (status == EFI_SUCCESS) { // shouldn't happen.
Print(L"First call to GetMemoryMap should never succeed... ???");
} else {
Print(L"(First) GetMemoryMap usage failure.");
}
return EFI_SUCCESS;
}
Re: UEFI GetMemoryMap success with odd results
Posted: Wed May 23, 2018 4:55 am
by zaval
Thanks for your reply.
Now it's clear - the UEFI implementation you are working with has screwed up the descriptors.
And this is really pissing off.
Indeed, the size of the descriptor version 1 should be 40 bytes.
Code: Select all
/*
* typedef UINT64 EFI_PHYSICAL_ADDRESS;
* typedef UINT64 EFI_VIRTUAL_ADDRESS;
*/
typedef struct _EFI_MEMORY_DESCRIPTOR{
UINT32 Type;
UINT32 _Pad;
EFI_PHYSICAL_ADDRESS PhysicalStart;
EFI_VIRTUAL_ADDRESS VirtualStart;
UINT64 NumberOfPages;
UINT64 Attribute;
}EFI_MEMORY_DESCRIPTOR, *PEFI_MEMORY_DESCRIPTOR, **PPEFI_MEMORY_DESCRIPTOR;
Re: UEFI GetMemoryMap success with odd results
Posted: Wed May 23, 2018 5:57 am
by linuxyne