running 32-bit code in LM64

[Octocontrabass]

So - negative indexes will indeed be an issue then? And I either need to map 4-8 GiB to 0-4 GiB or I need to switch to CM32, right?

Correct. (And maybe more than that, if you use a scaled index.)

I guess it would have been good if the former - mapping the 4-8 GiB region - was the default on UEFI unless you do a call to activate the high memory.

It would have been good for you, but bad for everyone else.

The unfortunate thing is this "preferred address" of executables being defaulted to somewhere in the 4-8 GiB region instead of either 8-12 (or even higher) or 0-4.

UEFI already requires all binaries to be relocatable, since it can't guarantee any "preferred address" will be available.

In hindsight, this might have been the way to support 32-bit executables running in a 64-bit environment.

You would still need to recompile existing 32-bit programs, since the instruction encoding is not completely backwards-compatible. Keeping the instruction encoding fully backwards-compatible would have made 64-bit code larger.

[kerravon]

So - negative indexes will indeed be an issue then? And I either need to map 4-8 GiB to 0-4 GiB or I need to switch to CM32, right?

Correct. (And maybe more than that, if you use a scaled index.)

Thanks for that additional consideration!

I guess it would have been good if the former - mapping the 4-8 GiB region - was the default on UEFI unless you do a call to activate the high memory.

It would have been good for you, but bad for everyone else.

Why would anyone be affected by this? What's wrong with keeping the 4-8 GiB virtual (not physical) region free?

The unfortunate thing is this "preferred address" of executables being defaulted to somewhere in the 4-8 GiB region instead of either 8-12 (or even higher) or 0-4.

UEFI already requires all binaries to be relocatable, since it can't guarantee any "preferred address" will be available.

Cool. Then they could make that change now, going forward, if I can convince people that it is a good idea. An unrelated development (a public domain 64-bit Windows compiler) means that I will likely to be switching to proper 64-bit soon, probably for a long time.

In hindsight, this might have been the way to support 32-bit executables running in a 64-bit environment.

You would still need to recompile existing 32-bit programs, since the instruction encoding is not completely backwards-compatible. Keeping the instruction encoding fully backwards-compatible would have made 64-bit code larger.

Sure - I don't want to interfere with 64-bit. I want 32-bit to be the one to suffer. Although if this "standard" had been around in 1986, some of the people back then might have complained that I was more concerned about 2005 (when Win64 came out) than 1986 or 1995 or whatever.

[Octocontrabass]

Why would anyone be affected by this? What's wrong with keeping the 4-8 GiB virtual (not physical) region free?

If you do that, virtual addresses are no longer equal to physical addresses. If virtual and physical addresses aren't equal, any software that works with physical addresses (such as a bootloader setting up page tables or a firmware driver using DMA) would need additional logic to convert between virtual and physical.

[kerravon]

Why would anyone be affected by this? What's wrong with keeping the 4-8 GiB virtual (not physical) region free?

If you do that, virtual addresses are no longer equal to physical addresses. If virtual and physical addresses aren't equal, any software that works with physical addresses (such as a bootloader setting up page tables or a firmware driver using DMA) would need additional logic to convert between virtual and physical.

If they're working at that level, can't they just switch the virtual memory mapping to what they want, rather than the (proposed) UEFI default?

Or perhaps UEFI could provide a call to change that mapping (but that mapping is already mismatched, isn't it? My preferred address above 4 GiB isn't necessarily being honored already).

Although people would then need to choose which set of software they wish to run. My 32-bit software won't work if they do the above. But that's probably fine. The people doing the above are probably also creating a CM32 environment.

[Octocontrabass]

If they're working at that level, can't they just switch the virtual memory mapping to what they want, rather than the (proposed) UEFI default?

That's still bad for everyone except you.

[kerravon]

If they're working at that level, can't they just switch the virtual memory mapping to what they want, rather than the (proposed) UEFI default?

That's still bad for everyone except you.

Ok - I was just about to reply - I finally realized. UEFI default is probably virtual = physical already, and that is why they force executables to be relocatable because they intend to relocate it rather than break their simple mapping.

Fine. They could provide a call then for people to be able to simply remap the 4-8 GiB region, and that this needs to be executed before negative indexes are used.

I have a similar situation on the mainframe. I have C code that executes prior to me setting up the page tables that I need. I have two choices - stay in AM31 until the page tables are ready, or hope that the C code doesn't generate any negative indexes.

Negative indexes are actually fairly rare at least the way I am doing things on the mainframe, so I do the latter currently, but may well switch to the former as it is more "pure".

I don't know if there is a direct equivalent concept in x64. Perhaps UEFI having a call to switch to CM32? Or starting in CM32? I don't think that's the same.

[kerravon]

BTW, it just occurred to me (inspired by the mainframe), that I could have made the 64-bit PE UEFI executables pure 32-bit code so long as the high halves of the 64-bit registers were 0 on entry. However, there is one big problem. The calling convention uses R8 and R9 which don't exist on the 80386. That's a great shame. If they had used rsi and rdi or something (Linux used those as well as the others) I could have stuck with 32-bit code all the way. In order to display "hello world" I only need 2 parameters, so I could achieve that, but I can't open either a file or a disk with only 2 parameters for UEFI calls.

[kerravon]

BTW, it just occurred to me (inspired by the mainframe), that I could have made the 64-bit PE UEFI executables pure 32-bit code so long as the high halves of the 64-bit registers were 0 on entry. However, there is one big problem. The calling convention uses R8 and R9 which don't exist on the 80386. That's a great shame. If they had used rsi and rdi or something (Linux used those as well as the others) I could have stuck with 32-bit code all the way. In order to display "hello world" I only need 2 parameters, so I could achieve that, but I can't open either a file or a disk with only 2 parameters for UEFI calls.

Actually, it occurred to me that I would want different calling conventions to call 64-bit UEFI compared to when I was calling my own 32-bit functions (in LM64) using stack trickery. That stack trickery still gives bog standard cdecl. If I need to call a UEFI function I need a different calling convention, and that calling convention is only relevant in LM64. So. I could have (even back in 1986) had a C compiler that recognizes yet another calling convention (on top of cdecl, stdcall, watcall, pascal), perhaps "lmcall", which would use ecx and edx for the first two parameters and e8 and e9 for the last two parameters, even though e8 and e9 don't exist on the 80386. I just need the compiler to generate the appropriate opcodes.

Next question - are e8 and e9 actually things, or do only r8 and r9 exist, and either way - are there any showstoppers that would prevent a 32-bit compiler from implementing lmcall so long as the code is never actually executed unless a LM64 environment exists? So e.g. in 1986, a 32-bit compiler has generated a bootx64.efi but it just sits there idly until a 64-bit-UEFI-only machine is actually manufactured.

[nullplan]

BTW, it just occurred to me (inspired by the mainframe), that I could have made the 64-bit PE UEFI executables pure 32-bit code so long as the high halves of the 64-bit registers were 0 on entry.

Why? In long mode, every 32-bit instruction zeroes out the top 32 bits of the 64-bit registers. You can clear the top half using any conventionally nonsense instruction such as

Code: Select all

mov ecx, ecx

.

Next question - are e8 and e9 actually things, or do only r8 and r9 exist,

If you mean the name for the low halves of r8 and r9, those are r8d and r9d. Not that it matters to a compiler. In machine code they are addressed as registers 0 and 1 (not sure off the top of my head which ones those are), but the REX prefix sets one of the extension bits.

You can typically get even a 32-bit assembler to generate REX prefixes by way of a "db" or ".byte" or "dc.b" pseudo-instruction. In this way, you can get any compiler to generate any machine code you should want.

[kerravon]

r8d and r9d

Ok, thanks. So, would:

mov eax eax

clear the top 32 bits of rax?

If so, how about this for a 32-bit C compiler:

1. __lmcall extension to say that the 64-bit ABI is being used for this function.

2. It is assumed that bootx64.efi will be loaded below 4 GiB, and the linker will provide an image base below 4 GiB to encourage that to happen. If it doesn't happen, you would need to buy a different computer.

3. The entry to bootx64.efi should be some assembler code that ensures all high 32 bits are cleared by doing mov eax, eax and mov r8d, r8d etc

4. In the absence of a UEFI call to set up ideal paging tables, it is necessary to disable interrupts and change the paging tables to map the 4-8 GiB region to 0-4 GiB, then reenable interrupts. No negative indexes can be used until this process completes. Negative indexes are pretty rare so shouldn't be an issue. Ideally there should be a way to see if UEFI has been updated to provide a suitable call to do this automatically (or even already be set by default) so that this system-related code normally never gets exercised.

5. Compiler is now able to use the default cdecl calling convention, by using stack manipulation instead of push/pop.

6. Other than any interfacing with UEFI (which wouldn't be done on a legacy system anyway - there would be conditional execution one way or another if that environment is detected - this is all valid 80386 code.

7. The 80386 code would likely call the win32 api, and more likely the msvcrt.dll in particular. A component external to the normal application will arrange for that to be translated into something appropriate - perhaps eventually calling 64-bit UEFI.

8. The 32-bit compiler needs to be selective in what instructions it generates. There are probably some situations where push and pop can be used (where it doesn't matter if that pushes 4 bytes or 8 bytes), but it wouldn't be the norm. Scaled indexes may never be appropriate (I don't know what the rules would be). Are there any other show-stoppers? Probably not, but some of these things need to be tested before there is a high degree of confidence.

There's probably no benefit to this over using CM32, unless you don't want interrupts disabled for a lengthy period of time. But it's at least a theoretical, interesting (to some) option, that would be nice to prove is actually viable.

Anything I've missed?

[Octocontrabass]

It is assumed that bootx64.efi will be loaded below 4 GiB, and the linker will provide an image base below 4 GiB to encourage that to happen. If it doesn't happen, you would need to buy a different computer.

Even if you can coerce the firmware into loading your binary below 4GB, you have no control over where the firmware puts your stack.

You already know with absolute certainty that bootx64.efi will never run on a 32-bit CPU, so there's no reason to limit any of the code in it to 32-bit addressing. If you're going to load a 32-bit binary from there, you can set up the 32-bit environment using 64-bit code.

4. In the absence of a UEFI call to set up ideal paging tables, it is necessary to disable interrupts and change the paging tables to map the 4-8 GiB region to 0-4 GiB, then reenable interrupts.

You can't re-enable interrupts until you restore the original UEFI page tables (or install your own ISRs that will restore the original UEFI page tables and call the original UEFI ISRs).

Ideally there should be a way to see if UEFI has been updated to provide a suitable call to do this automatically (or even already be set by default)

It's safe to assume UEFI will never be updated to provide either of those things.

There's probably no benefit to this over using CM32, unless you don't want interrupts disabled for a lengthy period of time.

Either way you can't enable interrupts unless you install your own ISRs to wrap the firmware's ISRs.

[kerravon]

It is assumed that bootx64.efi will be loaded below 4 GiB, and the linker will provide an image base below 4 GiB to encourage that to happen. If it doesn't happen, you would need to buy a different computer.

Even if you can coerce the firmware into loading your binary below 4GB, you have no control over where the firmware puts your stack.

I don't care where the stack is, do I? Any reference to esp will effectively be rsp, won't it?

You can't re-enable interrupts until you restore the original UEFI page tables (or install your own ISRs that will restore the original UEFI page tables and call the original UEFI ISRs).

I see - thanks.

Ideally there should be a way to see if UEFI has been updated to provide a suitable call to do this automatically (or even already be set by default)

It's safe to assume UEFI will never be updated to provide either of those things.

There is an open source UEFI used by Mr Chromebox on Chromebooks. I can potentially update it myself to get the function I want. If it doesn't work on non-Chromebooks, so be it.

[Octocontrabass]

I don't care where the stack is, do I?

Won't you have variables allocated on the stack? Won't you have pointers to those variables?

Any reference to esp will effectively be rsp, won't it?

Only in memory operands. In register operands, ESP is ESP.

If it doesn't work on non-Chromebooks, so be it.

If you don't care whether other people can use your software, that's fine, but I usually operate on the assumption that someone else might want to run the software I write, and they might not have access to the same hardware I do.

[kerravon]

I don't care where the stack is, do I?

Won't you have variables allocated on the stack? Won't you have pointers to those variables?

Any reference to esp will effectively be rsp, won't it?

Only in memory operands. In register operands, ESP is ESP.

Ok, so I need a UEFI implementation that puts the stack below 4 GiB too.

If it doesn't work on non-Chromebooks, so be it.

If you don't care whether other people can use your software, that's fine, but I usually operate on the assumption that someone else might want to run the software I write, and they might not have access to the same hardware I do.

My code won't work on an Amiga either.

If the "official UEFI" people aren't willing to cooperate - so be it. The people who may wish to run my software have an unreasonable UEFI vendor, they will indeed need to switch to a Chromebook or whatever does have a reasonable vendor.

Or complain to their UEFI vendor, rather than me.

But regardless - where would be a good place to put the proposed UEFI extension?

[Octocontrabass]

I don't think it matters where you put it since you're the only one who will implement it.