Page 1 of 1

[SOLVED] Switch calling convention from UEFI to kernel

Posted: Wed Apr 24, 2024 7:21 am
by scippie
I have written lots of kernels but they were all for MBR.
Now I am finally working on an UEFI loader for my kernel, and it works perfectly: it sets up all the necessary data and passes it on to the kernel it loads from a dedicated partition.

Both of them are written in C, UEFI is compiled with mingw and uses the UEFI calling convention, the Kernel is compiled with gcc to an ELF binary and uses the Linux calling convention.

The call works, but the parameter is passed incorrectly, which is logical as it is passed on through register RCX instead of RDI. The call returns properly and as both conventions expect, the return value is in RAX.

I tried adding __cdecl to the typedef of the entry function in the hopes that mingw would change the calling conventions for that function, but it doesn't help:

Code: Select all

typedef int __cdecl (*KERNEL_MAIN)(KernelConfig *config);
...
KERNEL_MAIN kernel_main = (KERNEL_MAIN)(kernel_location + elf->ProgramEntryOffset);
int return_value = kernel_main(&config);
The config is not being passed correctly.

So I improvised and came up with:

Code: Select all

__asm__ __volatile__ ("mov %0, %%rdi\n" : : "r"(&config) : );
int return_value = kernel_main(&config);
This means both RDI and RCX are passing the config data, and this works perfectly.

However, it feels like I am hacking. What if the next version of the C compiler decides that it needs to fiddle with RDI in between of those two lines for some reason? It will no longer work. I could of course also add the call to the kernel_main function in assembler to fix that.

But there must be a better way... why doesn't the __cdecl attribute help with this? Am I doing it wrong? Is there something else I need to do?

Re: Switch calling convention from UEFI to kernel

Posted: Wed Apr 24, 2024 8:02 am
by Octocontrabass
scippie wrote:why doesn't the __cdecl attribute help with this?
It's only valid for 32-bit x86.
scippie wrote:Is there something else I need to do?
Use the sysv_abi attribute instead. Your code should look something like this:

Code: Select all

typedef int (__attribute__((sysv_abi)) *KERNEL_MAIN)(KernelConfig *config);

Re: Switch calling convention from UEFI to kernel

Posted: Wed Apr 24, 2024 8:07 am
by scippie
Octocontrabass wrote:

Code: Select all

typedef int (__attribute__((sysv_abi)) *KERNEL_MAIN)(KernelConfig *config);
That was easy, thanks!

However, I find it strange that they couldn't keep the original attribute. The compiler knows it is compiling for 32-bit or 64-bit, so it could know what I mean.

But it works, thanks!

Re: Switch calling convention from UEFI to kernel

Posted: Wed Apr 24, 2024 9:38 pm
by Octocontrabass
scippie wrote:The compiler knows it is compiling for 32-bit or 64-bit, so it could know what I mean.
How would the compiler know what you mean? There's no such thing as 64-bit cdecl.

Re: Switch calling convention from UEFI to kernel

Posted: Thu Apr 25, 2024 1:34 am
by scippie
Octocontrabass wrote:
scippie wrote:How would the compiler know what you mean? There's no such thing as 64-bit cdecl.
I'm speaking more high level. Obviously the compiler doesn't.
But if in 32-bit cdecl means "use the calling conventions of Linux", then when 64-bit came around, I would have chosen to make cdecl mean "use the 64-bit calling conventions of Linux".

But obviously, another path has been chosen, and I am getting to know it. The cdecl keyword is something from the olden days, while this __attribute__ keyword offers much more options. I then even wonder why the cdecl keyword is still accepted in 64-bit.

Re: [SOLVED] Switch calling convention from UEFI to kernel

Posted: Thu Apr 25, 2024 8:46 am
by nullplan
scippie wrote:But obviously, another path has been chosen, and I am getting to know it. The cdecl keyword is something from the olden days, while this __attribute__ keyword offers much more options. I then even wonder why the cdecl keyword is still accepted in 64-bit.
According to the documentation, the sysv_abi attribute ought to be equivalent to the cdecl attribute. How true that is, I don't know. Experimenting on godbolt a bit showed me that __cdecl is recognized as a keyword only sometimes. clang warns about it on ARMv7, while GCC doesn't recognize it at all. But on ARMv8, it is accepted without warning or error. On loongarch64, it didn't work at all, so it is not a 64-bit thing.

I would expect that where supported, __cdecl is equivalent to __attribute__((sysv_abi)). But no, that is not true, because you have a case where it is simply doing nothing. So I presume this is for source compatibility only, although adding -D__cdecl= to the command line would have done the same thing.

Re: Switch calling convention from UEFI to kernel

Posted: Thu Apr 25, 2024 7:53 pm
by Octocontrabass
scippie wrote:But if in 32-bit cdecl means "use the calling conventions of Linux", then when 64-bit came around, I would have chosen to make cdecl mean "use the 64-bit calling conventions of Linux".
But cdecl is the default calling convention on 32-bit Windows too. Win32 API functions override the default to use stdcall. (On 64-bit Windows, everything uses ms_abi.)
scippie wrote:I then even wonder why the cdecl keyword is still accepted in 64-bit.
Source compatibility. If you have a bunch of code that was written for 32-bit Windows, that's one less thing you need to change when you compile it into a 64-bit binary.