OSDev.org

It came to my attention a while ago that there are major differences in the Intel implementation and AMD implementation of x86-64.
Intel processors do not have SYSCFG, TOP_MEM, etc; not a very big deal as one can live without them. But the one thing that irritates me most is the way of doing system calls.
Intel processors allow SYSENTER and SYSEXIT in both sub-modes of long-mode; and allows SYSCALL and SYSRET only in 64bit mode.
AMD do not allow SYSENTER and SYSEXIT in any sub-mode of long mode.

If I understand correctly, that would create a problem for writing the standard library as one will have to create 2 versions of it. One using SYSENTER/SYSEXIT, the other using SYSCALL/SYSRET. Also, the OS will need to install the correct version.
Is there a good way of getting around this problem other than using INT?

Try break down the problem into kernel and user land.

For kernel, it's probably good and trivial to setup all syscall stub entries if available (syscall, sysenter, optionally INT)

For userland, you may have one single syscall() interface in a system library, and possible multiple versions for different mechanism (eg. /usr/arch/i386/lib/syscall.so and /usr/arch/x86_64/lib/syscall.so), the correct version can be linked to application on load time.

Linking the correct version at load isn't required, the correct version needs to be copied during the install process.
Basically, copy the SYSENTER/SYSEXIT version if Intel processor; copy the SYSCALL/SYSRET version if AMD processor.

By version, I mean whether the library uses SYSENTER/SYSEXIT or SYSCALL/SYSRET.

If you feel good when the theoretical user upgrade his computer (which was using sysenter, set by installer) to a new computer (which cannot use sysenter)...

IMO, that code is tiny and it's trivial to support them all with some detections, you want to do it for CPU extensions anyway (MMX, SSE, etc).
It may be a be tricky to do it elegantly.

By the way, the other approach is to catch the #UD and emulate it.

One does not simple upgrade to a new processor without reinstalling the OS.

Why don't you just use SYSENTER/SYSEXIT in i386 mode and SYSCALL/SYSRET in amd64 mode?

Why don't you just leave support for pmode away?

trinopoty wrote:One does not simple upgrade to a new processor without reinstalling the OS.

This is not required by Windows, Linux(2.6+), BSD(6+), Mac, and Solaris 9+ (intel version). And my personal experience is that my server machine running FreeBSD 6.2 has be upgrade passed a few processor generations without reinstalling nor "update world", until I was bored enough to upgrade it to 9 recently.

Anyway, if you are talking about One does not simple upgrade to a new processor without reinstalling the(your) hobby OS - I agree and it is acceptable restriction, although this restriction can be avoided easily.

It's trivial to place all the system calls together in a table, and initialize them according to what is needed. You can even do all this in C.

Well, I agree that it is easy to keep all of it in the kernel. As for the user-mode library, I think i will write two versions of the syscall() function.

AMD were stupid in designing their version of AMD64.
They decided that SYSENTER/SYSEXIT won't work in compatibility mode, but will work in legacy mode (32 bit operating system).
There was no excuse for that design decision.

You have 32bit cpl3 code using SYSENTER that works perfectly ok when an AMD processor is running a 32 bit protected mode OS.
But the exact same 32bit cpl3 application code won't work ON THE SAME CPU when in 64 bit compatibility mode.
Absolutely atrocious. AMD can not make any justification for this.

One Possibility is to have 1 Memorypage for the Systemcall in userspace
and 3 Pages with different Syscall/sysenter/int Options in your Kernelspace.
And - dependent on your CPU - you map one of those 3 Pages from your kernel into Userspace.
i'm not sure, but i think this is the way Windows kernel does.
(google Kernelbase.dll)

Best regards