Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Has anyone successfully implemented SYSCALL/SYSRET fastcall as the syscall mechinism in their OS?
I'm struggling to find a sensible way to use them. The obvious problem of not switching to the kernel stack is a pain. In an SMP system it isn't fun to find the kernel stack... without a stack... and would lead to a lot of uglyness in the syscall entry point. I can't just map the per thread kernel stack in a predefined location as all my threads in the same process to share the same PML4.
I read an old posting on a linux kernel development list where two developers were discussing SYSCALL/SYSRET. They mentioned changing the CS after a SYSCALL and before the SYSRET (ie. the thread being preempted while in the kernel) caused the processor to set some internal flag and raise a GP fault. You couldn't then iret to a different thread. Surely this can't be true?
With more googling I found a presentation that mentions a hidden part of the GS register available to use as a pointer to the kernel stack when using SYSCALL. Anyone know anything about this?
A software interrupt is looking better by the minute...
Basically your create a segment that points to the base of some kind of kernel processor specific information structure, then access the fields using the gs register prefix and manual offsets. So you could essentially store a processor specific SYSCALL stack in this structure then access it in your syscall handler and likewise storing the user RSP. Certain threading systems do the same thing with the FS register iirc. GS should have 2 MSRs to set it's base address in long mode, as well as the "swapgs" instruction which can switch to a kernel specific base address.
Pretty much, be careful about actually loading the GS register with a selector value though. Doing so will automatically zero the GS base register regardless of the MSR values due to how long mode segmentation functions. If you swapgs afterwards it'll also store that value in the Kernel GS base and possibly a myriad of problems down the line. It's easy enough to work around though.
