OSDev.org

I'm trying to boost performance of my operating system, and one area I noticed I could possibly improve is signal dispatching. This is happening on x86_64.

When a signal is dispatched to a thread, the kernel does the following:

1) Subtracts 128 bytes from the user stack pointer, to preserve the red zone.
2) Pushes the values of all registers in the interrupted userspace state onto the user stack.
3) Pushes a fake return address onto the stack, called TRAP_SIGRET.
4) Passes control to the signal handler, in userspace.

Now, when the signal handler returns, this causes a jump to TRAP_SIGRET. This is an invalid address, which triggers a page fault, the kernel sees that RIP is set to TRAP_SIGRET, and so performs the signal return by restoring all registers from the user stack, then passes control back to the interrupted state.

I want to get rid of the overhead of having the kernel involved in the signal return.

I have a special area of memory known as the "user support page", which is shared by every process, and is read-only and executable in userspace. Currently it contains some code that helps with multithreading.

I thought that perhaps instead of using the TRAP_SIGRET trick, I actually push a return address inside of the user support page, which would point to a signal return procedure that executes in userspace.

The problem is, I can't think of any method, in userspace, of restoring ALL the registers. Note that this involves also restoring the stack pointer and the instruction pointer, and that there is a red zone, plus there may be an alternative signal stack etc.

I was thinking of perhaps somehow using the IRET instruction, but from what I see in the intel instruction set reference, it doesn't restore RSP when running in userspace. Besides, it sets CS, which should not be allowed in userspace, so it seems that it is privilieged althought I didn't see the manual stating that explicitly.

So I ask, is it possible, in userspace, to restore all registers from the stack, including the stack pointer itself, and the instruction pointer?

While I'm not sure if iret is a useful tool for you (yes, it doesn't update ss:rsp when you stay in the same privilege level), setting cs is actually allowed in userspace as long as you select a ring 3 code segment - of which you probably only have a single one, so cs is just reloaded with the same value.

Kevin wrote:You can just assign rsp like any other general purpose register, and then use a register indirect jmp to restore rip.

And while I'm not sure if iret is a useful tool for you (yes, it doesn't update ss:rsp when you stay in the same privilege level), setting cs is actually allowed in userspace as long as you select a ring 3 code segment - of which you probably only have a single one, so cs is just reloaded with the same value.

Register-indirect jmp wouldn't work, as that would destroy one of the registers.

Noticed this myself and edited it out, but you were too quick.

The reason why it works for setjmp/longjmp is that you have some caller saved registers there that you can use, but for signals, that's obviously not true.

mariuszp wrote:So I ask, is it possible, in userspace, to restore all registers from the stack, including the stack pointer itself, and the instruction pointer?

You can restore the stack pointer and instruction pointer at the same time using RET imm16.

1) Subtract 128+size_of_saved_registers from the user RSP to preserve the red zone
2) Push the userspace RIP and RFLAGS to the stack
3) Move the userspace registers to the space reserved for them
4) Pass control to the userspace signal handler
5) The signal handler restores the saved registers and pops RFLAGS
6) The signal handler uses RET 128+size_of_saved_registers to pop RIP and restore the previous RSP

This also avoids using pop to restore most registers (push/pop are relatively slow instructions).

Octocontrabass wrote:

mariuszp wrote:So I ask, is it possible, in userspace, to restore all registers from the stack, including the stack pointer itself, and the instruction pointer?

You can restore the stack pointer and instruction pointer at the same time using RET imm16.

1) Subtract 128+size_of_saved_registers from the user RSP to preserve the red zone
2) Push the userspace RIP and RFLAGS to the stack
3) Move the userspace registers to the space reserved for them
4) Pass control to the userspace signal handler
5) The signal handler restores the saved registers and pops RFLAGS
6) The signal handler uses RET 128+size_of_saved_registers to pop RIP and restore the previous RSP

This also avoids using pop to restore most registers (push/pop are relatively slow instructions).

Ah, I didn't know about the existence of the "ret imm16" instruction. The only problem is that this still wouldn't work for alternate signal stacks, but I guess for now I could handle that as a separate, less efficient case.