SYSCALL/SYSRET exact details

[Candy]

I was mucking about with my syscall/sysret stuff and one result of that was the definition of my system calls. They're all like:

Code: Select all

syscall_something:
  mov rax, syscall_id_of_something
  syscall

This is then hidden in a C function that pretends to be normal so that the compiler will just push all arguments, like so:

Code: Select all

void syscall_something(int x, int y);

I then add this to handle the system call:

Code: Select all

syscall_handler:
  cmp rax, 23
  jae no_syscall
  mov rbx, [syscall_table + rax]
  call rbx
no_syscall:
  pop rcx
  sysret

Now I was thinking about this sysret in particular. It ignores the stored result of the syscall itself (which is in rcx) and uses the top value of the stack (which is the value in user-land) to return. This might cause a hackable system where somebody inserts an arbitrary pointer in there causing it to execute that code instead (which might be kernel code etc.). I've checked the AMD manuals and they assure me that CPL3 is set before the RIP is loaded. Is this also true for Intel machines and has anybody tested anything with either of these instructions?

[YeXo]

If you're not sure whether a pentium sets CPL3 before returning, why not check the return value yourself. You can just pop the value of the stack in for example rax, check if the value is in userspace and than push it on the stack again.

[Candy]

If you're not sure whether a pentium sets CPL3 before returning, why not check the return value yourself. You can just pop the value of the stack in for example rax, check if the value is in userspace and than push it on the stack again.

The point was within the instruction. If it tries to load the next instruction before actually switching to CPL3, I'll get a kernel-space exception, which would not directly terminate the application that caused it. If it does switch to CPL3 before loading the next instruction it'll just be a normal user-space exception which means the programmer didn't do what he should've and it'll just crash normally (as it should). The AMD-style handling is the one I need for no crashes without extra logic. I don't know about the intel implementation however (nor do I know for sure that AMD did this right).

[B.E]

I Don't if it is the same for 64-bit, but i just looked at the 32-bit intel manual and it says

Code: Select all

Prior to executing the SYSEXIT instruction, software must specify the privilege level 3 code segment and code entry point, and the privilege level 3 stack segment and stack pointer by writing values into the following MSR and general-purpose  registers:
? SYSENTER_CS_MSR?Contains the 32-bit segment selector for the privilege level 0 code segment in which the processor is currently executing. (This value is used to compute the segment selectors for the privilege level 3 code and stack segments.)
? EDX?Contains the 32-bit offset into the privilege level 3 code segment to the first instruction to be executed in the user code.
? ECX?Contains the 32-bit stack pointer for the privilege level 3 stack. The SYSENTER_CS_MSR MSR can be read from and written to using the RDMSR and WRMSR instructions. The register address is listed in Table 4-3. This address is defined to remain fixed for future IA-32 processors. 

When the SYSEXIT instruction is executed, the processor does the following:
1. Adds 16 to the value in SYSENTER_CS_MSR and loads the sum into the CS selector register.
2. Loads the instruction pointer from the EDX register into the EIP register.
3. Adds 24 to the value in SYSENTER_CS_MSR and loads the sum into the SS selector
register.
4. Loads the stack pointer from the ECX register into the ESP register.
5. Switches to privilege level 3.
6. Begins executing the user code at the EIP address.

[Candy]

Code: Select all

Prior to executing the SYSEXIT instruction, software must specify the privilege level 3 code segment and code entry point, and the privilege level 3 stack segment and stack pointer by writing values into the following MSR and general-purpose  registers:
? SYSENTER_CS_MSR?Contains the 32-bit segment selector for the privilege level 0 code segment in which the processor is currently executing. (This value is used to compute the segment selectors for the privilege level 3 code and stack segments.)

Sysenter/sysexit != syscall/sysret.

[B.E]

Sysenter/sysexit != syscall/sysret.

Sorry I thought they were the same. SYSCALL/SYSRET in both processors do set CPL before RIP is set.
Here is what intel does from the manual.

Code: Select all

IF (OPERAND_SIZE = 64) THEN (* Return to 64-Bit Mode *)
EFLAGS = R11;
CPL = 0x3;
CS(SEL) = IA32_STAR[63:48] + 16;
CS(PL) = 0x3;
SS(SEL) = IA32_STAR[63:48] + 8;
SS(PL) = 0x3;
RIP = RCX;

here is what AMD does from the manual.

Code: Select all

IF (OPERAND_SIZE = 64) // Return to 64-bit mode.
{
CS.sel = (MSR_STAR.SYSRET_CS + 16) OR 3
CS.base = 0x00000000
CS.limit = 0xFFFFFFFF
CS.attr = 64-bit code,dpl3
temp_RIP.q = RCX
}

SS.sel = MSR_STAR.SYSRET_CS + 8 // SS selector is changed,
// SS base, limit, attributes unchanged.
RFLAGS.q = R11 // RF=0,VM=0
CPL = 3
RIP = temp_RIP

[Candy]

OK, nice, thanks