SYSRET ignores the Ring 3 segment selector?

[fbkr]

Hi all,

I'm trying to use sysret to get to ring 3, but I've ran into a huge roadblock. For some reason, sysret just ignores what I put in the STAR MSR, and loads a fixed value to CS and SS.

Even though I put 0x1b for "SYSRET CS and SS" field of the star MSR, I can see in bochs that CS is loaded with 0x13 and SS with 0xb. Interestingly, I would expect to get these values if I put 0 in the star register.

Running it under qemu (with and without kvm) and bochs works, and I can use syscall/sysret back and forth without a problem, but upon taking a timer interrupt from ring 3 -> 0, I get a GPF at iretq, possibly related to this error.

Am I missing something?

This is what I have in the GDT:

Code: Select all

Global Descriptor Table (base=0x0000000000252000, limit=63):
GDT[0x0000]=??? descriptor hi=0x00000000, lo=0x00000000
GDT[0x0008]=Code segment, base=0x00000000, limit=0x00000fff, Execute/Read, Non-Conforming, Accessed, 64-bit
GDT[0x0010]=Data segment, base=0x00000000, limit=0x00000000, Read/Write, Accessed
GDT[0x0018]=??? descriptor hi=0x00000000, lo=0x00000000
GDT[0x0020]=Data segment, base=0x00000000, limit=0x00000000, Read/Write
GDT[0x0028]=Code segment, base=0x00000000, limit=0x00000fff, Execute/Read, Non-Conforming, 64-bit
GDT[0x0030]=32-Bit TSS (Busy) at 0x00261000, length 0x00068
GDT[0x0038]=??? descriptor hi=0x00000000, lo=0x00000000

Not sure why it says Non-Conforming here or 32-Bit TSS, but I could not find a proper resource on addressing those.

This is what I use to initialize the entire syscall/sysret stuff:

Code: Select all

void switch_to_user() {
    wrmsr(msrs::ia32_efer, rdmsr(msrs::ia32_efer) | 1ULL);

    wrmsr(msrs::star, (uint64_t(0x18ULL | 0x3ULL) << 48U) | (uint64_t(0x8ULL) << 32U));
    wrmsr(msrs::lstar, reinterpret_cast<uint64_t>(&raw_syscall_entry));
    wrmsr(msrs::sfmask, 0);

    asm volatile("pushf\n"
                 "popq %r11");
    sysret(reinterpret_cast<void*>(user_code));
}

Assembly:

Code: Select all

  0000000000200adf <switch_to_user()>:
  200adf:       b9 80 00 00 c0          mov    $0xc0000080,%ecx
  200ae4:       0f 32                   rdmsr  
  200ae6:       48 83 c8 01             or     $0x1,%rax
  200aea:       b9 80 00 00 c0          mov    $0xc0000080,%ecx
  200aef:       0f 30                   wrmsr  
  200af1:       48 b8 00 00 00 00 08    movabs $0x1b000800000000,%rax
  200af8:       00 1b 00 
  200afb:       b9 81 00 00 c0          mov    $0xc0000081,%ecx
  200b00:       0f 30                   wrmsr  
  200b02:       b8 10 00 20 00          mov    $0x200010,%eax
  200b07:       b9 82 00 00 c0          mov    $0xc0000082,%ecx
  200b0c:       0f 30                   wrmsr  
  200b0e:       b9 84 00 00 c0          mov    $0xc0000084,%ecx
  200b13:       31 c0                   xor    %eax,%eax
  200b15:       0f 30                   wrmsr  
  200b17:       9c                      pushfq 
  200b18:       41 5b                   pop    %r11
  200b1a:       b9 0a 0a 20 00          mov    $0x200a0a,%ecx
  200b1f:       48 0f 07                sysretq 
 

[zhiayang]

I'm a little confused:

Hi all,
... and I can use syscall/sysret back and forth without a problem, but upon taking a timer interrupt from ring 3 -> 0, I get a GPF at iretq, possibly related to this error.

What are you doing, actually? You mention that syscall/sysret works without a problem, so I doubt the issue is with CS/SS not being loaded from the MSR correctly. The code itself looks fine (and it should be fine, because it works for you). What is your actual error scenario? Do you get a GPF when userspace services a ring 0 interrupt and you try and iret? That, afaik, has nothing to do with sysret or the MSRs.

Again, your code that sets the MSRs looks fine, as does your GDT (TSS notwithstanding --- that could be an issue if you did not set it up correctly)

[Gigasoft]

WRMSR uses EDX:EAX for the value to write, not RAX.

[Octocontrabass]

Not sure why it says Non-Conforming here or 32-Bit TSS, but I could not find a proper resource on addressing those.

Have you tried the Intel or AMD architecture manuals?

Code: Select all

    wrmsr(msrs::sfmask, 0);

You probably want to write 0x600 here instead of 0.

Code: Select all

    asm volatile("pushf\n"
                 "popq %r11");

You probably want to put 0x200 into R11 instead of the current RFLAGS value. Also, this needs to be handled as a parameter to sysret instead of a separate inline assembly block.

Code: Select all

  200af1:       48 b8 00 00 00 00 08    movabs $0x1b000800000000,%rax
  200af8:       00 1b 00 
  200afb:       b9 81 00 00 c0          mov    $0xc0000081,%ecx
  200b00:       0f 30                   wrmsr  

Your inline assembly for wrmsr (and presumably also rdmsr) has wrong constraints.

[fbkr]

WRMSR uses EDX:EAX for the value to write, not RAX.

Ugh, this was it. I just realized I only ever used wrmsr/rdmsr in the 32-bit loader side, and didn't pay attention to them in long mode. Thank you!

What are you doing, actually? You mention that syscall/sysret works without a problem, so I doubt the issue is with CS/SS not being loaded from the MSR correctly.

Yes, somehow syscall/sysret worked with the wrong descriptors. After enabling IRQ0, I was able to handle the interrupt, but could not return. While debugging it, I saw the pushed SS and CS were wrong so I figured setting them must've been wrong.

Have you tried the Intel or AMD architecture manuals?

Just took a look and got it. I thought non-conforming meant I set something wrong, rather than being a "thing".

[Octocontrabass]

Yes, somehow syscall/sysret worked with the wrong descriptors.

That's how it's supposed to work. SYSCALL/SYSRET use fixed values instead of reading the descriptors from the GDT to speed up the control transfer. You get odd behavior when they're combined with other instructions (like IRET) that do use the descriptors from the GDT if the descriptors in the GDT don't match those fixed values.

[fbkr]

Yes, somehow syscall/sysret worked with the wrong descriptors.

That's how it's supposed to work. SYSCALL/SYSRET use fixed values instead of reading the descriptors from the GDT to speed up the control transfer. You get odd behavior when they're combined with other instructions (like IRET) that do use the descriptors from the GDT if the descriptors in the GDT don't match those fixed values.

That makes sense. I'm coming from an (modern-ish) ARM background so I just keep getting confused by the complexity of x86 stuff :/