[SOLVED] Getting into ring3 on x86_64

[bluemoon]

I have a piece of code on test application:

Code: Select all

_start:
    ud2
    dd      0xdeadbeef
    jmp     _start

    xor     rbp, rbp
    mov     edi, 0
    mov     rsi, env
    call    main

    xor     rdi, rdi
    call    _exit
    ret

If I execute such code directly in ring0, #UD as expected:

Code: Select all

PSTUB   : FFFFFFFF:8012D740 Executing [/initrd/testapp]
PSTUB   : FFFFFFFF:8012D740 Program entry: 00000000:001000D0 -> F8EBDEAD:BEEF0B0F
INT06 : #UD Invalid Opcode Exception. RIP: 00000000:001000D0

However, if I do switch to ring3, it seems do not execute at all and only repeating process switch happens.

Code: Select all

SCHED : Current Process: FFFFFFFF:8012E740 Next Process: FFFFFFFF:8012F150, CR3: 00000000:01FEE000 Remain: 10
SCHED : Current Process: FFFFFFFF:8012F150 Next Process: FFFFFFFF:8012E740, CR3: 00000000:01FF1000 Remain: 10
SCHED : Current Process: FFFFFFFF:8012E740 Next Process: FFFFFFFF:8012F150, CR3: 00000000:01FEE000 Remain: 10
SCHED : Current Process: FFFFFFFF:8012F150 Next Process: FFFFFFFF:8012FB60, CR3: 00000000:00000000 Remain: 10
KMAIN : kthread(pid=3): echo #2
SCHED : Current Process: FFFFFFFF:8012FB60 Next Process: FFFFFFFF:8012E740, CR3: 00000000:01FF1000 Remain: 10
SCHED : Current Process: FFFFFFFF:8012E740 Next Process: FFFFFFFF:8012F150, CR3: 00000000:01FEE000 Remain: 10
SCHED : Current Process: FFFFFFFF:8012F150 Next Process: FFFFFFFF:8012E740, CR3: 00000000:01FF1000 Remain: 10

8012E740 and 8012F150 are two process of testapp, and nothing happen even the code to be executed is ud2
however, other kthread seems not disturbed and run normally.


my code to get ring3:

Code: Select all

; void enter_ring3  ( unsigned long ring3_ip, unsigned long ring3_sp );
enter_ring3:
    ; jmp     rdi
    mov     ecx, SEG_DATA64_3 +3
    ;mov     ds, cx
    ;mov     es, cx
    push    rcx
    push    rsi
    push    0x0202          ; rflags
    push    SEG_CODE64_3 +3
    push    rdi
    iretq

What do I miss?

ps. cpu exception (INT00-13) has IST=1, and PIC timer has IST=2.
I have kernel stack per each thread, and swap rsp0 on TSS upon reschedule.


EDIT:solved

[bluemoon]

I also tried sysret, it also "nothing happen", no cpu exception, no sign of execution...

Code: Select all

; void enter_ring3  ( unsigned long ring3_ip, unsigned long ring3_sp );
enter_ring3:
    ; jmp     rdi
    mov     rcx, rdi
    mov     rsp, rsi
    mov     r11, 0x0202
    sysret

I must have missed something obvious...

[xenos]

Have you tried single-stepping through your switch to ring 3 in bochs debugger or similar?

[bluemoon]

Not yet, I used to debug with qemu-gdb but it has problem on 64-bit target (the famous g packet too long), so I lost debugging facilities.
Now I'm building bochs on mac and see what happen.

[iansjack]

Just as a matter of interest, the gdb business is easily solved. Start the program, interrupt it (once it has transferred to 64-bit mode) and "load" a 64-bit executable.

But this probably won't solve your current problem as you won't be able to stop at the correct point to single-step. With my OS I wait until I get the command prompt for my shell, interrupt that, and then I can debug any further code.

May I recommend SimNow from AMD? This is an emulator rather than a simulator, so is a little slow, but is great for single-stepping assembler code in a situation like this. You can set a breakpoint in the debugger before booting your kernel and then single-step from there on.

I can't think what your problem is. A "sysret" should do the trick (except perhaps it should be "sysretq"?).

[bluemoon]

The ring3 process seems to come to a hang state. I dump the "Resume IP" from the scheduler as it read:

Code: Select all

SCHED : Current Process: FFFFFFFF:8012D740 Next Process: FFFFFFFF:8012D740, EIP: 00000000:001000D0 Remain: 10
SCHED : Current Process: FFFFFFFF:8012D740 Next Process: FFFFFFFF:8012D740, EIP: 00000000:001000D0 Remain: 10
SCHED : Current Process: FFFFFFFF:8012D740 Next Process: FFFFFFFF:8012D740, EIP: 00000000:001000D0 Remain: 10
SCHED : Current Process: FFFFFFFF:8012D740 Next Process: FFFFFFFF:8012D740, EIP: 00000000:001000D0 Remain: 10

And for qemu-gdb with 64bit, I tried set architecture i386:x86-64 with no luck.

I can't think what your problem is. A "sysret" should do the trick (except perhaps it should be "sysretq"?).

I tried sysret and it successfully returned to compatibile mode with correct descriptor (I can check this with qemu-gdb)
I also tried REX.w sysret, which returns to 64-bit code, and it seems freeze there.

[bluemoon]

OK Finally I catch the bug.

my PF handler unconditionally set entry to page +3, which obviously can't access by ring3.
somehow the CPU fault at 0x001000D0 but cannot resolve the handler, and i don't know what happen anyway.

Now I changed to

Code: Select all

prot = pt[MMU_PT_INDEX(addr)] & MMU_PROT_MASK;
pt[MMU_PT_INDEX(addr)] = page | prot | MMU_PROT_PRESENT;

and the ring3 code successfully trigger #UD.


So I learnt that a lot of seemingly un-related and weird things can happen with bug in MMU code