Happy Xmas and New Year! While you are having leisure time, would you help me a bit?
I just tried to study X86 64 SMP trampoline code, for this I tried one experimental OS, Kiwi from http://kiwi.alex-smith.me.uk/.
The compilation was OK and I can run it with single core. But I got problem when I set it to run on 2 cores with both QEMU and VMware.
The following is my problem and questions (I wrote to the owner of that project Mr. Alex Smith but got no reply, seems he is happy with his Christmas!):
1. If I set the virtual machine to have 1 processor with 1 core, then everything works fine; I could see the shell with the beautiful Logo.
2. Then I set the virtual machine with 1 processor and with 2 cores to match my host machine. Now I got a problem that I can only start the BSP, the AP always timeout; And then the fatal error trapped me in the KDBG (another nice thing I like). From the Vmware log I can see the following:
[
Dec 27 13:42:52.759: vcpu-1| ------MONITOR RING BUFFER START(entries=256, indexUnwrapped 6 entrySz=64)-----
Dec 27 13:42:52.759: vcpu-1| 005 --- CS:XIP 0700 0000004c SS:XSP 0000 0000fffe #TRIPLE 000d CPL0 PROT 16-bit fluff=0000 brCnt=-1
Dec 27 13:42:52.759: vcpu-1| 004 --- CS:XIP 0700 0000004c SS:XSP 0000 0000fffe #DF 000d CPL0 PROT 16-bit fluff=0000 brCnt=-1
Dec 27 13:42:52.759: vcpu-1| 003 --- CS:XIP 0700 0000004c SS:XSP 0000 0000fffe NESTED #GP error= 006a CPL0 PROT 16-bit fluff=0000 brCnt=-1
Dec 27 13:42:52.759: vcpu-1| 002 --- CS:XIP 4010 fffffffffc25a6aa SS:XSP 4018 fffffffffc008b78 excNum 00e eflags 00010646 errCode 0000 brCnt=-1 la 40000004068
Dec 27 13:42:52.759: vcpu-1| 001 --- CS:XIP 0700 0000004c SS:XSP 0000 0000fffe #GP error= 0008 CPL0 PROT 16-bit fluff=0000 brCnt=-1
Dec 27 13:42:52.759: vcpu-1| 000 --- CS:XIP 0700 0000004c SS:XSP 0000 0000fffe FARCALL/JMP PRE SELECTOR=0008 CPL0 PROT 16-bit fluff=0000 brCnt=-1
Dec 27 13:42:52.759: vcpu-1| ------MONITOR RING BUFFER END----------
]
So it seems something serious happened during a JMP.
3. I then launched QEMU (which also has the same problem,worse, it just continuously reboots to the GRUB menu again and again). I then attached GDB (through DDD) to the system; with some debugging I found the AP trampoline code can be entered (which means IPI is working). If I add a "hlt" before " ljmpl $0x08, $0", then it will not reboot (but will drop me to the "FailShell"!); If I put the "hlt" just before " jmp *entry_addr(%edx)", then it will still reboot. So it must be that " ljmpl $0x08, $0" is failing. I checked a while but I can not find any issue with the code (may be due to I am not experienced with X86 yet, and that is why I'd like to study with this!)
The code can be seen from:
http://bzr.alex-smith.me.uk/kiwi/trunk/ ... /ap_boot.S
4. I have some questions in the following code (questioned below):
Code: Select all
.type __ap_boot, @function
__ap_boot:
xorl %edx, %edx
/* Set the data segment and save the load location. */
mov %cs, %dx
mov %dx, %ds
shl $4, %edx
/* Load the GDT. */
leal __gdt(%edx), %eax
movl %eax, (__gdtp + 2)
lgdtl (__gdtp)
/* Enable protected mode. */
movl %cr0, %eax
orl $SYSREG_CR0_PE, %eax
movl %eax, %cr0
/* Fix the jump address. */
leal .Lpmode(%edx), %eax
movl %eax, (1f + 2) <================= Why is all this needed? Is the %eax needed for the ljmpl below?
1: ljmpl $0x08, $0 <=================== It tries to jump to 32 bit CS:0, right? But I don't think that address is .Lpmode in 32 bit pmode, am I right?
.Lpmode:
.code32
/* Load data segment. */
movl $0x10, %eax
mov %ax, %ds
/* Get the kernel entry point and jump to it. */
jmp *entry_addr(%edx)
.size __ap_boot, .-__ap_bootCode: Select all
#include <arch/x86/sysreg.h>
.code16
.global _start
_start:
jmp __ap_boot
.align 4
/** Where the kernel stores the kernel entry address. */
entry_addr: .long 0
__ap_boot:
cli
cld
xorl %edx, %edx
/* Set the data segment and save the load location. */
mov %cs, %dx
mov %dx, %ds
shl $4, %edx
/* Fix the jump address. */<======I changed this section of code a lot, trying to map the 32 bit code with a dedicated segment so that the last ljmp can jump at offset 0.
movl %edx, %eax
addl .Lpmode,%eax
movw %ax, (__gdt + 8 + 2)
shr $16, %eax
movb %al, (__gdt + 8 + 4)
movb %ah, (__gdt + 8 + 7)
/* Load the GDT. */
movl %edx, %ebx
xorl %eax,%eax
addl (__gdt),%eax ==========>It is strange at this point %eax is 0 seen from GDB,which I wanted to be the GDT physical base
addl %eax, %ebx
movl %ebx, (__gdtp + 2)
lgdtl (__gdtp)
/* Enable protected mode. */
movl %cr0, %ecx
orl $SYSREG_CR0_PE, %ecx
movl %ecx, %cr0
;hlt <============If I hlt here, the system will not reboot, if I do not hlt here, it will reboot just following the ljmpl below
ljmpl $0x08, $0 <=================once jump, fail!
.code32
.align 4
.Lpmode:
hlt <===========================it can not jump here
/* Load data segment. */
movl $0x10, %eax
mov %ax, %ds
hlt
/* Get the kernel entry point and jump to it. */
jmp *entry_addr(%edx)
.align 4
__gdt:
.quad 0x0000000000000000 /**< NULL descriptor (0x00). */
.quad 0x00CF9A000000FFFF /**< 32-bit code (0x08). */
.quad 0x00CF92000000FFFF /**< 32-bit data (0x10). */
.L__gdt_end:
.align 4
__gdtp:
.word .L__gdt_end-__gdt-1
.long 0
With best regards,
Cory
