Returning to Real Mode from a 64-bit linked kernel in PMode

[zhiayang]

Hello again, it's been a long time. (not really)

Anyways. I've this problem here; I want to support more than just the BGA (so the OS can run on other emulators like VMWare etc), so I'm looking into VBE support.

I boot the OS using GRUB1, using something sortiecat told me on IRC; that is to link the kernel as a 64-bit ELF, but objcopy it to a 32-bit one to fool GRUB. (If you ask why I'm using GRUB1 instead of GRUB2, i can never get that stuff to compile. Ever.)

Here's the problem: because I'm linking it to a 64-bit object, I can't output 32-bit .o's along the line, which means I can't do far jumps (According to NASM, which says 'addr:addr' jumps are not valid in 64-bit mode)

Basically, I cannot compile most of the PMode-RMode switching code I find (even from Real_Mode)...
It complains of relocations being truncated or something along those lines.

Note: I get put in a 32-bit PMode environment from which I set up paging, the 64-bit GDT and then jump to it... Kind of a unique situation here and I don't think anything I've found so far fits.


Thanks!

[sortie]

I had a similar problem when porting to x86_64 - the key is that you need to do an indirect far jump. In GNU as, the syntax is a bit different (need to add a * or something), I suspect the situation is the same in nasm.

[zhiayang]

I had a similar problem when porting to x86_64 - the key is that you need to do an indirect far jump. In GNU as, the syntax is a bit different (need to add a * or something), I suspect the situation is the same in nasm.

1. Is there a working example I could look at? (:

2. I've managed to get everything to compile and *not* crash while getting to RMode... but then I doubt I'm actually in RMode.
I get this obscure (admittedly it's pretty obvious what's happening, but not why) error from QEMU:

Code: Select all

qemu: fatal: Trying to execute code outside RAM or ROM at 0x00000000000b4eea

EAX=0000ff0f EBX=0002ffaf ECX=00056c43 EDX=00004000
ESI=00056c05 EDI=00056bf9 EBP=00067e5c ESP=00067e36
EIP=ffff06ba EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00cf9300
CS =c483 000c4830 ffffffff 00cf9a00
SS =0010 00000000 ffffffff 00cf9300
DS =0010 00000000 ffffffff 00cf9300
FS =0010 00000000 ffffffff 00cf9300
GS =0010 00000000 ffffffff 00cf9300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     00009108 00000027
IDT=     00000000 000003ff
CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000ff0f CCD=0003feaf CCO=ADDB    
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
make: *** [all] Abort trap: 6
[Finished in 3.5s with exit code 2]

Here's my current code (shamelessly ripped from wiki)

Code: Select all

[bits 16]

idt_real:
	dw 0x3ff		; 256 entries, 4b each = 1K
	dd 0			; Real Mode IVT @ 0x0000

global Entry16
Entry16:
        ; We are already in 16-bit mode here!

	cli			; Disable interrupts.

	; Need 16-bit Protected Mode GDT entries!
	mov eax, 0x10	; 16-bit Protected Mode data selector.
	lgdt [dword gdt]
	mov ds, eax
	mov es, eax
	mov fs, eax
	mov gs, eax

	; Disable paging (we need everything to be 1:1 mapped).
	mov eax, cr0
	and eax, ~1	; Disable paging bit & enable 16-bit pmode.
	mov cr0, eax

	jmp dword 0x0:GoRMode		; Perform Far jump to set CS.

GoRMode:
	mov sp, 0x8000		; pick a stack pointer.
	mov ax, 0		; Reset segment registers to 0.
	mov ds, ax
	mov es, ax
	mov fs, ax
	mov gs, ax
	lidt [dword idt_real]
	sti

	mov ax, 0x13
	int 0x10


	cli
	hlt			; Restore interrupts -- be careful, unhandled int's will kill it.



gdt:
; null
   dw 0x0000
   dw 0x0000
   db 0x00
   db 0x00
   db 0x00
   db 0x00
; 16 bit code (0x08)
   dw 0xffff
   dw 0x0000
   db 0x00
   db 0x9A
   db 0xf
   db 0x00
; 16 bit data (0x10)
   dw 0xffff
   dw 0x0000
   db 0x00
   db 0x92
   db 0xf
   db 0x00
;

[Combuster]

JMP ptr16:32 and JMP ptr16:16 have been actually removed from long mode; it's not an assembler issue.

As for the other bit, EIP is set with the top bits set to FFFF, and CS has an invalid limit (and all the other segments are left in a broken state as well). The latter should be a good pointer for fixing, but I'm not sure how you managed to get that broken EIP with CR0.PE = 0, but it looks like you didn't properly pass through 16-bit compatibility mode on your way out and borked the emulator with undefined processor use.