[Solved] Returning to real mode from protected mode

[piscus]

Hello! I am new to OSdev. My question is in regard to returning to real mode.

Symptom:
I return to real mode, but when I call interrupts with the intention of running BIOS interrupt handlers, my system hangs (cursor in the top left of screen). This same behavior is observed on both Qemu as well as on my Thinkpad T410 Laptop. However, as long as I don’t execute the INT instruction in real mode, I can return back to protected mode and the execution makes it to my kernel’s main loop.

Details:
I am using the multiboot feature of GRUB as is suggested by OSDev tutorials. As soon my kernel is handed execution from GRUB, I believe the state to be in as what’s described in the multiboot specification (detailed here: https://www.gnu.org/software/grub/manua ... state.html). I have confirmed that this is the case by dumping registers as well as running gdb with Qemu.

State: Interrupts are disabled. I have not re-programmed the PIC or PIT yet. I have not enabled paging. I have statically defined two 16 bit entries in the GDT. I have a real mode interrupt descriptor that should be pointing to the BIOS’ IVT (0x00-0x3ff). I execute the following sequence of instructions:
1. Set %ebp and %esp to 0x7000
2. Load the GDT
3. Far jump to load the cs with the index of the descriptor -> 0x00009a000000ffff. (I have tried making entries in my normal GDT, as well as creating a second 0xffff limit only GDT)
4. Load rest of segment selectors with index of the data segment descriptor -> 0x00cf92000000ffff.
5. Load the real-mode IDT with the address of a memory location that contains the value -> 0x03ff00000000 . I thought the original IVT is at 0x00-0x3ff
6. Clear PE flag to return to real mode.
7. Far jump to real mode with cs segment selector as 0
8. Load ds,es,fs,gs,ss with 0’s
9. Enable interrupts.
10. Throw INT (it hangs here). If I don’t throw INT and let code re-enter protected mode I can make it to my kernel main. I am trying to run the meme820 test.

Resources:
Intel manual on Switching Back to Real-Address Mode, section 9.9.2: https://software.intel.com/sites/defaul ... -3abcd.pdf
OSDev tutorial on switching from protected real mode (http://wiki.osdev.org/Real_mode). However, this guide does not mention re-programming the PIC to the original setting. Due to the lack of this information, I am wondering if there are other “gotchas” for my case that I am missing.

Why do I want to return?
I want to interface with the BIOS for learning purposes. I feel that I should be able to return to real mode and am confused why I can’t get it to work. This indicates to me that I don’t understand something about my system’s state, and I am making some mistake or getting caught on a "gotcha." I would like to have this nailed down for a greater understanding. Does the multiboot specification trash the ivt?

I have tried to accurately capture my steps and was hoping to not post code just yet unless I am really stuck. Any help with this matter is appreciated

[alexfru]

4. Load rest of segment selectors with index of the data segment descriptor -> 0x00cf92000000ffff.

What about SS? Do you keep it 32-bit? If so, why?

If you use task switching via TSS it's also good to clear the task switched flag with CLTS. Or you're risking an exception when executing an FPU instruction if the flag is set.

[iansjack]

Try executing a subroutine rather than an interrupt. This will demonstrate whether your stack is valid or not.

You don't need to post code, just a link to a repository containing it.

[MichaelPetch]

Posting you code to a it repository (or similar) would be useful. Debugging this type of problem may be easier in something like BOCHs.

[piscus]

Thanks for the responses! alexfru, good catch. That was I mistake in my copying. I am working on getting my current branch up to github to link in this thread.

In the meantime, I will post a code snippet. I kept iterating and might have made a little progress. But things still go wild. Following is a snippet of my code. During my meme820 test, I give three options and describe above each option the strange behavior I see. For each option I have put either some gdb output or qemu crash dump below my code snippet.

Code: Select all

.align 16
gdt:
	# Index 0x00
	# Required dummy
	.quad	0x00

	# 0x08
	# Unused
	.quad	0x00

	# 0x10
	# protected mode code segment
	# bit 63			bit 32
	# 000000000000000 | 1001101000000000
	# 000000000000000 | 1111111111111111 (limit)
	# bit 31 (base)		bit 0
	#.word	0xFFFF
	#.word	0x0000
	#.word	0x9A00	# 1001 1010 0000 0000
	#.word	0x00CF	# 0000 0000 1100 1111
	.quad 0x00cf9a000000ffff	

	# 0x18
	# protected mode data segment
	#.word	0xFFFF
	#.word	0x0000
	#.word	0x9200 # 1001 0010 0000 0000
	#.word	0x00CF # 0000 0000 1100 1111 
	.quad 0x00cf92000000ffff	

	# 0x20
	# 16 bit code segment
	#.word	0xFFFF 	limit (0-15)
	#.word	0x0000	base (16-31)
	#.byte	0x00	base (32-39)
	#.byte	0x9a	access byte (40-47) - 10011010
	#.byte	0x0f	flags + limit (48-55)
	#.word	0x00	base (56-63)
    .quad 0x00009a000000ffff

	# 0x28
	# 16 bit data segment    
	#.word	0xFFFF 	limit (0-15)
	#.word	0x0000	base (16-31)
	#.byte	0x00	base (32-39)
	#.byte	0x92	access byte (40-47) - 10010010
	#.byte	0x0f	flags + limit (48-55)
	#.word	0x00	base (56-63)
    .quad 0x000092000000ffff

gdt_end:
gdt_info:
	.word	gdt_end - gdt - 1	# Size of GDT
	.word	0x0000			# Upper 2 Bytes of GDT address.
	.word	0x0000			# Lower 2 Bytes of GDT address.

# IDT for protected mode
idt_info:
	.word	0x0000
	.word	0x0000
	.word	0x0000

# IDT for real mode
idt_real_info:
	.word	0x03ff
	.word	0x0000
	.word	0x0000

.code32
.text
.globl setup_32
setup_32:
	# Setup stack.
	movl	$0x00007000, %esp
	movl	%esp, %ebp

    # Set up GDT
	movl	$gdt, (gdt_info + 2)

    # Interrupts should already be disabled...
	cli
	lgdt	gdt_info

    # *********************
    # Return to real mode *
    # *********************

    # Load a new segment with a limit of 0xffff 
    # This is the segment limit required in real-
    # address mode.
	jmp	$0x20, $loadRMSeg

.code16
loadRMSeg:
	movl	$0x28, %eax
	movl	%eax, %ds
	movl	%eax, %gs
	movl	%eax, %fs
	movl	%eax, %es
	movl	%eax, %ss

        # This line allows the interrupt to work.
        lidt	idt_real_info

	# Preserve %cr0
	movl	%cr0, %eax

	# Turn off PE bit, bit 0 to return to real mode
        andl	$0xffffffffe, %eax
	movl	%eax, %cr0

	jmp	$0x00, $set_rm_segment_regs
set_rm_segment_regs:
	movw	$0x0000, %ax
	movw	%ax, %ds
	movw	%ax, %gs
	movw	%ax, %fs
	movw	%ax, %es
	movw	%ax, %ss

	# Enable interrupts for memory checking.
	sti

    
meme820:
	xorl	%ebx, %ebx

        # Option 1:
        # Using GDB and qemu I set a breakpoint at bail820 below.
        # I can get past int $0x15 and hit the breakpoint, int %0x15 indicates success.
        # (CF not set) and %eax has the magic value that it should.
        #
        # The code then continues on, and I return to protected mode. 
        # As I step through the code in GDB every single interrupt begins to
        # fire and the code completely goes off the rails. I never see main hit.
        # $smapBuffer is defined in another file and at address ~0x6000
	movw 	$smapBuffer, %di

       # Options 2:
       # Using GDB and qemu I set a breakpoint at bail820 below.
       # Interrupt is called and returns. Memory test indicates that it
       # failed. Code makes it to my main and life appears to be wonderful.
       #
       # For some reason, %esp is no longer at $0x7000 as I set it at the top.
       #movw    $0x500, %es:(%di) 

       # Options 3:
       # Using GDB and qemu I set a breakpoint at bail820 below.
       # My breakpoint is never hit. Qemu crashes horribly with the
       # dump below.
       #movw    $0x2d0, %di

	movl	$0x0000e820, %eax
	movl $0x534D4150, %edx
	movl	$20, %ecx
	int	$0x15
bail820:

	cli

	# **************************
	# Return to protected mode *
	# **************************
        
        < code here >
      

Option 1 GDB. Notice no carry flag and the correct magic number in eax. But ebx did not increase and the stack went wild.

Code: Select all

(gdb) info reg
eax            0xe820	59424
ecx            0x14	20
edx            0x534d4150	1397571920
ebx            0x0	0
esp            0xfffe	0xfffe
ebp            0x7000	0x7000
esi            0x375	885
edi            0x6840	26688
eip            0x2970	0x2970 <bail820>
eflags         0x200006	[ PF ID ]
cs             0x0	0
ss             0x0	0
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0

Option 2 GDB dump. Notice that all signs point to e820 failure, and the stack is slightly off:

Code: Select all

Breakpoint 1, bail820 () at setup_32.s:165
165		cli
(gdb) info reg
eax            0x1e3a	7738
ecx            0x14	20
edx            0x534d4150	1397571920
ebx            0x0	0
esp            0x6ffe	0x6ffe
ebp            0x7000	0x7000
esi            0x1	1
edi            0x0	0
eip            0x2972	0x2972 <bail820>
eflags         0x200013	[ CF AF ID ]
cs             0x0	0
ss             0x0	0
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0
(gdb) x/32x 0x6ff0
0x6ff0:	0x00000000	0x00000000	0x29720000	0x02460000
0x7000:	0x00000000	0x00000000	0x00000000	0x00000000
0x7010:	0x00000000	0x00000000	0x00000000	0x00000000

Qemu Crash dump from option 3. Break point never hit.

Code: Select all

qemu: fatal: Trying to execute code outside RAM or ROM at 0x00000000000b8600
EAX=00000000 EBX=00004600 ECX=00000046 EDX=534d00ea
ESI=00000111 EDI=0000465c EBP=000000fa ESP=000000ea
EIP=00004600 EFL=00204606 [D----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =00b4 00000b40 0000ffff 00009300
CS =b400 000b4000 0000ffff 00009a00
SS =0000 00000000 0000ffff 00009300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     00000000 000028f2
IDT=     002153b8 00006000
CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
DR3=0000000000000000 
DR6=00000000ffff4ff0 DR7=0000000000000400
CCS=00000000 CCD=00000111 CCO=INCW    
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000

[piscus]

And to confuse matters more, if I replace my code from meme820 to bail820 with a call to the BIOS routine that prints a character...

Code: Select all

movb    $0x0e, %ah
movb    'a', %al
int     $0x10

LABEL_am_i_reaching_here:

The behavior I described in my first post is exhibited again. Code runs until the interrupt is hit, but nothing seems to get hit after the interrupt.

[MichaelPetch]

Option 1 GDB. Notice no carry flag and the correct magic number in eax. But ebx did not increase and the stack went wild.

Code: Select all

(gdb) info reg
eax            0xe820	59424
ecx            0x14	20
edx            0x534d4150	1397571920
ebx            0x0	0
esp            0xfffe	0xfffe
ebp            0x7000	0x7000
esi            0x375	885
edi            0x6840	26688
eip            0x2970	0x2970 <bail820>
eflags         0x200006	[ PF ID ]
cs             0x0	0
ss             0x0	0
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0

With option 1 I'd be curious what QEMU says the 512 16-bit words from 0x0000 to 0x03ff are BEFORE the int 0x15. I'm really curious if there really is a proper real mode IVT in base of memory still. The fact the stack is 0xfffe i very suspect. That's almost like someone set SP to 0x0000 and then pushed a value. Either the real mode IVT is borked or the buffer ES:DI is pointing at is colliding with the stack (could happen if both were close together)

Regarding Option 2:

Code: Select all

movw    $0x500, %es:(%di)

. That is wrong. It seems you intended to try and read the memory map to ES:0x500 (Just above the BDA).But that's not what this instruction does. You moved the value 0x500 to memory address ES:DI which according to the register info you gave was likely 0x0000:0x0000 smack dab at the beginning of the real mode IVT. And since this instruction does't change DI the buffer you ended up passing into int 0x15 was 0x0000:0x0000 which again is writing on top of the IVT.

Regarding Option 3:

Code: Select all

movw    $0x2d0, %di

not sure what the intent was but this would have put the value 0x2d0 into DI. Int 0x15 would have tried to write to 0x0000:0x02d0 which is in the middle of the IVT potentially corrupting interrupts.

[MichaelPetch]

In this code:

Code: Select all

movb    $0x0e, %ah
movb    'a', %al
int     $0x10

LABEL_am_i_reaching_here:

I assume you meant to use

Code: Select all

movb   $'a', %al

Your code would have moved the byte from memory address DS:'a' which is 0x0000:0x0061 and put it in AL.

[piscus]

MichaelPetch, thanks for the continued responses! You are correct, $'a' was what I meant.

In regard to your question on option #1, I set the breakpoint at the _start symbol and dumped registers and bunch of Bytes starting at 0x00. Note the instruction my breakpoint is at,it jumps directly to the code snippet I posted:

Code: Select all

Breakpoint 1, _start () at boot.S:28
28	    jmp setup_32
(gdb) info reg
eax            0x2badb002	732803074
ecx            0x0	0
edx            0x0	0
ebx            0x10000	65536
esp            0x7ff00	0x7ff00
ebp            0x0	0x0
esi            0x0	0
edi            0x0	0
eip            0x2bc9	0x2bc9 <_start>
eflags         0x200046	[ PF ZF ID ]
cs             0x10	16
ss             0x18	24
ds             0x18	24
es             0x18	24
fs             0x18	24
gs             0x18	24
(gdb) x/128x 0x00
0x0:	        0x464c457f	0x00010101	0x00000000	0x00000000
0x10:	0x00030002	0x00000001	0x00002bc9	0x00000034
0x20:	0x00009b0c	0x00000000	0x00200034	0x00280007
0x30:	0x00140015	0x00000006	0x00000034	0x00000034
0x40:	0x00000034	0x000000e0	0x000000e0	0x00000005
0x50:	0x00000004	0x00000003	0x00003fa0	0x00003fa0
0x60:	0x00003fa0	0x00000013	0x00000013	0x00000004
0x70:	0x00000001	0x00000001	0x00000000	0x00000000
0x80:	0x00000000	0x00004244	0x00004244	0x00000005
0x90:	0x00001000	0x00000001	0x00005000	0x00005000
0xa0:	0x00005000	0x0000006c	0x00001bc0	0x00000006
0xb0:	0x00001000	0x00000002	0x00005004	0x00005004
0xc0:	0x00005004	0x00000068	0x00000068	0x00000006
0xd0:	0x00000004	0x6474e550	0x00004018	0x00004018
0xe0:	0x00004018	0x0000022c	0x0000022c	0x00000004
0xf0:	0x00000004	0x6474e551	0x00000000	0x00000000
0x100:	0x00000000	0x00000000	0x00000000	0x00000007
0x110:	0x00000010	0x00000000	0x00000000	0x00000000
0x120:	0x00000000	0x00000000	0x00000000	0x00000000
0x130:	0x00000000	0x00000000	0x00000000	0x00000000
0x140:	0x00000000	0x00000000	0x00000000	0x00000000
0x150:	0x00000000	0x00000000	0x00000000	0x00000000
0x160:	0x00000000	0x00000000	0x00000000	0x00000000
0x170:	0x00000000	0x00000000	0x00000000	0x00000000
0x180:	0x00000000	0x00000000	0x00000000	0x00000000
0x190:	0x00000000	0x00000000	0x00000000	0x00000000
0x1a0:	0x00000000	0x00000000	0x00000000	0x00000000
0x1b0:	0x00000000	0x00000000	0x00000000	0x00000000
0x1c0:	0x00000000	0x00000000	0x00000000	0x00000000
0x1d0:	0x00000000	0x00000000	0x00000000	0x00000000
0x1e0:	0x00000000	0x00000000	0x00000000	0x00000000
0x1f0:	0x00000000	0x00000000	0x00000000	0x00000000
(gdb) 

(gdb) x/128x 0x00
0x0: 0x464c457f 0x00010101 0x00000000 0x00000000
0x10: 0x00030002 0x00000001 0x00002bc9 0x00000034

Heading to work, so can't evaluate fully at the moment. But, I would guess the elf header shouldn't be at 0x00

[MichaelPetch]

You are correct, that is an elf header and I can tell the entire IVT is corrupt since none of the addresses you'd expect to be pointing at the BIOS region in the upper part of the first 1mb are present. Given that 0xbadb002 is still in EAX I have to assume that not much code has executed from the start of your kernel to the point you start setting up the jmp to real mode. The fact `_start` is in the first megabyte has me curious what your linker script looks like. Are you using multiboot modules as well to load files into memory?

Clearly as you suggest you need to find out what is overwriting the base of memory and the real mode IVT wth an elf file. If I had to hazard a guess I think what is going on is related to how you are getting code loaded into lower memory (less than 1mb). You've managed to load an elf executable into base of memory (starting at 0x000000000 which had me curious if you were using mulitboot boot modules to load secondary code along side your operating system)

[alexfru]

A20 disabled while loading ELF file at 1MB?

[MichaelPetch]

A20 disabled while loading ELF file at 1MB?

Ah, thanks for mentioning it. I had intended to mention that originally until I realized that some of the register output suggests he's using a multiboot compliant loader (He says he's using GRUB). A20 should be enabled. Also doesn't explain why the ELF header would have been loaded in memory at 0x100000 ? It's a good question nonetheless.

[piscus]

Thanks for the responses everyone. I am using GRUB's multiboot feature to boot my kernel. In my last debug output, no kernel code has executed by the time my breakpoint was hit... only BIOS then GRUB code; I placed my breakpoint at my kernel's entry point, _start.

I have not tested to to see if A20 is enabled, but according to the multiboot standard system state link I posted above, it should be. As to whether or not it has been enabled by the time GRUB loads my code -- I don't know. I have treated GRUB like a magical black box... perhaps I can't do that anymore.

I am not using multiboot modules to load other files and executables other than my kernel into memory ( to my knowledge ), so I am expecting that ELF image starting at 0x00 to be my kernel. I don't know enough at this point about GRUB to guess why it could be loading my kernel's ELF image ( soon to be verified ) at 0x00.

I will post back with the results of my investigation or a solution if I come across one. Thanks again

[MichaelPetch]

so I am expecting that ELF image starting at 0x00

I think that is where you are going wrong. Can you post your linker script? If you have set your Virtual memory address starting point at 0x00000 (something like . = 0 wold be bad) this causes serious butt hurt for multiboot loaders since they generally expect the entry point at 0x100000. Can you also show your assembly file with your multiboot header up to the point of the start label?

If I use mutliboot and need code some code to be located below 1mb (if for example i need real mode code in lower memory) I usually create a specially crafted linker script that has some sections with a VMA where the code expects to be run from (that is different from the memory location where it was loaded in memory (above 0x100000). Then in the entry point code I copy (rep movsw can be useful) the special sections from above 0x100000 to their proper location in lower memory below 1mb. Another option is for the real mode code to be placed in a kernel module that the mutliboot loader will load into memory for you. You then copy it to lower memory where it needs to be placed. The real mode code in the module would be built separate from the multiboot code/kernel and can be a binary file generated from the ELF executable. That conversion can be done with objcopy.

If you enable paging you can avoid copying by mapping pages below 1mb to the code/data above 0x100000.

[piscus]

OK these are the two files:

linker script:

Code: Select all

ENTRY(_start)

SECTIONS
{
	/* Begin putting sections here */
	. = 0x1000;

	/* First put the multiboot header followed by .text section. */
	.text BLOCK(4K) : ALIGN(4K)
	{
		*(.multiboot)
		*(.text)
	}

	/* Read-only data. */
	.rodata BLOCK(4K) : ALIGN(4K)
	{
		*(.rodata)
	}

	/* Read-write data (initialized) */
	.data BLOCK(4K) : ALIGN(4K)
	{
		*(.data)
	}

	/* Read-write data (uninitialized) and stack */
	.bss BLOCK(4K) : ALIGN(4K) 
	{
		*(COMMON)
		*(.bss)
	}
	
	/* Put other sections the compiler generates here */
}

Entire multiboot header file with start label

Code: Select all

# Declare constants used for creating a multiboot header.
.set ALIGN,    1<<0             # align loaded modules on page boundaries
.set MEMINFO,  1<<1             # provide memory map
.set FLAGS,    ALIGN | MEMINFO  # this is the Multiboot 'flag' field
.set MAGIC,    0x1BADB002       # 'magic number' lets bootloader find the header
.set CHECKSUM, -(MAGIC + FLAGS) # checksum of above, to prove we are multiboot

# Declare a header as in the Multiboot Standard. Putting this in a special
# section forces the header to be in the start of the final program.
.section .multiboot
.align 4
.long MAGIC
.long FLAGS
.long CHECKSUM

# The linker script specifies _start as the entry point to the kernel and the
# bootloader will jump to this position once the kernel has been loaded.
.section .text
.global _start
.type _start, @function
_start:
	
    jmp setup_32

	# Should never get here	
	cli
	hlt
.Lhang:
	jmp .Lhang