Hey friends, I'm working on a challenge in a toy x86 OS that's run with qemu. The OS successfully transitions from real mode to protected mode, initializing the GDT, IDT, and remapping the PIC along the way. I'm trying to use an amalgam of inline assembly and compiled assembly programs to return to real mode and execute a program that relies on BIOS interrupts (0x10, 0x16, etc). I started with the the guide provided on the wiki for the transition in between modes and cross-referenced examples I found elsewhere, but I'm running into issues with the far jump instruction and am stuck in unreal mode as far as I can tell. I've gone through multiple iterations attempting this transition so I won't start by posting code, but this is the general process I've followed:
- copy my target program to 0x7c00 (tested in a boot sector initially to ensure valid code)
- disable interrupts
- reset cr0 without the paging or PE bits
- reload the IVT limits to 0x0 - 0x3ff
- clear the GDT registers
- remap the PIC
- set ESP to 0x7b00
- clear all registers (CS/DS/GS/SS/etc)
- far jump to 0x0000:0x7c00
When I trace this transition with gdb, I can verify that cr0, IVT, and GDT registers look like they do when I examine them prior to the real mode -> protected mode transition. However, the CS/DS/GS... registers still have limits at 0xffffffff instead of their initial 0xffff. When the code at 0x7c00 is executed, even though it was compiled with the bits 16 specifier, I can see that gdb is handling 32-bit addresses instead of the 16-bit addresses I expected. Lastly, whenever a syscall occurs, I can see that the segment isn't being processed correctly (the IVT address for 0x10 is 0xc0005663, but the memory location jumped to is 0x5663 instead of the expected 0xc5663 with proper segment:offset addressing. I also verified that there is valid handler code at 0xc5663). Consequently, I don't think the far jump is properly resetting the CS:IP limitations. Am I missing a critical part of the transition process or is there a standard way apart from a far jump to reset those segment limitations?
Returning to Real Mode with Far Jump Failing
Re: Returning to Real Mode with Far Jump Failing
To properly return to real mode from 32-bit protected mode, you are going to have to go to 16-bit protected mode in the middle there. You will need a 16-bit code segment and also a 16-bit data segment in your GDT and load those into CS and SS while the CPU is still in protected mode.
Reason is that the CPU does not load the attributes of the segment registers in real mode, so they keep their last value. If you are in 32-bit PM, the code segment keeps the 32-bit attribute even in RM and there is no way to clear it. Same for the address size of the stack segment.
On the whole, doing this dance just to get back to 16-bit RM is so tedious and error prone that you are usually better off doing whatever it is you need done with your own drivers.
Reason is that the CPU does not load the attributes of the segment registers in real mode, so they keep their last value. If you are in 32-bit PM, the code segment keeps the 32-bit attribute even in RM and there is no way to clear it. Same for the address size of the stack segment.
On the whole, doing this dance just to get back to 16-bit RM is so tedious and error prone that you are usually better off doing whatever it is you need done with your own drivers.
Carpe diem!
Re: Returning to Real Mode with Far Jump Failing
Okay, that makes sense (even though the process is super convoluted, haha). So does reloading the GDT with 16-bit code & data segment values and then far jumping reload the CS/SS registers, including their hidden segments?
(The challenge for this OS exists because this process is such a beast, so as much as I wish I could take another route, I'm stuck with this approach.)
(The challenge for this OS exists because this process is such a beast, so as much as I wish I could take another route, I'm stuck with this approach.)
Re: Returning to Real Mode with Far Jump Failing
You can just add the 16-bit segments to the GDT you already have. It is perfectly happy to contain both 32-bit and 16-bit segments. Even 64-bit segments can be in there, if you want to have them. I would suggest using an instruction that allows you to load CS and SS at the same time, else you have a (small) section with mismatched code and stack segments. Luckily, such an instruction exists: iret. If you synthesize an iret frame, you can just jump to 16-bit protected mode very quickly.
Carpe diem!
-
Octocontrabass
- Member
- Posts: 5768
- Joined: Mon Mar 25, 2013 7:01 pm
Re: Returning to Real Mode with Far Jump Failing
Even if you have a BIOS (modern x86 PCs do not), this is a bad idea. Why can't you port some kind of virtual machine to your OS and run the program that way?
Where are you seeing the wrong address? Last I checked, GDB assumes the CS/DS/ES/SS bases are always 0, so it'll only show you the offset portion of most addresses.
Re: Returning to Real Mode with Far Jump Failing
Ahh gotcha, thanks for the pointers. I'll give this a shot!nullplan wrote: ↑Wed Apr 16, 2025 1:44 pmYI would suggest using an instruction that allows you to load CS and SS at the same time, else you have a (small) section with mismatched code and stack segments. Luckily, such an instruction exists: iret. If you synthesize an iret frame, you can just jump to 16-bit protected mode very quickly.
I wholeheartedly agree that this is a bad idea, but it's not supposed to be easy or practical; it was purely an intellectual challenge. And regarding the GDB bases, I'm not actually sure about that. I thought I recalled seeing more than just address offsets but I'm rarely in asm view, so you are likely correct.Octocontrabass wrote: ↑Wed Apr 16, 2025 9:09 pmEven if you have a BIOS (modern x86 PCs do not), this is a bad idea. Why can't you port some kind of virtual machine to your OS and run the program that way?
Where are you seeing the wrong address? Last I checked, GDB assumes the CS/DS/ES/SS bases are always 0, so it'll only show you the offset portion of most addresses.