[Solved] Returning to real mode from protected mode

[piscus]

If I try to start the sections at a later address, say, 0x10000, I get several linker errors:

Code: Select all

setup_32.s:119:(.text+0x85): relocation truncated to fit: R_386_16 against `.text'

Presumably due to the use of smaller register sizes in my code posted way above.

[MichaelPetch]

By doing this you are effectively asking multiboot loader to load your kernel pages into low memory potentially on top of the multiboot loaders own code and data areas. This is unwise, as multiboot was designed to load your kernel at 0x100000 and above. One of the reasons for this is so that the kernel the loader is placing in memory doesn't clobber the multiboot loader in the process of doing work. Likely this is causing serious issues and probably resulted in things like an elf header being loaded on top of the real mode interrupt vector table. This is just a guess.

[MichaelPetch]

I wrote a Stackoverflow answer that involved someone using Multiboot to boot but then wanted to switch back to real mode to run some VESA code. You can probably just look at the Preferred Solution and then the complete example. The key thing was to create a linker script that dealt with real mode code that would eventually be run from low memory (starting at 0x1000). The linker script is where most of the complexity was. The kernel started by copying the realmode code and data from where it was loaded above 0x100000 to the memory location starting at 0x1000. The VESA driver stuff was in a separate assembly file. Special sections called .data.realmode and .text.realmode contained all the real mode code and data. All the 32-bit code would be in the regular .text and .data sections.

The C code contains a function called realmode_setup that copies all the real mode code and data to low memory from where it was loaded by multiboot above 0x100000. The example also sets up a GDT (replaces the temporary multiboot GDT). It contains 32-bit and 16-bit data/code segment descriptors.

The code might give you some ideas on how to tackle your issue from a different perspective.

You can access the files for this Stackoverflow answer on my webserver

[piscus]

Awesome, thanks for the resource! Let me putz with it and see if I can get this thing to work. I hope to post back soon

[piscus]

Update:

I commented out the 16 bit code that I posted above and changed the linker script to the following (which allowed it to build since the 16 bit code was gone):

Code: Select all

ENTRY(_start)

SECTIONS
{
	/* Begin putting sections here */
	. = 0x100000;

	/* First put the multiboot header followed by .text section. */
	.text BLOCK(4K) : ALIGN(4K)
	{
		*(.multiboot)
		*(.text)
	}

	/* Read-only data. */
	.rodata BLOCK(4K) : ALIGN(4K)
	{
		*(.rodata)
	}

	/* Read-write data (initialized) */
	.data BLOCK(4K) : ALIGN(4K)
	{
		*(.data)
	}

	/* Read-write data (uninitialized) and stack */
	.bss BLOCK(4K) : ALIGN(4K) 
	{
		*(COMMON)
		*(.bss)
	}
	
	/* Put other sections the compiler generates here */
}

The elf header disappeared from address 0x00. Here is a gdb dump from my _start:

Code: Select all

(gdb) info reg
eax            0x2badb002	732803074
ecx            0x0	0
edx            0x0	0
ebx            0x10000	65536
esp            0x7ff00	0x7ff00
ebp            0x0	0x0
esi            0x0	0
edi            0x0	0
eip            0x101b66	0x101b66 <_start>
eflags         0x200046	[ PF ZF ID ]
cs             0x10	16
ss             0x18	24
ds             0x18	24
es             0x18	24
fs             0x18	24
gs             0x18	24
(gdb) x/128x 0x00
0x0:	0xf000ff53	0xf000ff53	0xf000e2c3	0xf000ff53
0x10:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x20:	0xf000fea5	0xf000e987	0xf000d676	0xf000d676
0x30:	0xf000d676	0xf000d676	0xf000ef57	0xf000d676
0x40:	0xc0004d65	0xf000f84d	0xf000f841	0xf000e3fe
0x50:	0xf000e739	0xf000f859	0xf000e82e	0xf000efd2
0x60:	0xf000d69b	0xf000e6f2	0xf000fe6e	0xf000ff53
0x70:	0xf000ff53	0xf000ff53	0xf00068e4	0xc00084a8
0x80:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x90:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0xa0:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0xb0:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0xc0:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0xd0:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0xe0:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0xf0:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x100:	0xf000ec59	0x9fc0003d	0xf000ff53	0xc00062a8
0x110:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x120:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x130:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x140:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x150:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x160:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x170:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x180:	0x00000000	0x00000000	0x00000000	0x00000000
0x190:	0x00000000	0x00000000	0x00000000	0xf000ff53
0x1a0:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x1b0:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53
0x1c0:	0xf000d65b	0xf000d67f	0xf000d67f	0xf000d67f
0x1d0:	0xf000d664	0xf000d66d	0xf000d652	0xf000d67f
0x1e0:	0xf000ff53	0x00000000	0xf000ff53	0xf000ff53
0x1f0:	0xf000ff53	0xf000ff53	0xf000ff53	0xf000ff53

[MichaelPetch]

The real mode IVT you posted as a followup when the kernel is built with a VMA and LMA of 0x100000 in the linker script is what I would expect.

I had suggested earlier the fact you were using a VMA (wich will alter the Load Memory address by default) of 0x1000 in the linker script was causing the multiboot loader to potentially corrupt itself and do something that wasn't intended. Having the multiboot loader load pages into low memory potentially clobbering multiboot itself is problematic. At least you know now that it in fact it is a matter of how you build the file and what virtual memory addresses you use. With the VMA at 0x100000 that it where multiboot expects to be loading things so it shouldn't be a problem (that's how it is was intended to be used). You have options, one I gave links to as an example. Another option is using multiboot modules to have the multiboot loader load a secondary binary file into memory containing the real mode code and dat that can be copied down to low memory by your own code.

[piscus]

Yeah, I made this change based on your suggestions. I am going to attempt your stack overflow approach when I get some time over the next few days.

I am excited to finally be in the right direction. Thanks for all your help so far!

[bellezzasolo]

My (personal) approach is to have the multiboot loader load my real mode code as a module. In fact, the first module it loads is my real mode loader, which I place at 0:0x1000 (stack at 0x8000:0xFFFE). The modules have a header with their load address and entry point.
The real module loader deals with entering and exiting real mode, and is hence specific to x86/x64, but other modules aren't.
There is a mechanism for data transfer between the kernel (read: driver) and the module via a memory block, which is application specific.

[piscus]

Reporting back...

I have been able to get the real mode code working using MichaelPetch's suggestion. Thank you bellezzasolo for responding with your suggestion too

I tried to adapt some parts of your Stackoverflow linker script Michael and found a working formula. I have been able to:
0. Copy realmode code from 0x100000 to 0x1000
a. Jump to 0x1000 and return to real mode and execute an int 0x10 to print $'a' to the screen
b. Run one iteration of the meme820 test successfully.
c. Return to protected mode and do the pic, pit, interrupts, paging etc.

This is my current linker script. They still seem like magic but I tried to pick out the relevant lines from your example. It didn't take cherry picking lines long before I got something that seemed to do as what you describe:

Code: Select all

OUTPUT_FORMAT("elf32-i386")
ENTRY(_start)

PHYSICAL_BASE_ADDRESS = 0x00100000;
REAL_BASE_ADDRESS = 0x1000;

SECTIONS
{
	/* Set the counter to 0x100000 */
	. = PHYSICAL_BASE_ADDRESS;

    /* Find the distance between 0x100000 - 0x1000 => 0xff000 */
    __physreal_diff = . - REAL_BASE_ADDRESS;

    /* Tell the linker that the .realmode section should have virtual addresses */
    /* generated at 0x1000, but is loaded at 0x100000 */
    .realmode REAL_BASE_ADDRESS : AT(ADDR(.realmode) + __physreal_diff) {

        __realmode_vma_start = .;

        /* LOADADDR is the LMA of the specified section */
        __realmode_lma_start = LOADADDR(.realmode);
        
        *(.text.realmode);
        *(.data.realmode);
        *(.multiboot);
    }
    
    /* Align at 4 Bytes and define some new symbols in our image we can refernce */
    . = ALIGN(4);
    __realmode_vma_end = .;
    __realmode_secsize = ((__realmode_vma_end)-(__realmode_vma_start));

    /* Set virtual address counter to 0x100000 */
    . += __physreal_diff;

    .text ALIGN(4K) : AT(ADDR(.text))
	{
		*(.text);
	}

	/* Read-only data. */
	.rodata BLOCK(4K) : ALIGN(4K)
	{
		*(.rodata)
	}

	/* Read-write data (initialized) */
	.data BLOCK(4K) : ALIGN(4K)
	{
		*(.data)
	}

	/* Read-write data (uninitialized) and stack */
	.bss BLOCK(4K) : ALIGN(4K) 
	{
		*(COMMON)
		*(.bss)
	}
	
	/* Put other sections the compiler generates here */
}

I had a some issues that I somehow found ways around, but don't fully understand at the moment and will probably have to go back and work through:

1) I could not get the multiboot section before the .realmode section as you did in your linker script. The multitboot header was consistently moved to be the last section in the output binary (verified by readelf, IDA (free version) and running it!) if I mimicked your script. I could not figure out why this is occurring. I can prevent this is if I included the *(.multiboot) before or after the .realmode.* sections (latter is currently in my linker.ld example). My concern with this is that I am either stepping around or including the multiboot header when I copy the real mode code. My attempts to define symbols to exclude it have failed.

2) Since I was originally loading the kernel to 0x1000, I had identity mapped the first (1024*4096) Bytes and everything worked fine when I turned paging on. Now that my kernel was loaded to 0x100000, it didn't work I mapped the the whole 0xc0000000 page table to 0x100000+ addresses attempting to map .rodata, .bss, .text etc. I think the text segment works fine. However, I am getting page faults in 0x100100 - 0x1001ff range. I figure that not all code might be relatively addressed and that some addresses might be absolute 0x100000's since I generated these virtual addresses. I guess this is normal? Though I haven't nailed down exactly what is causing the page fault yet.

Thanks again!

[MichaelPetch]

What environment do you build this on? Linux? MinGW? Cygwin? MacOS? What GCC do you use. Cross compiler or native compiler?

[piscus]

I am building this on Linux x86_64 (Ubuntu and sometimes debian), but using a gcc i686 cross compiler built with crosstool-ng to produce my binary.

EDIT: I am going to open a new thread regarding a "disappearing" multiboot header when I try to generate generate Virtual Addresses: 0xc010000+, Load Address: 0x100000, and integrating Michael Petch's real mode code solution. This thread's topic can be considered solved! See Michael's stackoverflow solution above (and other great suggestions from other users)