kernel load address > 1MB access in realmode

[mtbro]

My current status: I wrote a bootloader that searches for custom GPT bootpartition where my gboot loader resides. gboot searches for ext2 root partition, finds /boot/kernel and elf loader loads it to the memory. gboot then jumps to my kernel.
All works as expected.

Problem: I was thinking about an option to load my kernel above 1MB. If anything for a test if that can be done in RM. A20 is enabled.
I modified my loader just to do a test: boot1.S. This test works in qemu but fails on my test Celeron 500MHz HW. Machine gets frozen, no other code gets executed. I did go through e820 map so I know I'm landing my %es:(%edi) or %es:(%di) in free memory range. My free range is from 0x0 - 9fc00 and 0x100000 - 0x2000000.

While I could probably switch to PM before copy and then back to RM afterwards but that doesn't seem like a smart idea. At this early stages of kernel development I'm probably fine with being < 1MB but I'm wondering why system froze. I'd assume I'd see some sort of exception..

edit: I understand the limit of RM and that I can't get much more from segment:offset address than that. Part of the test was to see if I can load data above though. I wonder how mem extenders worked under DOS then (I only remember that code still had to below 1MB, only data could have been above). As I read my question as posted I feel I should erase it. But I will keep it..

[nullplan]

With A20 gate being enabled, you can already access memory all the way up to 0x10ffef (1MB + 64 kB - 16 B). You cannot get farther than that. If you want to access memory beyond that address, the easiest is to use function 87h of interrupt 15h. That gives you relatively quick access to the low 16MB of address space. That ought to be enough for your purpose.

The only alternative to that is to utilize unreal mode, but unfortunately that doesn't really work once you call back into BIOS, as any BIOS function can undo real mode. Another alternative is to hope that your BIOS supports EDD 3.0, because then you can make the BIOS load the sector to any 64-bit address.

While I could probably switch to PM before copy and then back to RM afterwards but that doesn't seem like a smart idea.

That is exactly what the above stated function does. If you want to keep using BIOS (e.g. to load more sectors), this is going to be the only real way.

Kernel.org has a minimal Linux boot loader that runs from MBR or VBR and uses a block list to load the kernel and initrd. And it also uses that function to load everything. Load a block, copy a block, adjust destination address, repeat.

[Octocontrabass]

works in qemu

QEMU's TCG doesn't enforce segment limits. You're trying to access memory beyond the segment limit.

While I could probably switch to PM before copy and then back to RM afterwards but that doesn't seem like a smart idea.

Actually, it's a pretty good idea. Another option is INT 0x15 AH=0x87, but this function may unconditionally disable A20 before returning, so be careful with that. A third option is to install a #GP handler that switches to unreal mode; you can use your existing copy routine this way, and you don't have to worry about the BIOS changing the segment limits back to 64kB.

I'm wondering why system froze. I'd assume I'd see some sort of exception..

Did you install an exception handler? The BIOS exception handler returns without doing anything, which causes the same exception again.

I wonder how mem extenders worked under DOS then (I only remember that code still had to below 1MB, only data could have been above).

Usually they worked by running in protected mode, with lots of interesting hacks to make it look like everything is still in real mode from a software perspective.

[mtbro]

Did you install an exception handler? The BIOS exception handler returns without doing anything, which causes the same exception again.

For some reason I assumed it's set by BIOS already. I did boot1.S and now I can see exception being triggered.

I've a question I'm a bit afraid to ask but I'll do: is unreal mode and the virtual 8086 mode referring to the same mode?

Thank you both for the int 15/87h - that does seem like a solid option. My curiosity stemmed from an idea that if I load the kernel above 1M I may have clean state for virtual 8086mode which I could probably use in early stages of my kernel. But maybe I'm getting too ahead of myself.

[Octocontrabass]

I've a question I'm a bit afraid to ask but I'll do: is unreal mode and the virtual 8086 mode referring to the same mode?

No. Virtual 8086 mode is part of protected mode. Unreal mode is real mode with higher segment limits.

My curiosity stemmed from an idea that if I load the kernel above 1M I may have clean state for virtual 8086mode which I could probably use in early stages of my kernel.

What would you use it for?

[mtbro]

Thanks for the clarification.

What would you use it for?

Mostly I'm curious to see how that works.
I spent maybe too much time in bootloader but I wanted to solve that issue myself (find a partition, find a kernel on ext2, load the elf kernel). I'm in that naive state where I even don't know what I don't know. Having an option to call BIOS services sounded like an ok idea.

[Octocontrabass]

Having an option to call BIOS services sounded like an ok idea.

It sounds good, but it only works with a very limited set of BIOS services. It's better to have your bootloader collect everything you need from the BIOS before it switches to protected mode and jumps to your kernel.

[mtbro]

I'll stick to that approach as you mentioned (right now I'm passing only smap structure to kernel).

I tried unreal mode as part of the test - boot1.S - it does work on HW too. I'm not sure how long that state lasts but it's interesting idea to have as a backup too.

Thank you both for the answers; you made this clear for me.

[Octocontrabass]

I tried unreal mode as part of the test - boot1.S - it does work on HW too.

On some hardware it may not work correctly because you're not using far jumps to enter/exit protected mode.

I'm not sure how long that state lasts but it's interesting idea to have as a backup too.

It lasts until something switches the CPU back into protected mode and sets new segment limits. The BIOS could do that at just about any time, and then you'll receive #GP again. If you set up a #GP handler that switches to unreal mode, you don't have to worry about it: your handler will be called whenever you need it.

[mtbro]

The BIOS could do that at just about any time, and then you'll receive #GP again. If you set up a #GP handler that switches to unreal mode, you don't have to worry about it: your handler will be called whenever you need it.

Perfect, thanks, this does make sense to me.

[rdos]

My boot loader solves this by temporarily switching to protected mode to do the copy and then switch back to real mode to read more data through BIOS. This method works on all machines. Unreal mode might be an alternative, but I really don't see the point in it. You risk that BIOS will reload selectors. The switching back & forth method cannot be sabotaged by BIOS. Additionally, the amount of code for switching between real mode and protected mode and back is more or less identical to enabling unreal mode.

[mtbro]

More I thought about the idea of jumping back and forth more I liked it. Especially as I was encouraged here it's a normal thing to do. I ended up doing just that in my libsa16.S library.

It seems to be working fine. My "kernel" is now loaded above 1M.
Thanks for sharing.

[neon]

Hi,

Suppose can vouch as this is what my loader does: it runs in 32 bit c but a separate asm file implements an interface that drops down to real mode to call the bios at will. Although my version is modeled after the old int86 function the idea is the same: load in real mode, copy and work with it in protected mode

[rdos]

For just implementing the copy function, it's not necessary to "obey" all the rules of switching to protected mode. For instance, the only selector that needs loading is the one used as the destination for the copy, like ds or es. The source address (which need to use a different selector) can be used with real mode attributes, and cs and ss doesn't need to be loaded at all. Interrupts should be switched off to avoid having to setup the IDT. Actually, the GDT only needs to contain a flat data selector and the null selector.

[mtbro]

Actually, the GDT only needs to contain a flat data selector and the null selector.

If I don't do jump to load cs selector loading data selectors doesn't seem to work. E.g. code I shared above where 0x10 is the 32b data:

Code: Select all

        lgdt (sa_gdt_desc)
        movl %cr0, %eax
        orl $1, %eax
        movl %eax, %cr0

        movw $0x10, %ax
        movw %ax, %ds

doesn't work. I'm starting to question my code above I did as PoC for unreal mode in boot1.S. I had to do jump which was not a far jump. Isn't that undefined behavior when it comes to what is happening to selectors?

I was not able to make my pm_copy() work if I didn't have 16bit code/data selectors defined in my GDT either. That's why I'm "rolling" from PM to RM by first selecting those selectors. Without that it always triple faulted.
Checking out the wiki's unreal mode proper PM cs selector is loaded with far jump. Why there would be 64k code limitation then? It's not in its GDT definition (0xffff limit).
I think I may have overthought this post though..