Another of those "can't enter long mode" post...

[kzinti]

Thanks, I really appreciate you spending time looking at this.

1) I use the default GCC that comes with Linux Mint, which is a variant of Ubuntu. This is just for convenience and I understand the downsides of not using a cross compiler. This is why my makefile do support cross compiling: I want to make sure I move away from my hosted GCC at some point. Maybe I simply need to add command line options to disable PIC so that it works on Debian systems. The kernel certainly do not use / need PIC.

2) When I cross compile using CROSS_COMPILE=x86_64-elf-, I run into the same issue as you: my cross-compiler doesn't have a 32 bits version of libgcc.a to link the bootloader. I need to update my build process to allow one to specify a different cross compiler for the 32 bits bootloader and the 64 bits bootloader (or drop the libgcc dependency). I haven't got to it yet and I hack the Makefiles as you did. Either that, or I need to figure out how to build a multilib cross compiler, which I wanted to avoid for now.

[MichaelPetch]

Now I do realize the code works in Bochs, but I am curious about whether this was intended with your allocator and memory mapping (it looks peculiar and is the output from BOCHs):

Code: Select all

0x0000000000000000-0x00000000ffffffff -> 0x000000000000-0x0000ffffffff
0xffffff0000000000-0xffffff0000000fff -> 0x000000000000-0x000000000fff
0xffffff0000001000-0xffffff0000001fff -> 0x000000200000-0x000000200fff
0xffffff0000002000-0xffffff0000002fff -> 0x000000400000-0x000000400fff
0xffffff0000003000-0xffffff0000003fff -> 0x000000600000-0x000000600fff
0xffffff0000004000-0xffffff0000004fff -> 0x000000800000-0x000000800fff
0xffffff0000005000-0xffffff0000005fff -> 0x000000a00000-0x000000a00fff
0xffffff0000006000-0xffffff0000006fff -> 0x000000c00000-0x000000c00fff
0xffffff0000007000-0xffffff0000007fff -> 0x000000e00000-0x000000e00fff
0xffffff0000008000-0xffffff0000008fff -> 0x000001000000-0x000001000fff
0xffffff0000009000-0xffffff0000009fff -> 0x000001200000-0x000001200fff
0xffffff000000a000-0xffffff000000afff -> 0x000001400000-0x000001400fff
0xffffff000000b000-0xffffff000000bfff -> 0x000001600000-0x000001600fff
[snip this continues on for *most* of the addresses]

You appear to identity map the first 4GiB(the first entry) then you seem to map the higher half to lower. It is worth noting that on the left the range is 4KiB pages but the physical addresses on the right are for 4KiB pages every 2MiB. Almost like one was mapping physical addresses as 4KiB pages but was done so like 2MiB pages . Now maybe this is what was expected, but it just looks peculiar. Maybe your physical page allocator works this way when mapping, but I haven't even attempted to wade through the actual code yet.

It is possible BOCHs has a bug here as well in displaying this. I haven't investigated that but i doubt it.

[kzinti]

The intention is to identity-map the first 4 GB using 2 MB pages and map the kernel to 0xFFFFFFFF 80000000 using 4K pages.

[MichaelPetch]

Something I just noticed is that your BOCHs is set up with 2GB of memory but QEMU seems to be 8G? If I drop QEMUFLAGS to use 2GB it seems to run fine (well it doesn't crash, whether it is running as expected I'm not 100% but the output seems to be as expected)

[MichaelPetch]

The intention is to identity-map the first 4 GB using 2 MB pages and map the kernel to 0xFFFFFFFF 80000000 using 4K pages.

If that is the intention then that isn't what happened. BOCHs doesn't show whether 4KiB or 2MiB were use to do the identity mapping, but it does seem to suggest that whatever you did in fact identity mapped the first 4GiB correctly. The Bochs output suggests that for the higher half you are mapping 4KiB pages (in virtual space) every 2MiB in physical space which according to what you intended is not what is suppose to happen.

[kzinti]

It does indeed work when I set QEMU to use 2 GB of RAM. Very interesting. It's not like I map large pages when there is 8 GB and normal ones when there is 2 GB. When I print the values I map in map_page(), they are 4 KB apart. But that doesn't mean I am writing the values in the right places...

There is something really interesting happening here.

I wonder if there is some sign-extension going on somewhere that i don't see.

[kzinti]

If I don't load the kernel (meaning I don't map it), I still get the page fault. This suggests the issue has to do with the identity mapping (or something else entirely). Basically it is not the kernel mapping.

[kzinti]

If I force the page allocator to allocate the page tables at addresses under 0x7FFFFFFF it works fine. But if the page tables are allocated at 0x80000000 or above, it crashes. Sounds like it is a sign extension issue.

[kzinti]

Bingo, that's the problem:

Code: Select all

    pml4[0] = (uint64_t)pml3 | PAGE_WRITE | PAGE_PRESENT;

pml3 is allocated at say 0x80000000. The cast to uint64_t sign-extends that value to 0xFFFFFFFF 80000000 which is not what I want. This doesn't happen when compiling the code in 64 bits mode, but it does in 32 bits mode.

The fix is to change the code to:

Code: Select all

    pml4[0] = (uintptr_t)pml3 | PAGE_WRITE | PAGE_PRESENT;

Thanks again to both of you, your help is much appreciated,

[MichaelPetch]

Glad you got it going!

[bzt]

That's very interesting. I didn't know this. I use the section headers to find the symbols in order to apply relocations (my kernel is a relocatable executable aka a shared object / .so). Is there another way to go about relocating my binary?

Well, it's not like that. You have to relocate your binary the same way, it's just the information is in a different format (a more compact and easier to parse format my I add).

1. look for the PT_DYNAMIC table in the program headers
2. dynamic table contains fix sized records (similar to section headers)
3. instead of string comparision (sectionname==".symtab") you simply compare integers (sectiontype==DT_SYMTAB)
4. data relocation info can be found likewise, using DT_RELA. Relocation entry size is stored in DT_RELAENT.
5. function relocation info pointed by DT_JMPREL.
6. this table also lists the needed shared libraries, with DT_NEEDED (see "ldd" output)

This dynamic table is mandatory for relocatable binaries and for binaries that use shared libraries. For full list of available records see:
https://docs.oracle.com/cd/E23824_01/ht ... 42444.html Table 13-8.

Now locating relocation info is a bit tricky, because someone misread the original SCO ELF spec (page 89-90), and Linux kernel is not really compliant. Unfortunately these days GNU ld (save SPARC port) and LLVM lld developers use the Linux kernel as a reference, therefore now it's perfectly valid to have one relocation table which contains both data and jump relocation records (as the spec mandates), and also to have two seprated relocation tables (as Linux kernel implements). This is highly dependent on linker script defaults now, but you can force one table with a specific linker script. There was an LLVM lld bug about this which I have repoted and which they fixed in recent versions (I love those guys, they really care about the quality of their linker).

If you use the ELF spec literally, you'll have only one table, DT_JMPREL pointing somewhere in the middle of it:

Code: Select all

+---+ start of relocation table, pointed by DT_RELA
|   | (each record DT_RELAENT bytes)
|   |
+---+ pointed by DT_JMPREL, contains code relocations from now on
|   |
+---+ end of table, DT_RELA + DT_RELASZ.

Cheers,
bzt

[kzinti]

Got it. This is actually what I do when self-relocating the EFI bootloader (my code is based off gnu-efi).

I will update my code to do relocations properly.
Thanks!

[kzinti]

Funny thing: I don't even use a .so anymore and this relocation code is now obsolete. I'll just remove it.