Making the kernel a dynamically loaded executable (pie)

[sebihepp]

Dear Forum,

I am working on making my kernel a dynamicall loaded executable, to use limines KASLR, for some days now and still can't get it to work properly.
This is my repository: https://github.com/sebihepp/HeppOS

I expect the problem to be in the linker script, specifically the PHDRs and sections. Depending on how I arrange the linker script, I can get qemu to work and bochs to triple fault in the middle of the kernel, both to triple fault directly when my kernel gets executed, or limine to print errors like "PT_DYNAMIC is of size 0, etc." on both emulators.

I totally lack the understanding of my linker and linker script in respect to PIEs (Position independent executables). I searched the last days for information what limine expects from such an executable, info on which sections are needed for PIEs and how to get the PHDRs right, but found absolutely nothing.

Could you please assist me? Maybe just a pointer into the right direction? Which documentation to read, etc.

Best regards
Sebi

[Octocontrabass]

I expect the problem to be in the linker script, specifically the PHDRs

Why does your linker script specify the PHDRs? What happens if you let the linker automatically generate them?

[sebihepp]

Why does your linker script specify the PHDRs? What happens if you let the linker automatically generate them?

If I don't specify them, then ld throws the error: PHDR segment not covered by LOAD segment

[nullplan]

I managed to fix all of your issues, which was no small feat without the limine header, but I just wrote a mockup. It didn't have to work after all, only compile.

First issue was the lack of a VMA. Without that, ld assumes the VMA is 0, but you actually want to set it to SIZEOF_HEADERS. Second issue was the overalignment of the input sections. I got rid of all the alignment specifications in the linker script save for the one for .data, since that is the break between executable and writable memory. Third issue was I had to add "-Wl,-z,max-page-size=0x1000" to the linker command line to get ld to use the correct page size. Otherwise it would end up loading everything in the same segment.

I had to move the init_array, fini_array, and preinit_array blocks out of .rodata. In PIE, these sections are writable, since they get relocated. Technically, in PIE they should go to the start of the writable section, with another page break afterward, so they can all be write protected after relocation. Putting them into their own section also allows the linker to assign the correct section types.

I also got a warning about a textrel in a PIE, because your GDT loading code was not written with position-independent code. You can even just leave it they way I left it, because the same code works for either mode.

In short, I have posted a pull request on github (https://github.com/sebihepp/HeppOS/pull/1) Have fun.

[sebihepp]

Thank you very much @nullplan. It an honor to get helped by such experts. The same is true for @octocontrabass of course.

I merged your change request and it compiles and limine loads the kernel correctly. Thank you very much.

I could have guesses, that init and fini array had to be written to in a pie - sorry I missed that.
While I know what telling the linker the page size means, I am guessing BLOCK and ALIGN wasn't enough, because header sections, etc. needed to be page aligned as well.
I also guess the VMA needed to be set to SIZEOF_HEADERS to allow the headers in front of the code inside the executable? Its a weak guess though.

For the GDT part I understand your assembly, though I am not sure why my absolute address isn't getting patched by the dynamic linker (limine loader)?
Why is _ReloadCS a relative address instead of a absolute one? Is it because of the missing dollar-sign? Know what? I will find the answer by myself reading the docs of gas.

Again, thank you very much. The next step for me is to find out, why bochs triple faults and qemu runs fine. But that another topic.

[nullplan]

While I know what telling the linker the page size means, I am guessing BLOCK and ALIGN wasn't enough, because header sections, etc. needed to be page aligned as well.

The problem is that x86_64 ld just likes to use 2MB pages for no reason, and you actively have to tell it to stop. Otherwise it sees sections with read, write, and execute format, linked into the same 2MB page, and creates and RWE segment instead of splitting them up.

But you also had too many ALIGN keywords. Take .text: At that point, I set the VMA to SIZEOF_HEADERS, which is 0x60 or so. With your alignment requirement of 4K, the linker has to insert almost 4k of padding before the .text section can even begin. Worse: With the page size set to 4k, the ELF header and program headers might not be loaded at all.

I also guess the VMA needed to be set to SIZEOF_HEADERS to allow the headers in front of the code inside the executable? Its a weak guess though.

It is to put the ELF and program headers on the same page as the start of the text section, to encourage the linker to put that stuff in a LOAD segment. Otherwise the VMA is set to 0, and you get the first load segment with p_vaddr = 0 and p_offset = 0x1000 (so similar to the above effect), and the headers aren't in it. Which is a condition that at least my ld detects and then complains about.

For the GDT part I understand your assembly, though I am not sure why my absolute address isn't getting patched by the dynamic linker (limine loader)?

Actually, it might be, since the relocation at this point is an absolute 64-bit address type (which Limine might support), and maybe Limine does not enable write protection before processing the relocs. However, textrels are just bad form and indicate lazy assembly. There are warnings all over the place if you have them, and it is best to eliminate them wherever possible. In userspace they cause crashes or wasted memory.

Why is _ReloadCS a relative address instead of a absolute one? Is it because of the missing dollar-sign?

No, it is because for IP-relative addresses, the assembler defaults to relative addressing. It is quite counterintuitive. The code would normally be

Code: Select all

leaq _ReloadCS - .(%rip), %rax

But when they ported the assembler to x86_64, they decided to make that "- ." implicit when the base register is RIP. The missing dollar is correct, because this is not loading a literal, this is indeed a memory reference, albeit one with predictable outcome.

[sebihepp]

No, it is because for IP-relative addresses, the assembler defaults to relative addressing. It is quite counterintuitive. The code would normally be
CODE: SELECT ALL

leaq _ReloadCS - .(%rip), %rax
But when they ported the assembler to x86_64, they decided to make that "- ." implicit when the base register is RIP. The missing dollar is correct, because this is not loading a literal, this is indeed a memory reference, albeit one with predictable outcome.

Ohh, i didn't know that. I wondered why it adds the virtual address of the label to rip. But now it is clear you subtract the current address (of the linker), getting an offset calculatable at compile time. Then add it to rip and voila, you got the absolute address for the lretq.

The problem is that x86_64 ld just likes to use 2MB pages for no reason, and you actively have to tell it to stop. Otherwise it sees sections with read, write, and execute format, linked into the same 2MB page, and creates and RWE segment instead of splitting them up.

Okay, thats weird, that it still tries to use 2mb segments. But i understand now. And an RWE segment lacks the write protection of the code.

It is to put the ELF and program headers on the same page as the start of the text section, to encourage the linker to put that stuff in a LOAD segment. Otherwise the VMA is set to 0, and you get the first load segment with p_vaddr = 0 and p_offset = 0x1000 (so similar to the above effect), and the headers aren't in it. Which is a condition that at least my ld detects and then complains about

Okay, i think i misunderstand the way the loader works. I thought the entire file is getting loaded into physical memory and then the LOAD segments (.text .rodata .data .bss) are getting copied into physical memory again and then paging is applied for the copy. There would be no need for the elf and program headers to be loaded, as they are in the loaded file, which already has been loaded, and not used by the program itself, only by the loader.