ELF segment with p_offset 0

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
nullplan
Member
Member
Posts: 1790
Joined: Wed Aug 30, 2017 8:24 am

ELF segment with p_offset 0

Post by nullplan »

Hi all,

playing around with some stuff today I happened upon a very weird thing. I'm linking my kernel as 64-bit ELF with a code segment and a data segment. But if the kernel contains absolutely no initialized data, the linker will generate a data segment with p_filesz == 0 (OK, that would be expected), and p_offset == 0 (and that was really not expected). I have BSS for days, but so far had no need of initialized data (I solved the problem for now by adding an initialized int, but that is a bit hacky) But with p_offset being zero, my memory zeroing code would actually zero out the ELF header, which leads to the entry point being overwritten with 0, leading to a fault shortly after the jump. Also, of course, a lot of the initial code of the kernel would be overwritten with 0, also likely leading to faults.

Anyone else see that? How can I prevent that? Here's the linker script:

Code: Select all

ENTRY(_kstart)
OUTPUT_FORMAT("elf64-x86-64")

PHDRS {
    headers PT_PHDR PHDRS;
    text PT_LOAD FILEHDR PHDRS;
    data PT_LOAD;
}

SECTIONS {
    . = 0xffffffff80000000 + SIZEOF_HEADERS;
    .text : {
        *(.text)
        *(.text.*)
    } :text
    .rodata : {
        *(.rodata)
        *(.rodata.*)
    }

    /* Normally, the overlap between text and data section is handled by having
     * two different pages for the last bits of text and the first bits of data.
     * That way, if the last bits of text are overwritten, it won't affect the
     * text that is actually used. Unfortunately, for the kernel this is not
     * possible. The whole file is loaded into memory en bloc, so the same page
     * would be mapped twice. Therefore, a write access to the writable page
     * would end up being visible in the non-writable side of things. Therefore,
     * we must actually page-align here.
     */
    . = ALIGN(2M);
    .data : {
        *(.data)
        *(.data.*)
    } :data
    .bss : {
        *(.bss)
        *(COMMON)
        *(.bss.*)
    }
}
Carpe diem!
Korona
Member
Member
Posts: 1000
Joined: Thu May 17, 2007 1:27 pm
Contact:

Re: ELF segment with p_offset 0

Post by Korona »

What is the output of "readelf -l" when applied to your kernel?

Note that p_offset is the file offset, not the memory offset, so I am not sure why you want to target it in a memset() call (unless I am missing something?). A p_offset (near) 0 is expected (and useful) for the first segment (such that it includes the ELF headers). p_offset = 0 is required by the undocumented __ehdr_start, this allows you to access the ELF header from within the program.

Example where __ehdr_start is used (in a DSO, but that does not make a difference here, it only affects the meaning of p_vaddr):

Code: Select all

$ readelf -l system-root/usr/lib/libc.so 

Elf file type is DYN (Shared object file)
Entry point 0x14a60
There are 6 program headers, starting at offset 64

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x00000000001051cc 0x00000000001051cc  R E    0x200000
  LOAD           0x00000000001051d0 0x00000000003051d0 0x00000000003051d0
                 0x00000000000015c0 0x00000000000027f8  RW     0x200000
[...]
EDIT: You also do not need the PHDRS directive to generate PHDRS. I would advise against using it since its more-or-less broken for all but very few use cases.
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].
nullplan
Member
Member
Posts: 1790
Joined: Wed Aug 30, 2017 8:24 am

Re: ELF segment with p_offset 0

Post by nullplan »

Korona wrote:What is "readelf -l"?
The command that shows you the ELF headers? Oh, you mean the output. Here you go:

Code: Select all

Elf file type is EXEC (Executable file)
Entry point 0xffffffff80000750
There are 3 program headers, starting at offset 64

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  PHDR           0x0000000000000040 0xffffffff80000040 0xffffffff80000040
                 0x00000000000000a8 0x00000000000000a8  R      0x8
  LOAD           0x0000000000000000 0xffffffff80000000 0xffffffff80000000
                 0x0000000000001850 0x0000000000001850  R E    0x200000
  LOAD           0x0000000000000000 0xffffffff80200000 0xffffffff80200000
                 0x0000000000000000 0x00000000000040b8  RW     0x200000

 Section to Segment mapping:
  Segment Sections...
   00
   01     .text .eh_frame
   02     .bss
Korona wrote:Note that p_offset is the file offset, not the memory offset, so I am not sure why you want to target it in a memset() call (unless I am missing something?).
You indeed are, but then, I didn't explain it, so it may be understandable. I have a 32-bit loader kernel (multiboot 1 compliant), that gets the 64-bit kernel as a module. GRUB does the heavy lifting of getting the files into memory. The loader kernel then maps the main kernel into a new paging structure and transitions to 64-bit mode. In order to implement BSS correctly (and with a heavy dose of hacking, I admit), I map the segments by mapping physical base plus offset to the given virtual address for the entire memory size, then memset the difference between the file size and memory size to 0, hoping to remain inside valid RAM as I do so. I am well aware that the main kernel might be loaded just before an end of RAM (there are enough holes in a real memory map, after all), and that the memory size may exceed even the entire rest of the file, so yes, this needs revisiting at some point. But it works for the moment. However, with the offset of the second segment being zero, the target of the memset will be the start of the file, rather than the start of the data section. Not entirely sure what i should trigger on here, maybe file size 0?
Korona wrote:A p_offset (near) 0 is expected (and useful) for the first segment (such that it includes the ELF headers).
Yes, and indeed it is zero in the first segment as well, but there it is expected. It being zero in the second segment trips me up somewhat.
Carpe diem!
Korona
Member
Member
Posts: 1000
Joined: Thu May 17, 2007 1:27 pm
Contact:

Re: ELF segment with p_offset 0

Post by Korona »

nullplan wrote:The command that shows you the ELF headers? Oh, you mean the output.
Yeah, I noticed that my sentence was confusing after submitting the post :D

What happens if you remove the PHDRS directive?
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].
User avatar
eekee
Member
Member
Posts: 891
Joined: Mon May 22, 2017 5:56 am
Location: Kerbin
Discord: eekee
Contact:

Re: ELF segment with p_offset 0

Post by eekee »

nullplan wrote:But if the kernel contains absolutely no initialized data, the linker will generate a data segment with p_filesz == 0 (OK, that would be expected), and p_offset == 0 (and that was really not expected). [...] But with p_offset being zero, my memory zeroing code would actually zero out the ELF header, which leads to the entry point being overwritten with 0, leading to a fault shortly after the jump. Also, of course, a lot of the initial code of the kernel would be overwritten with 0, also likely leading to faults.
I have 2 issues with this:
  • The section has 0 size, so your zeroing code should be writing 0 bytes. The fact that its writing zeros at all for a 0-size section is a bug.
  • Why are you trying to zero your data segment? It defeats the segment's purpose.
Kaph — a modular OS intended to be easy and fun to administer and code for.
"May wisdom, fun, and the greater good shine forth in all your work." — Leo Brodie
User avatar
bzt
Member
Member
Posts: 1584
Joined: Thu Oct 13, 2016 4:55 pm
Contact:

Re: ELF segment with p_offset 0

Post by bzt »

@nullplan: your segment 00 looks interesting. I believe you should remove "headers" from the PHDRS block, leave only "text" with the headers (now you have them listed in two segments, that ain't good) and maybe add ":text" after the .rodata section (I can't see the .rodata section in the mapping, but this might be insignificant), and ":data" with NOLOAD after .bss section.

Code: Select all

PHDRS {
    text PT_LOAD FILEHDR PHDRS;            # note, no "headers" line
    data PT_LOAD;
}

SECTIONS {
...
    .rodata : {
...
    } :text                                 # note, added segment
...
    .bss (NOLOAD) : {                       # added NOLOAD and...
...
    } :data                                 # ...explicit segment
I'm not sure if these help, but definitely worth a try. This tells the linker to place the bss in the data segment, which follows the text segment, so its p_offset can't be 0 for sure.
nullplan wrote:It being zero in the second segment trips me up somewhat.
Since you have an extra first segment, I guess you meant the text segment by "second segment". To solve that, use

Code: Select all

    . = 0xffffffff80000000;
    .text . + SIZEOF_HEADERS : {
or (note the dot before the colon, that's important)

Code: Select all

    . = SEGMENT_START("text-segment", 0xffffffff80000000) + SIZEOF_HEADERS;
    .text . : {
...or keep the "headers" segment but do not list FILEHDR and PHDRS in "text" segment in the PHDRS block. If by "second segment" you meant the data segment, then my best guess is explicitly setting the segment in the SECTIONS block as above.

I'm not sure, was your first real segment (the program headers) intentional? I see not much point in it, because you'll need the program headers to list the segments anyway, so why should you put the program headers segment in the program headers block? You should just use the ELF header to locate the program headers.
eekee wrote:The section has 0 size, so your zeroing code should be writing 0 bytes.
Why are you trying to zero your data segment? It defeats the segment's purpose.
Nope, he's trying to zero out the uninitialized part. The initialized part has the length of 0 (he's not zeroing that, no matter the size). In readelf's output, FileSiz tells the initialized size of the segment (bytes stored in ELF), and MemSiz - FileSiz the uninitialized part (not stored in ELF, must be zeroed out on load, 0x40b8 in his case).

Cheers,
bzt
Octocontrabass
Member
Member
Posts: 5568
Joined: Mon Mar 25, 2013 7:01 pm

Re: ELF segment with p_offset 0

Post by Octocontrabass »

I feel like I'm missing something. Why do you expect a segment not present in the object file to have a meaningful value for p_offset? Shouldn't you be using p_vaddr here instead?
nullplan
Member
Member
Posts: 1790
Joined: Wed Aug 30, 2017 8:24 am

Re: ELF segment with p_offset 0

Post by nullplan »

bzt wrote:@nullplan: your segment 00 looks interesting. I believe you should remove "headers" from the PHDRS block, leave only "text" with the headers (now you have them listed in two segments, that ain't good)
The first segment type is PT_PHDR, not PT_LOAD. There is no overlap between any PT_LOAD segment. I have no clue what I might need a PT_PHDR segment for, but I'm leaving it in for the moment. It has absolutely nothing to do with p_offset in the second PT_LOAD segment being zero. Unless it does and ld is being buggy. Which unfortunately I can't rule out. I will test it, but don't get your hopes up. Probably won't happen until after Christmas.
bzt wrote: and maybe add ":text" after the .rodata section
According to the documentation, the segment remains the same until overridden. Adding or removing :text there has no effect.
bzt wrote:(I can't see the .rodata section in the mapping, but this might be insignificant),
There is no .rodata section yet. I got ahead of myself and implemented the GDT and IDT stuff and only then noticed I didn't even have a debug printer yet (so no need for strings at all).
bzt wrote: and ":data" with NOLOAD after .bss section.
Both of these only explicitly add things that ld has already correctly deduced. But fine, I will test them all the same.
bzt wrote:This tells the linker to place the bss in the data segment, which follows the text segment, so its p_offset can't be 0 for sure.
As you can see, it has already correctly placed the BSS section into the second LOAD segment.
bzt wrote: I guess you meant the text segment by "second segment".
No, I actually did mean the second LOAD segment (the data segment). The PHDR segment is little more than an annotation. I might, for instance, add a stack segment later to tell the loader explicitly how much stack to allocate for the first thread.
bzt wrote:I'm not sure, was your first real segment (the program headers) intentional? I see not much point in it, because you'll need the program headers to list the segments anyway, so why should you put the program headers segment in the program headers block? You should just use the ELF header to locate the program headers.
Yeah, I don't know, either, but it was in all the examples. And it is in all the executable files I could find, so I figured it couldn't hurt.
Octocontrabass wrote:I feel like I'm missing something. Why do you expect a segment not present in the object file to have a meaningful value for p_offset? Shouldn't you be using p_vaddr here instead?
Because all PT_LOAD segments otherwise have a working p_offset value? I have looked at other ELF loaders on the interwebs, and so far I have failed to find any that would handle this case. I was simply unaware that having a garbage p_offset is even a possibility. Oh wait, that's a lie, I did find one that would handle this by accident. It will map everything from p_offset to p_offset + p_filesz, out of the file (so here it would fail to do anything), and then map anonymous pages for the remainder of p_memsz. That is what I aim to do eventually, but right now it is not implemented yet.
Carpe diem!
User avatar
bzt
Member
Member
Posts: 1584
Joined: Thu Oct 13, 2016 4:55 pm
Contact:

Re: ELF segment with p_offset 0

Post by bzt »

nullplan wrote:I have no clue what I might need a PT_PHDR segment for, but I'm leaving it in for the moment. It has absolutely nothing to do with p_offset in the second PT_LOAD segment being zero. Unless it does and ld is being buggy. Which unfortunately I can't rule out.
In theory you're right. But as you've said it, we cannot rule out buggy linker. I also had many issues until I come up with a linker script that actually works with both GNU ld and LLVM lld the same way. I've seen quite a lot of oddities :-(

FYI if you don't care about Offset, and you only use VirtAddress in the program headers then you're good to go (that one is correct, being 0xffffffff80200000). Frankly with FileSiz 0 the Offset shouldn't matter anyway as there's nothing to copy from the ELF (where GRUB loaded it) to the segment.

Cheers and have a merry Xmas!
bzt
Post Reply