How to actually mmap() ELF segments into memory?

[mariuszp]

Recently I;ve been working on properly implmenting mmap() under my OS, and I'm really confused on how to map ELF segments into memory using the standard call.

Here's the "readelf -l test" for a simple executable:

Code: Select all

Elf file type is EXEC (Executable file)
Entry point 0x4000b0
There are 2 program headers, starting at offset 64

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000400000 0x0000000000400000
                 0x00000000000000b5 0x00000000000000b5  R E    200000
  LOAD           0x00000000000000b8 0x00000000006000b8 0x00000000006000b8
                 0x0000000000000004 0x0000000000000130  RW     200000

 Section to Segment mapping:
  Segment Sections...
   00     .text 
   01     .data .bss 

I need to of course map the 4 bytes from the file into memory, and then make the rest zero by mapping anonymous memory.

But the entire section is on a single page!

According to http://pubs.opengroup.org/onlinepubs/00 ... /mmap.html the mapping is to be performed in page units (which is obvious even without looking at the standard). But does that mean if I just map the 4 bytes of the file into memory, the entire page from the file at that position gets mapped? That would not make sense, since that portion of the file does not contain all zeroes and hence violates the rule that the BSS sections must be zeroed.

If, on the other hand, I'm only supposed to read the 4 bytes in, and zero out the rest of the page, that of course means I don't map entire pages from the file. In this case, what is the best way to store file contents in memory and share them (using copy-on-write) between processes? If one process maps those 4 bytes, and another maps 8 bytes from that region, do I need to allocate a separate frame even if that area is never written to, since it is a partial page?

I hope I explained it clearly.

[Combuster]

Mapping entire pages straight from disk only works as an optimisation when individual sections are actually large enough to contain entire 4K chunks. For small applications you'll only get the corner cases where you have to load less than a page full of bytes and have to clear the remainder for security reasons. In essence, you have to start with the start- and endcases and maybe later you get to add the optimisation for the large middle parts.

After you have loaded a page into the process, you should still be able to use it with the copy-on-write system as they are supposed to be separate mechanics.

[mariuszp]

Thank you for the information.
And yes, i am able to use copy-on-writeon thosepages when the process is forked etc, i was asking about loading the original data once, keeo it on a shared read-only page, then only copy the "file page" to a new frsme if a process.with a private mapping writes to it.

[Schol-R-LEA]

Keep in mind that mmap() is meant primarily to map the .text and (perhaps) .rodata sections from the file, not mutable data. Otherwise, changes to the mapped page would change the ELF file itself, which is almost certainly not what you want. As Combuster said, using mmap() and copy-on-write for large sections of data is an optimization technique that only suits certain situations, namely large data sets that mostly go unused and unchanged over the course of the process run - a very unusual circumstance. While it should be possible to write a loader to do so programmatically, it is a fairly advanced technique that may well cost more processing time at loading than it would save during the process execution. If I were writing such a loader, I would want the more basic functions to work before adding that, and even then set a very high size threshold (4MiB or higher) for the mmap()ed data.

To be honest, I would be more inclined to use copy-on-write mmap()ing for loading things like audio and video files, except that this would only work for data that could be loaded raw to the memory in a usable form, so it would rarely if ever be feasible even then - I am thinking of doing something like that, but for semi-persistent caching of data, rather than permanent storage (basically, rather than swap space, I would have a more general caching mechanism that allows recent pages to remain page-mapped across system reboots, with different types of data categorized for faster retrieval).