Elf relocatable object vs. shared object

[Techel]

I'm reading into linking and stuff with ELF.
-> if the file is an executable, sections contain data that may be loaded into memory.
program headers define segments, ie. which memory segments should have which permissions (r/w/x etc), right?
Why don't just use the sections and leave the phs out? They have exact the same flags (r/w/x etc)
-> A relocatable object only contains several section containing data which can be rebased and their symbold be resolved and linked
with other ones; A shared object is the same; besides it contains information about execution. Why not just use relocatable objects
instead of shared ones, they can be linked and executed as well?

[Nutterts]

I'm reading into linking and stuff with ELF.

Same here, so I'm far from an expert. Just writing a simple elf64 executable loader.

Why not just use relocatable objects instead of shared ones, they can be linked and executed as well?

Yeah I've been wondering that too. I think when it comes to kernel drivers/modules it doesn't matter that much. If I had to write that part today with what I know now I'd probably go for a relocatable object. Afaik, and take this with a grain of salt as it's mostly guess work, it's easier with a shared object to extract and link parts of into another executable. Shared objects are usually not executed themselves. So it seems more a must-have for user space, like the libc library etc.

[rianquinn]

I'm reading into linking and stuff with ELF.
-> if the file is an executable, sections contain data that may be loaded into memory.
program headers define segments, ie. which memory segments should have which permissions (r/w/x etc), right?
Why don't just use the sections and leave the phs out? They have exact the same flags (r/w/x etc)

The program headers are what your going to use when loading the ELF file into memory prior to relocating everything. The current text in the wiki is pretty misleading. But basically, the ELF loader will loop through each segment, and load each segment into memory that is marked "LOAD". Once that is done, you will use the section information to figure out how to perform the relocations. Once that is done, you should be able to execute your program. If you have no relocations, you probably only need the program headers.

-> A relocatable object only contains several section containing data which can be rebased and their symbold be resolved and linked
with other ones; A shared object is the same; besides it contains information about execution. Why not just use relocatable objects
instead of shared ones, they can be linked and executed as well?

A buddy of mine actually did this, loading relocatable object files into memory, relocating and linking manually, and executing from there. It can be done, but it's a lot more work. The shared object file keeps things pretty simple. Once I figured out how everything worked, I was able to write a custom ELF loader in a day using shared libraries. If you looking for an example, feel free to take a look at what we did: https://github.com/Bareflank/hypervisor ... elf_loader

[Techel]

Thanks!

[kscguru]

I'm reading into linking and stuff with ELF.
-> if the file is an executable, sections contain data that may be loaded into memory.
program headers define segments, ie. which memory segments should have which permissions (r/w/x etc), right?
Why don't just use the sections and leave the phs out? They have exact the same flags (r/w/x etc)

Elf linkers are vague about the difference between a section and a segment, but that difference is critical. A section is information for a LINKER telling it a bunch of symbols are handled similarly (e.g. .text, .data) and cross-referenced (e.g. .sym for the symbol table); the linker will merge similar sections. A segment is information for a LOADER telling it, in simple mechanical steps, how to map the file into memory (e.g. this segment r-x, this segment rw-, apply these relocations with formula *a = *a + offset). ELF files can contain both formats, but they are targetted at different uses. A linker tends to be fairly large and needs lots of support information (e.g. a linker script, even if default); a complete loader can be implemented in a few hundred lines of code, the format is so simple.

One more analogy: sections are like assembly, they are (somewhat) human readable but need post-processing by an assembler, segments are like machine code, unreadable but can be executed directly. After linking a final executable, a linker would discard section information as no longer useful, just like the source code is no longer carried by the ELF file once it is compiled.

-> A relocatable object only contains several section containing data which can be rebased and their symbold be resolved and linked
with other ones; A shared object is the same; besides it contains information about execution. Why not just use relocatable objects
instead of shared ones, they can be linked and executed as well?

The amount of code necessary to link versus load. It's certainly possible to write a sophisticated loader that actually links relocatable objects directly into memory, but linking is complicated: there's a large symbol table (lots of string lookups) and the algorithms are O(n^2) in either time or memory. Loaders are much smaller and faster - O(1) in time and memory, and can be implemented in a few hundred lines of code.

There are places where relocatable objects are used - kernel modules are a common case (Linux .ko, for example). There, the symbols exported by a kernel are still in symbolic form (symbol X with string name Y at address Z) - kernels can change size and offsets without recompiling the associated modules, and don't exactly have system calls available to "help". A shared object where the symbolic references are already resolved loads faster - but only as long as its assumptions are satisfied.

[Techel]

I'd like to implement kernel modules that can be placed at any location and may call every other module's functions etc. Should I implement these as shared or relocatable objects?

[Schol-R-LEA]

I would recommend relocatable modules, for the reasons kscguru outlined earlier - since the sizes and offsets for kernel modules are only known at runtime, and in some cases the modules may need to be reloaded or relocated on-the-fly, having the symbol table available is likely to be necessary, so a shared executable probably isn't adequate. It is one of the cases where a relocatable image is the better solution.

[jnc100]

Additionally, shared libraries are (at least in the SysV ABI) expected to be position-independent code so that they can be loaded at different locations in a multiple of address spaces. Kernel modules do not have this requirement, so you can get the performance benefit of using non-PIC code with the flexibility of loadable modules by having them as relocatable objects instead.

To answer the previous point about segments vs sections: segments are designed for quick loading of 'executable files'. By this, ELF means either executables or shared libraries. It would actually be technically wrong here to use the section table to load the file as there is nothing in the specs to state that executable files need a section table at all (albeit all current tools do provide one). Sections are designed for the linker solely to use. If, however, you use relocatable object files as your modules, you will need to include a simple linker in your kernel. This need not be complex: simply kmalloc/mmap a memory region for each section, store all global symbols in a large string->offset map (e.g. hash table) and then perform the relocations (there are actually very few x86/x86_64 relocation types that will be emitted by your tools).

Regards,
John.

[Techel]

Nice, thank you all very much.