ELF interpreter and porting software

[mariuszp]

My kernel is currently able to load ELF programs, as long as they have been statically linked to be loaded at the correct place in memory and they use no shared libraries. Now I want to actually port some software, which would obviously at least use the C library, and for that I'll need dynamic linking and relocation.

If I understood correctly, then to do this, I need a "program interpreter", which is reference by a Program Header of type PT_INTERP. I was wondering, do I have to make that dynamic linker (program interpreter) myself, or is it possible to port one? And how can I change the program interpreter (using a linker script, for example)? I guess I have to make it myself, because the way arguments are passed to it is implementation-specific (I did not see anything in the ELF specification that said exactly how the arguments are to be passed).

Any help?

[Brendan]

Hi,

My kernel is currently able to load ELF programs, as long as they have been statically linked to be loaded at the correct place in memory and they use no shared libraries. Now I want to actually port some software, which would obviously at least use the C library, and for that I'll need dynamic linking and relocation.

If I understood correctly, then to do this, I need a "program interpreter", which is reference by a Program Header of type PT_INTERP. I was wondering, do I have to make that dynamic linker (program interpreter) myself, or is it possible to port one? And how can I change the program interpreter (using a linker script, for example)? I guess I have to make it myself, because the way arguments are passed to it is implementation-specific (I did not see anything in the ELF specification that said exactly how the arguments are to be passed).

Any help?

I'm guessing (I've never wanted to support ELF and haven't researched it); but I'd assume that "program interpreter" would:

• determine if the executable is for your OS and the correct architecture (e.g. not for Linux, and not for Itanium or something)
• examine the sections in the executable and figure out where each section gets mapped into virtual memory. This would include sanity checks (e.g. if an application is linked to wherever your kernel is, then it's a good idea to refuse to overwrite the kernel).
• determine which shared libraries it needs and pre-check to see if they actually exist (there's not much point doing much more work and then finding out the executable can't be used anyway)
• for sections that are in the file (e.g. ".text" but not ".bss") either load the section into the virtual address space or memory map that part of the file into the virtual address space; including respecting the section's attributes (e.g. setting page attributes to make sure a "read-only" section can't be written to, etc).
• for sections that aren't in the file (e.g. ".bss" but not ".text") either allocate RAM (and fill it with zero) or tell the virtual memory manager to do "allocation on demand" for that area; including respecting the section's attributes (e.g. setting page attributes to make sure a "no execute" section can't be executed, etc).
• map shared libraries into the virtual address space (which may include loading shared libraries into pages if no other process has already loaded them into pages).
• do dynamic linking
• pass control to the executable's entry point

I'd probably do the steps in the order above (e.g. starting from "determine if the executable is for your OS"); partly because what I'd learn doing the earlier steps would make it easier to figure out the later steps.


Cheers,

Brendan

[mariuszp]

In an ELF file, there is a program header which specifies an external program that is responsible for dynamic linking. In Linux it is usually /lib/ld-linux.so.2. And my main question is, how do I change that (preferably using a linker script)?

[jnc100]

You can specify the contents of the .interp section by passing the --dynamic-linker <linker_name> option to ld. Verify with readelf to see if it does what you want.

Regards,
John.

[gerryg400]

Now I want to actually port some software, which would obviously at least use the C library, and for that I'll need dynamic linking and relocation.

I don't believe this is true. Porting software does not require dynamic linking and relocation. I've ported cairo, freetype, gtk, libpng, pixman, zlib, libstdc++ and dash without yet implementing dynamic loading.

[mariuszp]

Hang on, when I compile a library I pass the -fPIC option to gcc. Doesn't that mean Place Independent Code? If so, does that mean I don't need relocation to load a library?

But either way, I need dynamic loading (otherwise, how would I load a dynamic library?). I managed to change the dynamic loader now, but I've ran into another problem - if I want to link (dynamically) with a library built for my system, I can't just pass an option like "-lmyos" to ld, because it seems to be searching my linux library paths. In this case, how do I make ld "pretend" that it found a library, such as "/lib/libmyos.so" and allow references to it? Or is it better to make an OS-specific toolchain and build using that? If so, does anyone know how that is done? (I know there's a wiki article about an OS-specific toolchain, but is the information it provides enough to build programs for my OS with shared libraries?)

[Owen]

GCC's -sysroot option.

By now you really should be using a cross compiler. When you're doing that, you can teach e.g. ld to write a custom interpreter specification to your output files by default

[linguofreak]

Hang on, when I compile a library I pass the -fPIC option to gcc. Doesn't that mean Place Independent Code? If so, does that mean I don't need relocation to load a library?

Yes. Position independent code can be executed at any address without relocation.

[NickJohnson]

Hang on, when I compile a library I pass the -fPIC option to gcc. Doesn't that mean Place Independent Code? If so, does that mean I don't need relocation to load a library?

Yes. Position independent code can be executed at any address without relocation.

Not quite, as I have found. You could in theory have a PIC binary generated by GCC that would not need relocations, because it uses IP-relative addressing for everything, but in practice GCC will always generate a GOT for PIC, even when the code does not reference anything external, which will need to be relocated for the code to work properly. This relocation is very easy, and will be in the writable section of the executable image, but it is definitely necessary.

[Owen]

Also, in some cases you can write code which cannot possibly be made PIC. For example,

Code: Select all

void* x = &y

where y is an external symbol cannot possibly be PIC (except via the use of an initializer), so will require a load time relocation

[bluemoon]

In an ELF file, there is a program header which specifies an external program that is responsible for dynamic linking. In Linux it is usually /lib/ld-linux.so.2. And my main question is, how do I change that (preferably using a linker script)?

ELF defines the loading procedure, it suggest, but it do not enforce, the existence of individual ld.so.
If you think using an external dynamic linker(ld.so) to do dynamic linking is a dead loop,
You may static link the code of (ld.so) into kernel, which is the code to handle address space, inject modules, and do relocation, and this make things a lot simpler for getting things started.

[mariuszp]

In an ELF file, there is a program header which specifies an external program that is responsible for dynamic linking. In Linux it is usually /lib/ld-linux.so.2. And my main question is, how do I change that (preferably using a linker script)?

ELF defines the loading procedure, it suggest, but it do not enforce, the existence of individual ld.so.
If you think using an external dynamic linker(ld.so) to do dynamic linking is a dead loop,
You may static link the code of (ld.so) into kernel, which is the code to handle address space, inject modules, and do relocation, and this make things a lot simpler for getting things started.

To be honest, it will probably by easier to have an external loader, because I am writing a microkernel (so I want to keep as many things as possible in userspace), and also if the loader is faulty, debugging it would be easier, as it is outside of the kernel.

However, do you think it's a better idea to write it from scratch, or port an existing one (if possible)?

[bluemoon]

I think it's easy to write a limited edition from scratch, since need a few features to get things working.
Then, in later version of the OS, review and consider to port / pull in more features covered by the full specification.

[dozniak]

Take a look at Android ld.so. It's fairly limited and might be a good starting point for figuring what needs to be done.