How to tell g++ to use only relative instructions?

[sebihepp]

Hello,

I currently think that besides my kernel every driver is loaded as a module by GRUB. Each Driver is a binary file and has a identifier as first dword, then 3 dwords which identitfy the type of driver (ATA, Filesystem, or so) and then a offset to a init routine. I can't tell where the driver would be in memory, so I need to make sure only relative instructions are used. How can I tell g++ and ld only to use rel. instr.? Or how do you solve that? I mean, you can't know the address of the module. And if you link the Module to a specified address, there is a problem if two or more modules overlap in memory.

best regards
sebihepp

[jal]

If you stick with the ELF format you create a shared library (Google for "how to create shared libraries" or the like, without the quotes). You cannot create a position independent flat binary I think, since the x86 doesn't natively support PIC, and therefore some tricks are necessary.


JAL

[Solar]

Check out -fpic. It requires some support from the surrounding system, as jal said.

Generally speaking, it's a bad idea to have "every" driver being a GRUB-loaded module, exactly because of the limitations of this approach. Rather have a single "boot support" module that gives the kernel access to the boot drive (ATA, SCSI, USB, ...), with all "boot support" module linked for a specific address - and then switch to "proper", kernel-controlled driver loading. At that point, you are in total control of relocation etc., and won't have to patch around what GCC does and GRUB expects.

[Kevin]

What exactly is wrong with doing proper driver loading right from the start? GRUB doesn't do anything sophisticated with multiboot modules. It just loads a file into memory without doing any parsing. You do with it whatever you like (most probably feeding it to your ELF loader) and free the memory afterwards. I can't see how this is much different from loading a file later (except for the fact that you already have it in memory and don't need to do the disk access yourself).

[Solar]

He still needs his code to be PIC to have his ELF loader relocate it, doesn't he?

What I meant is that having GRUB load "every" driver means he's either loading everything and the kitchen sink, or that the GRUB config must be adapted for every added / removed driver. Not nice.

[sebihepp]

The reason is, that I want most things outsourced in drivers. If my kernel have to load the drivers, there must be at least a ATA/ATAPI and Filesystem driver, but this is what I want in an externel module.

Thanks for the hint: position independent code (pic). I will search for more information on this.

But even if my kernel loads all drivers I have a problem. Lets say I have two drivers. Both have to be linked at an specific address if they are not independent. I cannot make sure, that the they overlap in the memory. Neither I know at compile time, how many drivers I will have nor do I know about the size. So I think this is a bad idea.
I wonder why Windows doesn't have any trouble with his dlls. All user dlls are compiled to the same address. But if I write a programm that's using 2 dlls there is no conflict. I really don't want to set up paging for each driver (I don't want paging at all) because I can't then just call a function via a interface class. I would have to update PageDir and/or PageTables, save every register...

[Kevin]

He still needs his code to be PIC to have his ELF loader relocate it, doesn't he?

What I meant is that having GRUB load "every" driver means he's either loading everything and the kitchen sink, or that the GRUB config must be adapted for every added / removed driver. Not nice.

Sure, I completely agree there. I was just confused about why you recommend bootstrapping with a special driver which is statically linked to a constant address if you need to do the real thing anyway and therefore could just use a "normal" driver.

[jal]

I wonder why Windows doesn't have any trouble with his dlls. All user dlls are compiled to the same address.

Because it does load-time relocation? Also, I remember in the early days (Win95), dll's were not all compiled to the same address, and you could get conflicts, which in worst case meant a DLL had to be loaded more than once (for different processes).


JAL

[sebihepp]

okay, I see. I have to use relocatable elf format.
well, then lets search around for information on relocatable elf and try to load them =)

[sebihepp]

Well, what is the difference between shared libraries and dynamic libraries?
Okay, I think shares libraries are loaded only once and all programmes can use them, while dynamic libs are loaded for each
programm. But what's the difference in exporting symbols and/or relocating?

So, if drivers were realised as shared libraries, every process could access them.
Therefore if I want a stage between drivers and processes, an interface for example, then I have to use
dynamic libraries?

[jal]

Well, what is the difference between shared libraries and dynamic libraries? (...) Therefore if I want a stage between drivers and processes, an interface for example, then I have to use dynamic libraries?

No. "shared libraries" or "shared objects" is a UNIX/Linux term, while "dynamic loadable libraries" (or DLLs) is a Windows term. They amount to the same thing.


JAL

[sebihepp]

Hey, I have another idea.

I use C++ and my kernel should be the interface every part of the os is working with. If I use an abstract Interface class for my kernel and for every driver I call main with the address of my kernel class, then there should be no relocations, because no data/code from outside is used.
And the v-table of the kernel class points to the right methods. I only have to search through the symbol table to find the main function and
eventually other ones.

[Solar]

Disclaimer: I am not *quite* sure I got your idea correctly, so I might be off here.

If I use an abstract Interface class for my kernel...

Yes...

...and for every driver I call main with the address of my kernel class...

I guess you mean: during driver initialization, your kernel calls the main() function of the driver, passing the address of the kernel interface class as parameter. (So that the driver could call back the kernel.)

Nice idea.

One note: Don't call it main(), call it driver_init() or init(). The function main() has a defined special meaning, and even if the environment makes it clear that your driver is not a hosted executable, your choice of function naming should also reflect this.

...then there should be no relocations, because no data/code from outside is used.

Wrong.

Your driver consists of class X and class Y. It has been compiled for address 0xdeadbeef, so class X sits at 0xdeadbeef and class Y sits at 0xdeadfeed.

Unless you load your kernel binary exactly to 0xdeadbeef, you have to do relocation, otherwise all references to class X and class Y will be off.

Same goes for any data object inside the driver binary.

[sebihepp]

@Solar:
Yes, you understood it. In future I call the driver init function init(), okay?

Wrong.

Your driver consists of class X and class Y. It has been compiled for address 0xdeadbeef, so class X sits at 0xdeadbeef and class Y sits at 0xdeadfeed.

Unless you load your kernel binary exactly to 0xdeadbeef, you have to do relocation, otherwise all references to class X and class Y will be off.

Same goes for any data object inside the driver binary.

Damn, that's not good. I thought class X and Y could be referenced with relative instructions. But if not, I have use either relocatable elf or set up an individual segment(I don't use paging) for each driver. the first is much work and the second is very slow. Oh god, why does elf not have a section with all Offsets in the File and I only have to go to these offsets and add the MemoryBase.
If I compile the driver with options -pic -fpic, which sections I have to parse? I know of .test, .bss, .rodata and .data. And I only want to do those relocations. I am not interessted in long names and so on.

[pcmattman]

Oh god, why does elf not have a section with all Offsets in the File and I only have to go to these offsets and add the MemoryBase.

Technically, it does. That's how one type of relocation works. Relocation can realistically be done in around a hundred lines of code. It's not that much work, and the end result is a massive gain. Also, there are relocations that need to be made anyway in applications that have already passed final link (eg, shared libraries or external variables) - you can't really escape

In our kernel we load kernel modules which are basically just ELF object files (linked with "-r"). They are relocated at runtime by the module loader, which finds space in the area of memory reserved for modules and then loads the module into that region. It performs relocation and linking against both the kernel symbols and the symbols of other modules that are present.

As for the entry point being called "main" or something... Each driver exposes 4 global variables which the module loader uses to determine:

1. Module name
2. Module dependency list (char** array)
3. Entry point
4. Exit point

This means we don't use the entry point in the ELF at all but rather the addresses and information presented as variables. An added bonus is that rather than just "main" the conventional function names are "entry" or "exit" (granted, exit is defined in POSIX, but we usually put the module name out the front anyway). Defining the variables is hidden behind MODULE_* macros.

We currently load all drivers at boot-time in an initrd (passed as a GRUB module), but we have written the support code and applications for dynamic module loading and unloading at runtime. This seems to also be one of your goals, and it is quite simple to achieve if you design your module loader properly.

I'm not sure about much of your kernel design, but maybe you could pick something up from our design and make it work for you?