Modular Kernel - struggling on compiler and ELF.

[ostylk]

Hello,
So I spent a lot of time designing my OS and thinking about how I could implement various things.

I've came up with a modular design as described in the wiki. Some important and necessary modules for the boot process are going to be loaded by an initrd.
So my requirements are:

• Kernel procedures should be accessible by the module. It should also be compatible if the symbols inside the kernel change locations so the module doesn't crash(happens when recompiling the kernel but not the modules?). I thought about dynamic linking at runtime?(then I need to somehow export symbol information into my kernel executable so I can use these when loading modules). Also I want to avoid passing a crazy "kernel procedures" structure table to the module(I think it's hacky and "unclean")
• Modules should be relocatable so I can load them wherever I want to. After reading about ELF and ELF relocation I'm a bit confused because I only want to change the program's base address and not the offsets between sections. So why are there so many relocation types and do I need to implement all?
• Also modules should be libraries which you can access by another module. So in addition to the above relocation I need a "shared relocatable library".

All modules are running in the same address space(kernel is mapped to 0xC0000000 and all modules will be after 0xE0000000).

I'm asking for some opinions if this is a feasible way and how to create the executable the way I've described(compiler flags etc.?). Is there even something like a "shared relocatable library" and if it exist how do I instruct my compiler to create such. How can I relocate ELF executables to a specific address at runtime(address is also determined at runtime). Point me into the right direction(links, wiki pages etc.)

I hope I've described my problems in a way you'll be able to understand cause English is not my native language.

Greetings,
ostylk.

[Ycep]

Wrong forum. You probably need "OS design & theory". http://forum.osdev.org/viewforum.php?f=15.
Every second person on this website english isn't it's native language.
If you really want to make modular kernel then you should firstly create storage driver, then floppy driver, then ELF parser and loader, then you should implement user mode and if you didn't already you need to create HAL.
Modular kernels aren't good start for you. Better first start with most simplest ones.
Most compiler flags depend on your OS archiceture. They also depend on which compiler you use.
0xC0000000 - it's somewhere at 3.2GB, and you definitely can't place your kernel there. 32-bit archiceture don't support this much RAM. For start just place it on low address.
0xE0000000 - also incredibly big. I think that 4MB should be enough for modules.
In my opinion kernel should be library, modules should be drivers and similar for kernel and shell should be program.

[Techel]

After you loaded all module images to memory and provisionally set up a physical memory, virtual memory and exception managenent, you can start parsing them and insert them into the virtual address space. In case of elf you might want a relocatable object file or shared library. When loading a module, put it's sections to suitable places, link it with other modules (but don't link other ones with it) and call it's entry point. If some symbold could not be resolved, proceed with others. Now it's safe to link others with this module.
The bootloader may also expose temporary functions which get linked, like retrieving the memory map or page directory of the provisionally set up memory managenent.
Note there's no 'kernel' module - the entire thing is the kernel.

[ostylk]

Wrong forum. You probably need "OS design & theory". http://forum.osdev.org/viewforum.php?f=15.

Well. I think that's not completly true. It says in the description of this forum.

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.

And I'm asking for the best way to implement something. But nevermind I think it's hard to tell where to post.

0xC0000000 - it's somewhere at 3.2GB, and you definitely can't place your kernel there. 32-bit archiceture don't support this much RAM. For start just place it on low address.
0xE0000000 - also incredibly big. I think that 4MB should be enough for modules.

Correct me if I'm wrong but isn't the whole 4GB address space available when paging is enabled?(The physical memory doesn't need to be 4GB large because pages can be swapped out) Also I want to keep my kernel at 0xC0000000. If I want to change it I'll simply change 2 values in my source so it actually doesn't matter.

After you loaded all module images to memory and provisionally set up a physical memory, virtual memory and exception managenent, you can start parsing them and insert them into the virtual address space. In case of elf you might want a relocatable object file or shared library. When loading a module, put it's sections to suitable places, link it with other modules (but don't link other ones with it) and call it's entry point. If some symbold could not be resolved, proceed with others. Now it's safe to link others with this module.
The bootloader may also expose temporary functions which get linked, like retrieving the memory map or page directory of the provisionally set up memory managenent.
Note there's no 'kernel' module - the entire thing is the kernel.

Thank you. That sounds good.
I'll probably go with compiling every module to a shared library and dynamically link other dependencies onto it.
I've summed it up to make sure I've understood this.
Loading a module:

• Read the program headers and place the section at the right virtual address(also with an offset for relocation)(modules are built with base address to 0x0)
• Go through relocation entries to relocate binary(REL, RELA sections)
• Resolve dependencies against kernel(dynamic linker)
• Resolve dependencies against other modules(error if a symbol isn't found)
• Jump to the entry point(init()-function)

I'm now going to try to implement that feature.

[onlyonemac]

Correct me if I'm wrong but isn't the whole 4GB address space available when paging is enabled?(The physical memory doesn't need to be 4GB large because pages can be swapped out) Also I want to keep my kernel at 0xC0000000. If I want to change it I'll simply change 2 values in my source so it actually doesn't matter.

Yes, you get 4GB of address space with paging enabled. So there's no reason why you can't map your kernel at that address, but in physical memory it will almost certainly need to be somewhere else.

Currently I'm putting the kernel below the 1MB boundary and using the memory from 16MB upwards for everything else, although this may need to change when my kernel gets bigger.