OSDev.org

Hi!

Im writing an x86 OS in C and NASM. So far I have done the bare bones such as:
- GDT
- IDT
- Console output
- Reading scancodes from keyboard
- Physical memory manager - bitmap based
- Paging with kernel being relocated to 3GB mark
- Kernel Heap, linked list based

My plan is to create a modular kernel where only core bits like the ones I have so far + scheduling and going into
userspace will be in the core of the kernel and rest such as filesystem drivers etc. will be loaded as modules using grub.

I've been reading around the subject a lot but I have a vague idea on how to approach it.
What I don't know is where to actually start. I know the modules I'll write will be separate from the kernel so they
(or the kernel) won't have idea of eachother - for that I presume I'll need the kernel symbol table to somehow magically
link the two together once the kernel and modules are being loaded by grub.

I do know Im not being specific here but at this stage Im uncertain if the things I ask about are even making sense
to people that are experienced with this.

Could anyone explain this a bit more to me? Specifically if my assumptions on how this should be done are
remotely correct.

Thanks!

Your kernel has to include the code that will allow it to load additional modules. At least it will need read-only file system drivers to load other stuff, and default hardware drivers. It could contain full default drivers that you could later replace by secondary drivers to update additional or better functions at run time. If any secondary driver fails, you will always have the default built-in drivers to restore whatever failed.

You can have a header file with the definitions of ordinal function numbers. In the kernel, those ordinal numbers correspond to a function entry address, and that address can be associated with a function name and C prototype to be used in your external modules and programs. The external module or program will have an imports table, a table of address pointers, in a known location for some file format, or specified to the system via an INT service to fill the imports. When loaded, the system will just inspect the imports table (which can import functions, variables, or anything you can think of). The imports table can simply contain the ordinal numbers of the functions and maybe the name of the library, subsystem, etc., from where you want to import those functions. Then the system will simply replace each ordinal number of a function in the imports table by the actual address of the function or element you want to use. Now you can simply access those pointers in your program, which should be given the proper names of the external objects, as any other element pointer you have defined yourself. It will be good if you can export as many things from your kernel so that you can write external test code without polluting your kernel code base as the first means to do tests, and that will also cut on the times you will need to reboot your system just to add static code to your kernel.

The other thing you need is a way to test some sort of header signature and probably a footer one too. In this way, and later also using a hashing algorithm, you can know if you have loaded your program properly. If not, you can avoid crashing your system by not executing a bad code buffer.

You can later add relocations, but that would require you to properly interpret the specified elements and instructions to have their offsets relocated, almost as in a full emulator. But to begin with, you can use paging instead of relocation, and in 64-bit mode the code is supposed to be more position-independent than in 32-bit mode, although the labels of data elements will be fixed, specially external ones.


I think that you could integrate all of the functions you will always need in your kernel without converting them to modules. You can abbreviate the system even more if you make a fully privileged Ring 0 half and another user-space Ring 3 part for programs, in the same kernel image.

In this way you can map the privileged things from your kernel for your user-space kernel functions much more easily. Then you will have a kernel for the privileged system tasks, and a kernel for user mode in the same image, mapping the things you want to give access to the two levels.


You need at least access to 28-bit LBA ATA PIO read of single sectors and a LBA FAT32 driver to scan directories and read files for being able to load modules and programs, or floppy access at least.

So in the meantime you can add a directory for temporary functions in your kernel that you want to put in modules when you have full disk access. In this way you can keep adding the necessary functions and then reorder and redistribute them as you need later, without waiting to make applications and system functions that will allow testing and actually using your system model.

Right...

Looking at what you wrote I am starting to realise how little I know.
I don't have much experience with kernel development or development in general so its kind of hard
for me to grasp the concepts still.

So far my kernel runs on bochs, qemu etc. from an iso that grub-mkresque generates for me.
I haven't touched on disk access or how the kernel would eventually stay on the disk
and on each boot get loaded into memory and run from there.
Currently it is being run as a cd-rom image.
I think the bottleneck in my understanding is how the above would work - more specifically
how the kernel relocates its binaries to the disk and then runs from there?

The ideal scenrario would be:
- kernel somehow magically appears on a hard drive
- on boot, grub loads it from there into memory together with some modules
- kernel runs and loads modules
- party time!

The big question is: How do I get from a simple "kernel" that I have right now,
to something that the ideal scenario describes.

To be completely honest I might have the whole sequence of things wrong.
A really helpful answer would probably involve giving me a rough idea
on how you would go about writing a modular kernel.
Rough steps such as:
- ... have a bootable kernel
- ... paging
- ... scheduling
- ... writing a module loader
- ... writing some modules
- ...
- ... kernel appears on disk and boots from there
- ...
- ... kernel becomes skynet...

I think thats the best I can describe my thinking process.
It would really help me if you could give me such rough idea on how you think about things.

Thanks!

I have at least 1 test machine only intended to test my OS but also with another OS to actually develop and create a real file system test environment. If something gets corrupt, I will back up before any attempts to write the file system, but before that, there's nothing to worry.

Take a look at my kernel project. It never launched until I decided that I would use an existing partition to copy my OS to a root directory instead of making things too expensive trying to reserve a disk chunk for a system that still would have almost no functions:

http://devel.archefire.org/forum/viewto ... 4265&hl=en

I use a test machine with an OS in it, files, fragmentation... I have a backup of any new files I get in the test machine. I also develop mainly there. It allows me to develop an initial file system driver for the file system where the "normal" OS is installed. I just boot with a portable USB or floppy disk with which I load my system from a subdirectory in the root directory, for example from "C:\OS\".

The computer never gets altered, my OS simply runs as a portable program that needs to be booted as the kernel.

The easiest initial file system drivers are FAT12 in a floppy, and FAT32 LBA in an ATA disk plugged to a default ATA port.

For all the present partitions in a running system, there must only be 1 copy of my OS in only 1 root directory. Otherwise, we will have trouble determining which of the copies of our OS was chosen to run.

To load modules you must understand the object file format you're gonna be using, such as ELF. On my OS, modules are simply linked into a single relocatable ELF64 object, the kernel chooses a virtual memory location to load it into, performs relocations and calls an initialisation function. It can then remove the module by calling an exit function and then removing it from memory.

Okay that makes sense.

~ wrote:You can have a header file with the definitions of ordinal function numbers. In the kernel, those ordinal numbers correspond to a function entry address, and that address can be associated with a function name and C prototype to be used in your external modules and programs.

Or, for a more type-safe implementation that ends up being more-or-less the same thing under the hood; just use a "struct" that contains a list of named function pointers. Then pass the address of an initialsed struct to the module somehow (I just pass it as a parameter to the module's entrypoint).

Calls just end up as "table->function(params)"; I've even gone so far as to write a header file containing inline "stubs" so that module souce can just say "function(params)", provided they strore a the table pointer in a specifically named global variable.