My project and the pros and cons of a modular kernel design

[RadikalRane]

Before I'll give you all an explanation of my new project I'll give you a little background information about myself (as this is my first post).

I'm an engineering student with a specialization in computer sciences, and I've been interested in OS design for a while now.
I've previously done 3 other attempts at building an OS, the first one failing horribly, the second one getting to the point where I had a memory manager, a VFS and something resembling multitasking, and the third one getting to the point where I had a simple console which accepted a couple of standard commands.
I've been following these forums for a while now and have gained a lot of knowledge by just reading through the wiki and lots of threads.
I learned C by messing around with different linux distributions since I was only 12 years old, and I have a basic knowledge of ASM. (I do have a lot of experience in a couple of high-level languages as well).

The project:
At the beginning of this academic year I had the idea of starting a new OS project after a taking a break from OS-deving for a couple of months to focus on doing some small projects in C# (my "weapon" of choice :p), but I didn't want to mindlessly start coding like I did in my previous projects, so I started planning. At first I hadn't a clue where to begin, because I really didn't know what I wanted to achieve with this project, but that all changed about 2 weeks ago.

I wrote out a plan for a completely modular kernel with a solid and clean kernel core which I could easily build modules for (I've done some high-level modular projects before) so I could actually experiment with different approaches and models for different functions without having to edit the main kernel code when I would want to change a certain function.

I think I covered most of the required functions for a stable modular kernel in my planning, but I'm still doing some research before I actually start writing the code since I really don't want to start over in the middle of my project when I suddenly realize I made an error in my planning.

This is the reason why I decided to make this thread, as I still have some questions about modular kernels in general

The Pros and Cons of a modular kernel approach
I know that to have an optimal modular kernel you need to have a solid kernel core with a solid and efficient system for loading and managing your modules. I'm not interested in building a system able of hot-swapping modules (except maybe for driver modules) as I can see some serious complications with this approach, but I just want to build a kernel core which I can experiment with and play with (that's why I call this project "Sandbox").

Now I wanted to ask you if there are any complications with modular kernels I should keep in mind performance-wise or speed-wise (as we live in a time where everyone wants an instant-on machine) and what I could do to prevent these complications or how I could work around them.

I plan on taking my time for this project, as I'm not in a hurry to get this completed, and I have good prospect for this project as it seems that every project I do improves drastically compared to its predecessor.

If you kept on reading this post until this point, I'd like to thank you (:p) and I hope I can get some good feedback from this.

[NickJohnson]

Welcome to the forums!

It seems like you have a good idea of where you're going, and how to get there.

I know that to have an optimal modular kernel you need to have a solid kernel core with a solid and efficient system for loading and managing your modules. I'm not interested in building a system able of hot-swapping modules (except maybe for driver modules) as I can see some serious complications with this approach, but I just want to build a kernel core which I can experiment with and play with (that's why I call this project "Sandbox").

Although the module loader is a key part of a moduar monolithic kernel, its speed is really not important at all. If everything ends up in kernelspace, you will get effectively no overhead in using the modules once they are loaded, and even the most inefficient design shouldn't even take a noticeable amount of time to work. The speed of a modular kernel comes from the code that makes it up, not the module loader. You should focus on memory management and task switching if you wanted more speed. Still, when in doubt, use brute force, especially when the goal is reliability.

You can also just design the kernel in a modular way, but have the resulting binary be one big blob, if you don't care about module loading at runtime, which requires a working userspace system to be of much use. Hot-swapping modules is really the same as loading and registering them too, although both are somewhat complex and require relocation.

I really don't want to start over in the middle of my project when I suddenly realize I made an error in my planning.

I really doubt that even if you go off with a bad design, you will have to throw away much code. Early on in my system's development, I "started over" many times because I thought the design I really don't want to start over in the middle of my project when I suddenly realize I made an error in my planning.was bad. In reality, 80% of the code was the same from attempt to attempt, just with different organization. My best advice would be to keep things short and simple, so you can refactor easily instead of starting over.

Also, if you want an even greater degree of modularity, you may want to look at a microkernel design. It takes longer to get to a "usable" result, but the interfaces are much cleaner and the code generally much shorter. For example, my kernel is only 1500 lines of C, and has a built in scheduler and memory manager, and can support a full multitasking userspace environment with drivers and IPC, yet it took over six months to get it working right, and there are no drivers.

[RadikalRane]

You can also just design the kernel in a modular way, but have the resulting binary be one big blob, if you don't care about module loading at runtime, which requires a working userspace system to be of much use. Hot-swapping modules is really the same as loading and registering them too, although both are somewhat complex and require relocation.

I think I want to have the option to load modules at runtime availabe as it could come in handy with loading drivers and other components, and it could result in a faster boot (if I would choose to build a complete OS ofcourse).

Also, if you want an even greater degree of modularity, you may want to look at a microkernel design. It takes longer to get to a "usable" result, but the interfaces are much cleaner and the code generally much shorter. For example, my kernel is only 1500 lines of C, and has a built in scheduler and memory manager, and can support a full multitasking userspace environment with drivers and IPC, yet it took over six months to get it working right, and there are no drivers.

I might have a look at a possible microkernel design, I have enough time available to code this thing and if it would better suit my needs it'd be a big advantage in general.

[Brendan]

Hi,

Now I wanted to ask you if there are any complications with modular kernels I should keep in mind performance-wise or speed-wise (as we live in a time where everyone wants an instant-on machine) and what I could do to prevent these complications or how I could work around them.

For speed, modularity shouldn't cost very much (but probably will cost an extremely tiny amount of extra overhead, due to restricting the compiler's optimiser, and use of dynamic linking and/or function pointers).

IMHO there's no other sane choice though. A non-modular monolithic kernel must either contain every possible driver (huge amount of bloat) or must be compiled specifically for the target machine (which kills "user-friendliness" for most end users).

The other benefit is that it forces you to write the code in a way that's easier to maintain - no hidden dependencies between pieces of code, and much easier to see the boundaries when you want to rewrite a piece.

For "instant on", modularity won't really make much difference. The key to "instant on" is lazy coding combined with parallel initialisation. For e.g. only mount a file system the first time that software tries to access a file on that file system, and initialise the hard disk controller when the file system is mounted (rather than during boot) and then only detect the disk drive being mounted (rather than all disk drives attached to that controller), and make sure you can do this while the OS is also initialising anything else (and that devices, etc can be initialised in any order). On top of that you can also pre-load some things that definitely will be needed during boot - e.g. when the boot loader loads the kernel into RAM it could also load something like a RAM disk image, so that modules and other files can be used before the kernel has initialised the hard disk/s or mounted any partitions. For example, you could pre-load the video, keyboard and mouse drivers and the GUI's login screen, and initialise hard drives, etc while the user is logging in.

Of course after the necessary stuff is started and the OS has nothing better to do, it can start initialising anything that was postponed (e.g. mounting file systems that haven't already been mounted, etc).

OSs like Windows and Linux seem to take ages to boot because they initialise all hardware during boot, regardless of if/when that hardware is used (and it's not limited to just hardware/devices - things like file system code, DHCP/networking and other "important but not necessarily needed" software too).

I'd also point out that it's easy to measure speed, but it can be extremely hard to measure other important factors (e.g. security, reliability, fault resistance and fault tolerance, a kernel's ability to detect bugs in other software, how easy it is to write/debug device drivers, how "feature rich" a kernel is, etc). For example, it's easy to do some measurements and say "this code took 123 ms to execute" but impossible to do some measurements and say "this code has a reliability rating of 123 thingies". Because of this, speed is typically over-emphasised and everything else it typically under-emphasised. In reality most computers spend 99% of their time doing nothing anyway, so it can make sense to sacrifice some speed for other less easily measured advantages; and a slower OS (with lots of other advantages) can easily be better than a faster OS (with no other advantages). Of course I probably should mention that everyone who knows me realises that I'm biased in favour of micro-kernels....


Cheers,

Brendan

[octavio]

Brendan wrote:

On top of that you can also pre-load some things that definitely will be needed during boot - e.g. when the boot loader loads the kernel into RAM it could also load something like a RAM disk image, so that modules and other files can be used before the kernel has initialised the hard disk/s or mounted any partitions. For example, you could pre-load the video, keyboard and mouse drivers and the GUI's login screen, and initialise hard drives, etc while the user is logging in.

OSs like Windows and Linux seem to take ages to boot because they initialise all hardware during boot, regardless of if/when that hardware is used (and it's not limited to just hardware/devices - things like file system code, DHCP/networking and other "important but not necessarily needed" software too).

Usually i agree 100% with Brendam so no need to post anything else.But this is not the case.
In general hardware initialization can be done very fast ,some mechanical (and slow) devices become ready before the Os starts, so for my OS (octaos) on my computer (acer aspire one) 0.5 seconds is enough to setup all the supported hardware and mounting various filesystems.Most of this time is just because the mouse setup takes 0.4 seconds.
If some Os take a lot of time to boot is because the software is bloated.
Instead of loading a ram disk image,is better to initialize the disks and filesystems and then load the optional modules like a gui or a keyboard driver if they are needed.A server always requires a filesystem but not a GUI.
About the first post: Modern Oses are modular .But modular kernel is not the same that microkernel.Microkernel also has memory protection betwen modules ,it is more secure but makes things dificult thats why many Oses are modular but monolitic,wich means that the same memory space is used by all modules.

[Brendan]

Hi,

Usually i agree 100% with Brendam so no need to post anything else.But this is not the case.
In general hardware initialization can be done very fast ,some mechanical (and slow) devices become ready before the Os starts, so for my OS (octaos) on my computer (acer aspire one) 0.5 seconds is enough to setup all the supported hardware and mounting various filesystems.Most of this time is just because the mouse setup takes 0.4 seconds.

The annoying thing about electric motors is that they can consume large amounts of electricity when they're first turned on ("inrush current"). Now, consider a SCSI array with a large number of disk drives. First you need to initialise the controller, then each hard drive needs to be started one at a time because starting all of them at once puts far too much strain on the power supply. Even when no disks are attached you need to check each possible device and wait for a response with a time-out. The end result? A RAID controller that supports 16 devices, can takes 2 seconds per potential device, or 32 seconds to initialise it properly. PATA/SATA/ATAPI is faster (and there's usually a lot less drives, so no need for "one at a time" device spin-up) but you get similar problems.

Initialising network devices can take a little while too. I boot using PXE a lot, and it takes most computers a few seconds just to get a DHCP packet (after the ethernet card itself is initialised); even though nothing else is running at the time, and even for fast computers with gigabit ethernet talking to an over-powered (but mostly idle) server.

Even something like initialising the PS/2 controller (before detecting and initialising the devices attached to it - keyboard & mouse) should take about 500 ms. There's similar small delays for lots of other things too (serial, parallel, FDC, etc); and all these delays add up.

In my experience, OSs that boot very fast are OSs that make the mistake of assuming that the firmware (BIOS) setup everything correctly for them and/or they're OS that don't have any/many device drivers to begin with. Basically they boot fast because they don't actually do anything properly.

I haven't seen your OS, so I can't comment on how thorough the hardware initialisation is. I did download your "CD.ISO" and try it in Bochs, but unfortunately the only thing it does is triple fault:

Code: Select all

00123102390p[CPU0 ] >>PANIC<< exception(): 3rd (12) exception with no resolution
00123102390i[CPU0 ] CPU is in real mode (active)
00123102390i[CPU0 ] CS.d_b = 16 bit
00123102390i[CPU0 ] SS.d_b = 16 bit
00123102390i[CPU0 ] EFER   = 0x00000000
00123102390i[CPU0 ] | RAX=0000000000000008  RBX=0000000000000000
00123102390i[CPU0 ] | RCX=0000000000000000  RDX=00000000000001f0
00123102390i[CPU0 ] | RSP=0000000000000001  RBP=0000000000000001
00123102390i[CPU0 ] | RSI=00000000ffff0000  RDI=00000000000853f8
00123102390i[CPU0 ] |  R8=0000000000000000   R9=0000000000000000
00123102390i[CPU0 ] | R10=0000000000000000  R11=0000000000000000
00123102390i[CPU0 ] | R12=0000000000000000  R13=0000000000000000
00123102390i[CPU0 ] | R14=0000000000000000  R15=0000000000000000
00123102390i[CPU0 ] | IOPL=0 id vip vif ac vm RF nt of df IF tf sf zf af pf cf
00123102390i[CPU0 ] | SEG selector     base    limit G D
00123102390i[CPU0 ] | SEG sltr(index|ti|rpl)     base    limit G D
00123102390i[CPU0 ] |  CS:f000( 0004| 0|  0) 000f0000 0000ffff 0 0
00123102390i[CPU0 ] |  DS:0000( 0005| 0|  0) 00000000 0000ffff 0 0
00123102390i[CPU0 ] |  SS:0000( 0005| 0|  0) 00000000 0000ffff 0 0
00123102390i[CPU0 ] |  ES:0800( 0005| 0|  0) 00008000 0000ffff 0 0
00123102390i[CPU0 ] |  FS:0000( 0005| 0|  0) 00000000 0000ffff 0 0
00123102390i[CPU0 ] |  GS:0000( 0005| 0|  0) 00000000 0000ffff 0 0
00123102390i[CPU0 ] |  MSR_FS_BASE:0000000000000000
00123102390i[CPU0 ] |  MSR_GS_BASE:0000000000000000
00123102390i[CPU0 ] | RIP=0000000000000635 (0000000000000635)
00123102390i[CPU0 ] | CR0=0x60000010 CR2=0x0000000000000000
00123102390i[CPU0 ] | CR3=0x00000000 CR4=0x00000000
00123102390i[CPU0 ] 0x0000000000000635>> push ax : 50

Cheers,

Brendan

[Owen]

Initialising network devices can take a little while too. I boot using PXE a lot, and it takes most computers a few seconds just to get a DHCP packet (after the ethernet card itself is initialised); even though nothing else is running at the time, and even for fast computers with gigabit ethernet talking to an over-powered (but mostly idle) server.

I'm reminded of Ubuntu trying to mount my network drive. Before it initialized networking.

With a 120 second timeout.

Damn, that was annoying!

(They fixed it in 9.10 it seems )

[octavio]

In my experience, OSs that boot very fast are OSs that make the mistake of assuming that the firmware (BIOS) setup everything correctly for them and/or they're OS that don't have any/many device drivers to begin with. Basically they boot fast because they don't actually do anything properly..

If even manufacturers don't setup their hardware correctly, don´t expect that a hobby Os will do.
So the linux linpus lite i'm using now must be doing everything wrong since it starts in about 10 seconds instead of 1 minute for ubuntu.
I don't use emulators or 64 bit machines,and Octaos usually fails on half the machines i try it the first time.But works well on all my computers after i correct the bugs.
Actually motors have electronic controllers that can limit the maximum power of the motor ,but i agree that in general scsi is a ****.

[NickJohnson]

If even manufacturers don't setup their hardware correctly, don´t expect that a hobby Os will do.

But frequently, the devices don't set themselves up (for speed reasons), and require the OS drivers to do it instead. To have written a correct driver for them, you have to have initialize those devices. Brendan's point was that most OSes start up fast not because they are written well, but instead because they have few drivers or incorrectly written drivers.

So the linux linpus lite i'm using now must be doing everything wrong since it starts in about 10 seconds instead of 1 minute for ubuntu.

That's because it has fewer drivers to start, and probably relies less on services like hald and dbus, which take some tome to start up. Also, you have to take into account disk speed and partition position - initialization is much more I/O than computation. The init system on Ubuntu is probably almost entirely to blame here - rearranging the rc.conf or equivalent will probably make things much more even in speed, but remove functionality. That's why there are gazillions of linux distros out there, that just have different things installed and initialized by default.


EDIT: Brendan changed/corrected the name on the second quote.. [/EDIT]

[Brendan]

Hi,

If even manufacturers don't setup their hardware correctly, don´t expect that a hobby Os will do.

But frequently, the devices don't set themselves up (for speed reasons), and require the OS drivers to do it instead.

From my perspective there's lots of problems:

• devices that no BIOS ever configures properly (sound cards, RDTSC/local APIC timer calibration, second video card, AGP speed on the first video card, additional CPUs).
• devices that BIOSs are meant to configure "wrong" for backward compatibility (e.g. A20 gate, PIC, #FERR)
• devices that some BIOSs do configure properly but some don't (e.g. PCI USB controllers in old computers that came with PCI slots but without USB support; older BIOSs that don't support modern/huge hard drives, etc)
• BIOSs that are just plain buggy
• Devices that have bad ROM software (e.g. inadequate/inefficient initialisation done by SCSI controller ROMs)
• Devices that have faults (hopefully the least common problem, but things like disk drives do fail, and if you've never spilled some sort of beverage into your keyboard or seen a computer full of dust then...)

Of course NickJohnson is also right - there's good reasons to avoid initialising some devices before the OS starts (mostly performance). Again, SCSI is a good example here - some (most?) decent SCSI/RAID controllers have a configuration option to only initialise some of the devices, so that the computer can boot faster (and initialise any remaining devices later).

Then there's things like "PS/2 emulation for USB keyboard/mouse", where an OS has to do a PCI bus scan, initialise the USB controllers, do a scan of USB devices and disable any "PS/2 emulation" before it can detect if there actually is a PS/2 keyboard or mouse, or not.

Now think about this: '...for my OS (octaos) on my computer (acer aspire one) 0.5 seconds is enough to setup all the supported hardware and mounting various filesystems.Most of this time is just because the mouse setup takes 0.4 seconds.". To me that means you detect and initialise all hardware except the mouse in 100 ms.

You (octavio) are right in one way - it's silly to expect (some/most?) hobby OSs to do full hardware initialisation (and do it properly). But in the same way it's also silly to compare these hobby OS/s to something like Linux/*BSD/Windows/Solaris that isn't intended to be a hobby OS, and then claim that all of these OSs are "slow and bloated" because they don't boot in 500 ms. Even if your specific Acer Aspire One is using SSD instead of a hard drive, it's extremely hard to believe that it's possible to detect and initialise all hardware in that amount of time, even with "parallel initialisation".

For a "reduce feature, non-hobby" (or "Lite") OS, assuming you're booting from SSD, I can believe around 5 seconds on that sort of hardware (with "fully parallel initialisation"). To get any faster than that (while still doing proper hardware initialisation) you'd need to start looking at some extreme options, like getting rid of the BIOS and installing the OS directly in ROM (so you can do more initialisation in parallel and/or postpone more of it until after the OS is usable; including postponing most of the POST memory test).


Cheers,

Brendan

[octavio]

when i say that the OS boots in 0.5 seconds i do not include the bios setup (7 seconds on this machine) or external devices like a usb keyboard (2 seconds but the system is already usable with the built in keyboard).I do parallel initialisation so the init time depends on the slowest device,that on this machine is the mousepad. Octaos loads quickly because is 1000 times smaller than linux,but i think that linux if optimized can be 10 (or more) times smaller and faster.

[NickJohnson]

Octaos loads quickly because is 1000 times smaller than linux,but i think that linux if optimized can be 10 (or more) times smaller and faster.

Size != speed; frequently, the opposite is true. The linux source tree is almost entirely device driver code (~80% iirc), most of which is not used on a single machine. Fancy algorithms are more scalable, but take more code. Non-fancy ones, like those used in hobby OSes, are faster for small inputs and take less code. Also, speed depends most on a few internal loops, not on the size of the whole codebase. And initialization speed is far from everything, especially on an OS like linux, which is used on many many servers, where uptimes are in months or years.