Best way to boot a microkernel?

[NickJohnson]

I now have the majority of a working kernel for my operating system, but I've come across an area that I hadn't put much thought into, namely booting all of my driver processes. My design is in part a microkernel, where all I/O related systems (vfs included) are run as very privileged userspace processes. Tasking and memory managment are in the kernel however. I have a working initrd based on the tar format, as well as a non-linking ELF loader. The issue is that I want to be able to have reasonable device detection, but I don't want to put device-specific information into the kernel.

I have a few ideas on a method to get everything started, but I want to know which would be the best and easiest to implement.

1. Load an "init" process, which then spawns drivers based on device detection. However, this process must have drivers to access the disk and filesystem.

2. Load an "init" process, which contains all of the driver logic. It then forks into the basic drivers as needed. The downside is all drivers needed for booting must be compiled into init.

3. Load a set of generic driver processes for each possible detected device, which then either adapt or are removed based on the actual devices' requirements.

4. Load all included drivers and let them figure out which devices to map themselves to. This would cause a lot of duplication of process in the drivers, however.

Are there any other methods? And how would I start making one of these work?

[Troy Martin]

1. Load an "init" process, which then spawns drivers based on device detection. However, this process must have drivers to access the disk and filesystem.

Interesting, but might be difficult to support strange devices.

2. Load an "init" process, which contains all of the driver logic. It then forks into the basic drivers as needed. The downside is all drivers needed for booting must be compiled into init.

Isn't this sort of monolithic?

3. Load a set of generic driver processes for each possible detected device, which then either adapt or are removed based on the actual devices' requirements.

I personally think that this would take a LONG time to write and use at runtime.

4. Load all included drivers and let them figure out which devices to map themselves to. This would cause a lot of duplication of process in the drivers, however.

Smart plan, not sure about the memory usage though.

[neon]

1. Load an "init" process, which then spawns drivers based on device detection. However, this process must have drivers to access the disk and filesystem.

I follow this method in my system. However, all boot class drivers and associated kernel mode dynamic libraries are pre-loaded at system startup by my OS loading program. Thus there is no need for minidrivers for the process. The OS loading program for my system itself doesnt have any drivers - it uses the bios through the systems boot library. This completely eliminates the need for additional loading software as the kernel has preloaded boot class drivers for disk and file system access.

Hope this gives you another idea to look into...

[xenos]

My microkernel is loaded by GRUB, which also loads a few drivers, like the disk driver and filesystem drivers. The init process looks for these drivers, starts them and uses them to load other drivers as they are needed. This approach is quite flexible since all pre-loaded drivers are specified in GRUB's boot menu, so it can easily be adapted to floppy / CD / hard disk boot and different filesystems.

[NickJohnson]

Well, I do have the resources already to load a set of drivers from the initrd by forking from idle. I could load one driver for the disk and one for the terminal along with init, and let init do the rest (without a vfs). How generic can IDE or floppy drivers be? Would I be able to write one simple driver to handle all different boot devices, or would the user have to intelligently choose proper drivers to include in the initrd?

[Love4Boobies]

Why not make your boot loader do most of the work? What I do is make my custom boot loader load a process which detects which devices are present and loads the appropriate drivers. The boot loader also loads the kernel and when the kernel starts, the the other process becomes a daemon which I find useful for situations like hot-swapping or using the USB when you'll need device detection while the OS is running.

[Brendan]

Hi,

Would I be able to write one simple driver to handle all different boot devices, or would the user have to intelligently choose proper drivers to include in the initrd?

You would be able to write one driver that handles all different boot devices, but it'd be like half a monolithic kernel.

The user shouldn't need to intelligently choose the proper drivers to include in the initrd - you could have (for e.g.) a generic boot CD for installing the OS, where the initrd includes all the drivers you might need to access a CD (ATA/SATA controllers, USB controllers, SCSI controllers, plus any CD drives). Then once the OS has booted from this CD (and after device detection, etc has been done) the user only has to tell the OS which device to install the OS on, and the OS should be able to figure out which device drivers it would need to include in the initrd for the boot device. For example, if the user wants to install the OS on a SCSI hard disk, then the OS installer would create an initrd that includes the right SCSI controller driver, the right hard disk driver and the right file system driver, and then the installer would put that initrd on the hard drive.

However, this means you need to be able to start device drivers in any order - for example, start whatever happens to be in initrd first, then start whatever is left over later.

Because of this (and for other reasons) I'm entirely in favour of the "parallel pull" model, where (during boot) any pieces of code that can be started are started as soon as possible.

For example, during boot you scan the PCI bus and find out you need a driver for a SCSI controller, a driver for a USB controller and a driver for a video card, so you try to start them immediately. If initrd only includes a driver for the SCSI controller then you can't start the driver for the USB controller or the video card. When the SCSI controller driver is ready it finds out what devices are connected to it, and arranges for device drivers for those devices to be started. If there's a hard disk and a CD-ROM connected to the SCSI controller, then the device driver for the SCSI controller would try to start a SCSI hard disk driver and a SCSI CD-ROM driver, and if initrd only includes a SCSI hard disk driver then the SCSI CD-ROM driver won't be started. When the SCSI hard disk driver is ready it'll arrange for partitions to be discovered. If the code that discovers partitions detects a FAT file system and an ext3 file system, then it'll try to start appropriate file system drivers. If initrd doesn't include a FAT file system driver then only the ext3 file system driver will be started. Once the ext3 file system driver is started it'll tell the VFS it's ready, and the VFS will tell everything else that a new file system has become available, and everything else will try to start anything it couldn't start before (the PCI bus scanning code will try to start the USB controller driver and the video card driver again, the SCSI controller will try to start the SCSI CD-ROM driver again, and the code that discovers partitions will try to start a FAT file system driver again).

Also, you might want to put extra stuff in initrd to improve boot times. For example, if initrd included a video driver and a keyboard driver then maybe the user will be able to start logging in before any of the file systems are ready to use; and maybe it'd be a good idea to put ethernet card driver/s and DHCP in initrd too.


Cheers,

Brendan

[RevivalDBM]

Although I only recently finished my microkernel, and am currently working on the basic servers, I will be including simply a few drivers in my initrd, just enough to read my install CD and perform the installation. The actual installation will have an initrd created during installation with it's basic drivers included. Those basic drivers are enough to load the rest.

[NickJohnson]

Well, more of what I was asking is: is it possible to write a truly "generic" floppy/ATA/USB driver, i.e. one that doesn't even need to know what model of device it is interfacing with, so that I can write something that needs minimal device detection? Because the feasibility of that would be the difference between having init spawn everything and the kernel spawn everything. It would be easier for me, assuming you can make generic drivers like this, to have init use it's own drivers, then replace them after things are set up right.

I also have nothing against making things work kind of like a monolithic kernel during bootup. I'm trying not to classify my design in the sort of black and white monolithic / microkernel debate. I don't make any sort of decisions about my design just because I dislike the other design, because following anything blindly is a *very* bad idea.

[Brendan]

Hi,

Well, more of what I was asking is: is it possible to write a truly "generic" floppy/ATA/USB driver, i.e. one that doesn't even need to know what model of device it is interfacing with, so that I can write something that needs minimal device detection? Because the feasibility of that would be the difference between having init spawn everything and the kernel spawn everything. It would be easier for me, assuming you can make generic drivers like this, to have init use it's own drivers, then replace them after things are set up right.

Floppy drive controllers and floppy drives are mostly standard, especially if you limit it to the first floppy drive controller (so you don't need to guess I/O ports, etc). ATA is fairly standard too if you don't use things like bus mastering to speed it up, but you would need to determine if you need to use CHS or LBA or LBA48, and have a hard disk driver and a CD-ROM driver (ATAPI). AFAIK SATA emulates ATA so you shouldn't need anything extra.

For USB you'd need to scan the PCI bus to find any USB controllers. Then you'd need 3 drivers for the USB controller/s (EHCI, OHCI and UHCI). After that you'd need to find out which devices are connected to the USB controller/s. From here I think a generic "storage device driver" should cover most things.

For SCSI there's no standard for the controller itself and every SCSI controller needs a different driver (which is why SCSI controllers have their own ROM, so the BIOS can use them). After you've done the PCI bus scan and found the controllers you'd need to start the right driver for the controller, but after that I think the SCSI devices (hard disks and CD-ROMs) are mostly standard (I'd expect you'd need something like LBA or LBA48 detection here too though).

There's no standard for network cards either (PCI bus scan plus a different driver for each type of card); but once you've got basic packet drivers done the protocols (e.g. DHCP, and TFTP or FTP or even HTTP) are standard.

Also, if you support older computers (e.g. without PCI), you'd need code to probe for ISA SCSI and ISA network cards.

With all of the above, you'd be able to do what the BIOS could have already done for you, and possibly a little more (as not all network cards support PXE). However, you'd be looking at writing around 100 device drivers plus some other code (e.g. for PCI scan, etc); and it's extremely unlikely that any of these device drivers will be useful after the OS has booted. For example, you'd end up with a device driver for a specific SCSI controller used during boot (with full access to hardware and no bus mastering, etc) and another device driver for the same SCSI controller that's used after boot (which allocates/frees memory using the OS's APIs, relies on messaging, etc). Only a small part of the code could be reused (e.g. store the results of your PCI scan somewhere so that it doesn't need to be done a second time).

One problem is that all those drivers, etc would probably add up to several MiB of code and data; so it won't fit on a floppy. You could get around that by not including most of the other drivers when you are booting from floppy. Another problem is that most of this code and data is only there in case it's needed, and most of it won't actually be used. This will slow down boot times a little, but probably more importantly it'll cost you several MiB of RAM, which may mean your OS is unable to boot on older machines (e.g. with 8 MiB of RAM). Then there's the ironic part of it - you'd be loading large amount of code and data that you use to be able to load a smaller amount of code and data (for e.g. load 3 MiB of stuff early during boot so that you can load 1 MiB of stuff later during boot).

To get around all these problems you could use a different initrd for different situations, but in that case you might as well put your kernel in initrd (and just use the OS's normal drivers instead of special "during boot" drivers)...

If you restrict the range of hardware supported then it might make some sense. For example, if your OS doesn't support old computers (no ISA) and doesn't support booting from network or booting from SCSI. However, out of the 25 computers I've got here those restrictions would make your OS useless for 8 of them (over 30%) - one is ISA, 5 are SCSI and 2 have no CD/floppy/hard disk (they can only boot from network). I admit some of the computers I have here are a little unusual, but even if your OS can't boot on 15% of computers it's still a large sacrifice.


Cheers,

Brendan

[JackScott]

Another very good post by Brendan. But I'll just nitpick a bit.

SATA has two (or three, depending on how you count) modes:

• IDE Mode - The controller and the drive behave as if they were Ultra-DMA IDE drives, and the programmer notices no difference.
• AHCI Mode - The SATA 'native' mode, where a few extra features are available, such as hot-swapping and such. This mode has nothing to do with IDE, however (I believe) that the AHCI mode is a standard in it's own right. This is how Vista can install to SATA drives that aren't in IDE mode: as long as they conform to the AHCI standard, it'll work. XP didn't have an AHCI driver included, so it needed an F6 driver.
• RAID Mode - While it's not part of the whole SATA thing, it's worth mentioning for completeness. Under RAID mode, no standard covers the controller, and it does vary wildly. But you do get the cool effects like RAID5 and so on. Even Vista needs the F6 boot driver for this type of disk (since there are so many different controllers).

[NickJohnson]

Well, assuming I'm using the design with init having simple drivers, I was thinking of just having a couple of drivers (floppy and ATA) so that I can do usermode autodetection and load other things. Those drivers can run within init, and could support only FAT32 or ext2 or something, and load from a small "boot" partition. I don't care much if I'm initially stuck booting from only floppies/ATA disks, or compiling custom inits that can handle SCSI, USB, networking etc.

I definitely see your point about not being able to support other computers, but I'm assuming that whoever installs (or packages) the system will be able to compile a custom init for their box.