Reading the HDD before the kernel is setup

[tlf30]

Hello,
I am looking for a little advice.

My initial loader image hands me off into my kernel initialization code in 64bit protected mode. I need to be able to read the current hdd that the OS is on so the kernel can properly setup itself. After the kernel loads, it can use its own device drivers, but during the loading process, they are not available.

I am wondering what would be the best way to read the hdd that the system is booted to to load the kernel. I was looking at using ATA 48bit PIO. Is this a good way of doing things? Or should I use CHS?
I would just use the BIOS interrupts, but I have no intentions of switching in and out of 64bit mode a dozen times.

Thanks,
Trevor

[Roman]

You can transfer a RAM disk to your kernel through the loader. ATA PIO is good, but it will break with non-ATA devices (e. g. SD cards, USB sticks).

[Brendan]

Hi,

My initial loader image hands me off into my kernel initialization code in 64bit protected mode. I need to be able to read the current hdd that the OS is on so the kernel can properly setup itself. After the kernel loads, it can use its own device drivers, but during the loading process, they are not available.

I am wondering what would be the best way to read the hdd that the system is booted to to load the kernel. I was looking at using ATA 48bit PIO. Is this a good way of doing things? Or should I use CHS?
I would just use the BIOS interrupts, but I have no intentions of switching in and out of 64bit mode a dozen times.

There's only 4 choices:

• Build everything into the kernel (drivers, file systems, etc) so that kernel is able to access file systems and disk drives properly. In this case kernel is likely to grow to 5 MiB or more; which means that you'll need to switch in/out of real mode and some other mode (protected mode) just to get enough RAM to load kernel into.
• Make boot loader load everything using firmware (e.g. "initial RAM disk" containing kernel, drivers, file system code, etc); so that OS can get everything it needs from RAM after you've discarded the firmware (until drivers, file systems, etc are running). In this case the "sum of all things you load" is likely to grow to 5 MiB or more; which means that you'll need to switch in/out of real mode and some other mode (protected mode) just to get enough RAM to load it all into.
• Build device enumeration (e.g. scanning PCI and USB buses) and miniature device drivers for ATA (including 48-bit LBA, 24-bit LBA, CHS, etc, because different disks/controllers may support one and not another), AHCI, ATAPI, about 30 different SCSI controllers, 4 different USB controllers and USB "mass storage device" into your boot code (e.g. maybe into a second stage), so you can use them to load stuff from disk (plus about 50 network card drivers and network stack maybe, to load stuff from network). In this case those miniature device drivers won't be able to use your kernel (for things like memory allocation, IRQ handling, scheduling, etc) so they'd be discarded as soon as you've used them and replaced with real drivers later during boot (after kernel is working). Also, all that stuff (e.g. the "second stage") is likely to grow to 2 MiB or more; which means that you'll need to switch in/out of real mode and some other mode (protected mode) just to get enough RAM to load all the temporary/discarded drivers into.
• Have about 100 different boot loaders; where one contains a miniature ATA driver, one contains a miniature AHCI driver, etc. This is similar to the previous option; but replaces the "need to switch between real mode and some other mode" with something much worse (having to manage/maintain 100 different boot loaders and ensure the right one is installed, which is messy and makes it impossible to have something like a generic "live CD" or USB image).

Note that the third and fourth choices are hideously idiotic.

I use the second option. More specifically; I pack all the files (kernels, drivers, etc) into a single file (sort of like "tar"), then (optionally) compress that file. Boot code loads this file (using firmware), then (if it was compressed) decompresses the file, then detects things (e.g. CPU capabilities, etc) and decides which kernel to start (plain 32-bit, 32-bit with PAE, 64-bit). Then, later during boot, all those files (that were loaded into RAM by boot loader) are used to populate the VFS cache, where all those files are marked as "potentially needs to be written to disk" in the VFS cache. After file systems are mounted, OS checks to make sure that these files are actually in the file system, and if they aren't it writes the files (that were loaded by boot loader) onto disk. This means that if something trashes your main file system the OS is able to "self heal" a little. Finally; I also keep track of which files were actually used during boot, so that the OS can use the information to optimise/minimise the "file containing many files" and update it.

All of this combined mean that I can have a generic "OS installer" CD that has a huge "file containing many files" (with many kernels, many drivers, etc), and when the CD is booted the boot code keeps track of which kernel and which drivers are needed, and the OS installer can generate a smaller "file containing many files" to install on the hard disk. Then (possibly after the user has been using the OS for 3 days and the "OS installer" CD is no longer present); if/when the user creates a file system that "self healing" stuff will automatically unpack/install the files into the file system (the OS installer doesn't need to do this).


Cheers,

Brendan

[Rusky]

Having the boot loader load the drivers along with the kernel is probably the most straightforward way, as Brendan explained. Further, if your firmware is anything better than the old PC BIOS (EFI, coreboot, etc) it doesn't require any shenanigans with real mode either.