OSDev.org

I'm curious as to whether or not the following design qualifies as a microkernel. The kernel itself has necessary base services for managing the rest of the system (memory manager, scheduler, and processes/threads). Suppose I want to improve the performance of my system, so I implement kernel-mode drivers. Each kernel-mode driver, however, is its own process but with a privilege level of 0. (Or perhaps a slightly higher (1, maybe 2) privilege level in protected mode, as opposed to long mode.) Additional drivers (such as a file system driver or digital camera driver) would be run in user space. I theorize that it is still possible (quite easily) to crash the system with this design, but a simple exception (e.g., divide by zero) would only cause that driver process to restart, with appropriate kernel support

I'm not sure as to whether this design this falls under for a variety of (what I believe to be obvious and/or understandable) reasons. Modular? Hybrid? Or a true microkernel?

I wish to improve performance for basic tasks such as bus I/O, but maintain stability for less critical tasks such as printing a document. On that note, are there any suggestions for keeping a (mainly) microkernel design while improving performance for these tasks?

Thanks in advance for your time in reading (and hopefully considering) this post.

Cheers,
-Aaron

In my opinion, a microkernel doesn't contain drivers. That's just my opinion and in fact I don't think the name of a design matters.

BUT, I'm curious about how having kernel mode drivers improves the performance of a system. If a filesystem runs in userspace, wouldn't it be better speed-wise for example if the disk drivers were DLL's loaded by the fs ?

The performance aspect derives from the reduction of one or more context switches. The driver in question would be able to perform as many I/O operations as it wishes (e.g., outb and so on) without the need to ask the kernel for permission. While this is unsafe, the installation of bus drivers is (in theory) not very frequent. I feel an improvement in speed here overpowers the stability aspect.

I don't believe it is wise to allow FS drivers the ability to load disk drivers; disk drivers are meant to provide the data an FS driver uses to manage the on-disk structure. Or were you referring to the VFS service, which FS drivers (might) register with?

Thank you for your response.

Cheers,
-Aaron

I'd consider it a hybrid, since you are mixing concepts of a modular kernel with those of a microkernel.

Alright, thanks.

Let's say we agree to have the fs in user space. Consider the 2 schemes (there are other schemes) -

a) the fs loads dll style drivers.
b) the fs calls the kernel to talk to the driver.

IMHO, from a performance point of view you should choose a). It will generally be faster since we don't have to enter the kernel.

However, you are also worried about safety/permissions. The disk driver must be a trusted process/module/dll or we would never let it near the data on the disk. And IOPL can be used to make sure untrusted drivers don't get access to the hardware. As long as it can only be loaded by trusted (perhaps even signed) filesystems, I don't see the problem. In a microkernel, lot's of decision making is moved to user space. Permission's handling is itself often a userspace program.

What I'm saying is that I think performance will be improved by keeping the fs and the driver together. In userspace (or in the kernel).

I'd like to reiterate the functional flaw(s) in doing so. The FS requires disk drivers to even be able to recognize it-self. So, lets say we have a computer with 2 hard drives in them. One hard drive is an IDE and the other is a serial ATA. The IDE drive has 3 partitions "Linux (ext/3)," "Windows XP (ntfs)," and "Windows 7 (ntfs)." The SATA drive has two partitions, "Source (ext/2)," and "Files (ntfs)." (Actually, this was the exact setup I had a while ago.) Now, let's assume the OS has several file system drivers, not counting the VFS. These file system drivers include extfs (ext/2/3/4), ntfs, vfat, reiserfs, zfs, sfs, and so on. From what it sounds like, you are proposing that each FS driver attempt to load the disk driver (IDE or SATA or another) associated with its own FS. In order to accomplish this (somewhat) efficiently, the kernel would have to somehow know each FS that is used in the system and load those drivers. Those FS drivers would then load the appropriate storage drivers and then try to find itself on the media.

What I'm proposing is the internal storage devices be enumerated, and (for those which are to have certain partitions mounted) their appropriate drivers loaded. After which, their partition tables would be parsed (in whatever format they are, e.g., GUID or MBR). The results of the partition parsing would allow the OS to choose the proper FS drivers to then load, as opposed to loading them all. This sort of a system is, IMHO, the easiest of the two to program and work with, as well as the most efficient (again, in my opinion).

My currently proposed design would improve performance for several reasons. 1. The storage drivers have a kernel privilege level, and can therefore directly access hardware ports and registers, without having to ask for permission. 2. Less memory would be used, thus improving overall system efficiency.

I believe storage drivers will also require communication with the kernel in order to have their ports programmed. This way may cause less system port calls. Although, that wouldn't necessarily be true if all the driver is doing is accessing some physical memory it requested to carry out the transactions. When I think about it, actually, the majority of disk drivers will only be doing that. But I don't believe that's what all drivers will be doing.

This is open to debate though. I appreciate, greatly, your response; it has invoked a deeper thought process into the matter, for me. Perhaps my whole kernel mode argument can be tossed after all?

Cheers,
-Aaron

From what it sounds like, you are proposing that each FS driver attempt to load the disk driver (IDE or SATA or another) associated with its own FS. In order to accomplish this (somewhat) efficiently, the kernel would have to somehow know each FS that is used in the system and load those drivers. Those FS drivers would then load the appropriate storage drivers and then try to find itself on the media.

Not quite. I have a single fs process (actually called blockman let's call it fs-cache from now on) that is, I guess, the local filesystem cache. Right now I have my device drivers (to be honest I've only got an IDE driver so far) and filesystems (only FAT so far) as separate processes. My kernel doesn't do any device enumeration. It's all handled by user processes that execute from a startup script. That startup script, fs-cache and the ide driver are in the initrd filesystem so there is no chicken and egg problem. I also have a vfs that ties together the various filesystems (e.g. initrd, nfs, local fs, devices, ftp etc.) into a single file system. The kernel knows absolutely nothing about any of this because the vfs is also a process.

Permissions are (actually "will be" because I haven't implemented this part yet) handled by the normal Posix means. The on disk partitions also have normal Posix permissions associated with them (mostly root for hard drives.) Therefore only processes started by root can ever read or write directly to the hard drive and even then they need to ask the kernel to modify their IOPL. Possibly later I will make that a bit more fine-grained by using an IO bitmap.

My currently proposed design would improve performance for several reasons. 1. The storage drivers have a kernel privilege level, and can therefore directly access hardware ports and registers, without having to ask for permission. 2. Less memory would be used, thus improving overall system efficiency.

In a microkernel, if usermode device drivers are given access to ports based on permissions/signing using a processor mechanism (like IOPL) then it's not necessary to have them in the kernel is it?

I too appreciate this discussion. My girlfriend is sick of hearing about it !!

Well, Intel Manual volume 2B page 4-17 states a #GP will occur "If the CPL is greater than (has less privilege) the I/O privilege level (IOPL) and any of the corresponding I/O permission bits for the I/O port being accessed is 1." To be honest, I had forgotten about the IO permission bitts in the TSS. I will have to review this to determine whether it is efficient to update the TSS for each process with a different IOPL. (Implementation not yet thought out.)

Cheers,
-Aaron

Hi,

AaronMiller wrote:Well, Intel Manual volume 2B page 4-17 states a #GP will occur "If the CPL is greater than (has less privilege) the I/O privilege level (IOPL) and any of the corresponding I/O permission bits for the I/O port being accessed is 1." To be honest, I had forgotten about the IO permission bitts in the TSS. I will have to review this to determine whether it is efficient to update the TSS for each process with a different IOPL. (Implementation not yet thought out.)

There's several ways to use the I/O permission bitmap efficiently - maybe even having one TSS used for "no I/O port access" plus a TSS per device driver (to avoid copying I/O permission bitmaps into the TSS during task switches).

For performance, a much larger problem is "Each kernel-mode driver, however, is its own process". By this I assume that each device driver runs in it's own address space; and you need to do a full task switch (including reloading CR3 and invalidating TLB entries, etc) before any of the driver's code is run. From my perspective, if performance is more important than isolation then you should have device drivers mapped into all address spaces (regardless of CPL or what you do with I/O ports) to avoid the "full task switch"; and if isolation is more important than performance you should probably run device drivers as CPL=3 processes (because the extra overhead of using CPL=3 instead of CPL=0 is negligible compared to the "full task switch" overhead).

Of course if isolation is more important than performance, then you can attempt to minimise the number of task switches. For e.g. use asynchronous interfaces to allow you to postpone the task switches (e.g. make 20 requests then do one task switch and handle the 20 requests before switching back vs. do 2 task switches for each of the 20 requests).


Cheers,

Brendan

Thank you Brendan, I greatly appreciate your comment. What you have said makes sense. Perhaps, to make a compromise, I could have one process for all drivers and one thread for each driver within the process. That way, it's easier to just restart that thread. Any opinions?

Hi,

AaronMiller wrote:Thank you Brendan, I greatly appreciate your comment. What you have said makes sense. Perhaps, to make a compromise, I could have one process for all drivers and one thread for each driver within the process. That way, it's easier to just restart that thread. Any opinions?

In some cases (e.g. USB controller driver and drivers for USB devices connected to that USB controller) it'd help. In other cases (e.g. disk controller driver, ethernet card driver and video driver) it won't help because these drivers don't really talk to each anyway. In all cases (whether it helps performance or not) it'd weaken the isolation (e.g. one faulty driver could trash any other drivers).

Basically there's no right way and no wrong way - it all depends on what sort of compromise between performance and isolation (and complexity) you want, and designing something that meets your goals.

Note: an example increasing complexity to improve performance without sacrificing isolation would be to have the kernel could monitor communication between pieces, and dynamically shift pieces into the same address space if it notices that the pieces talk to each other a lot. Also, this might not be limited to drivers only - if there's a lot of communication between a few normal processes (e.g. a text editor and a spell checker?) then they could be grouped together too.

You might want to look into the way L4 implemented "small address spaces". The idea is to split a normal address space into a medium address space and several smaller address spaces using segmentation. For example, 1 GiB of "kernel space", 1 GiB of "medium user space" and sixteen 128 MiB "small user spaces" all sharing the same 4 GiB address space; where processes start using a 128 MiB small address space and are migrated to a 1 GiB address space if they need more space (and are then migrated to a 3 GiB address space of their own if they need more than that). This allows you to separate pieces/processes into the same large address space to reduce TLB flushing while also isolating them (using segmentation). Unfortunately, it won't work in long mode though (where segmentation isn't usable).


Cheers,

Brendan

That's interesting, thank you for the information.

I think I'll just do something fairly simple; kernel mode drivers (KMDs) are simple dynamic libraries loaded by the EDI runtime system service, which is a kernel mode module. User mode drivers (UMDs) are separate processes which communicate with the runtime via system calls. I think this layout is easiest to manage and probably the most efficient for my needs. I'm open to suggestions, however.

Cheers,
-Aaron