OSDev.org

Hello,
I have been doing a lot of thinking, and I am wondering how considerable the advantages of microkernels are. I find a microkernel an interesting thing to research, but they are slow and very hard to develop. My microkernel is not stable at all and slow. I have long been a fan of these things, and always will be a big fan of Managarm (which I like better then Linux) and Hurd. But if a monolithic or hybrid kernel only allows modules to be loaded by the root user, then the security concern, for all practical purposes, goes away. Stability is still a concern, but just as speed is a concern for microkernels. I plan on working on NexOS until it can run a shell with file I/O, but will recode NexNix as an x86_64 monolithic kernel OS. I think a hybrid kernel is best, and when NexNix can run a Debian user space like Pedigree, then I will make a hybrid kernel OS. I personally think that if done right (like Managarm), then a microkernel is better. But I will make a monolithic kernel. and research microkernels. What is your opinion on microkernels, monolithic or hybrid kernels, and even exokernels?

A microkernel is more complex regarding inter-kernel communication.
Especially Mach (used by Hurd) is a quite complex thing.

Monolithic kernel are more complex regarding having an overview of the
whole kernel. The Linux makefile for example is quite complex.

I try to go into the direction of a layered OS, making the kernel and
system libs layered, like a graphics stack, a network stack and so on with
process managment and I/O.

Greetings
Peter

nexos wrote:Hello,
I have been doing a lot of thinking, and I am wondering how considerable the advantages of microkernels are.

Modularity by clearly separated layers, manageability, stability, flexibility, increased security.

nexos wrote:I find a microkernel an interesting thing to research, but they are slow and very hard to develop.

Take a look at the L4 microkernel family. They are not slow. And I wouldn't call the development hard, but it is considerably more difficult than a monolithic kernel, that's true (mostly because you can't access anything, you have to plan interfaces). But once you have a good and efficient messaging system, there's not much difference, except you'll try to minimize the number of messages naturally, while in a monolithic kernel you wouldn't care about minimizing the number of memory access.

nexos wrote:But if a monolithic or hybrid kernel only allows modules to be loaded by the root user, then the security concern, for all practical purposes, goes away.

This part is not true at all. Modules are the worst thing that could happen from security point of view. Basically you load code from unverified userspace into kernelspace, where it's going to run with supervisor privileges (ring 0)... A quick summary:

Modular monolithic:
- loads code into the kernel space
- module code can access any part of the system, including kernel structures and privileged instructions
- no run-time checks possible (modules are simply linked on load)
- a crash in a module crashes the kernel, and the whole system along with it

Microkernel:
- loads code into separate, user level address spaces, so called servers
- server code is a plain simple user level code, no special access available
- all access are checked in run-time (by validating messages sent by the server)
- a crash in a server is fully recoverable without the user even noticing

nexos wrote:What is your opinion on microkernels, monolithic or hybrid kernels, and even exokernels?

Depends what goal you want to achieve.

For a special-purpose OS with one task, go for monolithic. For stability and security, go for microkernel. For simulating other OS' features within the same environment, go for exokernel.

Cheers,
bzt

I think hybrid kernels are the best. You get the microkernel structure and flexibility with monolithic speed. You still have stability and security concerns, but I will address those.

bzt wrote:- a crash in a module crashes the kernel, and the whole system along with it

In a basic monolithic or hybrid kernel, yes, but Linux shows an oops error and unloads the faulting module. You could even restart the module too! It just doesn't come free with the architecture. I doesn't come free with microkernels either. A server must be restarted if it crashes.
Overall, their are pros and cons to both. I will implement a hybrid and a microkernel, and come back to say what I think is better.

nexos wrote:Hello,
I have been doing a lot of thinking, and I am wondering how considerable the advantages of microkernels are. I find a microkernel an interesting thing to research, but they are slow and very hard to develop. My microkernel is not stable at all and slow.

That was not my experience. A microkernel is in general smaller and therefore more maintainable. However, microkernels are slower than monolithic kernels, mainly because of more task switching. However, you always pay for features which is usually slower performance. The question is more is the feature worth the degradation of performance. I certainly think so and I find my microkernel having adequate performance. There are ways to speed up microkernels.

nexos wrote:I have long been a fan of these things, and always will be a big fan of Managarm (which I like better then Linux) and Hurd. But if a monolithic or hybrid kernel only allows modules to be loaded by the root user, then the security concern, for all practical purposes, goes away. Stability is still a concern, but just as speed is a concern for microkernels. I plan on working on NexOS until it can run a shell with file I/O, but will recode NexNix as an x86_64 monolithic kernel OS. I think a hybrid kernel is best, and when NexNix can run a Debian user space like Pedigree, then I will make a hybrid kernel OS. I personally think that if done right (like Managarm), then a microkernel is better. But I will make a monolithic kernel. and research microkernels. What is your opinion on microkernels, monolithic or hybrid kernels, and even exokernels?

The border between microkernels and hybrid kernel is not well defined. I probably have designed a hybrid kernel according to some and others not.

What you should think, is this feature worth it or not. I've found that process isolation is great and having drivers in user space is great and worth it despite the performance degradation. Disabling the MMU will make your CPU faster but is it something that most people want? There is always a price.

They are a pain in the @$$ that should be avoided. They introduce a lot of unnecessary overhead and are way more complicated to code.
There is a reason why all the big OS'es are monolithic.
Dealing with all the IPCs and servers and crap is just not worth it. That so called "protection" is not even noticeable to an end user.
Plus they are slower.
And stop saying it is secure blah blah, nobody is going to hack your shitty little OS.

nexos wrote:In a basic monolithic or hybrid kernel, yes, but Linux shows an oops error and unloads the faulting module.

What can Linux do if a faulty module doesn't release a lock, or clears out an other module's memory or sends the kernel into an infinite loop? Or allocates resources like a fork-bomb by mistake? Or issues a CLI+HLT combo? Or jumps into the middle of an instruction in another module (so that the error is triggered in another module's code space)?

What you say is only possible if the error raises and exception, and even then only a small amount of errors are recoverable. On the other hand you can restart a microkernel server if it's not responding to messages, which includes all kinds of problems, even otherwise undetectable errors. With separated address spaces a faulty server can't mess up other server's data like a faulty kernel module can.

Octacone wrote:There is a reason why all the big OS'es are monolithic.

This isn't true. Windows - or more precisely the NT kernel - is not monolithic. The most widespread OS, Minix (thanks to Intel embedding Minix without permission into every single Intel CPU) is not monolithic either.

Octacone wrote:And stop saying it is secure blah blah

Why? That's the truth. Even a badly implemented microkernel is much more secure than the most bullet-proof monolithic one.

Octacone wrote:nobody is going to hack your shitty little OS.

Famous last words? Just because you have low self-esteem and you think your own OS is shitty is not a reason to forget about security. With that mentality your OS is always going to be just a "shitty little OS". Plus there are other factors to security than an attacker hacking: a misbehaving driver issuing CLI+HLT for example, or accidentally zeroing out the number of mounted disks in the VFS etc. etc. etc. There are nearly infinite number of ways how things can go wrong in a monolithic kernel.

Cheers,
bzt

Before this turns into another Tannebaum Torvalds debate, I will give my final piece. Both sides have pros and cons. I think a well designed monolithic kernel is more stable then a poorly designed microkernel. Stability takes work on both sides, as the Wiki article about microkernels points out.

nexos wrote:Before this turns into another Tannebaum Torvalds debate, I will give my final piece. Both sides have pros and cons. I think a well designed monolithic kernel is more stable then a poorly designed microkernel. Stability takes work on both sides, as the Wiki article about microkernels points out.

Okay, one final thought: stability != security.

For example one of the first mainstream microkernels, AmigaOS was pretty stable, but it wasn't secure at all. seL4 kernel is the most secure microkernel out there, but it's not necessarily stable.

Cheers,
bzt

nexos wrote:they are [...] very hard to develop.

Having built a monolithic kernel and now working on a microkernel, I would say making a microkernel is working out to be simpler because my code can be more generic.

For example:

• My microkernel only does work on a system call or interrupt. The work is very lightweight (send a message, allocate a page, unschedule a thread, ...) that I don't foresee a reason to preempt the kernel, so there is no kernel multithreading (and kernel multiprocessoring can be implemented with simple spinlocks.) The kernel code can stay clean and simple.
• Other than the initial boot modules, the application loading code can exist in userland. You could support new executable formats, JITs, loaders that inject their own runtime (scripting languages and virtual machines), etc. without needing kernel support.
• Device management is done in userland space. So someone could start building TV tuners without needing to update the kernel to support TV tuners. It might be helpful to publish a standardized TV tunner interface though.

My reasoning is anecdotal. There are some things that you need to do get started such as building IPC support. This can be simple (pass a fixed sized struct), or complicated (creating an interface definition language.) At some point, even monolithic kernels may be interested in adding IPC, so it's work you'll probably do one day anyway.

As I see it, microkernels are better *in theory*, but they don't pan out on existing hardware. Every major operating system that has been marketed at some point as using a microkernel has eventually moved some significant class of drivers into kernel space for performance reasons.

The reason for this is that a kernel is essentially a collection of libraries that need to have private memory where they can keep data that userspace processes (and, ideally, other kernel components) can't see.

Modern hardware generally makes it convenient to have a single private memory region (kernelspace) that is isolated from userspace processes, but does not provide any useful facility to allow drivers operating in that memory region to be isolated from each other. Monolithic kernels are the kernels that just accept this and run all the code that needs to be isolated from userspace in kernelspace.

Microkernels, on current hardware, isolate their different components from one another by giving up on having those components be libraries and turn them into whole separate userspace processes instead. This provides the necessary isolation, but introduces significant overhead by turning what would otherwise effectively just be function calls into IPC.

However, if we had hardware with some sort of capability-based-addressing scheme, it would be possible to gain the desired isolation while having calls to drivers still basically be function calls, which might make microkernels more competitive. The bad news there is that most historical or proposed hardware capability systems I've seen have tried to have really fine granularity, often byte or word level granularity, which has resulted in lots of overhead in the hardware. As a result, historical capability systems have tended to underperform and fail in the market, and the proposed systems I've taken a look at probably will as well.

So many good posts!

bzt wrote:stability != security

No, it does not. But I think it influences it. For example, an OS maybe bulletproof in security, but if it is unstable a malicious driver can figure out what buttons to push to wreak havoc.

AndrewAPrice wrote:Having built a monolithic kernel and now working on a microkernel, I would say making a microkernel is working out to be simpler because my code can be more generic.

The kernel, yes, but I am talking about the whole thing. For example, the kernel may give bus controlling to a user process. That makes bus controlling for the kernel easier. It just talks to the bus driver via some protocol and gets the info it needs. But a bus controller still needs to be made. This user mode bus controller would be slower, as reading or writing to a port would be slower because it needs to make a system call to use the out or in instructions, which are privileged. Luckily, MMIO is used nowadays. But then you need to securely map MMIO regions. You need to be careful not to allow apps to map any address for security. So security requires work in a microkernel.

linguofreak wrote:As I see it, microkernels are better *in theory*, but they don't pan out on existing hardware. Every major operating system that has been marketed at some point as using a microkernel has eventually moved some significant class of drivers into kernel space for performance reasons.

Very true, on paper, microkernels sound perfect, until you see speed problem. Perhaps a good idea would be to put some stuff (VFS, bus controllers, etc) in kernel mode and put untrusted stuff in user mode. That would be a sort of hybrid approach.

nexos wrote:This user mode bus controller would be slower, as reading or writing to a port would be slower because it needs to make a system call to use the out or in instructions, which are privileged.

You can use the IOPB for that.

Yes, but is IOPB available for x86_64? I've been wondering, as that would greatly speed up my microkernel.

Yes, it is! It works the same as in 32-bit protected mode, including all of the strange quirks.