Pinion - a kernel for your kernel

[NickJohnson]

Hi,

As part of this project, I've written a sort of microkernel that, unlike a normal microkernel, is designed purely to do memory management and threading for a larger monolithic kernel, which runs under it in ring 1 (but with full IO privileges.) It is only written for single-processor x86, but there is nothing in the ABI that prevents things like SMP; the semantics are also general enough that ports for other architectures could be easily developed. The idea is to have a common base that contains all of the tricky parts of the OS but imposes no real constraints on the rest of the kernel, and that also has a very well-defined ABI, so other implementations following the specification can be drop-in replacements. Basically, an "OS engine" in the spirit of a game engine.

Since it seems like it could be legitimately useful for other hobby OS developers, I've set up a separate project called Pinion to contain the specification for the ABI as well as this reference implementation. The reference implementation contains a lot of Rhombus kernel code, which has been tested pretty thoroughly, and the new threading model is based on (and checked against) a formal state machine.

I'm not officially releasing the draft of the specification and the full documentation until April 19. At that point, I'd really like feedback on this, especially from people who might actually like to base a new system on it. My new project will use it, and I will likely backport Rhombus to it, so it should be actively developed for quite a while. It's also effectively complete even at this point, so there is no danger of vaporware.

Before then, feel free to ask any questions about the design here.

Some features of the design:
- Preemptive multithreading in both kernelspace and userspace
- Ability to pause threads and inspect/modify their states
- Handling of stack overflows/page faults/GPFs/etc. from kernelspace
- Event queues that multiple threads can wait on (e.g. multiple worker threads can handle one type of IRQ or event in a round-robin fashion)

https://github.com/nickbjohnson4224/pinion

[Owen]

Whats the purpose of the ring 1 distinction? All I can imagine it doing is harming portability.

[NickJohnson]

Whats the purpose of the ring 1 distinction? All I can imagine it doing is harming portability.

Ring 1, unlike ring 0, and like ring 3, will load its stack information from the TSS when an interrupt fires. This means interrupts from ring 1 and ring 3 can be handled very similarly, and neither can cause any sort of kernel panic due to a mismanaged stack. It's not for protection reasons: using ring 1 is purely an x86-specific design decision that makes the implementation simpler and more robust (not to mention that ring 1 code can read and write kernel pages, so the idea that it is safer is moot.) If you wanted to port it to some architecture that does not have more than two protection levels, both Pinion and the rest of the kernel would operate in kernel mode.

[Owen]

It seems to me that this means that the kernel is going to have to pay similar prices to a traditional system call whenever it wants to invoke any Pinion service, and that fast system calls (i.e. SYSCALL/etc) are going to be *slower* than the traditional, slow methods of doing system calls because they then have to head back to ring 1 for servicing... (and may then have to be vectored back up to ring 0 if they need a Pinion service)

Realistically, I can't see any "stack mismanagement" issues with the whole kernel running in ring0 even with a 1 stack per CPU design. A "mismanaged stack" is a sign of a kernel bug anyhow.

[NickJohnson]

It seems to me that this means that the kernel is going to have to pay similar prices to a traditional system call whenever it wants to invoke any Pinion service, and that fast system calls (i.e. SYSCALL/etc) are going to be *slower* than the traditional, slow methods of doing system calls because they then have to head back to ring 1 for servicing... (and may then have to be vectored back up to ring 0 if they need a Pinion service)

The intent is not to make the system as fast as possible, but instead to make the important, system-independent components of a kernel reusable. The exact mechanism by which system calls are handled is implementation-specific. For the reference x86 implementation, everything is using software interrupts, which I know are slow. However, Pinion contains the scheduler and handles timer interrupts directly, so there is no added overhead for multitasking. In addition, it would be possible (if an implementation had the rest of the kernel in ring 0) to have Pinion calls be simple function calls if speed were paramount. Either way you cut it, the resulting system should be strictly faster than a microkernel, which already has acceptable performance.

Realistically, I can't see any "stack mismanagement" issues with the whole kernel running in ring0 even with a 1 stack per CPU design. A "mismanaged stack" is a sign of a kernel bug anyhow.

It would be a sign of a kernel bug, but not of a bug in Pinion. The current design should be able to recover from (or at least give useful debug information for) a number of different faults in kernelspace. Robustness is useful, particularly for the reference implementation. Also, although Pinion internally uses a 1-stack-per-CPU design, the kernel running under it has one stack per thread.

[eino]

Gonna try this one out, just waiting for some basic documentation (or perhaps implementation guidelines or something like that)

[ACcurrent]

This looks like the OSLib and OSKit that actually works!