Page 1 of 1

TanOS 0.1 - from-scratch Rust microkernel: ring-3 isolation, synchronous IPC, fault isolation (QEMU/KVM)

Posted: Sat Jun 13, 2026 3:32 pm
by TanOS
TanOS is a from-scratch x86-64 microkernel in Rust. It boots under QEMU/KVM and demonstrates:
4-level paging - each process gets its own address space
Ring 3 userspace - loads ELF binaries from an initrd via iretq
Synchronous IPC - cross-address-space call/receive/reply rendezvous
Fault isolation + restart-on-crash - a ring-3 crash kills only that process; the kernel can restart it from its image while others keep running. (Restart-from-image, not full reincarnation - clients aren't transparently re-bound to the restarted process.)
IPC latency - ~3,800 cycles/round-trip under KVM (unoptimized; two CR3 reloads, no PCID, spinlocks per trip)
Design decisions worth knowing:
Single kernel stack - every int 0x80 syscall saves the full register frame onto one reused kernel stack. Context switching copies that frame into the current PCB, restores another process's frame + CR3, and iretqs. This works safely because the syscall gate clears IF, preventing nested entries, so one stack always suffices.
Memory layout puts the kernel identity-mapped in the low 1 GB (2 MB huge pages) reachable from every address space, userspace starting at 0x40000000, and the user stack at 0xC0000000 - chosen specifically to avoid non-canonical address faults (we hit that bug; it's documented in the README).
What to look for when testing: build and boot per the README, then check /tmp/s.txt - you should see the driver crash, reincarnate, and the survivor confirm it stayed alive throughout.
No disk image yet - builds from source via cargo +nightly-2024-01-15. Full instructions in the README.
Disclosure: built with heavy AI assistance (Claude). Judge the output, not the prose.
GitHub: https://github.com/hawkaiglai/tanos

-----
EDITED

Re: TanOS 0.1 - Rust microkernel: ring-3 isolation, synchronous IPC, process reincarnation on QEMU/KVM

Posted: Sat Jun 13, 2026 7:56 pm
by Octocontrabass
TanOS wrote: Sat Jun 13, 2026 3:32 pmIt boots under QEMU/KVM
Does it work on bare metal or any other VMs?
TanOS wrote: Sat Jun 13, 2026 3:32 pmFault isolation + reincarnation — ring-3 crash kills only that process; kernel restarts it from its image while other processes survive
How does that impact IPC for the surviving processes? Since your kernel is a microkernel, presumably other processes will still try to communicate with the process that crashed.
TanOS wrote: Sat Jun 13, 2026 3:32 pmint 0x80 syscall
Why do your system calls use INT 0x80 instead of SYSCALL?
TanOS wrote: Sat Jun 13, 2026 3:32 pmkernel identity-mapped
This will prevent your OS from booting on many systems, possibly including QEMU depending on how it's configured. Normally you want your kernel to use the higher half of the address space, to keep it cleanly separated from userspace in the lower half. On x86-64 specifically, you'd also want to link the kernel to run in the highest 2GiB of the address space (at or above 0xFFFFFFFF80000000) since it allows the compiler to produce smaller code.
TanOS wrote: Sat Jun 13, 2026 3:32 pmJudge the output
Oh, I'm definitely judging it.

Re: TanOS 0.1 - Rust microkernel: ring-3 isolation, synchronous IPC, process reincarnation on QEMU/KVM

Posted: Sun Jun 14, 2026 3:46 am
by TanOS
Thanks - this is exactly the kind of scrutiny I was hoping for, and a couple of your points sent me straight back to the code. Going in order:

▎ Does it work on bare metal or any other VMs?

No - only QEMU (KVM and TCG). I haven't tested bare metal, VirtualBox, VMware, or Bochs, and I'd expect it to need real work first: it assumes a legacy PIC + PIT (no APIC/ACPI), relies entirely on a serial port for output (no framebuffer - so on a
machine with no COM port it'd boot to a blank screen), and makes the low-memory layout assumption you flag below. I've scoped the claim to "QEMU/KVM" deliberately rather than overstate the reach, and I've now written that out explicitly as a limitation
in the README.

▎ How does that impact IPC for the surviving processes? [...] other processes will still try to communicate with the process that crashed.

You were right that this was broken, and it was the most useful question of the lot - so I fixed it. Previously, if a process was blocked in a call awaiting a reply and its server crashed, it hung forever: the rendezvous still referenced the dead PID
and nothing woke the waiter.

Now the kernel tracks which process owes the reply (in_service), and on process death it wakes any caller blocked on the dead one with an error instead of leaving it stuck. Commit: https://github.com/hawkaiglai/tanos/commit/cfcd5be - there's an
ipc_crash_demo build (--features ipc_crash_demo) that shows a client recovering from a server that crashes mid-request.

To be precise about what that does and doesn't do yet, though: the caller gets an error and must retry - the kernel does not transparently re-bind it to the reincarnated server, which comes back with a new PID and no inherited service identity.
Proper client re-binding (stable service identity, dependency tracking, re-establishing endpoints) is the MINIX-3 reincarnation-server problem, and that part isn't done. So "reincarnation" here demonstrates the kernel-side mechanism plus not-hanging on
peer death - not the full client-recovery story.

▎ Why do your system calls use INT 0x80 instead of SYSCALL?

No principled reason - int 0x80 was the simpler bring-up (a DPL-3 IDT gate, no LSTAR/STAR/SFMASK setup, no swapgs). I actually had SYSCALL scaffolding and deleted it as dead code because it was never wired up. SYSCALL is faster and the right long-term choice; I haven't switched because it isn't the bottleneck - the ~3,800 cycles is dominated by the two CR3 reloads per round-trip, not the trap entry.

▎ [identity-mapped kernel] will prevent your OS from booting on many systems [...] use the higher half [...] 0xFFFFFFFF80000000

Agreed, and this is the most valuable pointer. The kernel is linked at 0x100000 and identity-maps the low 1 GB, so it runs in the low half - a deliberate bring-up shortcut (it mirrors the early boot tables and keeps all RAM trivially reachable while I
build page tables), but I know it's not the right design. Moving to a higher-half kernel (link at 0xFFFFFFFF80000000, -mcmodel=kernel, keep a low physmap for physical access) is on the list as a real fix, not a someday - I've scoped it and it's a
contained change in the linker script, the boot trampoline, and per-address-space mapping; mostly I'm wary of the silent-reset-during-the-transition debugging. It's also a prerequisite for the bare-metal answer above. I've noted the current low-half
layout as a known limitation in the README too.

▎ Judge the output - Oh, I'm definitely judging

Please do - it's already produced one real fix and a clear roadmap. Thank you for the time.

Re: TanOS 0.1 - Rust microkernel: ring-3 isolation, synchronous IPC, process reincarnation on QEMU/KVM

Posted: Sun Jun 14, 2026 6:27 am
by dseller
Octocontrabass wrote: Sat Jun 13, 2026 7:56 pm
TanOS wrote: Sat Jun 13, 2026 3:32 pmIt boots under QEMU/KVM
Does it work on bare metal or any other VMs?
TanOS wrote: Sat Jun 13, 2026 3:32 pmFault isolation + reincarnation — ring-3 crash kills only that process; kernel restarts it from its image while other processes survive
How does that impact IPC for the surviving processes? Since your kernel is a microkernel, presumably other processes will still try to communicate with the process that crashed.
TanOS wrote: Sat Jun 13, 2026 3:32 pmint 0x80 syscall
Why do your system calls use INT 0x80 instead of SYSCALL?
TanOS wrote: Sat Jun 13, 2026 3:32 pmkernel identity-mapped
This will prevent your OS from booting on many systems, possibly including QEMU depending on how it's configured. Normally you want your kernel to use the higher half of the address space, to keep it cleanly separated from userspace in the lower half. On x86-64 specifically, you'd also want to link the kernel to run in the highest 2GiB of the address space (at or above 0xFFFFFFFF80000000) since it allows the compiler to produce smaller code.
TanOS wrote: Sat Jun 13, 2026 3:32 pmJudge the output
Oh, I'm definitely judging it.
It looks like you're talking to an LLM here.

Re: TanOS 0.1 - Rust microkernel: ring-3 isolation, synchronous IPC, process reincarnation on QEMU/KVM

Posted: Sun Jun 14, 2026 1:56 pm
by Octocontrabass
TanOS wrote: Sun Jun 14, 2026 3:46 amTo be precise about what that does and doesn't do yet, though: the caller gets an error and must retry - the kernel does not transparently re-bind it to the reincarnated server, which comes back with a new PID and no inherited service identity.
So your "process reincarnation" is just restarting the process if it dies? That doesn't sound new or unique to me.
TanOS wrote: Sun Jun 14, 2026 3:46 amint 0x80 was the simpler bring-up
I suppose that's fitting.
TanOS wrote: Sun Jun 14, 2026 3:46 ammostly I'm wary of the silent-reset-during-the-transition debugging.
Perhaps it would be easier to debug if you wrote the code yourself.

Re: TanOS 0.1 - Rust microkernel: ring-3 isolation, synchronous IPC, process reincarnation on QEMU/KVM

Posted: Mon Jun 15, 2026 6:48 pm
by TanOS
I corrected the conflation in the title, the post, and the repo (link https://github.com/hawkaiglai/tanos/commit/68fb7ad).
- Thanks for pointing this Octocontrabass.