Zambesii kernel: SMP, ccNUMA, Uniform Driver Interface

[latentprion]

Yo,

So, I've been working on this kernel for some years now, and I've decided I can do a release now, mostly because I'm about to abandon the project soon, because my goals have changed and I'm going to be undertaking a new project based on the seL4 capability-based microkernel, for various reasons. Zambesii has some really neat features, and it was just about ready for a version 0.01 release, when I had finished the Uniform Driver Interface implementation. This was not a toy project. Zambesii was developed and tested to work on Bochs, Qemu, VMWare Player, Virtual-Box, and Microsoft Virtual-PC, as well as multicore real hardware machines. I was very meticulous about ensuring that it worked in as many environments as I was able to personally test on. I don't recall its working status since I haven't touched it in some months, but it's not one of those "It works in Qemu only" type of "GDT + IDT + Hello World" projects.

The kernel is portable and scalable, with an (almost) fully asynchronous API. The only architecture it's ported to is IA-32, but it was written and designed to be portable, so porting it to other architectures shouldn't be difficult (and should be easy, and was intended to be easy). Something perhaps interesting is that it's written in C++. The build system supports G++ and CLang++, but the binary that's produced by CLang++ doesn't run though. It fails around the constructor calling sequence, which probably means I just had to get crt*.o working and then do some extra debugging and I would have had CLang++ support.

It's very much an Eleanore Semaphore type of project, because you won't be seeing any kind of flashy graphics, etc, here. I was very focused on getting the internals right.

A non-exhaustive list of the features is:

• Symmetric multiprocessing, tested on real hardware.
• Cache-coherent Non-Uniform Memory Access, tested in Qemu only, because I have no access to a ccNUMA machine to test it on. But Geist did allow me to test on a NUMA machine at his workplace.
• Demand paged kernel heap.
• CPU hotplug (untested, but the code is there).
• Memory hotplug (untested, but the support is there).
• Multiprocessing, and multithreading.
• Soft real-time priority-based scheduler.
• Native Uniform Driver Interface environment implementation.
• AP CPUs boot in parallel asynchronously. When I got this done, I remember being pretty proud of it.
• Other stuff I don't remember

The UDI environment implementation is pretty much on the brink of completion: I have UDI processes able to be spawned and executed according to the UDI specification, and a udiprops binary compiler and parser, and other neat stuff. The remaining work for my UDI implementation was for me to write the UDI service calls: right now a UDI driver will be initialized, and begin executing, and then be unable to make any service calls to the OS, so the next step was to implement the UDI syscalls, and that would have hopefully been the spark that would have got UDI going in the hobbyist community and broken the "Chicken and egg" problem. I plan to use UDI in my next project, though.

I may or may not complete the UDI implementation in this kernel and possibly port this kernel to ARM before moving on, but that remains to be seen. Feel free to comment, ask questions, like, subscribe, upvote, and share. https://github.com/latentPrion/zambesii

--Latentprion

[Love4Boobies]

How can you not recall its features and working status after just a few months? You were involved with this project for a few years, as you say.

Also, why the new account? Did you get banned or anything?

[pecan]

Wow, this is a massively impressive project. I've been digging around the source and I like what I see. I'd like to tinker with it in the coming weeks/months/eventually.

i'm also really interested to see what your seL4 project turns into. You helped me get set up with learning seL4 a while back. Should be fun to see what you do with it.

[latentprion]

@L4B we don't mention my adolescent account lol, it wasn't banned, but I just felt like I wanted to leave it behind ^_^

@pecan: yup, I'm really sold on the seL4 kool-aid too, and I'm excited to pitch my tent with the giants, Gernot Heiser and Gerwin Klein, ze German geniuses :3

[pecan]

So far I've successfully built Zambesii on FreeBSD with minor modifications. I documented that here. It was pleasantly easy.

If I want to get a little acquainted with the internals, what might be a good thing to hack on?

[latentprion]

Yo,

Hey, so that's a really good question since I never really budgeted for people to actually be trying to use it until the v0.01 release, but the thing is, it's fairly close to that now anyway -- the problem is that userspace hasn't been bootstrapped (there's no "init" process, so to speak), so there's nothing fun to try or do in userspace yet: until I actually finish the v0.01 release, there's only heavy-lifting to be done in the kernel itself to bring it to the point where I can point at something cool in userspace and say, "Sure, try doing this, or that!". But, since there's a bit of interest being shown, I'll finish up the project and do a v0.01 release now

But, if you're interested in playing around with the kernel features, I'll go through the code tonight and try to think of some cool things you can try, by calling some functions from within the kernel "main()" function, basically to trigger some actions or other from the kernel that would let you observe the features at work

Gimme a bit of time to refresh myself on the codebase and I'll be back within the next 24 hours, give or take;
--Latentprion

[latentprion]

Yo,

For now, enjoy some photos of the kernel on real hardware, I forgot I'd made these for my resume:

http://imgur.com/a/44xV8#0
http://imgur.com/a/hlDt8/

Not very flashy, but cool I hope, nonetheless: most of them are photos of the kernel on real hardware, because I put a lot of effort into ensuring that it was a real-hardware viable kernel, and not just an emulator-only thing

--Latentprion