Announcing: Progsbase/Infracore: a unikernel-like OS/NoOS

[martinfjohansen]

I have been working on an entirely new kind of design for an OS for many years. The goal is to make an OS that is very easy to understand, control and make secure. The OS can be deployed to one machine or over many machines. The programs are developed in the same way regardless. The OS can also run inside another OS. It is very back-to-the-basics and old-school in certain respects, but cloud-native and modern in other respects.

The basic idea of the OS is as follows:

* All programs are developed and tested as single thread programs with mocks. This gives completely deterministic running of tests.
* The image for the computer is built and deployed as a whole, then booted into. (Like unikernels.) Or many images can be built for many computers to boot into and run as a unified, multi-core system.
* All parts of the computer are seen as devices:

- All available cores that can run a single thread is regarded as a Processing Unit (PU). Each of these run a single program.
- All disks are regareded as arrays of bytes.
- All screens are regarded as able to display an image at any one time.
- Etc.

* A single program occupies a single Processing Unit (PU) at all times. When running inside another OS, a single program runs in a process.
* Each PU is connected to devices. The devices connected are available to the program running on that PU.
* To make a runnable program, the device mocks must be replaced by drivers. For example:

- a disk device is supported by a block device.
- another PU is supported by shared memory and interrupts.
- another PU on another computer is supported by a network card driver.
- a screen is supported by a screen driver.
- etc.

* Example 1: A program can access a file system as follows: A disk is called "Disk 1". A program running on "PU 1" is connected to the disk. "PU 2" is connected to "PU 1". "PU 1" servers a file API to "PU 2" and accesses "Disk 1" as a raw block device. "PU 2" interacts with "PU 1" using the file system API exposed by it.

* Example 2: Four computers expose a webserver API on "Computer 1/PU 1", "Computer 2/PU 1" etc. Each of these are connected to a memory cache on "Computer 1/PU 2", "Computer 2/PU 2" etc. Each of these are connected to a computer with the disk on it, "Computer 5". Computer 5 has a sequencer on "PU 1". It handles one request from the memory caches at a time. It is connected to the for memory caches, and sequences requests onwards to a file system running on "PU 2" which is connected to a disk on "Disk 1".

* Example 3: Three disks are connected to "PU 1" running a RAID 1 program. "PU 2" is connected to "PU 1" and interacts with "PU 1" as a disk, a virtual disk.

Let me know if you would like to hear more examples of how IT-systems are built on this model. As far as I know, all kinds of IT-systems can be built on this model: Large distributes systems, network disks, redundancy and replication, load balancing etc. etc.

Development started in 2016.

Most code is open source and available under the MIT license.

I am unsure about the name, I have been toying around with calling it "progsbase", "infracore", "core" or "NoOS". (The idea of calling it a NoOS comes from the fact that there is no fixed set of code that need to be deployed, all code is selectively built into the image, as if having no operating system.)

Code available here for most parts developed so far under the MIT license:
- https://repo.progsbase.com/repoviewer/c ... structure/
- https://repo.progsbase.com/repoviewer/c ... singunits/
- https://repo.progsbase.com/repoviewer/c ... nce.disks/

A video introduction:
- https://www.youtube.com/watch?v=qylpgzn ... 1ErfHdjQ_f

An online platform for running the OS in the browser:
- https://www.progsbase.com/noos/

A bug bounty program for the OS components:
- https://www.progsbase.com/bug-bounty/

An ISA for coding the OS:
- https://www.progsbase.com/isa/

I hope you found it interesting, and let me know if you have questions or would like clarifications. The website https://www.progsbase.com/ is set up using the architecture as Example 2 above.

[IanSeyler]

Very interesting! This seems similar to the Parallax system I worked on years ago. Do you have this running on physical/virtual hardware directly?

[BigBuda]

Very interesting! This seems similar to the Parallax system I worked on years ago. Do you have this running on physical/virtual hardware directly?

Hey Ian, nice to see you here. What happened to Parallax? You stopped developing it?

[IanSeyler]

Hey Ian, nice to see you here. What happened to Parallax? You stopped developing it?

It was developed as a proof of concept for a company down in the USA. The funding never came so I stopped developing it to focus on BareMetal OS instead.

-Ian

[martinfjohansen]

Very interesting! This seems similar to the Parallax system I worked on years ago. Do you have this running on physical/virtual hardware directly?

Hi, glad to hear you found it interesting!

I have not prioritized making it run on physical/virtual hardware directly yet, but that would not take too much effort and would be quite interesting. All components are ready to be used in that mode already, only a few drivers needed. A major benefit of this would be security. One could run an extremely minimal set of components, just those needed to deliver the services in question.

What about having a chat about this and the OSes you have been involved in over a Google meet or similar? I'd be happy to discuss it here as well of course.

[martinfjohansen]

I had a talk with Ian Seyler, and it was clear that the Progsbase/Infracore components can run on top of BareMetal-OS. I have started making the required thin layer of abstraction to make that happen.

Hopefully, this can shortly realize the vision of making an image of just those software components needed to run a service, and absolutely no other bloat. That will be great!

After this is done, the plan is to put services into production. I run a cloud-provider company here in Norway where I live, so I have access to both customers and bare metal ressources to make that happen.