Announcing: Progsbase/Infracore: a unikernel-like OS/NoOS
Posted: Thu Jun 22, 2023 5:07 am
I have been working on an entirely new kind of design for an OS for many years. The goal is to make an OS that is very easy to understand, control and make secure. The OS can be deployed to one machine or over many machines. The programs are developed in the same way regardless. The OS can also run inside another OS. It is very back-to-the-basics and old-school in certain respects, but cloud-native and modern in other respects.
The basic idea of the OS is as follows:
* All programs are developed and tested as single thread programs with mocks. This gives completely deterministic running of tests.
* The image for the computer is built and deployed as a whole, then booted into. (Like unikernels.) Or many images can be built for many computers to boot into and run as a unified, multi-core system.
* All parts of the computer are seen as devices:
- All available cores that can run a single thread is regarded as a Processing Unit (PU). Each of these run a single program.
- All disks are regareded as arrays of bytes.
- All screens are regarded as able to display an image at any one time.
- Etc.
* A single program occupies a single Processing Unit (PU) at all times. When running inside another OS, a single program runs in a process.
* Each PU is connected to devices. The devices connected are available to the program running on that PU.
* To make a runnable program, the device mocks must be replaced by drivers. For example:
- a disk device is supported by a block device.
- another PU is supported by shared memory and interrupts.
- another PU on another computer is supported by a network card driver.
- a screen is supported by a screen driver.
- etc.
* Example 1: A program can access a file system as follows: A disk is called "Disk 1". A program running on "PU 1" is connected to the disk. "PU 2" is connected to "PU 1". "PU 1" servers a file API to "PU 2" and accesses "Disk 1" as a raw block device. "PU 2" interacts with "PU 1" using the file system API exposed by it.
* Example 2: Four computers expose a webserver API on "Computer 1/PU 1", "Computer 2/PU 1" etc. Each of these are connected to a memory cache on "Computer 1/PU 2", "Computer 2/PU 2" etc. Each of these are connected to a computer with the disk on it, "Computer 5". Computer 5 has a sequencer on "PU 1". It handles one request from the memory caches at a time. It is connected to the for memory caches, and sequences requests onwards to a file system running on "PU 2" which is connected to a disk on "Disk 1".
* Example 3: Three disks are connected to "PU 1" running a RAID 1 program. "PU 2" is connected to "PU 1" and interacts with "PU 1" as a disk, a virtual disk.
Let me know if you would like to hear more examples of how IT-systems are built on this model. As far as I know, all kinds of IT-systems can be built on this model: Large distributes systems, network disks, redundancy and replication, load balancing etc. etc.
Development started in 2016.
Most code is open source and available under the MIT license.
I am unsure about the name, I have been toying around with calling it "progsbase", "infracore", "core" or "NoOS". (The idea of calling it a NoOS comes from the fact that there is no fixed set of code that need to be deployed, all code is selectively built into the image, as if having no operating system.)
Code available here for most parts developed so far under the MIT license:
- https://repo.progsbase.com/repoviewer/c ... structure/
- https://repo.progsbase.com/repoviewer/c ... singunits/
- https://repo.progsbase.com/repoviewer/c ... nce.disks/
A video introduction:
- https://www.youtube.com/watch?v=qylpgzn ... 1ErfHdjQ_f
An online platform for running the OS in the browser:
- https://www.progsbase.com/noos/
A bug bounty program for the OS components:
- https://www.progsbase.com/bug-bounty/
An ISA for coding the OS:
- https://www.progsbase.com/isa/
I hope you found it interesting, and let me know if you have questions or would like clarifications. The website https://www.progsbase.com/ is set up using the architecture as Example 2 above.
The basic idea of the OS is as follows:
* All programs are developed and tested as single thread programs with mocks. This gives completely deterministic running of tests.
* The image for the computer is built and deployed as a whole, then booted into. (Like unikernels.) Or many images can be built for many computers to boot into and run as a unified, multi-core system.
* All parts of the computer are seen as devices:
- All available cores that can run a single thread is regarded as a Processing Unit (PU). Each of these run a single program.
- All disks are regareded as arrays of bytes.
- All screens are regarded as able to display an image at any one time.
- Etc.
* A single program occupies a single Processing Unit (PU) at all times. When running inside another OS, a single program runs in a process.
* Each PU is connected to devices. The devices connected are available to the program running on that PU.
* To make a runnable program, the device mocks must be replaced by drivers. For example:
- a disk device is supported by a block device.
- another PU is supported by shared memory and interrupts.
- another PU on another computer is supported by a network card driver.
- a screen is supported by a screen driver.
- etc.
* Example 1: A program can access a file system as follows: A disk is called "Disk 1". A program running on "PU 1" is connected to the disk. "PU 2" is connected to "PU 1". "PU 1" servers a file API to "PU 2" and accesses "Disk 1" as a raw block device. "PU 2" interacts with "PU 1" using the file system API exposed by it.
* Example 2: Four computers expose a webserver API on "Computer 1/PU 1", "Computer 2/PU 1" etc. Each of these are connected to a memory cache on "Computer 1/PU 2", "Computer 2/PU 2" etc. Each of these are connected to a computer with the disk on it, "Computer 5". Computer 5 has a sequencer on "PU 1". It handles one request from the memory caches at a time. It is connected to the for memory caches, and sequences requests onwards to a file system running on "PU 2" which is connected to a disk on "Disk 1".
* Example 3: Three disks are connected to "PU 1" running a RAID 1 program. "PU 2" is connected to "PU 1" and interacts with "PU 1" as a disk, a virtual disk.
Let me know if you would like to hear more examples of how IT-systems are built on this model. As far as I know, all kinds of IT-systems can be built on this model: Large distributes systems, network disks, redundancy and replication, load balancing etc. etc.
Development started in 2016.
Most code is open source and available under the MIT license.
I am unsure about the name, I have been toying around with calling it "progsbase", "infracore", "core" or "NoOS". (The idea of calling it a NoOS comes from the fact that there is no fixed set of code that need to be deployed, all code is selectively built into the image, as if having no operating system.)
Code available here for most parts developed so far under the MIT license:
- https://repo.progsbase.com/repoviewer/c ... structure/
- https://repo.progsbase.com/repoviewer/c ... singunits/
- https://repo.progsbase.com/repoviewer/c ... nce.disks/
A video introduction:
- https://www.youtube.com/watch?v=qylpgzn ... 1ErfHdjQ_f
An online platform for running the OS in the browser:
- https://www.progsbase.com/noos/
A bug bounty program for the OS components:
- https://www.progsbase.com/bug-bounty/
An ISA for coding the OS:
- https://www.progsbase.com/isa/
I hope you found it interesting, and let me know if you have questions or would like clarifications. The website https://www.progsbase.com/ is set up using the architecture as Example 2 above.