Page 1 of 1

NyanOS and XNC

Posted: Fri Oct 03, 2014 6:17 pm
by Roman
Hi, everybody. I want to announce my operating system project. It is called NyanOS. Nyan stands for Not Yet Another *Nix. The main goals are security, stability, full protection and separation.

Note: this OS is mostly just a concept for now.

Filesystems.
-Boot filesystem - stores the bootloader, the virtual machine and the kernel.
-Data filesystem - other software.

XNC virtual machine.
The kernel is executed under a virtual machine. It's purpose is to manage memory access. The architecture will be CISC-like.

Kernel.
I preferred the microkernel architecture for my kernel. The kernel starts userspace servers, which control hardware.

Root program.
The root program is the only program having full access to the real hardware (via VM calls) and virtual machine. It's memory region can only be accessed (executed, written or read) by itself, so no runtime kernel patching.

Memory region.
Every program has it. A memory region is a contiguous part of physical memory, access permissions can be controlled only by the root program. By default, only the owner has access to it.

Boot sequence.
Bootloader -> virtual machine -> kernel (aka root program or ROM) -> hardware search and driver probing -> system software -> other apps.

Context.
Every program has a context. It describes its memory region borders, holds register state, etc. Contexts are manipulated by the virtual machine.

How do programs interact with libraries?
When a library's code is loaded into memory, everything has RX access to it. If library requires writing, the kernel allocates memory in program's private address space.

Direct hardware access API (DHA).
Every operating system needs a way to extend hardware support, but how can a managed OS access real hardware? You can say, it's possible to use native code, but it is a huge security hole. DHA allows the root program to access real hardware (registers, physical memory).

Why DHA API is not a big security hole?
The reader probably wants to ask this question. The answer is, that the kernel controls, what servers (drivers) want to access. Servers cannot control same ports, memory, etc. For example, we have two servers: the first is good, when the second wants to do bad things. The kernel reads their manifests, it discovers, what hardware they want to control, if there's a match between two different servers, it loads the one, which was installed earlier. I think, this scheme partly eliminates the problem, but the main vulnerability is still in the user's brain. Software can't control it, prevent it from wanting to load a malicious driver or even typing "rm -rf /*". Also, DHA doesn't allow programs to access memory regions, they don't own (only ownerless), i. e. they can't use it for defeating memory protection.

Binary format support.
It will be easily extensible, just like hardware support.

PS. I know, that these concepts are not new to the world. I just wanted to tell you about, what I'm working on and hear your thoughts and opinions on this.


About the XNC (XNC's Not C) programming language.

The application root.
The application root is the descriptor of all parts of the program: GUI, TUI, back-ends.
Future code will look something like this:

Code: Select all

class ApplicationRoot: argv {
    func LoadGUI: {
        loadUI: ProgramGUI; //ProgramGUI should be in a separate file. It does all GUI-related: handles UI events, draws UI elements, etc...
        activateUI: ProgramGUI;

        return UI_LOAD_SUCCESS; //Or UI_LOAD_FAILURE. UI_LOAD_FAILURE doesn't cause the app to crash. Failure can mean, that program just doesn't have any GUI.
    }

    func LoadTUI: {
        loadUI: ProgramTUI; //ProgramTUI should be in a separate file.
        activateUI: ProgramTUI;

        return UI_LOAD_SUCCESS; //Or UI_LOAD_FAILURE. UI_LOAD_FAILURE doesn't cause the app to crash. Failure can mean, that program just doesn't have any TUI.
    }

    //CLUI (command line user interface) is not interactive, so it is not a view, but it still can load ProgramTUI's.
    func LoadCLUI: {
        CLUIMain: argv;

        return UI_LOAD_SUCCESS;
    }

    class  {
        loadBE: ProgramBackEnd;
        startUI: ;
    }
}
ProgramGUI pseudo-example:

Code: Select all

class ProgramGUI {
    UILabel helloWorld = [alloc UILabel];
    helloWorld.title = @8"Hello, World!"; //@8"" - UTF-8 string.

    UIEvent onKeyDown: keyCode {
        //Check keyCode, do stuff.
        ...
    }

    helloWorld.onClick = [alloc code]: {
        UIAlert helloAlert = [alloc UIAlert];
        helloAlert.title = @8"Hello, World!";
        [push helloAlert.buttons]: [alloc UIAlertButton];
        helloAlert.buttons#1.title = @8"OK";
        [display helloAlert];
        [wait helloAlert];
        if (helloAlert.buttons#1.pressed)
        {
            //Do something.
            [exit ProgramBackEnd];
        }
    }
}
ProgramTUI pseudo-example:

Code: Select all

//Similiar to ProgramGUI.
...
BackEnd pseudo-example:

Code: Select all

class ProgramBackEnd {
    func Exit:
    {
        //Exit the application. Resource deallocation is done by kernel.
        ...
    }
}
It is similar to Objective-C (and Java?), isn't it? :D

App manifests.
This is similar to OS X and iOS. In an application manifest you'll be able to select status bar style you want (or even hide it), specify an icon and a lot of other things. Even variables for the application, no struggling with parsing config files at all, everything is loaded just for you.

Localization bundles.
This is also reminds about OS X and iOS. This subsystem will be like in iOS.

I will post all my new thoughts here.

Re: NyanOS and XNC.

Posted: Sat Oct 04, 2014 3:09 am
by embryo
@Roman
Interesting work :)
Roman wrote:Filesystems.
-Boot filesystem - stores the bootloader, the virtual machine and the kernel.
-Data filesystem - other software.
You can extend it farther. Not only system can have it's personal disk area, but so can every program. It allows to isolate applications and prevent many security issues.
Roman wrote:The root program is the only program having full access to the real hardware (via VM calls) and virtual machine.
Here the difference between the root program and the virtual machine is unclear. What is the VM, what it is for?
Roman wrote:Also, DHA doesn't allow programs to access memory regions, they don't own (only ownerless)
Access to a free region is a security hole. The free region can hold some critical information that a finished process has used.
Roman wrote:About the XNC (XNC's Not C) programming language.
I see memory allocation operators here, but where are deallocators? Is there a garbage collector? If it is, then why there is allocation operators? It it is not, then where are deallocations?

And about the syntax - why it is as it is? What was a driver behind such design?

Re: NyanOS and XNC.

Posted: Sat Oct 04, 2014 6:29 am
by Roman
embryo wrote:@Roman
Interesting work :)
That's just only concepts for now.
embryo wrote:
Roman wrote:The root program is the only program having full access to the real hardware (via VM calls) and virtual machine.
Here the difference between the root program and the virtual machine is unclear. What is the VM, what it is for?
The core and the most low-level part of NyanOS is the VM. The root program (aka kernel) is ran under it.
embryo wrote:
Roman wrote:Also, DHA doesn't allow programs to access memory regions, they don't own (only ownerless)
Access to a free region is a security hole. The free region can hold some critical information that a finished process has used.
Drivers only need to access memory physically to control memory mapped hardware, right? Virtual machine's virtual address space (where all programs are loaded) doesn't include this memory, so drivers won't be able to access it (except for their own regions).
embryo wrote:
Roman wrote:About the XNC (XNC's Not C) programming language.
I see memory allocation operators here, but where are deallocators? Is there a garbage collector? If it is, then why there is allocation operators? It it is not, then where are deallocations?

And about the syntax - why it is as it is? What was a driver behind such design?
Yes, [alloc ObjectType] means allocate memory and also run object's initialization code. This program is small, so it just doesn't deallocate anything, memory will anyway be freed by kernel after program's finished (the memory region will just be removed), so no need for memory management in a hello-world-program. Of course, manual memory deallocation will be supported for big programs like drivers, heavy text editors, etc.

About the syntax, are you asking about things like application root, ProgramGUI and others? If yes, it is needed for making code more structured and beautiful. Also, if the OS is in graphical mode, the GUI is started, but if the app doesn't have it, OS spawns a terminal emulator, which runs the program in TUI mode, but if the program doesn't even have a TUI, it is started in CLUI mode. The user will be able to set priorities manually. In text mode CLUI is preferred by default.

Re: NyanOS and XNC.

Posted: Sat Oct 04, 2014 7:50 am
by max
Are you planning to have one big virtual machine process that contains all processes as threads? Or does each process have its own virtual machine instance? Are you planning to utilize memory security techniques like paging?

The issue I see is that for example a driver that crashes for some reason and corrupts memory could crash your entire system.

Re: NyanOS and XNC.

Posted: Sat Oct 04, 2014 8:44 am
by Roman
max wrote:Are you planning to have one big virtual machine process that contains all processes as threads? Or does each process have its own virtual machine instance? Are you planning to utilize memory security techniques like paging?

The issue I see is that for example a driver that crashes for some reason and corrupts memory could crash your entire system.
There will be only one "real" virtual machine process, which will manage all virtual processes. If a virtual process crashes, VM just frees it's region. Don't forget, that drivers will be written in managed code too, so they cannot corrupt memory.

Re: NyanOS and XNC.

Posted: Sun Oct 05, 2014 3:02 am
by embryo
Roman wrote:The root program is the only program having full access to the real hardware (via VM calls) and virtual machine.
And how applications will be executed? Will it be a compiled version (JIT) or some kind of interpreter VM? What form the programs will be in? Text, bytecode, binary?
Roman wrote:Drivers only need to access memory physically to control memory mapped hardware, right?
I don't know is it right for your OS or not. Any application (in theory) can use free memory, so I just don't understand why do you think about drivers only.
Roman wrote:Virtual machine's virtual address space (where all programs are loaded) doesn't include this memory, so drivers won't be able to access it (except for their own regions).
If one driver can access region used by another driver - can it read a critical information?
Roman wrote:About the syntax, are you asking about things like application root, ProgramGUI and others?
No, I see some text is a bit hard to read, like this:

Code: Select all

helloWorld.onClick = [alloc code] {
        UIAlert helloAlert = [alloc UIAlert].initWithTitle: [alloc UTF8String] "Hello, World!" Text: [alloc Nothing] OKButtonText: [alloc UTF8String] "OK" CancelButtonText: [alloc UTF8String] "Cancel" OtherButtonsText.[WhateverButtonText: [alloc UTF8String] "Whatever" AnythingButtonText: [alloc UTF8String] "Anything"];
        [display helloAlert];
        [wait helloAlert];
        if (helloAlert.OKButton.Pressed) //OKButton is created automatically by display.
        {
            //Do something.
            BackEnd.ExitApp();
        }
    }
Why here some code is surrounded by square brackets while another part is surrounded with curly braces and yet another part goes without any brackets? Why there is a lot of text in one line? What do the nested square brackets mean? Why C-like style is changed in such a manner? What is the reason for it?

Re: NyanOS and XNC.

Posted: Sun Oct 05, 2014 3:55 pm
by Roman
embryo wrote:
Roman wrote:The root program is the only program having full access to the real hardware (via VM calls) and virtual machine.
And how applications will be executed? Will it be a compiled version (JIT) or some kind of interpreter VM? What form the programs will be in? Text, bytecode, binary?
Program will be compiled into bytecode and of course, I'll add JIT.
embryo wrote:
Roman wrote:Drivers only need to access memory physically to control memory mapped hardware, right?
I don't know is it right for your OS or not. Any application (in theory) can use free memory, so I just don't understand why do you think about drivers only.
Basic applications use their memory regions. Drivers may need to access specific parts of memory.
embryo wrote:
Roman wrote:Virtual machine's virtual address space (where all programs are loaded) doesn't include this memory, so drivers won't be able to access it (except for their own regions).
If one driver can access region used by another driver - can it read a critical information?
It can't access it at all. Driver's code and data are stored in protected memory region, nothing else can access them.
embryo wrote:
Roman wrote:About the syntax, are you asking about things like application root, ProgramGUI and others?
No, I see some text is a bit hard to read, like this:

Code: Select all

helloWorld.onClick = [alloc code] {
        UIAlert helloAlert = [alloc UIAlert].initWithTitle: [alloc UTF8String] "Hello, World!" Text: [alloc Nothing] OKButtonText: [alloc UTF8String] "OK" CancelButtonText: [alloc UTF8String] "Cancel" OtherButtonsText.[WhateverButtonText: [alloc UTF8String] "Whatever" AnythingButtonText: [alloc UTF8String] "Anything"];
        [display helloAlert];
        [wait helloAlert];
        if (helloAlert.OKButton.Pressed) //OKButton is created automatically by display.
        {
            //Do something.
            BackEnd.ExitApp();
        }
    }
Why here some code is surrounded by square brackets while another part is surrounded with curly braces and yet another part goes without any brackets? Why there is a lot of text in one line? What do the nested square brackets mean? Why C-like style is changed in such a manner? What is the reason for it?
Fixed. Brackets mean, that something inside must be done first.

Re: NyanOS and XNC.

Posted: Mon Feb 09, 2015 12:30 pm
by bashcommando
It looks a LOT like Java. =D>

Re: NyanOS and XNC

Posted: Tue Feb 10, 2015 3:47 am
by Combuster
It looks a LOT like Java.
More like objective C with the class and function names reversed... even though that language in particular can write this a lot shorter.