New OS idea

[AxelDominatoR]

Hi all!

I'm new to this forum, and this is my first post.

I have in mind a new type of operating system, and I wanted to ask around if it could be done and how well it will work.

Here's a little list of main features and differences between my os and other oses:

It's not a monolithic kernel, it's not a microkernel. It's a self-recompiling kernel. ( more on this later... )
It would only use kernel mode. No user space.
No need to use MMU... it handles memory in a different way.

Ok so...
The kernel and all the programs that will run under it, are a SINGLE, self-recompiling program.
I'm currently developing a new macroassembly-style language to use it.

When the kernel loads, the compiler will automatically find what type of architecture it is running on, then first recompile itself ( so we'll have an optimized compiler that will compile only for that arch ) and then recompile the kernel on-the-fly for that architecture. Then, all the drivers or programs ( called simply routines/subroutines at the moment ) will be compiled and loaded on-the-fly.

I think ( I'm not a low-level programming expert ) that this will speed up things because: there will not be arguments passing, any program can access directly all the memory ( mapped as a single block ).
It will open doors to a new style of programming, because a program can have as much options as it needs, but be loaded only with the necessary at the moment. Then if you need another option, the compiler will compile and link it at runtime.

Naturally I must consider memory protection to make sure a malicious or bad program will not overwrite reserved memory areas... The compiler will take care of this.

It's only a general description, but I can reply with more and most in-depth arguments if asked.

What do you think?

Thanks for reading all this

[Curufir]

Think we dealt with most of the arguments in a recent post about an interpreted architecture (Look for the topic "A new direction").

Basically what it comes down to is that you can get away with no memory protection so long as you control the language and filesystem. Without a language specifically designed for segregation of code the compiler can't guarantee anything.

[AxelDominatoR]

I read the thread.

Well it seems that we had two similar ideas... ;D
My OS will not be an interpreter.
It will compile, then link and load.
Then the control will FULLY be transferred to the program. I'm thinking about how implement a multitasking environment this way... so...
How about specify jump instructions between programs?

Two small examples:

A single program is running:

Kernel start
bla blab la
A program is requested
Compiler compiles the program then loads and links it
There is a jump to the start of the routine ( program )
until the program ends, all the processor power will go to the program
Then it will return to the kernel.

Ok, this was a single program example. Realistically, an OS will have at least... 3-4 programs running concurrently... so here's how we could act:

Every program tells the compiler the priority it would like to run
The compiler reads the priority, and seeing if it's possible according to the free cpu and priviledge of the program, puts some jumps back to the "multitasking control". This program will only re-jump in turn to the other programs.
CPU used by any of the program will be decided by how much instruction there will be between a jump to the "multitasking control" and the next jump.

This should give full cpu horsepower if a single program is running ( IE: an embedded controller that does only one thing ).
Instead, if you run more than one program, you could specify how much cpu you would like. For example, a 3D rendering program would need most cpu cycles. But maybe a simple email notification will only need 1% of CPU cycles...
so the 3D rendering will have a jump-to-multitasking control once every... say... 100 instructions...
instead the mail will make only one or two instructions and give back to the other programs.

One thing... I honestly don't know how actually multitasking works on modern systems... Maybe it's the same, maybe it's better...
Just my idea

[Candy]

Hi all!

I'm new to this forum, and this is my first post.

I have in mind a new type of operating system, and I wanted to ask around if it could be done and how well it will work.

Here's a little list of main features and differences between my os and other oses:

It's not a monolithic kernel, it's not a microkernel. It's a self-recompiling kernel. ( more on this later... )
It would only use kernel mode. No user space.
No need to use MMU... it handles memory in a different way.

Ok so...
The kernel and all the programs that will run under it, are a SINGLE, self-recompiling program.
I'm currently developing a new macroassembly-style language to use it.

Try searching for all-in-one programs, they were popular during the 1980s somewhere, and the begin of the 1990. Windows 95 proved to the people that used them that the concept was dead and that you'd better have separate programs, because that allows a lot more work to be done. Search for protection and security, and possibly interface-definition and you might find out why I'm complaining.

When the kernel loads, the compiler will automatically find what type of architecture it is running on, then first recompile itself ( so we'll have an optimized compiler that will compile only for that arch ) and then recompile the kernel on-the-fly for that architecture. Then, all the drivers or programs ( called simply routines/subroutines at the moment ) will be compiled and loaded on-the-fly.

Try compiling a linux system from scratch, up to the graphical shell. Now think according to moore's law when a computer will be able to do that in a minute (well, times given: now, a few days, duration until that happens is 1440*3 ~= 4096, 4096 takes 12 doublings, 12 doublings takes 18 years (moore's law) and that ends up being 2022). Then think what the current types will be by then?

I think ( I'm not a low-level programming expert ) that this will speed up things because: there will not be arguments passing, any program can access directly all the memory ( mapped as a single block ).
It will open doors to a new style of programming, because a program can have as much options as it needs, but be loaded only with the necessary at the moment. Then if you need another option, the compiler will compile and link it at runtime.

Demand-paging, dynamic libraries functioning as plugins, these things have been invented and are working right now on your own computer! For programs with loads of useless options, look at GNU software. For limited programs with only one specific use, look at Windows software. (yes I'm sometimes surprised they need multiple tools for dvd ripping, in linux that's a single complex command, in windows a multitude of simple programs that all do practically nothing. Specifically when you want them to do a part of what they normally do, they can't.)

Naturally I must consider memory protection to make sure a malicious or bad program will not overwrite reserved memory areas... The compiler will take care of this.

It's only a general description, but I can reply with more and most in-depth arguments if asked.

char *something;
char *bad_pointer = 0x12345678; // this is a valid address, your compiler would be happy with this
int *something_else = (int *)(((char*)&something) + 4);
*something_else ^= 0xE0000000;
*bad_pointer = 0xFF; // now 0xF2345678 is overwritten. That's system memory.

Can you make a compiler that fishes out ALL possible assignments and checks them?

[AxelDominatoR]

You are right... if we were talking of a normal C programming language
My OS will be based on a proprietary language... sort of Macro Assembler

It's much much much faster in compiling than C
Kernel is much much much simpler than linux one ( come on... linux kernel is HUGE )
Consider that all the drivers will be compiled at runtime only when needed...
in linux you must compile all the modules before you use them...

It's all matter of viewing how the entire system will work

When you start a program in a modern OS, with precompiled binaries, you only loose time to load in memory and start the program.
My OS will compile them at run-time, and this is surely more time-consuming. But you will have an incredible more: efficient, fast, modular, etc system than any other.
Here's an example in simil-c

Let's suppose a program will have two functions. One of them is used almost every time. The other one is used only 2 or three times.

so:

main routine()
{
if ( we should use routine 1 )
{
routine1();
}
else
{
routine2();
}
}

Ok.. so, it's a small time loss the check for the routine to use.
Instead, let's suppose we could have a program that will always use the routine1, and if you want to use routine2 you will simply tell the compiler to take the right code and put in it...
Yeah, you lost about 100 times the time you need to do 1 if... but you must do it ONE time...
This is the philosophy behind my OS... naturally ALL the programs must be projected to reflect this idea...

[mpp.eox3]

time-consuming. But you will have an incredible more: efficient, fast, modular, etc system than any other.

Why will it be more efficient and faster?
I don't think that your Macro Assembler will beat the Intel C/C++ compiler. And if you try hard, compiling your kernel+apps will take much more time than you think you might loose with the traditional approach.
Also, have you already thought of writing optimizing code generators for different cpus like 486, pentium, ppro, p][, p///, p4,... + don't forget AMD..

[Legend]

And throughout this discussion we are back to a safe language (your macro assembler style language would have to be safe for this to work) and jit compiling, however, utilizing byte code might give you the ability to cut the edge to intel compilers, as the impressive results are obtained by multi-pass compiling (with some sort of profiling done between each pass), which should be more practical done on byte code then on any textfile ...

However, the automated cooperative multitasking is interesting, might contain traps. Like, what does the compiler do for complex loops?

[tt]for (int i = 0;i < 100000;i++) {
/* Some very small operation */
}[/tt]

Where to jump back now? I bet you will split that loop into some parts, but what is with loops for complex operations?

Note: What most other OS is called preemptive multitasking, it works without having to rely on the application for it to work, however, might take up more cycles.

[AxelDominatoR]

One method may be this:

for (int i = 0;i < 25000;i++) {
/* Some very small operation */
/* Some very small operation */
/* Some very small operation */
/* Some very small operation */
Jump to the controller
}

If the operation is VERY small...
Naturally it depends on the loop and on the compiler to make the right choices...

Yeah the compiler will be the most difficult part to think...

I think that there are much more efficient ones, anyway....

Returning to the compiler part...
A: My language isn't C... is a macro assembly which is simpler than C
B: Yeah, the kernel will take much time to compile... if you think of a linux kernel with all the drivers and such...
This kernel is designed for efficiency. It takes a little more time to compile every single module you want to load, but it gives away much more modularity, much more speed regarding the way it's designed and the way the programs will be designed...
My OS modules are not the same as a linux kernel module, or other OS DLL or other things...
they are routines and subroutines that are dinamically compiled and loaded into the kernel. I don't know exactly how is done by the linux kernel but here is how it would work for me:

Kernel
Instruction1
Instruction2
Instruction3

The compiler is called to compile and load a new subroutine that at the moment will replace Instruction 3

The compiler then compiles only the part it needs, load it into memory....
Kernel
Instruction1
Instruction2
Routine

Ok here are two ways of loading... the compiler could arrange the instructions sequentially to not make use of a jump
or it could make a simple jump to the new routine location... it looses an instruction, but it gains modularity...

However I don't know if actual modules are done this way...

I repeat, these are only ideas... for now

Thanks for the replies and the interest ^_^

[crc]

In some ways this sounds similar to my OS. Mine is built over a Forth compiler, and applications are compiled when they are loaded. Performance isn't too bad. The compiler isn't optomized much at all, but it can compile a 200k source file in under four seconds. That could be improved further without too much trouble.

ONLY the core functions for the runtime compiler and interpreter are precompiled. Everything else is compiled at startup or runtime. Most of the drivers are written in assembly and Forth. So are all of the applications. And most of the kernel itself is also written in Forth. (The machine code part is 2-4k, depending on the options you include).

I don't restrict memory access, but that would be trivial to do. Just create a list of 'restricted' areas, and spit out an error when the program tries to access them. As to overall security, my OS is very insecure. But I like it that way, because I can access all of my hardware directly and manipulate things at will. On the other hand, I'm considering writing a code scanner that can locate dangerous code sequences before compiling them. That would solve most of the existing security flaws (I think).

Loaded programs become part of the system. The routines that make them up become globally accessible when they are loaded and compiled. Organizing them can be done by loading each program into a named vocabulary (lists of functions and variables). Programs can also be unloaded when you're done with them.

[Legend]

I don't restrict memory access, but that would be trivial to do. Just create a list of 'restricted' areas, and spit out an error when the program tries to access them. As to overall security, my OS is very insecure. But I like it that way, because I can access all of my hardware directly and manipulate things at will. On the other hand, I'm considering writing a code scanner that can locate dangerous code sequences before compiling them. That would solve most of the existing security flaws (I think).

This sounds like reprogramming what the MMU would do for you. And in order to check this I guess you would have to
interpret the code, not run it directly on the cpu? ???

Loaded programs become part of the system. The routines that make them up become globally accessible when they are loaded and compiled. Organizing them can be done by loading each program into a named vocabulary (lists of functions and variables). Programs can also be unloaded when you're done with them.

In some way, you can do similar things with Java, too!
For testing purposed I included the jar-file with Sun's JMX implementation from one Sun Application Server (the one that comes with the J2EE 1.4.2 JDK), was very interesting to see that this worked that easy!

[crc]

This sounds like reprogramming what the MMU would do for you. And in order to check this I guess you would have to
interpret the code, not run it directly on the cpu?

IIRC, the MMU is used in paging or segments. I use a flat memory model (0x1000 = memory location 0x1000; no address translations are needed).

Source code is compiled to native x86 machine code. The idea is that the compiler can check the addresses being read/written to at compile time. Functions that read/write directly to memory locations could also check their arguments at runtime if the need arose. Of course it's possible to bypass checks with the current design, but if I added a code scanner, it wouldn't be. At the moment there are no checks. I won't even add them unless it proves neccessary in the future.