Virtualizing kernels anyone ?

[ineo]

I was wondering if some of you were working on a virtualizing kernel. I've written some parts of my kernel, and now I am interested in having enough abstraction to be hardware independant. I even think of it to create a distributed OS using virtualization.

Do you have some experience of that kind ? Have you any pointer that could help me ?

Thanks.

[mystran]

I was wondering if some of you were working on a virtualizing kernel. I've written some parts of my kernel, and now I am interested in having enough abstraction to be hardware independant. I even think of it to create a distributed OS using virtualization.

I believe that most of us probably think of either virtualizing or distribution at some point, but the fundamental truth to remember is that every layer of abstraction add some overhead. Ofcourse if the overhead is small enough (easy to implement, small compile-time penalty, no runtime-penalty) then it can be ignored, but otherwise one has to consider whether the benefits outweight the cost.

Virtualization as in basic hardware abstraction is more or less common practice in modern operating systems. The single piece of hardware dependency (ignoring drivers) is usually CPU. Even then, many OS's can be ported to new CPU types by simple writing some abstraction code, and compiling for the new instruction set.

To avoid the need to recompile for different instruction sets, you have to standardize on some non-native program representation (such as bytecode) and either interpret that, or compile it on the fly (called just-in-time or JIT). Example of this approach is Java, and while Java is just a platform, there are operating system based on the idea that the OS consist of the Java platform, plus the minimum amount of management code needed to support it, and provide hardware drivers. Such system is effectively hardware independent.

Distributed computing is a bit different, although related beast. All other issues aside, I think the biggest problem for general purpose operating systems going distributed is speed and reliability: networks are slow and unreliable, with unexpected delays and failures.

Building a distributed platform on top of some general purpose platform can be a nice idea though. There are some Java "OS" which provide a distributed virtual OS running on top of some normal Java-capable OS.

Finally, there's always the possibility of extending regular OS with some distributed features. Possibilities range from TCP/IP support, through component systems (such as SUNRPC (eg NFS), COM+, CORBA, ICE, ...) to distributed operating systems such as Plan 9, where single processes can be migrated from one system to another.

Distributed computing however is a difficult beast to tame. I'd say that in some ways it's even harder to write a distributed system than a regular operating system. In any case, I'd build a basic operating system AND a normal distributed system first, before trying to merge them.

[Legend]

For most pieces of a distributed system, it is a better task to provide clustering for the application level ..
For example your kernel should be responsible to get the local machine under control, and not a lot do even that well ...

Look for example at IBM WebSphere on top of Linux/AIX/Windows etc. ...