Managed code operating systems

[bloodline]

Hi all,

I noticed that there doesn’t seem to be any discussion on this forum (at least as far as the search function is concerned) regarding Managed Code operating systems.

The only well documented example I’m familiar with is Microsoft’s Singularity https://www.microsoft.com/en-us/researc ... ingularity, and I wonder if members here had any thoughts on the subject, if anyone has any experience I’d love to hear about the pros and cons they found!

Thanks

[nexos]

I don't think its a good idea, as that would make the kernel a lot slower.

[bloodline]

I don't think its a good idea, as that would make the kernel a lot slower.

My thinking is that only “user space” code would be managed, the kernel (a microkernel, or perhaps somewhat hybrid) would be coded more traditionally.

[PeterX]

It seems to me that managed code is simply interpreted bytecode. You would need a tiny core which interprets and the rest can be in that bytecode.

I think with managed code you could have analysis of what the code does (you could have that with native code, too). But in real practice this doesn't seem like a great idea.

Greetings
Peter

[bloodline]

It seems to me that managed code is simply interpreted bytecode. You would need a tiny core which interprets and the rest can be in that bytecode.

I think with managed code you could have analysis of what the code does (you could have that with native code, too). But in real practice this doesn't seem like a great idea.

Greetings
Peter

As I understand it, that is one approach. I can’t see why Compiled code wouldn’t be permissible, if compiled from a “safe” language like C#.

[nullplan]

I noticed that there doesn’t seem to be any discussion on this forum (at least as far as the search function is concerned) regarding Managed Code operating systems.

That would probably be because about half of the forum is trying to reinvent Linux (although some seem to be reinventing OpenBSD, as if it was any difference for this comparison), while the other half is trying to reinvent MS-DOS. That's for the established members. Other than that, we get a lot of newbies here that copied James Molloy's code and now they need help. Am I being hyperbolic?

Things aren't really moving here, is my point. Every new dawn only provides more of the same. The same OSes and the same questions. The biggest innovation currently under consideration is UEFI, an invention from nigh-on two decades ago. UEFI came out right around the time the switch to 64-bit and to multi-CPU happened, and you will notice a staunch refusal in some parts of this community to acknowledge either trend.

I for one would welcome a managed code approach to kernel development, but I fear that it would lead to vendor lock-in. Once your kernel is on .NET and can only receive messages from .NET applications, all of those kind of have to use .NET as well. Whereas, if the kernel is written in C, it can usually receive messages from anything with raw processor access, and can run .NET applications or Java applications or native applications. Or whatever you want, really.

I don't think its a good idea, as that would make the kernel a lot slower.

I think that is too shallow an analysis, and one that the research paper goes out of its way to discredit. Indeed, since it seems to me that privileges are handled entirely in software on that system, it avoids a lot of the hardware problems you have with traditional operating systems. If I understand the paper correctly, they run everything in supervisor mode, but disallow user applications from doing certain things (by not running them if they try to do them). But that means that there are no system calls, there are only normal messages. Therefore, there are no system call switching costs (which depending on architecture can be severe!) Also, they apparently use a single address space, which on a modern 64-bit CPU has no drawbacks to my knowledge (on 32-bit systems you would limit yourself to 4GB of address space for the whole system, and not all of that would be RAM). Therefore, a context switch is as costly as a setjmp()/longjmp(). How is that for speed?

Plus, that OS settles my debate with bzt by requiring processes to be sealed, and no code can be added to a process. A process can start another process, yes, but then they are independent.

My thinking is that only “user space” code would be managed, the kernel (a microkernel, or perhaps somewhat hybrid) would be coded more traditionally.

That is not what they describe in the paper. The kernel is written 90% in "Sing#" (which is modified C#, because as it turns out, C# does allow unsafe operations), the remaining tenth made up of assembly and C++.

It seems to me that managed code is simply interpreted bytecode.

Both IcedTea and Mono have been doing dynamic recompilation forever. That is a process by which native code is created dynamically from bytecode, and is then run at native speed. Since only one class at a time is translated, and only on demand, this approach is feasible even with Java 1.6 or later, and .NET with generics. Whenever a reference is made to a class that is not yet translated, the branch targets will point to the NULL page, causing an exception that is handled by translating that other class.

But in real practice this doesn't seem like a great idea.

I concur, but not for speed reasons. I don't think the vendor lock-in mentioned above is a good thing. And I really don't want the world to be run by Microsoft. The world run by Google is already scary enough.

For now, I think we will have to live with these unsecure systems, and bring our daily sacrifice at the altar of Address Sanitizer. Damn, that thing should be the default!

[PeterX]

That would probably be because about half of the forum is trying to reinvent Linux (although some seem to be reinventing OpenBSD, as if it was any difference for this comparison), while the other half is trying to reinvent MS-DOS. That's for the established members. Other than that, we get a lot of newbies here that copied James Molloy's code and now they need help. Am I being hyperbolic?

Things aren't really moving here, is my point. Every new dawn only provides more of the same. The same OSes and the same questions.

Talking about innovations, some are (differing widely in significance and intensity) :
- GPU improvements
- single-board computers
- internet of things
- quantum computers (unfortunately probably too expensive for people like us)
- virtual machines
- containers (I think especially of Docker which seems cool to me)
- Windows subsystem for Linux
- PiPhone (open source smartphone)
- blockchain

All the things I listed might influence your OSs and the design decisions you make for your OSs. Maybe someone writes a really futuristic OS...

Greetings
Peter

[bloodline]

I noticed that there doesn’t seem to be any discussion on this forum (at least as far as the search function is concerned) regarding Managed Code operating systems.

That would probably be because about half of the forum is trying to reinvent Linux (although some seem to be reinventing OpenBSD, as if it was any difference for this comparison), while the other half is trying to reinvent MS-DOS. That's for the established members. Other than that, we get a lot of newbies here that copied James Molloy's code and now they need help. Am I being hyperbolic?

Lol, yes I did find James’s tutorial and found it useful to help my understanding of the GDT and IDT, but the rest of it diverged from my OS model so I have no idea how good or bad the rest of it is. -edit- also I was annoyed that all his examples use Intel syntax and I’ve settled into using AT&T syntax for now... it’s the little things

I’m not interested in making a UNIX clone, I want to explore a few ideas I’ve had knocking around for a few years, and one of those was thinking about moving memory protection from the hardware domain to the software domain.

Things aren't really moving here, is my point. Every new dawn only provides more of the same. The same OSes and the same questions. The biggest innovation currently under consideration is UEFI, an invention from nigh-on two decades ago. UEFI came out right around the time the switch to 64-bit and to multi-CPU happened, and you will notice a staunch refusal in some parts of this community to acknowledge either trend.

I have yet to explore any of these three “innovations” yet... but I get that some people want to have a “tech ceiling”... I have friends who top out at the C64, and there isn’t really much more you can do without reinventing the wheel on that platform.

I for one would welcome a managed code approach to kernel development, but I fear that it would lead to vendor lock-in. Once your kernel is on .NET and can only receive messages from .NET applications, all of those kind of have to use .NET as well. Whereas, if the kernel is written in C, it can usually receive messages from anything with raw processor access, and can run .NET applications or Java applications or native applications. Or whatever you want, really.

Well, you can’t blame M$ for going down a route that would lead to vendor lock-in... that is the business model that have spent 40 years trying to maintain.

But that doesn’t stop us exploring some of the ideas they were researching.

I don't think its a good idea, as that would make the kernel a lot slower.

I think that is too shallow an analysis, and one that the research paper goes out of its way to discredit. Indeed, since it seems to me that privileges are handled entirely in software on that system, it avoids a lot of the hardware problems you have with traditional operating systems. If I understand the paper correctly, they run everything in supervisor mode, but disallow user applications from doing certain things (by not running them if they try to do them). But that means that there are no system calls, there are only normal messages. Therefore, there are no system call switching costs (which depending on architecture can be severe!) Also, they apparently use a single address space, which on a modern 64-bit CPU has no drawbacks to my knowledge (on 32-bit systems you would limit yourself to 4GB of address space for the whole system, and not all of that would be RAM). Therefore, a context switch is as costly as a setjmp()/longjmp(). How is that for speed?

So, this is what interests me, though I don’t like the “running everything in supervisor mode” concept. I would still expect two levels of privilege. But what is the minimum amount of hardware assisted protection one can get away with (supplemented by software enforced security) , for maximum speed.

The tricky part for me is thinking about how (third party) hardware drivers would work...

Plus, that OS settles my debate with bzt by requiring processes to be sealed, and no code can be added to a process. A process can start another process, yes, but then they are independent.

My thinking is that only “user space” code would be managed, the kernel (a microkernel, or perhaps somewhat hybrid) would be coded more traditionally.

That is not what they describe in the paper. The kernel is written 90% in "Sing#" (which is modified C#, because as it turns out, C# does allow unsafe operations), the remaining tenth made up of assembly and C++.

Microsoft are not the be all and end all of software development I was first introduced to the idea with their Singularity project... but that just got me started thinking about the idea, not wanting to just reimplement their project.

It seems to me that managed code is simply interpreted bytecode.

Both IcedTea and Mono have been doing dynamic recompilation forever. That is a process by which native code is created dynamically from bytecode, and is then run at native speed. Since only one class at a time is translated, and only on demand, this approach is feasible even with Java 1.6 or later, and .NET with generics. Whenever a reference is made to a class that is not yet translated, the branch targets will point to the NULL page, causing an exception that is handled by translating that other class.

One might imagine a system where webm “binaries” are the only supported native code...

But in real practice this doesn't seem like a great idea.

I concur, but not for speed reasons. I don't think the vendor lock-in mentioned above is a good thing. And I really don't want the world to be run by Microsoft. The world run by Google is already scary enough.

For now, I think we will have to live with these unsecure systems, and bring our daily sacrifice at the altar of Address Sanitizer. Damn, that thing should be the default!

[nexos]

That would probably be because about half of the forum is trying to reinvent Linux (although some seem to be reinventing OpenBSD, as if it was any difference for this comparison), while the other half is trying to reinvent MS-DOS. That's for the established members. Other than that, we get a lot of newbies here that copied James Molloy's code and now they need help. Am I being hyperbolic?

Lol, I originally had a mess of Brokenthorn, James Molloy, and Bran. I somehow thought I could make it into a microkernel . I spent most of 2019 wasting hours of coding on this belief, could never figure out the IDT, and barely knew of the Intel Manuals. Oh well, now I am actually making something . And I am able to say I am defying conventional hobby OSes. Back on topic now

My thinking is that only “user space” code would be managed, the kernel (a microkernel, or perhaps somewhat hybrid) would be coded more traditionally.

I have thought about doing that before, making a microkernel out of native code, and then making all drivers in user space using my own managed language. I will probably do that when my current OS is finished .

It seems to me that managed code is simply interpreted bytecode. You would need a tiny core which interprets and the rest can be in that bytecode.

Partially, but the big this about managed languages is a garbage collector, which prevents bad memory access, insures memory doesn't get corrupted, etc.
Personally, any good kernel shouldn't rely on behavior like this and try to do things well from the start.

[bloodline]

My thinking is that only “user space” code would be managed, the kernel (a microkernel, or perhaps somewhat hybrid) would be coded more traditionally.

I have thought about doing that before, making a microkernel out of native code, and then making all drivers in user space using my own managed language. I will probably do that when my current OS is finished .

Ok, you are definitely thinking along the same lines as me here! I hope we might be able to swap some ideas

[Schol-R-LEA]

This is a topic which has been beaten into the ground in the past, and the arguments for and against this haven't changed since the last time it was discussed. I would specifically suggest you review the flame war between Brendan and Embryo found here and here.

My own opinion on the topic - that 'managed code' is a marketing term with no actual technical meaning - is well known in this forum. While I have no problem with many of the techniques which are loosely referred to by the term, the term itself is frankly insulting in the same way such terms as 'cloud computing' and 'trusted computing' are.

[bloodline]

This is a topic which has been beaten into the ground in the past, and the arguments for and against this haven't changed since the last time it was discussed. I would specifically suggest you review the flame war between Brendan and Embryo found here and here.

My own opinion on the topic - that 'managed code' is a marketing term with no actual technical meaning - is well known in this forum. While I have no problem with many of the techniques which are loosely referred to by the term, the term itself is frankly insulting in the same way such terms as 'cloud computing' and 'trusted computing' are.

I fundamentally agree with you, and I’m not seeking to emulate a project like Singularity, but what I am thinking about is broadly, or at least conceptually, similar in its foundation. I used the term because I imagined that OS fans here would understand what I’m talking about if I used it.

I would prefer a better term, “Software Enforced Security” or maybe “Secure By Design”... I don’t know, I just want to explore a different way of curating the user space than the MMU crawling through page tables which have to swapped out at every context switch.

Though I am thinking about using similar techniques to the “Managed Code” designs, my end goal is very different than Microsoft’s.

[Schol-R-LEA]

One of the things I object to is the implication that these techniques are new, when in fact none of them are. Most of them date back to the 1970s or earlier - Hell, interpreted bytecode (of a limited sort, primarily specialized FP interpreters) date back to the 1940s. Slapping a new label on them won't change that fact.

[nexos]

I think writing a kernel in a managed code is not good, but my idea remains of a microkernel that runs driver in managed code, and the kernel is written in a native language.

[PeterX]

One of the things I object to is the implication that these techniques are new, when in fact none of them are. Most of them date back to the 1970s or earlier - Hell, virtual machines (of a limited sort, primarily specialized FP interpreters) date back to the 1940s. Slapping a new label on them won't change that fact.

I didn't mean VMs are an totally new invention. But I mean VMs have become a mass phenomenon only in the last few years. That's really different from the past. VMWare, KVM, Qemu, containers, Xen, Intel-CPU allowing 64bit code in 32bit OS, etc.

And, yes, marketing people love to give shiny new names to anything, be it new or old. There was no need to rename EFI to UEFI, just to name an example of marketing buzz-words. You probably all know more examples.

I think writing a kernel in a managed code is not good, but my idea remains of a microkernel that runs driver in managed code, and the kernel is written in a native language.

That's a cool feature I think. Not neccessary and sometimes unwanted (because some people like Asm drivers) but cool anyway.

Greetings
Peter