Managed Code OSes - Speed and Security (split)

[earlz]

Hobby OS's allow us to implement our dream system.

My goal is to write an operating system where programs are able to achieve maximum performance (hence I'm modelling it after a video-game OS). The reason I admire Dex and his OS is because it has a similar goal but the way we're going about it is completely different. In his OS you are given full access to the hardware and memory and do whatever you like, and there is no point recreating his OS otherwise I could just ask to join his team. Instead in my OS, we differ because I also admire the microkernel architecture, my OS will be multitasking (to the extent the Xbox 360 is - you can have background tasks) and I wish to allow applications to communicate through drivers rather than directly to the hardware (so applications are portable across a wide variety of hardware configurations and not recreate the problem DOS games had where every game had to support every piece of hardware).

As a result of selecting the microkernel architecture and sticking to my goal of maximum performance, I'm developing an IPC implementation so that a calling a function in another thread (that thread could be in another program) is not much less efficient than calling a local function, removing any need for ring changes or system calls. To do this safely (the microkernel goal) I'm taking the route of JITing 'safe' bytecode so I can ensure it doesn't access memory or call functions it shouldn't(I chose .Net because of the large number of languages you can develop in). My project differs from SharpOS and Cosmos since I my goal isn't to write a 100% .Net OS (my kernel and the JIT compiler are written in C++), as my choice of using .Net is limited to the user applications (including drivers and servers).

As a result of my decision to use safe bytecode I will be able to run any application or server (drivers excluded) on any platform I have ported the kernel to, including as a guest program on another operating system. This means I can debug the portable kernel code (such as the JIT compiler) as if were a native Windows application, and I can also test out my servers and applications without requiring a VM. Drivers as well as non-portable parts of my kernel are unfortunately excluded.

Wait so your building a high performance(Soft Real-Time-ish) OS where the applications run in .Net.. Isn't that like contradicting itself? That's like saying "I'm going to build the most secure OS" and then having all the applications run in ring 0. Sure you can get fast speeds in your OS... as fast as .Net can allow..

[AJ]

Wait so your building a high performance(Soft Real-Time-ish) OS where the applications run in .Net.. Isn't that like contradicting itself?

Why? Once the apps have been JIT compiled (or AOT compiled) it could theoretically run faster than a traditional kernel because you could run all the user apps in the same address space due to software isolation.

Cheers,
Adam

[AndrewAPrice]

Thanks, Dex!

Wait so your building a high performance(Soft Real-Time-ish) OS where the applications run in .Net.. Isn't that like contradicting itself? That's like saying "I'm going to build the most secure OS" and then having all the applications run in ring 0. Sure you can get fast speeds in your OS... as fast as .Net can allow..

.Net doesn't imply slowness, all code is ran natively on the CPU. Other than the overhead of the Just In Time compiler when launching an application which compiles the bytecode into machine code, I'm able to compile in machine-specific constants at load-time rather than compile-time, optimize for CPU extensions if they are available, and use software isolation to remove the need for CPU ring protection and ring switching (as AJ suggested).

I've never been a fan of AOT compiling. In some systems it has it's benefits, in mine it'll compromise software isolation which is the reason I chose the JIT path (unsafe code could be injected into a AOT'd binary). I do see advantages in preloading and caching as long as the kernel can verify it's integrity.

[AJ]

Hi,

A thought about this has just sprung to mind. You don't like AOT compilation because of the possibility of code injection. This means you are limited to JIT, which has a performance penalty the first time a piece of code is called. You then cache the result - in your cache, it really doesn't matter whether the code was JIT or AOT compiled.

How about having a background worker thread which compiles (and caches) code in an AOT sort of way? You still don't cache the result to long-term storage, but it also means that you always have a chance of hitting precompiled code rather than relying on JIT. If the algorithm to select which code to precompile is good, this could work quite effectively without compromising security too much.

Cheers,
Adam

[Edit: Because this had gone OT (my fault!), I've split it from -> here <-]

[JamesM]

(unsafe code could be injected into a AOT'd binary)

Why not sign the binary?

[Solar]

The AOT compiler signing that the binary... what? Doesn't do Bad Things (tm)?

Since the ability to sign a binary has to be built into the compiler, it can be reverse-engineered, and malicious code be signed as well.

But I don't really see how JIT would be a way to do away with memory protection. MP, if done right, protects each app from malicious neighbours. JIT, if done right, does the same. The point here is the if done right. Why drop one security measure just because you have another? I'd always combine those two.

And there is one compromise way that is too often overlooked, IMHO: Compile-at-installation. You get all the benefits of knowing exactly the hardware layout of the target system, having no performance penalty at load time, and the ability to sign the binary, using a checksum including some hardware values. No unsigned code being allowed to run, means that your installer (the signing authority) can double as a strong anti-virus checkpoint, and (if you are so minded) means that you cannot pack together the binaries on disk and redistribute them elsewhere.

[Thomas]

Since the ability to sign a binary has to be built into the compiler, it can be reverse-engineered, and malicious code be signed as well.

signing of an assembly is not done by the compiler , but by a separate utility ( In .NET its sn.exe ) . It generally uses the RSA encryption algorithm and signing is not done at compile time . Its after the source code is compiled into an assembly . See Assembly

-- Thomas

[rootnode]

We solve it that way that our JIT doesn't allow unsafe code, only in our Kernel and System Libraries. All other applications or libraries that are jitted, aren't allowed to use our system libraries, but have to use mono's assemblies. At least that's the rough idea we have.

[earlz]

Thanks, Dex!

Wait so your building a high performance(Soft Real-Time-ish) OS where the applications run in .Net.. Isn't that like contradicting itself? That's like saying "I'm going to build the most secure OS" and then having all the applications run in ring 0. Sure you can get fast speeds in your OS... as fast as .Net can allow..

.Net doesn't imply slowness, all code is ran natively on the CPU. Other than the overhead of the Just In Time compiler when launching an application which compiles the bytecode into machine code, I'm able to compile in machine-specific constants at load-time rather than compile-time, optimize for CPU extensions if they are available, and use software isolation to remove the need for CPU ring protection and ring switching (as AJ suggested).

I've never been a fan of AOT compiling. In some systems it has it's benefits, in mine it'll compromise software isolation which is the reason I chose the JIT path (unsafe code could be injected into a AOT'd binary). I do see advantages in preloading and caching as long as the kernel can verify it's integrity.

They say the same thing about Java and it still doesn't surpass my C compiler.


And as for the security bit. Whenever someone finds a bug in your JITer you'll kick yourself for leaving a root kit in your OS... Unless you don't care for the multi-user revolution.. in that case you just gave a virus unrestricted access to your system, which isn't much better.. ie, Windows 95.

[NickJohnson]

The thing about JIT is that it requires a program at least as complex as a traditional compiler - much more so if you want the produced code to run fast - in order to run anything on your machine. Few users of operating systems have a compiler, and embedded systems hardly want them either. Making the OS implementation thousands of times simpler and a reasonable degree faster with AOT, at the cost of punishing stupid programmers/admins more, is more than a good trade in my mind. C programs can be quite secure too: see OpenBSD.

@Solar: I also like the compile-at-installation idea. LLVM has shown that distributing bytecode but running machine code is not only fast but produces extremely fast executables too, and the security through signing executables would definitely be a plus. If you could make a compiler that would guarantee no absolute or illegal memory accesses (which would be hard with C - maybe another language?), and make it sign all executables, you could even use that method to run AOT optimized executables under a no-paging system like many managed OSes do - another speed boost.

[JamesM]

They say the same thing about Java and it still doesn't surpass my C compiler.

The speedups we're talking about here come when an application can be trusted. The underlying OS in a Java system (Linux/Windows) doesn't trust anything. It can't. So the Java program is very likely to be slower than the equivalent C program. Once the OS trusts the application though (which would involve the OS being written in a managed language itself or dealing solely with managed languages), IPC becomes *blindingly* fast.

The advantage to JIT here too is that if a program is executing say with root privileges, most of the security checks can be removed at compile time. Leaving the compile stage as late as possible opens up more optimisation opportunities.

With regards to AOT; We've opted against it for several reasons:

1) If we removed the compiler from the OS (everything was AOT'd), there would be no way to run the compiler and run new programs once the OS had booted, as the compiler is written in an unmanaged language. While this may be desirable for some situations (embedded devices) it will not be for others.

2) Using a bytecode that allows reflection lets datatypes be shared across modules/packages. However, with AOT you don't know what polymorphic type instantiations are going to be needed (think C++ template instantiations). So either specific instantiations must be forced, or you can't use polymorphic types over module boundaries.

3) Part of the project ties with my Masters thesis which is about microthreading (µthreading). Effective µthreading requires the compilation to depend on processor topology, which may be dynamic (if we assume that clustering is possible).

[AndrewAPrice]

When I said I wasn't fond of AOT I was meaning in the form of storing the compiled file on disk with the possibility of the user (or another program) tampering with it. As Solar said, you could reverse engineer to kernel to extract the key. Security through obsecurity is unmessy. I do like AOT in the form of preloading/caching as long as the kernel can ensure the integrity of what's in the cache.

@Solar: I wanted to do away with memory protection to avoid switching address spaces and rings purely for the speed.

If I got rid of one of those two I'd need a way to ensure code doesn't do something bad, which is what originally lead me down the path of JITing.

[JamesM]

As Solar said, you could reverse engineer to kernel to extract the key.

No, as with normal digital signatures the kernel would only have the public key; the private key would be required to actually sign the executable. Each user could have their own public/private key pairs, plus one for the distributing body (package management software). That way an attacker would need either the user's private key (used to sign his/her executables when compiled) or the package-vendor's private key (needed to distribute to other people) to forge a signature.

[AndrewAPrice]

As Solar said, you could reverse engineer to kernel to extract the key.

No, as with normal digital signatures the kernel would only have the public key; the private key would be required to actually sign the executable. Each user could have their own public/private key pairs, plus one for the distributing body (package management software). That way an attacker would need either the user's private key (used to sign his/her executables when compiled) or the package-vendor's private key (needed to distribute to other people) to forge a signature.

I was referring to extracting the key from the AOT compiler. There is really only one use to having a private AOT compiler with the key built in and that is being able to do kernel updates (they only code that isn't .Net).

[JamesM]

As Solar said, you could reverse engineer to kernel to extract the key.

No, as with normal digital signatures the kernel would only have the public key; the private key would be required to actually sign the executable. Each user could have their own public/private key pairs, plus one for the distributing body (package management software). That way an attacker would need either the user's private key (used to sign his/her executables when compiled) or the package-vendor's private key (needed to distribute to other people) to forge a signature.

I was referring to extracting the key from the AOT compiler. There is really only one use to having a private AOT compiler with the key built in and that is being able to do kernel updates (they only code that isn't .Net).

My point wasn't that the key would be stored in the compiler, more that each user would have their own private key which the kernel only let the compiler access. The kernel knows that the compiler application is trusted because it's signed too. Easiest way to explain this is with a timeline:

• All programs on the system are digitally signed. Remember with asymmetric encryption you require two keys: the private key one way, the public key the other. In this case the private key is the package manager's own, and is used to sign the executable once it has been AOT'd. This is important, because it provides the trust base that you can work from (you need to be able to trust your compiler!).
• When the AOT compiler compiles bytecode->native code, it asks the OS for the current user's private key, which could be randomly generated on system install time. It then signs the executable it produces with that key.
• Now, in order to execute a program, the OS/runtime has to check the signature of the executable. If it is signed correctly with either the package manager's or the user's key (use the package manager's public key and the user's public key to test this), you know the program has not been tampered with.

This works because:

• You trust the package manager.
• You therefore trust the compiler signed by the package manager.
• You therefore trust that the compiler will not do anything untoward with your private key while it is signing the binary.
• You can now trust properly signed binaries, checkable by the use of your public key, which does not need to be kept secret.

As with all asymmetric algorithms, this relies on no leaks of the private key. As a bytecode-only OS, you have the advantage that you never allow unsafe code to run, which makes a number of current OS attacks impossible.