Choosing the right language for kernel development

[ru2aqare]

Absolutely. The handle is only a number. This number is used to look up the kernel data. The handle could also be "typed", so the kernel knows the handle passed is associated with the data-type intended. The typing is kept in the kernel.

The number needs to be validated, which reduces performance. Strong typing eliminates this by (for example) making the constructor unavailable to user mode code, so user mode code can only pass either a null reference (which is easy to check for) or a handle it already knows about.

The handle is like the "this" pointer in C++. It refers to the object and is passed as a parameter to "methods". This means that all the user-level code knows about the object-data in the kernel is the handle, and nothing else. True encapsulation that is unbreakable, unlike C++ encapsulation.

This is true, however user-mode code can still pass arbitrary values to the kernel, which need to be checked. Strong typing eliminates this check.

Not between kernel and user-level. Only within the same program. It is easy to break any type-safe language with assembler code.

I think he meant that type-safety applies to the entire system, not only to the kernel. Strong typing applied only to the kernel, and allowing unsafe applications does not have an advantage over traditional kernel designs, I suppose.

[Walling]

The number needs to be validated, which reduces performance. Strong typing eliminates this by (for example) making the constructor unavailable to user mode code, so user mode code can only pass either a null reference (which is easy to check for) or a handle it already knows about.

Exactly. And on a bad day you could unintentionally implement a system call that didn't validate the handle. The compiler (for an unsafe language) can't enforce that no memory corruption / malicious pointers occur.

[Colonel Kernel]

Assembler and C++ is the optimal combination. C is useless "middle-ground". You can do everything faster in assembler than i C, and object-oriented interfaces are much easier constructed in C++ than in C. Therefore, C has no real advantage anywhere.

Hand-written asm is not always faster than the code produced by a good C optimizing compiler. The OO features of C++ help, if OO is what you need. On the other hand, C++ is a ridiculously complex language and I would hate to try porting a C++ compiler to a new platform!

After fighting with the evils of cross-platform C++ code for the better part of 10 years, I'm happy to go back to the simplicity of C for low-level stuff (my hobby) and the convenience of C# for high-level stuff (my day job). Different strokes for different folks, I guess.

Isolation is done by using handles in kernel code, and only exporting those to user-mode. This concept is 100% safe and doesn't need any compiler support. Isolation within the kernel is best achieved with hardware mechanisms (paging, segmentation, and whatever else is present).

There is no 100% safe.

I was not actually discussing the trade-offs between software and hardware isolation. I was just trying to clear up what it means for a language to be "safe" since this seems to cause people a lot of confusion. But if you're interested in learning more about the trade-offs, there are plenty of research papers to read! Specifically, take a look at "Deconstructing process isolation" which has a lot of interesting benchmarks comparing various mixes of hardware and software isolation. To summarize: MMU overhead is non-trivial!

No language that support pointers can be considered "safe".

Strictly speaking, you're correct. C# as a whole is not safe. However, its unsafe features are all limited by the type system such that they can only be used in methods or blocks of code that are specifically marked with the unsafe keyword. This means that if you write a C# program that uses pointers, the system will know about it when it verifies your IL and can refuse to run your code. This cannot be spoofed as it is a fundamental property of the IL being verified -- either it violates type-safety, or it doesn't.

It is easy to break any type-safe language with assembler code.

What do you mean exactly? That allowing inline asm makes a language unsafe? That's true. Do you mean that you can write arbitrary unsafe code that can run in the same memory space as a program written in a "safe" language and write all over its memory? If so, that defeats the purpose of using safe languages in the first place. Singularity does not allow arbitrary unsafe code to run. It verifies the code first, and refuses to run it if it is unsafe.

[rdos]

The number needs to be validated, which reduces performance. Strong typing eliminates this by (for example) making the constructor unavailable to user mode code, so user mode code can only pass either a null reference (which is easy to check for) or a handle it already knows about.

I don't think this will work. I don't know about how C# passes arguments, but I do know how C and Pascal does it. Safe types are passed as pointers in both Pascal and C/C++, and I'm sure they are in C# as well. This kind of security between kernel and user is only apparent. It cannot stop user-mode code from simulating the behavior of C#, and passing it malicious data.

My method might take a little evaluation, but it also offers superior protection from buggy user mode apps. Using handles is only one part of the protection. Here it how it works:

1. No stack-frames are used
2. Oridnary parameters are passed in registers.
3. Pointers to raw data must be loaded into segment registers, and cannot be passed by reference.
4. Passing data-structures between usermode and kernel is not allowed.
5. Handles are per process, and are typed. This means that an application cannot use handles in incorrect contexts, and cannot use the handles of other processes.

Flat mode applications cannot supply pointers to data structures in the kernel to a kernel API. This is because the flat mode user selector only covers user-mode addressable memory, and not kernel addressable. It has a linear base of 1000000h and a size of 0DF000000h. Because all kernel APIs work on 48-bit pointers, user- mode cannot pass bad pointers to kernel.

This is true, however user-mode code can still pass arbitrary values to the kernel, which need to be checked. Strong typing eliminates this check.

No difference to C#. These checks are needed there regardless because usermode and kernel are not linked together. Any separately compiled modules in the kernel will need these checks as well.

[rdos]

Exactly. And on a bad day you could unintentionally implement a system call that didn't validate the handle.

Not in my implementation. The dereferincing of the handle is needed to access the kernel data structure that is related to it. The handle number is meaningless both for usermode and kernel. There is an kernel-only API call that takes a handle and its type and returns the handle data if successful.

The compiler (for an unsafe language) can't enforce that no memory corruption / malicious pointers occur.[/quote

Segmentation can ensure that flat mode user mode (or segmented for that matter) only can pass pointers that reside in usermode memory, and that usermode has no access to (or knowledge of) kernel data structures.

[rdos]

Hand-written asm is not always faster than the code produced by a good C optimizing compiler. The OO features of C++ help, if OO is what you need. On the other hand, C++ is a ridiculously complex language and I would hate to try porting a C++ compiler to a new platform!

Not a problem for me. I use BC 5.5 (which is now free) as a compiler under Windows. I also support running TD, TD32, TASM, TASM32, TLINK, TLINK32 and BCC32 natively under RDOS.

After fighting with the evils of cross-platform C++ code for the better part of 10 years, I'm happy to go back to the simplicity of C for low-level stuff (my hobby) and the convenience of C# for high-level stuff (my day job). Different strokes for different folks, I guess.

I know the problem. I gave up on GCC a while back. Mostly because the patches seemed to take for ever to become accepted in the mainstream code.

I was not actually discussing the trade-offs between software and hardware isolation. I was just trying to clear up what it means for a language to be "safe" since this seems to cause people a lot of confusion. But if you're interested in learning more about the trade-offs, there are plenty of research papers to read! Specifically, take a look at "Deconstructing process isolation" which has a lot of interesting benchmarks comparing various mixes of hardware and software isolation. To summarize: MMU overhead is non-trivial!

True. Nor is segmentation overhead, but I think segment protection within the same process is more efficient than using paging to separate address spaces. Provided you don't use crappy C/C++ compilers that cannot handle segmentation properly.

[ru2aqare]

I don't think this will work. I don't know about how C# passes arguments, but I do know how C and Pascal does it. Safe types are passed as pointers in both Pascal and C/C++, and I'm sure they are in C# as well. This kind of security between kernel and user is only apparent. It cannot stop user-mode code from simulating the behavior of C#, and passing it malicious data.

My point was that if the entire system (including user-mode applications) is written in type-safe languages, then the checks are not requried. Obviously, if you allow even one application not written in type-safe languages, then you can't ignore the checks.

An issue with protection relying on segmentation is that it's not supported in long mode. If you target only x86, then it's not much of a problem, but for x64 you have to come up with something else.

[rdos]

My point was that if the entire system (including user-mode applications) is written in type-safe languages, then the checks are not requried. Obviously, if you allow even one application not written in type-safe languages, then you can't ignore the checks.

How do these type checks handle different versions of kernel & usermode libraries? It you use type-safe language calling conventions & data structures, they must "belong together" for things to work properly. When I create new kernel APIs I define them on the register level, and I always ensure backward compatibility (well, at least almost always).

An issue with protection relying on segmentation is that it's not supported in long mode. If you target only x86, then it's not much of a problem, but for x64 you have to come up with something else.

I know. I won't target x64 because it breaks the segmentation features and an x32 OS cannot easily act as a server OS for 64-bit applications. I'll run the new processors in x32 mode instead.

[ru2aqare]

How do these type checks handle different versions of kernel & usermode libraries? It you use type-safe language calling conventions & data structures, they must "belong together" for things to work properly.

I would guess the kernel publishes a set of interfaces, which are maintained in each new version of the kernel. The actual implementation is thus not visible outside the kernel. As for IL, the identity of a type is determined by its name, the name of the assembly it is declared in, the version number and the public key of the assembly. If the name remains the same, but any of the other three changes, the type is considered a different type, and cannot be used in place of the original type.

[Colonel Kernel]

After fighting with the evils of cross-platform C++ code for the better part of 10 years, I'm happy to go back to the simplicity of C for low-level stuff (my hobby) and the convenience of C# for high-level stuff (my day job). Different strokes for different folks, I guess.

I know the problem. I gave up on GCC a while back. Mostly because the patches seemed to take for ever to become accepted in the mainstream code.

Heh... There are worse C++ compilers than GCC, believe me. Try targeting C++ compilers up to 8 years old on all the major *nix platforms (and some barnacle-encrusted ancient *nixes too). I'm soooo glad to be developing on only one platform now.