What do you think about managed code and OSes written in it?

[embryo]

Why should endianess be a problem and why shouldn't I use a safe language?

Endiannes issues lead to another class of errors that can not be fought automatically in unmanaged language.

You can use safe language, but it requires another parts of managed environment like garbage collector, for example. So, you can find yourself using a complete managed environment if you switch to a safe language.

And unmanaged environment just doesn't care for any interdependencies

No, look at e.g. Qt. They do care in an unmanaged environment.

They care for SOME interdependencies. It is the difference.

What you're basically saying is: “When everyone would use my and only my platform (managed environment), developing for my platform would be platform independent”.

No. The world has Java runtimes, that are implemented for many platforms. So, when anybody would use my platform, then the Java's platform independence will be given for free.

[Rusky]

For the "almost 0% of bugs" that aren't detected before compiling, aren't detected during compiling, aren't detected during unit testing, and aren't detected by the managed environment the developers use while developing; you want to sacrifice performance for end users, while I don't think that (given all of the above) there's any sane reason that justifies it.

One thing is these bugs (admittedly much rarer if you use static checking, unit testing, fuzz testing, etc) can lead to security vulnerabilities. So it's not so much "trade a tiny bit of performance for nicer error messages," but "trade a tiny bit of performance for less frequent loss of security."

You want large libraries that programmers can use to reduce development time. I want services that programmers can use to reduce development time instead (and I don't want any libraries at all).

This is probably off topic, feel free to split the thread or something if you want, but I'm curious about your reasoning here. What exactly separates a service from a library? For example, would you replace a compression/decompression library like zlib with a service, even if it were just used to parse an internal file format or something?

[HoTT]

You can use safe language, but it requires another parts of managed environment like garbage collector,

Garbage collection is not unique to managed environments. There have been GCs long before that term (managed X) got coined. But why do you need a garbage collector to avoid errors that relate to endianess? Could you give an example of an error and how it is avoided using a GC?

They care for SOME interdependencies. It is the difference.

What I wanted to illustrate is, that you don't have to use a managed environment to write platfrom independent code.

No. The world has Java runtimes, that are implemented for many platforms. So, when anybody would use my platform, then the Java's platform independence will be given for free.

Not for free, someone has to implement that runtime.

[Brendan]

Hi,

Yes, the summary is interesting.

But I want one more summary

It's about managed vs unmanaged. I define the difference as a harsh change in developer's perceptions. But you prefer to define the change as a gradual transition when one component can be added after another is already in use for some time. And your current component acceptance is at the position of compile and debug time compliance with the managed environment approach. As you can see it is just one step apart from a complete managed environment with a runtime part also present.

I'm thinking of an unmanaged language combined with a managed environment for developers to use for debugging, like C and valgrind (but not C or valgrind); and I'm not thinking of a managed language (and managed environment) that wraps programmers in bubble wrap and prevents them from doing their job properly (which can include using things like assembly for performance critical pieces) like Java (or at least, like Java without JNI).

Another point, that you agree with, is some benefits of managed environment runtime, that can be helpful, but as you see it, just in a very small number of cases. And here is a whole picture - we both see some benefits of the managed approach, but employ them differently. The only reason you still refuse the complete managed environment (with runtime part) is runtime's minimal significance (as you see it). But the way, you already have behind, just tells me that it is some time in the future when you finally accept the whole environment without any exclusion. Just because it's benefits can be rethought and it will lead to some positive decisions.

The only reason I'd ever recommend a managed language to anyone is if I believe they're irredeemably incompetent and should never be allowed anywhere near real programming. This has never happened in the past (everyone has at least some ability to learn).

I only care about my OS and don't care (and don't want) applications intended for my OS to run on other OSs or applications intended for other OSs to run on my OS; but (I assume) you do care about that.

Yes, it is easier to care just about your OS and forget about others. But then you refuse to benefit from a great amount of a code, that is written for other environments. I prefer to use such benefit in form of Java code base. And another benefit is more probable community support in case of Java.

How do you convince end users to switch from existing OSs (that are more mature - faster, less buggy and with more drivers) to your OS, when the applications run on both OSs? "Same but worse" is unlikely to be an good marketing strategy.

I also want a language where unit testing is an inbuilt feature; and an IDE that automatically runs the unit tests in the background and highlights things that failed their unit tests while you type. You probably haven't thought about this, but I think it's important to detect all the bugs that can't be detected automatically (with or without run-time tests).

Here I miss the point of unit tests. Who will write them? If it is a developer, then such "inbuilt" feature works just as an ordinary unit test runner program. I mean I see no difference between the "inbuilt" and traditional approaches.

The most obvious benefit is integration - e.g. having mistakes highlighted for you while you're working on the source code in the IDE (and not having additional hassle, or steps to accidentally forget, or having to manually translate the unit test results into a location in the source code).

The less obvious benefit is that it provides "default scenarios" for debugging. For example, maybe there's a problem with an "isNumberPrime()" function buried deep within the program, and instead of running the entire program in the debugger you can just run an existing unit test in the debugger without all the unnecessary baggage.

And how it is possible "to detect all the bugs that can't be detected automatically"? I see it as if it tells that all bugs can be detected. But how can we detect all possible bugs?

I should've said "be able to detect bugs that can't be detected automatically". For example, consider this pseudo-code:

Code: Select all

bool isNumberPrime(int n) {
    if( (n & 1) == 0) return false;
    return true;
}

This bug can't be detected automatically (regardless of language or managed/unmanaged); but it's relatively easy to write unit test/s that find the problem for you. There's even people that think the tests should be written before any of the actual code (e.g. test driven development).

You want large libraries that programmers can use to reduce development time. I want services that programmers can use to reduce development time instead (and I don't want any libraries at all).

Any service is built around some code. If we call this code "a library", then I see the difference with your approach in the way the code is invoked. And I suppose you understand that communication is expensive, then I conclude your invocation method is still very close to traditional call conventions. And if it is then I just see no difference between your services and traditional libraries.

There are similarities (but I'm using asynchronous messaging where the service may be running on a completely different computer and there's some other things that come may into it).

I want to use the same IDE, tools and programming language for normal/portable applications and for things like boot code and kernel (that can't be 100% portable and can't be "managed"). I assume you want different languages (one "safe" language for applications and one more powerful/lower level language for boot code, kernel, etc) with twice as much to learn and different tool-chains to mess with.

Here we are more close than it is seen from your point of view. My way is about using the same language and IDE, but with some clearly defined separation between low level and high level code. One part of such separation is about defining low level code within a specially designated method. And second part is about assembler-like look and feel of the low level code. First part allows an easy way for a developer and a managed environment to recognize and to manage low level parts. Second part allows to leverage assembly knowledge base, that is accumulated by the humanity so far.

How does the managed environment "manage the low level parts" (e.g. prevent intentionally malicious assembly language code from doing unsafe things)?

So, I just don't see how it is possible to use your approach without learning assembly (twice as much to learn, as you say). And don't see another toolchain that you suppose I should mess with.

Inline assembly, with the same IDE, same tool-chain, same services/"libraries", same APIs, same memory model, same (unmanaged) execution environment, etc.

For the "almost 0% of bugs" that aren't detected before compiling, aren't detected during compiling, aren't detected during unit testing, and aren't detected by the managed environment the developers use while developing; you want to sacrifice performance for end users, while I don't think that (given all of the above) there's any sane reason that justifies it.

Ok, let it be our religious assessment of the bug percentage, that we can find after all tests are done. But my religion tells me that the percent is essentially more important than you assume. May be the exact number really can be less than 1%, but it's consequences for security and reliability can be too costly, and such cost can be much greater than <1% of a total development cost if we suppose that our environment is highly competitive.

I think you'll find that for all the security vulnerabilities most would've been prevented by a better unmanaged language and the remainder would not have been prevented by a managed language.


Cheers,

Brendan

[Brendan]

Hi,

For the "almost 0% of bugs" that aren't detected before compiling, aren't detected during compiling, aren't detected during unit testing, and aren't detected by the managed environment the developers use while developing; you want to sacrifice performance for end users, while I don't think that (given all of the above) there's any sane reason that justifies it.

One thing is these bugs (admittedly much rarer if you use static checking, unit testing, fuzz testing, etc) can lead to security vulnerabilities. So it's not so much "trade a tiny bit of performance for nicer error messages," but "trade a tiny bit of performance for less frequent loss of security."

Because managed languages will prevent bugs like this:

Code: Select all

   if ((err = SSLHashSHA1.update(&hashCtx, &serverRandom)) != 0)
        goto fail;
    if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0)
        goto fail;
        goto fail;
    if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0)
        goto fail;

It's a fact that nobody with a banana up their butt has ever been abducted by aliens; therefore it's important (to avoid being abducted by aliens) for everyone to put a banana up their butt - walking funny is a small price to pay for the added false sense of security.

You want large libraries that programmers can use to reduce development time. I want services that programmers can use to reduce development time instead (and I don't want any libraries at all).

This is probably off topic, feel free to split the thread or something if you want, but I'm curious about your reasoning here. What exactly separates a service from a library? For example, would you replace a compression/decompression library like zlib with a service, even if it were just used to parse an internal file format or something?

For me; services are processes (in their own isolated environment), and may be running on a different computer with a different architecture. For example; you might have a word processor running on 32-bit 80x86 that's using a "spell checker" service running on an ARM CPU in a different computer on the network. With a little trickery it should be possible for the kernel to send the same requests to multiple different "spell checker" services (possibly written by different people and running on different computers) so that if one fails (e.g. that computer explodes) the word processor doesn't even know (possibly including comparing the replies for "N-modular redundancy").


Cheers,

Brendan

[Marionumber1]

Hi,

Because managed languages will prevent bugs like this:

Code: Select all

   if ((err = SSLHashSHA1.update(&hashCtx, &serverRandom)) != 0)
        goto fail;
    if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0)
        goto fail;
        goto fail;
    if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0)
        goto fail;

It's a fact that nobody with a banana up their butt has ever been abducted by aliens; therefore it's important (to avoid being abducted by aliens) for everyone to put a banana up their butt - walking funny is a small price to pay for the added false sense of security.

Locking my door isn't guaranteed to stop someone from breaking into my house, so I should leave my door unlocked?

[Rusky]

It's a fact that nobody with a banana up their butt has ever been abducted by aliens; therefore it's important (to avoid being abducted by aliens) for everyone to put a banana up their butt - walking funny is a small price to pay for the added false sense of security.

Except I'm not proposing a banana up everyone's butt to prevent alien abductions, I'm proposing putting locks on people's doors to prevent burglaries. Carrying around a key and locking and unlocking your door is a small price to pay for increased safety of your home, your family, and your person. If you go back a few posts in this thread, I dug up some stats on vulnerabilities in the Linux kernel that would have been prevented by array bounds checking. "goto fail; goto fail;" is nowhere near the most common sort.

For me; services are processes (in their own isolated environment), and may be running on a different computer with a different architecture. For example; you might have a word processor running on 32-bit 80x86 that's using a "spell checker" service running on an ARM CPU in a different computer on the network. With a little trickery it should be possible for the kernel to send the same requests to multiple different "spell checker" services (possibly written by different people and running on different computers) so that if one fails (e.g. that computer explodes) the word processor doesn't even know (possibly including comparing the replies for "N-modular redundancy").

Ah, okay. I'm still not clear on your "no libraries" stance, though- I feel like there would still be tasks that are small enough, specific enough, and stateless enough, that setting up a separate environment would provide no benefit. At the smallest scale, obviously, you're still going to divide your services into functions that all run in the same process, and it will probably make sense to share some of those functions between services as libraries. I'll bring up my example of zlib again- why would I ever want to outsource the decompression done by a PNG loading service, or for that matter the PNG loading itself?

[Brendan]

Hi,

It's a fact that nobody with a banana up their butt has ever been abducted by aliens; therefore it's important (to avoid being abducted by aliens) for everyone to put a banana up their butt - walking funny is a small price to pay for the added false sense of security.

Except I'm not proposing a banana up everyone's butt to prevent alien abductions, I'm proposing putting locks on people's doors to prevent burglaries. Carrying around a key and locking and unlocking your door is a small price to pay for increased safety of your home, your family, and your person. If you go back a few posts in this thread, I dug up some stats on vulnerabilities in the Linux kernel that would have been prevented by array bounds checking. "goto fail; goto fail;" is nowhere near the most common sort.

As I've said, C and C++ are horrible (in terms of how possible it is to detect bugs at compile time), and bounds checking is solvable at compile time.

Locking your door is silly when there's armed guards patrolling the inside and outside of your home anyway.

For me; services are processes (in their own isolated environment), and may be running on a different computer with a different architecture. For example; you might have a word processor running on 32-bit 80x86 that's using a "spell checker" service running on an ARM CPU in a different computer on the network. With a little trickery it should be possible for the kernel to send the same requests to multiple different "spell checker" services (possibly written by different people and running on different computers) so that if one fails (e.g. that computer explodes) the word processor doesn't even know (possibly including comparing the replies for "N-modular redundancy").

Ah, okay. I'm still not clear on your "no libraries" stance, though- I feel like there would still be tasks that are small enough, specific enough, and stateless enough, that setting up a separate environment would provide no benefit.

Anything small enough, specific enough and stateless enough is too trivial to bother with.

At the smallest scale, obviously, you're still going to divide your services into functions that all run in the same process, and it will probably make sense to share some of those functions between services as libraries. I'll bring up my example of zlib again- why would I ever want to outsource the decompression done by a PNG loading service, or for that matter the PNG loading itself?

You forget my "file format converters" (which are mostly just services that the VFS uses). No application will ever see PNG.

If you want to read a compressed native image file, just tell the VFS to open it as a (decompressed) native image file and let it arrange the conversion/decompression (and also let it cache the decompressed file for next time, if there's enough disk space for that). Of course image files are a little special - most of the time you'd just tell the GUI to tell the video driver to load them as a texture (and let the video driver ask the VFS for the image file without caring if it's compressed or not).


Cheers,

Brendan

[Rusky]

As I've said, C and C++ are horrible (in terms of how possible it is to detect bugs at compile time), and bounds checking is solvable at compile time.

Continuing to assert this with no reasoning is pointless. Until you can show me a language design that actually solves all bounds checking at compile time, I will stick with my point that static bounds checking is in fact undecidable even with a really fancy type system, and for a general purpose operating system to be impossible to break through buffer overflows, etc. there will need to be some checks at run time. (As you have said, that should mostly be part of verifying user input.)

Locking your door is silly when there's armed guards patrolling the inside and outside of your home anyway.

Not locking your door is silly in any possible situation involving armed guards.

You forget my "file format converters" (which are mostly just services that the VFS uses). No application will ever see PNG.

Okay, so you can access your image files transparently through any instance of that service on your network. How does the file format converter load the PNG? It must have the compression/decompression logic implemented somewhere, and that logic is also used in many other places (other file formats, network protocols, your texture loader, etc). Compression and decompression are about as purely stateless as you can get at this scale- they take a chunk of input data and produce a chunk of output data based entirely on the input. So either you're duplicating that logic everywhere for no good reason, or you pull it into a service of its own which has way more overhead than just calling a function, or you put it in a library.

[HoTT]

As I've said, C and C++ are horrible (in terms of how possible it is to detect bugs at compile time), and bounds checking is solvable at compile time.

Locking your door is silly when there's armed guards patrolling the inside and outside of your home anyway.

It's not in the general case. Please google bounds checking and halting problem. You can have a language (and I think it's a neat idea) that rejects all array accesses where the bounds cannot be statically checked, but that's not the same as bounds checking is solvable in compile time.

[Brendan]

Hi,

As I've said, C and C++ are horrible (in terms of how possible it is to detect bugs at compile time), and bounds checking is solvable at compile time.

Continuing to assert this with no reasoning is pointless. Until you can show me a language design that actually solves all bounds checking at compile time, I will stick with my point that static bounds checking is in fact undecidable even with a really fancy type system, and for a general purpose operating system to be impossible to break through buffer overflows, etc. there will need to be some checks at run time. (As you have said, that should mostly be part of verifying user input.)

I've already explained (one of possibly many?) ways to do this. If you've intentionally chosen to be wrong, then I doubt repeating what I've already said will help.

You forget my "file format converters" (which are mostly just services that the VFS uses). No application will ever see PNG.

Okay, so you can access your image files transparently through any instance of that service on your network. How does the file format converter load the PNG? It must have the compression/decompression logic implemented somewhere, and that logic is also used in many other places (other file formats, network protocols, your texture loader, etc). Compression and decompression are about as purely stateless as you can get at this scale- they take a chunk of input data and produce a chunk of output data based entirely on the input. So either you're duplicating that logic everywhere for no good reason, or you pull it into a service of its own which has way more overhead than just calling a function, or you put it in a library.

So?

If DEFLATE was implemented as a service; then the cost of my messaging is around 1000 cycles per message (for a max. message size of 2 MiB; assuming both sender and receiver are on the same computer). To send 1 GiB of data to a "DEFLATE service" and receive 2 GiB of data back (assuming a 2:1 compression ratio) it's going to need to be split into 1536 messages and cost about 1536000 cycles for all sending and receiving. On an ancient 25 MHz 80486 (the oldest/slowest/worst CPU I intend to support) that works out to about 62 ms. Compared to the cost of compressing/decompressing the data, the overhead of messaging is almost entirely irrelevant. If it happened regularly nobody would care.

However; it won't happen regularly either. PNG files would be converted into the OS's native file format as soon as possible (preferably, before they're stored in the OS's file system) and never converted from the OS's native file format back into PNG. None of the OS's native file formats use DEFLATE (including the OS's native "compressed file format").


Cheers,

Brendan

[Brendan]

Hi,

As I've said, C and C++ are horrible (in terms of how possible it is to detect bugs at compile time), and bounds checking is solvable at compile time.

Locking your door is silly when there's armed guards patrolling the inside and outside of your home anyway.

It's not in the general case. Please google bounds checking and halting problem. You can have a language (and I think it's a neat idea) that rejects all array accesses where the bounds cannot be statically checked, but that's not the same as bounds checking is solvable in compile time.

Sigh.

For anything that could use an arbitrary sized array, you can use a pool of fixed size arrays instead. Fixed size arrays can be checked at compile time. It's not just solvable, it's trivial to solve.


Cheers,

Brendan

[HoTT]

Sigh.

For anything that could use an arbitrary sized array, you can use a pool of fixed size arrays instead. Fixed size arrays can be checked at compile time. It's not just solvable, it's trivial to solve.

That's neither the general case nor does this imply your indices are in your static bounds.

[Brendan]

Hi,

For anything that could use an arbitrary sized array, you can use a pool of fixed size arrays instead. Fixed size arrays can be checked at compile time. It's not just solvable, it's trivial to solve.

That's neither the general case nor does this imply your indices are in your static bounds.

We've been through variable range analysis earlier in this topic. It's not that hard to determine if the range of values a variable could possibly hold are all within the bounds of a fixed size array.

If you define "general case" as "all the cases where it's impossible" and then claim it's impossible; then you're right (in a purely pointless and irrelevant way).

More importantly, you're looking at the risk of security problems caused by C, making the mistake of assuming that all unmanaged languages have to be exactly like C and reaching the conclusion that all unmanaged languages have unavoidable security problems. It's like looking at a colander and deciding that it's impossible to make a bowl that holds water.


Cheers,

Brendan

[HoTT]

M3ore importantly, you're looking at the risk of security problems caused by C, making the mistake of assuming that all unmanaged languages have to be exactly like C and reaching the conclusion that all unmanaged languages have unavoidable security problems

I am not.

We've been through variable range analysis earlier in this topic. It's not that hard to determine if the range of values a variable could possibly hold are all within the bounds of a fixed size array.

It is not. Counter examples are given as well.

If you define "general case" as "all the cases where it's impossible" and then claim it's impossible; then you're right (in a purely pointless and irrelevant way).

The general case is where both your index and your array size may be the result of computation possibly involving input.

Of course you can say that all your arrays are of length 255 and are indexed by an 8-bit integer type. But then you introduced a big limitation already, one that I don't think is practical.