OSDev.org

Brendan wrote: I don't know why you think it's a problem - it's not.

1. Halting Problem
2. In practice it's not possible, see the works of Cousot, it's e.g. explicitly mentioned here
3. Easy to construct an example involving input.
4. Integer arithmetic is undecidable.

HoTT just beat me to it

Dependent typing can dramatically increase the number of bounds checks that can be elided by a compiler, but type checking is fundamentally undecidable and some runtime checks cannot safely be avoided.

Brendan wrote:The best solution for latency-sensitive things is unlikely to be DEFLATE - compression times aren't great.

Fine, use LZO like Linux's zram implementation. In any case, HTTP gzip compression also works fine, and I wouldn't be surprised if video/audio codecs for live streaming/communication also had problems with potentially-networked message passing. In fact, one of the biggest reasons to use compression is to trade computational power against network bandwidth, so offloading it and trying to send the uncompressed data over your LAN is worse than pointless. None of this stops you from putting the code in a service if you really want to, but it had better not be the only way to use it.

Brendan wrote:There's definitely pieces of code where the (potential) network latency is unacceptable. Some are tiny (easier to re-implement than to find a pre-existing implementation), some are too small to matter ("cut & paste" is good enough), some are overly-general, some I don't care about (XML parsing).

I can't think of anything that doesn't fit in one of those categories. This doesn't mean there are none; but does imply it won't be common enough to be important.

Various forms of compression, audio/video codecs, encryption, bignum libraries like libgmp, compiler support libraries like libc/libgcc for things like optimized memcpy, scientific and numeric computing libraries, file formats that don't make sense to have a standardized converter, libraries implementing protocols for services, Unicode handling, ....

Hi,

HoTT wrote:

Brendan wrote: I don't know why you think it's a problem - it's not.

1. Halting Problem

Wrong. For any case where it's impossible to determine the actual minimum range of values a variable could possibly hold, you just revert to the minimum range of the data type.

HoTT wrote:2. In practice it's not possible, see the works of Cousot, it's e.g. explicitly mentioned here
3. Easy to construct an example involving input.

Wrong. The only reason it's "impossible" is the language specification. If the language specification says "all array indexes are assumed valid unless the compiler can prove they're out of range" then it's impossible; and if the language specification says "all array indexes are assumed erroneous unless the compiler can prove they're not out of range" then it's trivial.

HoTT wrote:4. Integer arithmetic is undecidable.

Irrelevant. We're not doing mathematical theory, we only need to care about operators programming languages support, and all of those are entirely decidable.


Cheers,

Brendan

all array indexes are assumed erroneous unless the compiler can prove they're not out of range" then it's trivial.

This will leave correct problems untypeable, which makes it undecideable, but is the approach I like as well.

Hi,

HoTT wrote:

all array indexes are assumed erroneous unless the compiler can prove they're not out of range" then it's trivial.

This will leave correct problems untypeable, which makes it undecideable, but is the approach I like as well.

It means you can guarantee there's no overflows and that all static array indices are within bounds, at compile time.

The main consequence is there's a few "false negatives" where something is actually safe but the compiler can't figure that out and generates an error anyway. The other consequence is that as the compiler improves you get less false negatives, which means that source code that compiles without errors on "compiler version 2" may not compile on "compiler version 1". I'd consider both of these "quirks" rather than things that actually matter in practice - the false negatives are minor, and programmers rarely downgrade their compiler.

Note that I use the function's input and output argument types as a binding contract. For example, for "uint8_t foo() { return 3; }" my compiler assumes the function returns a value from 0 to 255 without caring what the range of the returned value actually is. This means that if you change the code (e.g. to "uint8_t foo() { return 9; }") but don't change the contract, then all calling code will still be fine. It also means that (once function declarations and global data have been determined) the compiler can analyse functions in isolation (in parallel). Finally, it removes a major cause of false negatives.

With that in mind; the only cause of false negatives is complicated loop conditions.

For something like "int y = 0; for(int x = 0; x < 20; x++) { y += 2; }" it's relatively easy to determine the range of all variables during and after the loop, and prove it's fine (and replace the entire loop with "x = 20; y = 40;" as an optimisation).

For something like "int y = 0; for(int x = 0; (x & 5) == 3; x = (x+100)/3) { y += 2; }" it's more complicated, so the compiler may just give up and assume that both variables have a range from INT_MIN to INT_MAX during the loop. In that case the compiler would decide that y could be INT_MAX and that "y += 2;" is a potential overflow, so you get a compile time error.


Cheers,

Brendan

Brendan wrote:It means you can guarantee there's no overflows and that all static array indices are within bounds, at compile time.

Come to think of it, I rarely use static arrays in desktop applications.

Hi,

MessiahAndrw wrote:

Brendan wrote:It means you can guarantee there's no overflows and that all static array indices are within bounds, at compile time.

Come to think of it, I rarely use static arrays in desktop applications.

For C, I typically just use "malloc()", even though I know it has multiple problems. For C++ I probably wouldn't use "malloc()" at all (but would use "new" instead, as that's what you do for that language).

For a language designed to encourage the use of static arrays (combined with an OS that you know does "allocation on demand", so you don't need to care about having a massive ".bss" for your pools of statically allocated arrays); it's likely people would do what is natural for that language (instead of trying to do whatever is natural for some other language).


Cheers,

Brendan

HoTT wrote:With dependent type systems I mean this. Javas ArrayList does not qualify, it's length is not part of its type.

Why object's property is not a part of it's type?

HoTT wrote:I don't know what you're aiming at. But your constantly attributing advantages that some managed ecosystems/evironments have to the fact that they are managed. Which is not true.

Well, then what is a reason for managed environments to have any advantages?

HoTT wrote:And I'll point out again that while managed might imply safe, safe does not imply managed. Same for GC.

For something to be safe we need some work to be done. But who will do the job?

HoTT wrote:What was the problem with endianess again?

It's not safe and you will pay a bug and development speed fee for it. Isn't it just obvious?

Why object's property is not a part of it's type?

In Java the property is part of the objects type, but it's value is not.

It's not safe and you will pay a bug and development speed fee for it. Isn't it just obvious?

Nope, never had a bug due to endianess. Care to elaborate how such thing could look like and how a managed environment prevents that?


And what do you have in mind if you call something safe or not safe? I'm thinking memory safety, i.e. no stack corruption, array access in bounds, etc.

Brendan wrote:The main reason I'm going for "services" is for scalability (e.g. being able to use many CPUs in many separate computers in a distributed system). It's not about "bubble wrap" at all.

If you mean there is no similarity between wrapping a code within a managed environment and wrapping parameters and return value within a message, then I just don't know how to explain it better.

But at least I see you are trying to trade some benefits (like scalability) for performance. And you just refuse to accept such trade when we talk about reliability and security.

May be you can implement some very restrictive compiler, that will refuse to compile if it is unable to infer a variable's value, but what development speed decrease it will lead to? Here we have another trade, when development speed is traded for (may be) better execution speed. But why the same thing (execution speed) is traded back and forth? And the answer is that your priorities dictate such trade. But if your priorities allow to trade the speed in whatever manner, then why, for example, my priorities can not trade the speed for something else? Do you see the importance of priorities? You just can't get everything better and need to trade something less important for something more important. And it's actually the same trade as is the case for managed systems, but with different priorities.

And if we speak about just execution speed and software reliability, then may be your approach is good enough. But if it is about things like development speed, then your approach just squeezes a programmer with help of a too restrictive compiler.

Brendan wrote:So, the exact same Java application will run more efficiently on your OS than it does on both Windows and Linux? That's nice...

Yes, it's nice. But may be it would be more correct if we see it as a general performance problem, including execution and development time with addition of software usage costs.

Brendan wrote:I sit in a room surrounded by a pool of 25+ computers on a LAN. Almost all of them are idle almost all of the time. Are you suggesting there isn't enough processing power here for one computer to be editing text while 10 other computers do unit tests?

No, you have enough computer power, but you are not the only person who has some power. And other people had decided long ago that a button "run unit tests" is a really good solution. But here again we should remember that your preferences can be different.

Brendan wrote:Of course there would be some logic built in. E.g. there's no point running the unit tests while the source is constantly being modified - you have some sort of time delay (if the code the unit test relies on hasn't been modified for 60+ seconds, then start the unit test). You also wouldn't repeat tests if the pieces of source code they test weren't modified since last time.

60+ seconds is too much. There is just one moment, when I want to start unit tests, and this moment usually happens a way before 60+ seconds after I have my code ready for testing. But if instead of 60+ seconds you switch to 10, 20, 30 or whatever, then again the moment when I want to see the unit tests results is still unpredictable and it happens just once per relatively big chunk of work. So, your 20-40 seconds interval definitely will make an annoyance for many developers, even if the difference between the moment a developer wants it and the actual tests start will be just 5 seconds. And if it will be more than 5 seconds, then all developers will cry "where our 'run tests' button?".

Brendan wrote:In that case, it'll be like UAC in Vista, where everything caused the dialog box to pop up and users clicked on it without thinking (until they found a way to disable UAC).

The UAC issue in fact is not so important. I have it turned on and see no problem if it disturbs me once in a week or even a month.

And about asking user in general - we just can select some better time to ask. For example, asking a user at the time he installs an application is just as natural as it can be on Windows, because all installers always ask something and those who don't are perceived as abusing user's trust.

HoTT wrote:Care to elaborate how such thing could look like and how a managed environment prevents that?

It looks like if you write a binary file on one platform and read it on another.

And because a managed environment manages the endiannes issues, it will never be possible to write and read a file in a different manner on different platforms using the same program.

HoTT wrote:And what do you have in mind if you call something safe or not safe? I'm thinking memory safety, i.e. no stack corruption, array access in bounds, etc.

Well, as it was said, we need another thread - a thread about safe languages.

But at least I see that memory safety is not the only problem. There is a process of a code execution that can jump to some wrong address, is it a memory issue? And endiannes is also can be seen as a memory issue, because on different platforms we can get different memory content.

And because a managed environment manages the endiannes issues, it will never be possible to write and read a file in a different manner on different platforms using the same program.

How do you read a binary format that happens to use the opposite endianess using a managed language?

Hi,

embryo wrote:

Brendan wrote:The main reason I'm going for "services" is for scalability (e.g. being able to use many CPUs in many separate computers in a distributed system). It's not about "bubble wrap" at all.

If you mean there is no similarity between wrapping a code within a managed environment and wrapping parameters and return value within a message, then I just don't know how to explain it better.

But at least I see you are trying to trade some benefits (like scalability) for performance. And you just refuse to accept such trade when we talk about reliability and security.

For what I'm doing (and for the intended usage of my OS), the advantages for end users (scalability) far outweigh the disadvantages for end users (message passing overhead).

For what you're doing, the advantages for end users (in very rare cases where "correct" code actually has bugs, crashing because a managed environment detected a bug at run-time instead of crashing differently because a bug wasn't detected at run-time) doesn't outweigh the disadvantages for end users (slower software and a much more bloated environment).

embryo wrote:May be you can implement some very restrictive compiler, that will refuse to compile if it is unable to infer a variable's value, but what development speed decrease it will lead to? Here we have another trade, when development speed is traded for (may be) better execution speed.

I'm not sure what you're comparing against here. Think of it like this:

• If compiler does compile-time checking:
  • If the resulting code is executed in an unmanaged environment:
    • Compiler takes a little longer to compile code
    • Fastest development time (least time spent finding bugs)
    • Fastest execution speed
    If the resulting code is executed in an managed environment:
    • Compiler takes a little longer to compile code
    • Fastest development time (least time spent finding bugs)
    • Slowest execution speed
  If compiler doesn't do the compile-time checking:
  • If the resulting code is executed in an unmanaged environment:
    • Compiler takes a little less time to compile code
    • Slowest development time (more time spent finding bugs)
    • Fastest execution speed
    If the resulting code is executed in an managed environment:
    • Compiler takes a little less time to compile code
    • Second fastest development time (because you have to actually execute the code before the managed environment can find the bugs, which might mean "100% code coverage" unit tests)
    • Slowest execution speed

Note 1: For fairness, I assume the language is the same in all cases (it'd be foolish to distort the comparison by comparing different languages like "assembly vs. python" or something equally silly).

Note 2: Things that effect the end user are in bold, as they're far more important.

embryo wrote:

Brendan wrote:So, the exact same Java application will run more efficiently on your OS than it does on both Windows and Linux? That's nice...

Yes, it's nice. But may be it would be more correct if we see it as a general performance problem, including execution and development time with addition of software usage costs.

I'm sorry - I failed to do anything to make it clear that I was being sarcastic. Microsoft have been improving their OS for about 20 years, and Sun/Oracle have been improving their JVM for about 20 years too. Both Microsoft and Oracle also have far more developers working on it than you do. Imagine a turtle and a cheetah both competing in a marathon, where the cheetah is given a 50 km head start and the turtle has 4 broken legs. The chance of the turtle catching up to the cheetah is about the same as the chance that your OS and JVM will catch up to Windows and Oracle's JVM.

embryo wrote:

Brendan wrote:Of course there would be some logic built in. E.g. there's no point running the unit tests while the source is constantly being modified - you have some sort of time delay (if the code the unit test relies on hasn't been modified for 60+ seconds, then start the unit test). You also wouldn't repeat tests if the pieces of source code they test weren't modified since last time.

60+ seconds is too much. There is just one moment, when I want to start unit tests, and this moment usually happens a way before 60+ seconds after I have my code ready for testing. But if instead of 60+ seconds you switch to 10, 20, 30 or whatever, then again the moment when I want to see the unit tests results is still unpredictable and it happens just once per relatively big chunk of work. So, your 20-40 seconds interval definitely will make an annoyance for many developers, even if the difference between the moment a developer wants it and the actual tests start will be just 5 seconds. And if it will be more than 5 seconds, then all developers will cry "where our 'run tests' button?".

That's easily fixed - just have a "bluild the executable" button, and do any pending unit tests before compiling. However; that too would miss the point.

Mostly what I want is real-time feedback for all bugs. I don't think this is possible in practice, so I'd want to get as close to that as possible in practice. Maybe every time you move the cursor away from a modified line of text the IDE cancels any effected unit tests and starts the new test/s immediately.

embryo wrote:

Brendan wrote:In that case, it'll be like UAC in Vista, where everything caused the dialog box to pop up and users clicked on it without thinking (until they found a way to disable UAC).

The UAC issue in fact is not so important. I have it turned on and see no problem if it disturbs me once in a week or even a month.

And about asking user in general - we just can select some better time to ask. For example, asking a user at the time he installs an application is just as natural as it can be on Windows, because all installers always ask something and those who don't are perceived as abusing user's trust.

UAC has improved a lot since it was originally introduced; but when it was first introduced it was useless because everyone ignored and disabled it.

If you make "managed" optional; then programmers will want faster performance for their software and force users to run the application as "unmanaged". There will be no advantages (compared to an unmanaged OS) because it will be an unmanaged OS (because "managed" is disabled for everything anyway).


Cheers,

Brendan

HoTT wrote:How do you read a binary format that happens to use the opposite endianess using a managed language?

First of all, a managed environment just never writes a file with opposite endiannes. So, there's just no reason for the problem to exist.