OSDev.org

HoTT wrote:The general case is where both your index and your array size may be the result of computation possibly involving input.

Of course you can say that all your arrays are of length 255 and are indexed by an 8-bit integer type. But then you introduced a big limitation already, one that I don't think is practical.

HoTT wrote:

You can use safe language, but it requires another parts of managed environment like garbage collector,

Garbage collection is not unique to managed environments. There have been GCs long before that term (managed X) got coined. But why do you need a garbage collector to avoid errors that relate to endianess? Could you give an example of an error and how it is avoided using a GC?

HoTT wrote:

No. The world has Java runtimes, that are implemented for many platforms. So, when anybody would use my platform, then the Java's platform independence will be given for free.

Not for free, someone has to implement that runtime.

Finally, what if I said "all arrays of type T are of length Tn and must be indexed by an integer type that is either only capable of storing values from 0 to Tn-1 or can be proven to only hold a value from 0 to Tn-1, for any value of Tn; but there can be many different array types each with their own independent Tn"?

Is it still too limited?

GC is part of a managed language. If there is no automatic memory management then the language in unmanaged. Once you have decided to use managed language (because of endiannes issues, for example), there is no choice but to use it with some memory management facilities like GC.

• a language with a GC is managed
• endianess problems can only be solved in a managed language.

Brendan wrote:I'm thinking of an unmanaged language combined with a managed environment for developers to use for debugging, like C and valgrind (but not C or valgrind); and I'm not thinking of a managed language (and managed environment) that wraps programmers in bubble wrap and prevents them from doing their job properly.

Brendan wrote:If DEFLATE was implemented as a service; then the cost of my messaging is around 1000 cycles per message (for a max. message size of 2 MiB; assuming both sender and receiver are on the same computer). To send 1 GiB of data to a "DEFLATE service" and receive 2 GiB of data back (assuming a 2:1 compression ratio) it's going to need to be split into 1536 messages and cost about 1536000 cycles for all sending and receiving. On an ancient 25 MHz 80486 (the oldest/slowest/worst CPU I intend to support) that works out to about 62 ms. Compared to the cost of compressing/decompressing the data, the overhead of messaging is almost entirely irrelevant. If it happened regularly nobody would care.

Brendan wrote:How do you convince end users to switch from existing OSs (that are more mature - faster, less buggy and with more drivers) to your OS, when the applications run on both OSs? "Same but worse" is unlikely to be an good marketing strategy.

Brendan wrote:

embryo wrote:

Brendan wrote:I also want a language where unit testing is an inbuilt feature; and an IDE that automatically runs the unit tests in the background and highlights things that failed their unit tests while you type. You probably haven't thought about this, but I think it's important to detect all the bugs that can't be detected automatically (with or without run-time tests).

Here I miss the point of unit tests. Who will write them? If it is a developer, then such "inbuilt" feature works just as an ordinary unit test runner program. I mean I see no difference between the "inbuilt" and traditional approaches.

The most obvious benefit is integration - e.g. having mistakes highlighted for you while you're working on the source code in the IDE (and not having additional hassle, or steps to accidentally forget, or having to manually translate the unit test results into a location in the source code).

The less obvious benefit is that it provides "default scenarios" for debugging. For example, maybe there's a problem with an "isNumberPrime()" function buried deep within the program, and instead of running the entire program in the debugger you can just run an existing unit test in the debugger without all the unnecessary baggage.

Brendan wrote:but I'm using asynchronous messaging where the service may be running on a completely different computer and there's some other things that come may into it.

Brendan wrote:How does the managed environment "manage the low level parts" (e.g. prevent intentionally malicious assembly language code from doing unsafe things)?

Brendan wrote:I think you'll find that for all the security vulnerabilities most would've been prevented by a better unmanaged language and the remainder would not have been prevented by a managed language.

HoTT wrote:I think it's to cumbersome to please a dependent type system for every use case

HoTT wrote:What not follows is that

• a language with a GC is managed
• endianess problems can only be solved in a managed language.

embryo wrote:

HoTT wrote:I think it's to cumbersome to please a dependent type system for every use case

There are examples of such (and very usable) type systems. In Java it is ArrayList class. It holds an array of a suitable size and changes the array every time the size is found to be unsuitable.

You can set the claims from above as your goal and next to implement it, but would it be an efficient approach?

If you have to fight one problem it doesn't mean that you should refuse to fight all other possible problems. It means if you are fighting endiannes with safe language then it can be much more efficient, than if you would fight the endiannes only.

Brendan wrote:If DEFLATE was implemented as a service; then the cost of my messaging is around 1000 cycles per message (for a max. message size of 2 MiB; assuming both sender and receiver are on the same computer). To send 1 GiB of data to a "DEFLATE service" and receive 2 GiB of data back (assuming a 2:1 compression ratio) it's going to need to be split into 1536 messages and cost about 1536000 cycles for all sending and receiving. On an ancient 25 MHz 80486 (the oldest/slowest/worst CPU I intend to support) that works out to about 62 ms. Compared to the cost of compressing/decompressing the data, the overhead of messaging is almost entirely irrelevant. If it happened regularly nobody would care.

However; it won't happen regularly either. PNG files would be converted into the OS's native file format as soon as possible (preferably, before they're stored in the OS's file system) and never converted from the OS's native file format back into PNG. None of the OS's native file formats use DEFLATE (including the OS's native "compressed file format").

HoTT wrote:

Finally, what if I said "all arrays of type T are of length Tn and must be indexed by an integer type that is either only capable of storing values from 0 to Tn-1 or can be proven to only hold a value from 0 to Tn-1, for any value of Tn; but there can be many different array types each with their own independent Tn"?

Problem marked bold. It's what I said prior in the thread. If it cannot be proven, you'll have to insert a runtime check somewhere to make it provable.

int32_t foo(uint8_t x) {
                           // the value in 'x' must range from 0 to 255 here
    uint32_t y = 4;        // the value in 'y' must range from 4 to 4 here
    uint32_t z = 0;        // the value in 'z' must range from 0 to 0 here

    y += x;                // the value in 'y' must range from 4 to 259 here
    if(y < 100) {
                           // the value in 'y' must range from 0 to 99 here
       z = x/2;            // the value in 'z' must range from 0 to 127 here
    } else {
                           // the value in 'y' must range from 100 to 259 here
       z = x + y;          // the value in 'z' must range from 100 to 514 here
    }
                           // the value in 'z' must range from 0 to 514 here
    if( (z & 1) == 1) {
                           // the value in 'z' must range from 0 to 514 here
        z = z & 0xFF;      // the value in 'z' must range from 0 to 255 here
        y = y ^ foo(z);    // the value in 'y' must range from 0 to 0xFFFFFFFF here
    }
                           // the value in 'z' must range from 0 to 514 here
                           // the value in 'y' must range from 0 to 0xFFFFFFFF here
    while(z > 0) {
                           // the value in 'z' must range from 0 to 514 here
       z--;                // the value in 'z' must range from 0 to 513 here
                           // the value in 'y' must range from 0 to 0xFFFFFFFF here
       y = y & 1234;       // the value in 'y' must range from 0 to 1234 here
    }
                           // the value in 'y' must range from 0 to 0xFFFFFFFF here
    return y;
}

HoTT wrote:

Is it still too limited?

Depends. Say strings are just arrays of utf8 codeunits and I want to write an algorithm that checks for any possible string if it only contains valid utf8.

How would it be typed?

embryo wrote:

Brendan wrote:I'm thinking of an unmanaged language combined with a managed environment for developers to use for debugging, like C and valgrind (but not C or valgrind); and I'm not thinking of a managed language (and managed environment) that wraps programmers in bubble wrap and prevents them from doing their job properly.

Well, here you refuse to accept "a bubble wrap" because it "prevents them from doing their job properly". But in the following:

Brendan wrote:If DEFLATE was implemented as a service; then the cost of my messaging is around 1000 cycles per message (for a max. message size of 2 MiB; assuming both sender and receiver are on the same computer). To send 1 GiB of data to a "DEFLATE service" and receive 2 GiB of data back (assuming a 2:1 compression ratio) it's going to need to be split into 1536 messages and cost about 1536000 cycles for all sending and receiving. On an ancient 25 MHz 80486 (the oldest/slowest/worst CPU I intend to support) that works out to about 62 ms. Compared to the cost of compressing/decompressing the data, the overhead of messaging is almost entirely irrelevant. If it happened regularly nobody would care.

You have accepted another kind of "bubble wrap", but now it is called "services" with message passing related overhead. And you are eager to argue that in some cases the overhead is acceptable, but you refuse to listen to arguments about another "bubble wrap" and it's related pretty acceptable overhead. So, I conclude that it is just your religion, that prevents you from accepting the same thing when it is introduced by somebody else but you. But hopefully, the religion canons can be relaxed in the future and you finally will accept managed approach as efficient enough to compensate any involved overhead with really useful features.

embryo wrote:

Brendan wrote:How do you convince end users to switch from existing OSs (that are more mature - faster, less buggy and with more drivers) to your OS, when the applications run on both OSs? "Same but worse" is unlikely to be an good marketing strategy.

It's more efficient. And it's users (and developers) are more productive.

embryo wrote:

Brendan wrote:

embryo wrote:Here I miss the point of unit tests. Who will write them? If it is a developer, then such "inbuilt" feature works just as an ordinary unit test runner program. I mean I see no difference between the "inbuilt" and traditional approaches.

The most obvious benefit is integration - e.g. having mistakes highlighted for you while you're working on the source code in the IDE (and not having additional hassle, or steps to accidentally forget, or having to manually translate the unit test results into a location in the source code).

The less obvious benefit is that it provides "default scenarios" for debugging. For example, maybe there's a problem with an "isNumberPrime()" function buried deep within the program, and instead of running the entire program in the debugger you can just run an existing unit test in the debugger without all the unnecessary baggage.

The approach (as you describe it) was implemented long ago with the help of different plugins for different IDEs. But it was found inconvenient to run unit tests concurrently with code writing activity. It slows an IDE and produces permanent distraction for a developer because the code is just not ready to be unit tested. So, as a final solution a button "run unit tests" was chosen.

embryo wrote:

Brendan wrote:How does the managed environment "manage the low level parts" (e.g. prevent intentionally malicious assembly language code from doing unsafe things)?

It doesn't prevent all forms of malicious behavior. But it manages such code in the following way - it detects such code fragments, next it notifies user that provided application will be run in dangerous mode, next it accepts user's decision, next it compiles managed and unmanaged parts of the program, next it runs resulting program under hardware protection (and prevents some malicious activity from being possible). And all specified actions are acts of management, so we can see the management is still in place.

Rusky wrote:

Brendan wrote:If DEFLATE was implemented as a service; then the cost of my messaging is around 1000 cycles per message (for a max. message size of 2 MiB; assuming both sender and receiver are on the same computer). To send 1 GiB of data to a "DEFLATE service" and receive 2 GiB of data back (assuming a 2:1 compression ratio) it's going to need to be split into 1536 messages and cost about 1536000 cycles for all sending and receiving. On an ancient 25 MHz 80486 (the oldest/slowest/worst CPU I intend to support) that works out to about 62 ms. Compared to the cost of compressing/decompressing the data, the overhead of messaging is almost entirely irrelevant. If it happened regularly nobody would care.

However; it won't happen regularly either. PNG files would be converted into the OS's native file format as soon as possible (preferably, before they're stored in the OS's file system) and never converted from the OS's native file format back into PNG. None of the OS's native file formats use DEFLATE (including the OS's native "compressed file format").

Neither of these assumptions hold true for all uses of DEFLATE. Dealing with files is not particularly latency-sensitive, but on-the-fly compression/decompression as part of a network protocol (a multiplayer game? real-time data stream of some sort?), or compressed-in-RAM virtual memory (as opposed to storing it on disk), or other high-performance uses of compression, might not work out so well.

Rusky wrote:Your signature (which I assume you'd apply here) says scalable slow code can beat unscalable fast code, but scaling goes both directions and at different granularities. I will preemptively agree with you that overly-generic code is harmful, but there will always be pieces of code that need to be shared with applications that can't handle the latency of potentially-networked message passing.

Brendan wrote:Finally, what if I said "all arrays of type T are of length Tn and must be indexed by an integer type that is either only capable of storing values from 0 to Tn-1 or can be proven to only hold a value from 0 to Tn-1, for any value of Tn; but there can be many different array types each with their own independent Tn"?

Surely you can see this is just another minor extension. It's still easy to prove there are no bounds problems. Is it still too limited?

fn insert(x) {
    int pos = ++size;

    // this check must be enforced or heap could be accessed past its end
    if (pos >= capacity)
        /* resize the array, or error */

    while (pos > 0 && x < heap[(pos - 1) / 2]) {
        heap[pos] = heap[(pos - 1) / 2];
        pos /= 2;
    }

    heap[pos] = x;
}

fn remove() {
    result = heap[0];
    heap[0] = heap[size--];

    int pos = 0;

    // this check must be enforced
    while (pos * 2 + 1 < size) {
        int child = pos * 2 + 1;

        // the first check here check must be enforced
        if (child + 1 < size && heap[child] > heap[child + 1])
            child = child + 1;

        if (heap[pos] < heap[child])
            return;

        swap(heap[pos], heap[child]);
        pos = child;
}