Brendan's OS Design Case Study

[embryo2]

This is the reply to this thread conversation.

If the user doesn't want the extra load caused by redundancy, then they wouldn't have enabled it in the first place.

Yes, so if your OS will have all required configuration capabilities and some means for a programmer to configure it's actions at run time, then I see no problem with it's unexpected/unwanted behavior.

I'm only pointing out that people who think VMs actually help rely on flawed logic - they assume it's impossible for programmers to write software without bugs while at the same time they assume that programmers can write a VM without bugs.

Yes, bugs are possible even in the VM code, but the maturity of the basic component of a system is always makes it less buggy than you expect. The situation is just like with any other service software from OS and to network support, databases, file systems and so on. The service layer adds some benefits to the system despite of possible bugs.

The goal is to allow a collection of 1 or more computers to behave like a single system; for the purpose of maximising performance and/or minimising hardware costs. For a simple example, instead of having an office for 20 people with one computer per user plus a server (21 computers) where most computers are idle most of the time, you could have an office for 20 people with 4 people per computer and no server that's faster (due to resource sharing and effective utilisation of resources) and a lot cheaper (5 computers and not 21).

Partially your goal was achieved by using virtualization, when low end (and cheap) computers are used as terminals and actual job is done by one powerful server. But your case is more complex and supposes some splitting of the job among all available computers. Such splitting (even with new programming paradigm) is a non trivial problem.

"Cloud" (paying for remote processing) is fine when you've got a massive temporary job. For a massive permanent job it's stupid (cheaper to buy the hardware yourself instead of paying to use someone else's and paying for internet costs);

For massive permanent job it's cheaper. It was discussed before, but I can repeat - the economy of scale can decrease costs a lot.

and for thousands of tiny separate little jobs (e.g. where the latency is far more important than the processing) it's stupid.

It depends on the latency. If it is measured in tens of seconds then yes, it's unacceptable. But if it's about one second or even less, then it can be perfectly acceptable for many kinds of jobs.

Of course even for the very rare cases where cloud might make sense in theory, you have to trust someone else with your data and (for sensitive data) it might not make sense in practice due to lack of trust.

Well, suppose you have a medium sized enterprise and use cloud of some big company. What do you expect the big company would do with your data? Would it sell it to competitors of the same size? But then your enterprise can buy the data of the competitors in the same manner they can buy your data and it is enough to start a successful process in the court. If the big company would sell the data to another big company then what is the purpose of the data for the buyer? Big boys need data about biggest part of all companies and next they need some very expensive department that is able to analyze the gathered data (without knowing it's structure and purpose), so it looks like a whole CIA staff will be working for such task to be completed. Isn't it look a bit overextended?

Corporate world has it's ways to get required information without Google or other cloud providers, so your trust doubts look a bit unreal.

Finally, for the extremely rare cases where "cloud" isn't stupid; there's no reason why someone using my OS wouldn't be able to rent some servers on the internet, put my OS on those servers, and make those servers part of their cluster.

For your OS to be used as a cloud it should include many configuration tools, easy to use and available on the net. But it's an easier part of your goal.

For normal software (e.g. "oversized monolithic blob" applications using procedural programming) the cost of distributing it across multiple machines is too high (and quite frankly, most of the victims of the "oversized monolithic blob using procedural programming" approach are struggling just to use 2 or more CPUs in the same computer). To be effecitve, the "oversized monolithic blob using procedural programming" stupidity has to be discarded and replaced with something suitable, like "shared nothing entities that communicate". The "shared nothing entities that communicate" approach is already used for almost everything involving 2 or more computers.

The communicating entities boil down to the ability of some entity to extract independent subtasks from a bigger one. Such parallelization efforts still deliver no viable results, so you are going to outperform the whole academic society.

Fred is using an application. That application communicates with Fred's GUI (sending video and sound to the GUI, and receiving keyboard, mouse from the GUI). Fred tells his GUI to send the application to Jane's GUI. Now the application communicates with Jane's GUI. No process was stopped or started, nothing was written to disk.

Or the application just serializes it's current state and sends it to the Jane's application. No process was stopped or started, nothing was written to disk.

The second type of teamwork is more like pair programming. Imagine a small group of developers that have headsets and are constantly talking/listening to each other and watching each other type and correcting each other's typos while they're typing. In this case team members do not work in isolation, but work on the same task at the same time. For programming; there are existing solutions for pair programming (including collaborative real-time editors), but they're not widespread.

The goal would be to provide both types of teamwork in a way that allows people to use either type of teamwork whenever they want however they want (including "ad hoc" where no prior arrangements are made); and seamlessly integrate it all into the IDE so that it's a single consistent system rather than an ugly and inflexible collection of separate tools.

But collaborative team work is done using one computer with one person typing and another looking and suggesting improvements. In your case both need headset and computer just to do the same job they did without the headsets and additional computer.

Why don't you think a maintenance tool that suits almost everyone is possible? Can you think of anything people might want in a maintenance tool that is impossible?

Everything is possible, but there's still no maintenance tool that suits almost everyone. One prefers red while another prefers green and satisfying both is at least very costly. And much worse, there are many existing maintenance tools that are integrated with some other corporate software, so your task now is to integrate your tool with all possible corporate software. Yes, it is possible, but you know about required time.

[Brendan]

Hi,

Yes, bugs are possible even in the VM code, but the maturity of the basic component of a system is always makes it less buggy than you expect. The situation is just like with any other service software from OS and to network support, databases, file systems and so on. The service layer adds some benefits to the system despite of possible bugs.

This can probably be expressed as a formula; like:

• bugs = lines_of_code * complexity_of_code * (1 - developer_skill) / (compile_time_bug_detection/3 + developer_bug_detection/3 + maturity/3)

Where all the values are between 0 and 1.0 except for lines_of_code.

Of course that's only for one piece. To find the total bugs you'd have to add up the bugs for all the pieces (in the code the application programmer wrote, in each library, in the kernel, whatever).

For example, for a simple "Hello World" application you'd expect zero bugs for the application code, maybe 1 bug in a library and maybe 5 bugs in the kernel. Then you add a VM on top because some fool decided it'd help "reduce" bugs and the total bugs for "Hello World" increases by several orders of magnitude. In this case, because "Hello World" is so simple, using a VM is incredibly idiotic.

For a medium sized application with medium complexity it's a little better; but most of the bugs are caught by compiler or unit tests or users; so using a VM (even a "mature" VM) still adds more bugs than it solves and just makes things worse.

For a VM to make sense; you need an extremely large and complicated application so that "bugs prevented by VM" is actually larger than "bugs introduced by VM". Applications like this are extremely rare.

However; this relies on a false assumption. I've incorrectly assumed that a VM prevents bugs. The fact is that the VM doesn't prevent bugs, and only makes a small number of bugs easier to detect, and only if the compiler failed to detect them in the first place.

Basically; there is no case where VM makes anything better. It's just a retarded waste of time that always makes things worse.

"Cloud" (paying for remote processing) is fine when you've got a massive temporary job. For a massive permanent job it's stupid (cheaper to buy the hardware yourself instead of paying to use someone else's and paying for internet costs);

For massive permanent job it's cheaper. It was discussed before, but I can repeat - the economy of scale can decrease costs a lot.

You were wrong before and you are wrong now. "Economy of scale" is minor and (for massive permanent jobs) things like provider markup/profit, internet costs, latency, configuration hassles, etc surpass any benefits by a wide margin.

and for thousands of tiny separate little jobs (e.g. where the latency is far more important than the processing) it's stupid.

It depends on the latency. If it is measured in tens of seconds then yes, it's unacceptable. But if it's about one second or even less, then it can be perfectly acceptable for many kinds of jobs.

For anything where users are involved you've got about 100 ms between the user pressing a key and the screen being updated before the system feels sluggish. For most of the internet I'm lucky to see "ping" report times that are better than 200 ms despite the fact that no processing is involved and the data is tiny. Basically; the internet is too slow to handle "doing nothing" with acceptable latency. For ethernet/LANs latency is typically better than 1 ms; so you can have distributed applications that feel like they respond to the user instantaneously despite the networking latency.

Of course even for the very rare cases where cloud might make sense in theory, you have to trust someone else with your data and (for sensitive data) it might not make sense in practice due to lack of trust.

Well, suppose you have a medium sized enterprise and use cloud of some big company. What do you expect the big company would do with your data? Would it sell it to competitors of the same size? But then your enterprise can buy the data of the competitors in the same manner they can buy your data and it is enough to start a successful process in the court. If the big company would sell the data to another big company then what is the purpose of the data for the buyer? Big boys need data about biggest part of all companies and next they need some very expensive department that is able to analyze the gathered data (without knowing it's structure and purpose), so it looks like a whole CIA staff will be working for such task to be completed. Isn't it look a bit overextended?

Take a look at things like (e.g.) the requirements for storing people's medical records or credit card information, and tell me if it's even legal to use "cloud" for these things in the first place.

For normal software (e.g. "oversized monolithic blob" applications using procedural programming) the cost of distributing it across multiple machines is too high (and quite frankly, most of the victims of the "oversized monolithic blob using procedural programming" approach are struggling just to use 2 or more CPUs in the same computer). To be effecitve, the "oversized monolithic blob using procedural programming" stupidity has to be discarded and replaced with something suitable, like "shared nothing entities that communicate". The "shared nothing entities that communicate" approach is already used for almost everything involving 2 or more computers.

The communicating entities boil down to the ability of some entity to extract independent subtasks from a bigger one. Such parallelization efforts still deliver no viable results, so you are going to outperform the whole academic society.

The same basic "asynchronous messaging" model has been in use by virtually all very scalable systems and virtually all fault tolerant systems for about 50 years now. Just because you don't think you can design software like this doesn't mean that everyone else can't.

Fred is using an application. That application communicates with Fred's GUI (sending video and sound to the GUI, and receiving keyboard, mouse from the GUI). Fred tells his GUI to send the application to Jane's GUI. Now the application communicates with Jane's GUI. No process was stopped or started, nothing was written to disk.

Or the application just serializes it's current state and sends it to the Jane's application. No process was stopped or started, nothing was written to disk.

Jane wasn't running any application before.

To serialise the data you'd need extra code to serialise it, extra code to start the process that receives the data, extra code in the application to receive and parse the data, extra bandwidth to transfer the data, and then extra code to terminate Fred's "now unused" old process.

If Fred was editing a movie and the application was working with 50 GiB of data, then the application would be transferred to Jane instantly.

The second type of teamwork is more like pair programming. Imagine a small group of developers that have headsets and are constantly talking/listening to each other and watching each other type and correcting each other's typos while they're typing. In this case team members do not work in isolation, but work on the same task at the same time. For programming; there are existing solutions for pair programming (including collaborative real-time editors), but they're not widespread.

The goal would be to provide both types of teamwork in a way that allows people to use either type of teamwork whenever they want however they want (including "ad hoc" where no prior arrangements are made); and seamlessly integrate it all into the IDE so that it's a single consistent system rather than an ugly and inflexible collection of separate tools.

But collaborative team work is done using one computer with one person typing and another looking and suggesting improvements. In your case both need headset and computer just to do the same job they did without the headsets and additional computer.

In my case; a person in a computer room in Australia (me) would be able to collaborate with a person that happens to be on a moving train in Europe and a third person that's in bed in America. The headsets were only part of an example (and would be more effective), but that does not mean headsets are required or that people couldn't communicate via. typing/reading instead (or any other means).

Why don't you think a maintenance tool that suits almost everyone is possible? Can you think of anything people might want in a maintenance tool that is impossible?

Everything is possible, but there's still no maintenance tool that suits almost everyone. One prefers red while another prefers green and satisfying both is at least very costly. And much worse, there are many existing maintenance tools that are integrated with some other corporate software, so your task now is to integrate your tool with all possible corporate software. Yes, it is possible, but you know about required time.

You think I should forget about everything because "desktop themes" are impossible and someone might like a different colour??

For integration with "other corporate software" you've got it backwards. That other corporate software would have to integrate with the OS (including the OS's maintenance tools) not the other way around.


Cheers,

Brendan

[embryo2]

Yes, bugs are possible even in the VM code, but the maturity of the basic component of a system is always makes it less buggy than you expect. The situation is just like with any other service software from OS and to network support, databases, file systems and so on. The service layer adds some benefits to the system despite of possible bugs.

This can probably be expressed as a formula; like:

• bugs = lines_of_code * complexity_of_code * (1 - developer_skill) / (compile_time_bug_detection/3 + developer_bug_detection/3 + maturity/3)

Where all the values are between 0 and 1.0 except for lines_of_code.

Ok, let's try the mathematical approach. I agree with the definition of the enumerator, but I disagree with the denominator. Developer bug detection and maturity are closely related and intermixed, so I suppose it is better to combine them in one value.

The compile time bug detection is another issue. I suppose it can be defined as a constant, because if we to compile a large chunk of code it usually contains relatively stable parts of complex and simple code, so the ratio of complex to simple is a constant. I suppose it is universally true for any big enough chunk of code, because whatever system is taken into account there is always a lot of a dumb code that implements some simple algorithms. Even if it is something like AI we still can see a lot of sorting, iterating, comparison, accessing data structures and other simple stuff. But to be precise we can introduce a complexity factor, which describes the ratio of the complex code to the simple one. But even with all this in mind we can remember that whatever part of bugs were detected by the compiler, those bugs are eliminated immediately, so, because there's no more such bugs, it is absolutely legal to remove them from the equation.

Also we should introduce some means of defining newly created bugs that are introduced during system updates. So, the final formula would look like this:

First we can show all caught bugs:

• bugs = lines_of_code * complexity_of_code * (compile_time_bug_detection_constant + developer_bug_detection_factor)

Here developer_bug_detection_factor depends on the skills the development team has while the compile_time_bug_detection_constant is always the same because of the seldom compiler updates.

Next we can show all remaining bugs:

• bugs = sum_for_all_code_chunks(lines_of_code * complexity_of_code) - sum_for_all_code_chunks(lines_of_code * complexity_of_code * overall_bug_detection_by_developers_users_tests * exponent_asymptotically_tending_to_1(1-code_age/max_age)) = sum_for_all_code_chunks(lines_of_code * complexity_of_code * (1- overall_bug_detection_by_developers_users_tests * exponent_asymptotically_tending_to_1(1-code_age/max_age)))

Now we can see that most influential part of the equation involves code age, while all other parts tend to fluctuate near some constants. So, more mature the code is, less bugs it contains and your words about useless system components are much less applicable when time passes.

For example, for a simple "Hello World" application you'd expect zero bugs for the application code, maybe 1 bug in a library and maybe 5 bugs in the kernel. Then you add a VM on top because some fool decided it'd help "reduce" bugs and the total bugs for "Hello World" increases by several orders of magnitude. In this case, because "Hello World" is so simple, using a VM is incredibly idiotic.

Usually the "Hello World" programs are written by beginners, it means that the simplicity of a language is very important. With the help of VM many languages become very easy to learn, so, your claim about "idiotic" use of VM is highly doubtful.

For a medium sized application with medium complexity it's a little better; but most of the bugs are caught by compiler or unit tests or users; so using a VM (even a "mature" VM) still adds more bugs than it solves and just makes things worse.

Here again you forget about developer skills. Developers en mass have relatively limited skills and VMs help them to write reliable applications. So, your claim about "most bugs are caught" is partially implemented by the VM and without the VM the number of bugs will be at unacceptable level.

For a VM to make sense; you need an extremely large and complicated application so that "bugs prevented by VM" is actually larger than "bugs introduced by VM". Applications like this are extremely rare.

And here you forget not only about developer skills, but also about many additional issues, the VM helps us to care of. It is security, reliability, user experience, compatibility, ability to catch bugs quicker and so on.

You were wrong before and you are wrong now. "Economy of scale" is minor and (for massive permanent jobs) things like provider markup/profit, internet costs, latency, configuration hassles, etc surpass any benefits by a wide margin.

Well, let's consider the very simple case - your personal web server. Your approach is about holding everything at home, so all your computers will run some parts of the server. My approach is about efficiency (including financial part), so I rent a virtual server in a cloud. Now let's compare costs. Your tens of computers consume at least 1000 watts while I don't care how much watts consumes the cloud, just because may payment is fixed - 4$ per month for small load server. Now you can multiply 24*30.5 kilowatt-hours by the price of the kilowatt-hour in your location and compare the number with the mentioned 4$.

For anything where users are involved you've got about 100 ms between the user pressing a key and the screen being updated before the system feels sluggish. For most of the internet I'm lucky to see "ping" report times that are better than 200 ms despite the fact that no processing is involved and the data is tiny. Basically; the internet is too slow to handle "doing nothing" with acceptable latency. For ethernet/LANs latency is typically better than 1 ms; so you can have distributed applications that feel like they respond to the user instantaneously despite the networking latency.

It was your words about uselessness of calling string comparison library with your messaging approach, but now you think that it is very useful to call a server on the net for every printed character. May be it worth to remember that some processing is still possible on the client side? And with this in mind we can safely assume, that even latencies on the scale of one second can be perfectly acceptable in many cases.

Take a look at things like (e.g.) the requirements for storing people's medical records or credit card information, and tell me if it's even legal to use "cloud" for these things in the first place.

I suppose it's legal when cloud provider satisfies the legal rules. If one private enterprise is allowed to store some sensitive data, then why another enterprise shouldn't be allowed to do the same? And legal rule satisfaction involves the responsibility in the court if something goes wrong. So, what is the difference between two enterprises with sensitive data if both implement all required regulating rules?

The communicating entities boil down to the ability of some entity to extract independent subtasks from a bigger one. Such parallelization efforts still deliver no viable results, so you are going to outperform the whole academic society.

The same basic "asynchronous messaging" model has been in use by virtually all very scalable systems and virtually all fault tolerant systems for about 50 years now. Just because you don't think you can design software like this doesn't mean that everyone else can't.

Ok, it has been in use, but your goal is about distributed processing, so you should define a way of efficient distribution of tasks among all available computers. Or you should sacrifice the efficiency and allow most of your computers to be idle while your smart-phone is straggling performing some calculations.

Or the application just serializes it's current state and sends it to the Jane's application. No process was stopped or started, nothing was written to disk.

Jane wasn't running any application before.

To serialise the data you'd need extra code to serialise it, extra code to start the process that receives the data, extra code in the application to receive and parse the data, extra bandwidth to transfer the data, and then extra code to terminate Fred's "now unused" old process.

If Fred was editing a movie and the application was working with 50 GiB of data, then the application would be transferred to Jane instantly.

In your case all the problems from above are also in play. You need some code to serialise a data, extra code to start the process that receives the data (there was no Jane's application), extra code in the application to receive and parse the data, extra bandwidth to transfer the data, and then extra code to terminate Fred's "now unused" old process (or application).

And also you need to send the 50GiB file to the Jane's computer, or else it will be a wonder if the computer would process the video without having it's bytes available.

You think I should forget about everything because "desktop themes" are impossible and someone might like a different colour??

You think the user's taste is limited to the colors only? You can remember holy wars among Windows, Linux and MacOS funs. The universal tool for everybody is impossible without some kind of monopoly that you have not.

For integration with "other corporate software" you've got it backwards. That other corporate software would have to integrate with the OS (including the OS's maintenance tools) not the other way around.

Ok, but now your turn to answer how it is possible to enforce corporations to switch to your system. If they switch, then yes, they will invest in integration, but would they switch? And the switch is such a beast that enforces you to forget about some perfectionism and to implement exactly what the corporations want, despite of if you like it or not. The last enforcement, it seems to me, is not as easy acceptable by you as it should for the corporations to start looking at your product. It's about marketing and advertising, but not about highest quality. Yes, the world often looks so ugly.

There should be some visible benefit, like reduction of the number of computers because of distributed processing, but such benefits are hard to achieve, because there's no simple way of distributing the processing. And even if you would manage to create such system, somebody from Microsoft will come to your client and tell them that todays graphical interfaces are helping to improve user's performance and to reduce costs and Microsoft has all required video drivers (developed by hardware vendors) while your system has just some very limited set of drivers for old hardware. And such advertising can make a deal for Microsoft. In short - it's too expensive to compete with big boys.

[Brendan]

Hi,

Yes, bugs are possible even in the VM code, but the maturity of the basic component of a system is always makes it less buggy than you expect. The situation is just like with any other service software from OS and to network support, databases, file systems and so on. The service layer adds some benefits to the system despite of possible bugs.

This can probably be expressed as a formula; like:

• bugs = lines_of_code * complexity_of_code * (1 - developer_skill) / (compile_time_bug_detection/3 + developer_bug_detection/3 + maturity/3)

Where all the values are between 0 and 1.0 except for lines_of_code.

Ok, let's try the mathematical approach. I agree with the definition of the enumerator, but I disagree with the denominator. Developer bug detection and maturity are closely related and intermixed, so I suppose it is better to combine them in one value.

I intended "developer detection" to be things like unit tests and QA testing that are designed to detect bugs before software is released; and "maturity" is mostly end users running the software and reporting bugs after software is released.

Please note that the exact formula is entirely irrelevant and only exists to illustrate my point. My point is that for small/simple programs a VM adds more bugs than it detects and is therefore incredibly idiotic; and for extremely huge/complicated programs a VM doesn't prevent bugs and only detects an insignificant number of bugs that should've already been detected and is therefore incredibly idiotic; and at some point in the middle there's a "point of equilibrium" where a VM is incredibly idiotic for both reasons in equal amounts.

You were wrong before and you are wrong now. "Economy of scale" is minor and (for massive permanent jobs) things like provider markup/profit, internet costs, latency, configuration hassles, etc surpass any benefits by a wide margin.

Well, let's consider the very simple case - your personal web server. Your approach is about holding everything at home, so all your computers will run some parts of the server. My approach is about efficiency (including financial part), so I rent a virtual server in a cloud. Now let's compare costs. Your tens of computers consume at least 1000 watts while I don't care how much watts consumes the cloud, just because may payment is fixed - 4$ per month for small load server. Now you can multiply 24*30.5 kilowatt-hours by the price of the kilowatt-hour in your location and compare the number with the mentioned 4$.

You can't compare 2 extremely different things (distributing applications within a LAN, and providing content to the general public). For your $4 per month, if you compile a large project how much does that virtual server help compile times? If you're playing a 3D game how much of the graphics/physics/AI processing is offloaded to that virtual server? If you search your entire file system for all files containing the word "efficient" how many files will the virtual server look at? If a friend drops by, can you plug an extra monitor and keyboard into your computer so that your friend can run an electronic circuit simulator while you're converting MPEG videos into a different format, even if your computer doesn't have enough RAM to do both at once?

My server is being used for HTTP and FTP; plus DHCP (so other computers can access the internet), TFTP (for network boot, part of my OS testing), samba and NTFS (file sharing) and NTP (so other computer can sync to the server instead of syncing to remote time servers); plus "general use" (web browsing, programming, compiling, IRC, etc). Running HTTP/FTP on this machine costs about $0 extra (maybe a few cents per year for the additional power consumption caused by HTTP/FTP alone) because the machine is necessary to keep the LAN usable and useful anyway. Also note that my web site is auto-generated by my project's build utility whenever I press F12, and this takes less than a second because it's doesn't need to be uploaded to a remote server.

Basically; for the actual purpose of my distributed systems your virtual server is completely useless; and for the purpose of providing content to the general public your virtual server is costing you $4 per month while mine is not.

For anything where users are involved you've got about 100 ms between the user pressing a key and the screen being updated before the system feels sluggish. For most of the internet I'm lucky to see "ping" report times that are better than 200 ms despite the fact that no processing is involved and the data is tiny. Basically; the internet is too slow to handle "doing nothing" with acceptable latency. For ethernet/LANs latency is typically better than 1 ms; so you can have distributed applications that feel like they respond to the user instantaneously despite the networking latency.

It was your words about uselessness of calling string comparison library with your messaging approach, but now you think that it is very useful to call a server on the net for every printed character. May be it worth to remember that some processing is still possible on the client side? And with this in mind we can safely assume, that even latencies on the scale of one second can be perfectly acceptable in many cases.

Did you even read what I wrote?

Mostly, for distributing load there's advantages (resource utilisation, doing more in parallel) and disadvantages (the overhead of communication). For tiny/trivial things (e.g. string comparison) the advantages are tiny and the disadvantages are large in comparison, and therefore it doesn't make sense to distribute the load. For cases where there's a massive amount of processing and very little data the advantages are huge and the disadvantages are insignificant (even when communicating with a remote server across the internet). Most things are in between, where it does make sense to distribute the load when the communication costs reasonable (e.g. across a LAN) but it does not make sense to distribute the load when the communication costs unreasonable (e.g. across the internet, where latency is several hundred times higher).

Take a look at things like (e.g.) the requirements for storing people's medical records or credit card information, and tell me if it's even legal to use "cloud" for these things in the first place.

I suppose it's legal when cloud provider satisfies the legal rules. If one private enterprise is allowed to store some sensitive data, then why another enterprise shouldn't be allowed to do the same? And legal rule satisfaction involves the responsibility in the court if something goes wrong. So, what is the difference between two enterprises with sensitive data if both implement all required regulating rules?

That's like saying "If unicorns existed and the moon was made of cheese then it should (maybe one day perhaps) be possible to....". The answer you're looking for is "No; the requirements are too strict, and even if it was permitted in theory a responsible company would decide the risk is too high".

The same basic "asynchronous messaging" model has been in use by virtually all very scalable systems and virtually all fault tolerant systems for about 50 years now. Just because you don't think you can design software like this doesn't mean that everyone else can't.

Ok, it has been in use, but your goal is about distributed processing, so you should define a way of efficient distribution of tasks among all available computers. Or you should sacrifice the efficiency and allow most of your computers to be idle while your smart-phone is straggling performing some calculations.

I'm not sure what you want defined (that hasn't already been both defined and discussed).

Or the application just serializes it's current state and sends it to the Jane's application. No process was stopped or started, nothing was written to disk.

Jane wasn't running any application before.

To serialise the data you'd need extra code to serialise it, extra code to start the process that receives the data, extra code in the application to receive and parse the data, extra bandwidth to transfer the data, and then extra code to terminate Fred's "now unused" old process.

If Fred was editing a movie and the application was working with 50 GiB of data, then the application would be transferred to Jane instantly.

In your case all the problems from above are also in play. You need some code to serialise a data, extra code to start the process that receives the data (there was no Jane's application), extra code in the application to receive and parse the data, extra bandwidth to transfer the data, and then extra code to terminate Fred's "now unused" old process (or application).

And also you need to send the 50GiB file to the Jane's computer, or else it will be a wonder if the computer would process the video without having it's bytes available.

Wrong, you've failed to read/understand what I wrote. The only thing I change is which GUI the process talks to. The application and all its data remain the same; and because I'm not serialising the application's data, transferring it all to a different computer, or parsing the data, I can switch an application from one user's GUI to another user's GUI "instantly" (regardless of how much data the application has).

You think I should forget about everything because "desktop themes" are impossible and someone might like a different colour??

You think the user's taste is limited to the colors only? You can remember holy wars among Windows, Linux and MacOS funs. The universal tool for everybody is impossible without some kind of monopoly that you have not.

Some people will use my OS and some won't. For the former I will have a monopoly (everybody that uses my OS will be using my OS). For the latter I will only offer my sympathy.

For integration with "other corporate software" you've got it backwards. That other corporate software would have to integrate with the OS (including the OS's maintenance tools) not the other way around.

Ok, but now your turn to answer how it is possible to enforce corporations to switch to your system. If they switch, then yes, they will invest in integration, but would they switch? And the switch is such a beast that enforces you to forget about some perfectionism and to implement exactly what the corporations want, despite of if you like it or not. The last enforcement, it seems to me, is not as easy acceptable by you as it should for the corporations to start looking at your product. It's about marketing and advertising, but not about highest quality. Yes, the world often looks so ugly.

I'll never be able to force corporations (or anyone else) to switch to my OS. The idea is to make the OS attractive enough that they want to voluntarily switch to my OS (despite any hassles involved in switching from existing OSs to mine, which won't be a small hurdle).

For marketing and advertising; to be honest I'm not worried at all - after everything I'm planning is implemented and the OS is ready to be released (which may never happen); a 30 minute video on youtube should be enough to stun the world.

There should be some visible benefit, like reduction of the number of computers because of distributed processing, but such benefits are hard to achieve, because there's no simple way of distributing the processing.

You're saying that my relatively simple way of distributing processing will be bad because there's no relatively simple way of distributing processing?

And even if you would manage to create such system, somebody from Microsoft will come to your client and tell them that todays graphical interfaces are helping to improve user's performance and to reduce costs and Microsoft has all required video drivers (developed by hardware vendors) while your system has just some very limited set of drivers for old hardware. And such advertising can make a deal for Microsoft. In short - it's too expensive to compete with big boys.

Video drivers (for setting video modes only) aren't that much of a problem - it's the GPUs. However; current hardware/CPUs are "good enough" for software rendering now (and will get more powerful in the 10+ years before the OS is ready to be released); and by supporting Intel's GPUs (which isn't easy but isn't impossible either) you'd cover about 80% of 80x86 computers anyway.


Cheers,

Brendan

[embryo2]

My point is that for small/simple programs a VM adds more bugs than it detects and is therefore incredibly idiotic; and for extremely huge/complicated programs a VM doesn't prevent bugs and only detects an insignificant number of bugs that should've already been detected and is therefore incredibly idiotic; and at some point in the middle there's a "point of equilibrium" where a VM is incredibly idiotic for both reasons in equal amounts.

If we consider all beginners who use the VM, then the number of bugs VM has is orders of magnitude less than number of bugs all beginners make. It means that entire developer community will get the benefits of VM's bug detection even if we consider only programs like Hello World. You can compare it with the cases when beginner C developers are struggling to write the same Hello World programs. Or if you consider your still not implemented smart compiler and think it will prevent at least all those bugs that VM prevents, then I can point at a simple thing - VM also can have a smart compiler that is run at install or run time, so, all your words about such compiler can be applicable to the VM.

You can't compare 2 extremely different things (distributing applications within a LAN, and providing content to the general public). For your $4 per month, if you compile a large project how much does that virtual server help compile times? If you're playing a 3D game how much of the graphics/physics/AI processing is offloaded to that virtual server? If you search your entire file system for all files containing the word "efficient" how many files will the virtual server look at? If a friend drops by, can you plug an extra monitor and keyboard into your computer so that your friend can run an electronic circuit simulator while you're converting MPEG videos into a different format, even if your computer doesn't have enough RAM to do both at once?

My server is being used for HTTP and FTP; plus DHCP (so other computers can access the internet), TFTP (for network boot, part of my OS testing), samba and NTFS (file sharing) and NTP (so other computer can sync to the server instead of syncing to remote time servers); plus "general use" (web browsing, programming, compiling, IRC, etc). Running HTTP/FTP on this machine costs about $0 extra (maybe a few cents per year for the additional power consumption caused by HTTP/FTP alone) because the machine is necessary to keep the LAN usable and useful anyway. Also note that my web site is auto-generated by my project's build utility whenever I press F12, and this takes less than a second because it's doesn't need to be uploaded to a remote server.

Basically; for the actual purpose of my distributed systems your virtual server is completely useless; and for the purpose of providing content to the general public your virtual server is costing you $4 per month while mine is not.

My goal is to minimize total costs and while all the compilations and video rendering take just a tiny fraction of the entire month the most persistent cost is the web site maintenance, including power consumption, administration, failure recovery and so on. For just 4$ I have freed myself from all the hassle of the 24*365 web site maintenance while you are still obliged to do all the administering, recovering and other stuff. And, of course, because most part of the month your computers do nothing except the web site task, the electricity bill is still higher than my.

The only thing I change is which GUI the process talks to. The application and all its data remain the same; and because I'm not serialising the application's data, transferring it all to a different computer, or parsing the data, I can switch an application from one user's GUI to another user's GUI "instantly" (regardless of how much data the application has).

Ok, now I have noticed the difference, your approach is about having a client-server application that processes even such things as a video file. But if we to concentrate on the client-server approach then it becomes obvious that such approach is already widespread across the world and most visible example of it is the world wide web with all it's servers and browsers. You have extended the approach a bit (like X Window has done) and pretend that it introduces some new benefits for users. But what exactly the benefits are? You still need to implement the server part just like many others did in case of web video streaming. And after you have such server then what is the difference between your server and a standard video streaming server that exists for a decade or even more? May be you think the difference is in your messaging, that allows you to easily decouple GUI from back-end, but here you are wrong because such decoupling is achieved with the help of a good application design and the underlying communication protocols are playing just a trivial role of a transport, while the actual task is always performed by the software that is decoupled from communication (except the cases of obviously bad designed software). So, you need to reimplement a streaming solution, that exists for many years, and in the end your users will have another item on the list of choices with very similar features. But would the world be better for users if instead of a list of 100 items they will have the list of 101 similar items?

The idea is to make the OS attractive enough that they want to voluntarily switch to my OS (despite any hassles involved in switching from existing OSs to mine, which won't be a small hurdle).

Having you are agree that it can take ages my point is about concentration on a niche where the success is much more probable. Such narrowed scope won't take ages and can be extended in case of a successful market reaction. But your maximalism won't allow you to get less than everything. So, your way is yours and I hope that at least some interesting parts of your vision still can be implemented.

For marketing and advertising; to be honest I'm not worried at all - after everything I'm planning is implemented and the OS is ready to be released (which may never happen); a 30 minute video on youtube should be enough to stun the world.

Unfortunately, it's not enough. But until you try it you won't believe.

You're saying that my relatively simple way of distributing processing will be bad because there's no relatively simple way of distributing processing?

I'm saying that your messaging is just a transport protocol, but not a distributed processing in any way. So, you're right when saying "there's no relatively simple way of distributing processing" and your solution is just not about processing, but about communication.

Video drivers (for setting video modes only) aren't that much of a problem - it's the GPUs.

Do you mean to use VBE instead of full featured drivers? But here again can be some obstacles in form of not implemented support for extended video modes by the BIOS vendors.

However; current hardware/CPUs are "good enough" for software rendering now (and will get more powerful in the 10+ years before the OS is ready to be released); and by supporting Intel's GPUs (which isn't easy but isn't impossible either) you'd cover about 80% of 80x86 computers anyway.

Yes, such way is seen as viable to me. Except some fancy effects that always will require additional computing power of the GPUs. But may be CPUs and GPUs will merge in the nearest 10-20 years.