SPICE: lots of theoretical wankery that may someday be an OS

[iansjack]

I didn't specifically mean to hold up the Windows approach as the ideal - just that there's more than one way to do it, and certainly that there are ways to do it that don't require a full-fledged package manager.

That there are other ways of doing a job doesn't mean that those ways are better - just different. My belief remains that the Windows scattergun approach is not so good as the Linux dedicated approach.

I'm all for new ideas in OS design; it's just that I haven't seen any proposed in this thread. As I said before, criticism of existing systems is easy but it becomes more meaningful when credible alternatives are suggested. Just repeating the Mantra that "Unix is bad" cuts little ice.

Anyway, I wish you well with your OS ideas and look forward to seeing them in practice.

[linguofreak]

A developer's website can tell you much, much more about a piece of software and the philosophy behind it than the little blurb in the repository;

Nothing prevents you from visiting the developer's website, it's just that if the software is in a repository you're using, you don't *have* to.

It's like the Linux community has convinced itself that it's not really a walled garden if it's the community building the walls.

Except there are no walls: You aren't restricted to using just your distribution's official repositories (I have my package manager set up to pull in software from quite a few third party repositories), and if the developer of a given package offers his software as an rpm or a deb, you can download that straight off his website and install it. And in the rare instance that a developer offers his software as a compressed folder containing the binaries, you can just unzip that into your home folder and not even bother with the package management system.

[commodorejohn]

Nothing prevents you from visiting the developer's website, it's just that if the software is in a repository you're using, you don't *have* to.

Except there are no walls: You aren't restricted to using just your distribution's official repositories (I have my package manager set up to pull in software from quite a few third party repositories), and if the developer of a given package offers his software as an rpm or a deb, you can download that straight off his website and install it.

As anyone who's been around kids can tell you, you don't need to actually have power over someone to be able to ostracize them. The repository system still gives a third-party intermediary a say in the interaction between the developer and the end user, and on an internet of weirdoes, trolls, and maladjusts, that's just begging for stupid drama and petty politics (again, I'll refer to that "non-free" repository nonsense.) And since it's done to solve problems that, in my view, stem either from people being unreasonably lazy or from the underlying system being overly complicated, I see no desperate need for it.

I'm all for new ideas in OS design; it's just that I haven't seen any proposed in this thread. As I said before, criticism of existing systems is easy but it becomes more meaningful when credible alternatives are suggested. Just repeating the Mantra that "Unix is bad" cuts little ice.

Anyway, I wish you well with your OS ideas and look forward to seeing them in practice.

Yes, as I said earlier, I'm getting to that. I would've earlier, but I had church this morning and then had to mow the lawn this afternoon. Now I'm free until I need to get supper on. On to the Ideas, starting with the abstract and moving to the concrete...

(Also, for the record, I wouldn't even claim these are new ideas - certainly nothing unique to me. They're just old ideas that have been buried by the orthodoxy.)

Philosophy
As you may have gathered, I'm not a fan of desktop Unix/Linux (let it run the servers, I don't care.) The single biggest reason for this is that it has no coherence or unity of design. I think that a lot of that is due to having no guiding philosophy (or at least not one that anybody followed - "do one thing and do it well" is hard to reconcile with projects like GNOME and KDE where something as simple as a text editor can require everything from SSL libraries to a notification framework to pieces of the media player kit!) It was built as an operating system for mainframes controlling dumb terminals, and was only hacked into some semblance of usability as a graphical workstation OS over the course of thirty-plus years and countless kludgy revisions, by many different people who didn't really seem to know or care where the whole thing was ultimately going. Consequently, it's a mess.

So, in an attempt to avoid that, here's what I consider to be the guiding principles for a good desktop operating system:

1. The only true user-friendliness is that which comes from a simple, elegant design.
   Trying to make an arcane, overly-complicated system "friendly" by hiding the complexity beneath shiny, dumbed-down UI is like trying to make an octopus an end table by putting a doily on it. No amount of interface will ever be a suitable substitute for a clean and simple underlying design.
   
   And "simple" and "elegant" refer to the whole architecture, not just to the individual pieces. It's quite possible for a million simple components to come together into an unholy, arcane mess of a structure (*cough*Xwindows*cough*)
2. "User-friendly" does not mean "idiot-proof." But it doesn't mean "correspondence course required," either.
   The job of the OS is not to save the user from himself, and trying to do so is only going to hamper him from becoming a more advanced user. At the same time, it is the job of a computer to empower humans, by enabling them to control complex systems with simpler controls. This can be a difficult balance to strike, but it is absolutely key to creating a good desktop operating system. If you write programs primarily to be understood by programmers, your programs will be used primarily by programmers; if you write programs to be understood by people, they will be used by everybody. As Einstein may not actually have said, "make things as simple as possible, but not simpler."
3. Simple and small is good. Complete and useful is better.
   I'm a fan of minimalism in cases where it's sensible, but it's important to keep in mind that doing one thing well is best when there's only one thing to be done. Even on modern systems, the overhead for launching programs is non-negligible, and for n components, the user may have to wrap their head around up to n² - n possible interactions; thus, a software framework built out of many small pieces is not always (or even often) the best solution.
4. Developers should always build on established behavioral conventions.
5. Developers should always build on established behavioral conventions.
6. No, seriously. Always.
   It astonishes me how little the principle of least surprise is respected in software development. It's the most basic principle of good UI design, and it's the most frequently quoted, but it's probably the least frequently heeded. Consistency matters. The principle of least surprise should always be respected.
7. Except when it would be stupid.
   It is possible to take it too far. Trying to forcibly apply a common interface to seriously disparate systems is how we wound up with Windows 8, after all. But unless there's a seriously good reason not to, it should be the first resort for developers choosing the behaviors of their software.
8. Failing in a visible, easily-diagnosible way is good. Failing in a recoverable way is better.
   I once lost a significant amount of work in Audacity because it ran out of memory in the middle of an operation and decided that the appropriate thing to do was crash and burn, instead of just cancelling the operation.
   
   That's retarded.
9. "Worse is better" is worst.
   Or at least as it's commonly used now. The original statement had to do with the importance of achieving a state of basic completeness; lately, it seems to be more used as an excuse for never advancing past a state of basic completeness. Building a complete, functional program is important; building a good complete, functional program is just as important.

So that's my basic abstract, philosophical thoughts on OS development. More concrete, interesting ideas will follow once I get dinner on.

[Combuster]

The job of the OS is not to save the user from himself

The majority of users are stupid, and need to be managed and prevented from doing damage - either of their own or through suggestions of others.

Similarly, considering all applications malicious is a good way of reasoning about them. It makes sense to do so when half of the apps you find on a cellphone are socially engineering the user to do something that's not part of the functionality they actually advertise.

[commodorejohn]

If you design a system for idiots, only idiots will use it. And more importantly, the idiots will never learn not to be idiots.

[embryo]

If you design a system for idiots, only idiots will use it. And more importantly, the idiots will never learn not to be idiots.

Your most important problem is your desire. System without idiot users will straggle to get any attention and most probably die. You have mentioned improper marketing and other Apple's mistakes - it is the consequence of ignoring idiots. Another ignorant systems are unixes - they keep idiots far from understanding themselves. But idiots use Linux distributions - is it a joke? One noticeable distribution is named Android and very soon will reach 2 billion users.

But your most important advantage also is your desire. It can help you build a good system. At least you can build it and if you consider everybody else as idiots then you have a great success - you will have very good (from your point of view) system which is not designed for idiots.

The two positions above are extremes of the developer goals. Something useful, of course, should be placed somewhere in between. And the question is - how to decide where in the range your system should be. For the decision to be called credible you have elaborated some philosophy a few posts before this one. In my opinion your philosophy can be summarized like this:

1. It should have simple and elegant design.
2. It shouldn't be too user friendly (not for idiots), but it should respect (your?) habits and allow to recover easily from idiotic (ok, uninformed) user actions.
3. It should be complete and useful.

The last point is for all users (including idiots), but first two are mostly for developers. So we see a system for developers with as rich capabilities as possible. As an ideal it is really interesting for developers. But here is another problem - how to implement such an ideal? And the answer can be found within another range - from Linux's way on the one extreme and up to Windows/Android way on the other. First extreme is a very long lived community process on the basis of a bazaar approach, which obviously leads to the number one philosophy point to be compromised. Second extreme is based on a cathedral approach with tightly controlled development process. But as time goes the cathedral should be reconstructed a lot of times to suite new needs of a cathedral owner and it obviously leads to the number one philosophy point to be compromised, because the system should serve idiot's needs (i.e. main customers, ordinary users) and the service should be delivered as quickly as possible and to be as inexpensive as possible.

And after looking at the possible implementation approaches it seems that all your philosophy is useless and in any way you should first make something "not so good" and try to increase it's quality in some iterative manner without deep thoughts about the philosophy. Because the philosophy like - the OS should be simple and good - is absolutely obvious and requires no thoughts. And the implementation details will always prevent you from building a real cathedral (an ideal).

I can advice something like building a team of developers with closely related vision of a future system and start working on something not very useful, but reflecting the team's vision. The main success condition is the team. If the team is here - you can make really good system using relatively simple iterative approach.

[Rusky]

So, Brendan seems to be advocating the Waterfall Model, which was originally conceived not as a legitimate methodology but as an example of a bad methodology. Software development and design requires iteration or you get locked into bad decisions- nobody can know all the problems they will encounter as part of a project, let alone solve them all perfectly without any experience using their design. Even Brendan has changed his design over time because of the experience he's gained working on previous versions and using other software.

Of course you should think through what you're going to do before you start coding away, but there's a middle ground. You have to be willing to change your design over time, even if for no other reason than to keep up with changing requirements from the real world. This is one reason Linux has "no official 'standard graphics interface'"- not because the people who made the older designs were idiots, but because they had different requirements, and because nobody has time to stop the whole world while they recreate the entire software stack and upgrade everyone's hardware to use the new interface every time it changes.

Another reason is that there are many sets of requirements in use even at a single point in time. There is no silver bullet, there is no single interface that will satisfy all uses cases optimally. A design that works on a supercomputer would be insane to use on a cell phone, and a design for a cell phone would be insane to use on a workstation. You cannot build a system that will satisfy everyone all at once now, let alone into the future.

One example is the insanity on zero-install in this thread. If an application has to be usable directly from a USB drive, and cannot store any data elsewhere, then all the following use use cases are impossible:

• You can't store an application separately from its dependencies, so you duplicate them all everywhere- maybe not a problem on your typical desktop with TBs of storage, but it still wastes RAM and thus destroys the cache
• You can't store your apps in read-only locations- useful for security, for running off CDs (as mentioned here!), for running off the network
• You can't separate config and data from the app itself (yes, this is a feature)- useful when you want to store the app and config on different media (fast/small vs slow/large; central app db on a network with user profiles on their devices; user profile on a USB drive with app on machine; app on USB drive with profile on machine; use case-specific set of config profiles rather than one per user; etc.)
• You can't keep your config around when you un/re-install/update the app, or easily move it to another install- one of many reasons for a separate /home partition

Another example is the OP's claim that any non-browser/network app should never be insecure or the developer is "doing it wrong." The fact is, your software will be insecure. It's much more important to figure out how to deal with vulnerabilities when they're discovered rather than sticking your head in the sand and singing "lalala I can't hear you." Even in a memory-safe language with a sophisticated type system and lots of static analysis- no, even in a mathematically impossible language that catches all security holes at the language level, your design and users will still have problems- phishing, scammers calling on the phone pretending to be tech support, social engineering, side channel attacks, etc.

Given that software must have dependencies, different software must use different dependencies, software must be updated both for changing requirements and for security, and both config and dependencies should be separable from their applications, it's not unreasonable to use a program to manage it all. It needs to make sure apps have the right dependencies installed, that you don't accidentally break your system trying to install conflicting dependencies (optimally by letting them coexist), that you don't get phished trying to install something by gallivanting around the Internet downloading unsigned packages from untrusted sources, and that your software is all up to date. Call it a package manager or not, but it's a very good thing whatever it is.

To clarify again, I'm not claiming any particular Linux distribution is perfect, or that we can't come up with a good zero-install system, or that unix is the pinnacle of good design. But throwing out all the wisdom and experience that's gone into those systems because you don't like some petty surface things is insane. There's even less of a chance that your magical recreate-everything design will be any good if you throw out the baby with the bathwater, let alone assert dominance without even attempting to provide solutions to these problems.

[onlyonemac]

I may have gone through with that anyway, if it weren't for the fact that I'd also been reading through Andy Hertzfeld's folklore.org, a collaborative history of the development of the Macintosh. I grew up on 68k Macs, and it was both a nostalgia trip and a revelation...

Personally I like Linux and other alternative operating systems (including Haiku, which I've been experimenting with over the past few days and have subsequently taken to - it's still a very efficient design and gets rid of the multiple layers problem which bloats Windows and Linux, although to a lesser extent in the latter due to it's design model), but I still maintain that the Classic Macintosh operating system is still the best designed operating system ever, from everything to the kernel model to the interface. (Interestingly enough I noticed some vague similarities between the Classic Macintosh operating system and Haiku - both on a technical and a non-technical level.) Why don't you just use an old Mac? Also, I'm eagerly awaiting the day when someone ports or at least rewrites the Classic Macintosh operating system to run on PCs (or perhaps the day when I finally shelve my current operating system project and write a loose Mac OS clone)...

By the way, are you the same commodorejohn that was on the 68kMLA forums?

[iansjack]

I still maintain that the Classic Macintosh operating system is still the best designed operating system ever

Novel though the concept of multitasking may be - and I realize that not having to implement this innovation makes OS design much easier - I'm not convinced that the world is ready to return to a single-tasking OS.

The proof of the pudding is in the eating. How many people now use these wonderful OSs - Classic Mac OS, AmigaOS, Beos, Haiku, whatever? And yet the granddaddy of them all - Unix -, strangely, is still going strong, thanks to a well thought out, elegant design, powering the Internet, allowing amateurs like Sir Tim to invent the world-wide web. It's a funny old world.

[Octocontrabass]

I still maintain that the Classic Macintosh operating system is still the best designed operating system ever

Every program runs in kernel mode. There are kilobytes of global variables, at fixed memory locations. Are you sure it's the best design ever?

[SpyderTL]

@commodorejohn,

Getting anyone on this site to agree with you about anything would be a true miracle. The problem is that everyone here already has their "perfect" OS design in their head, and everything that doesn't happen to line up with it is immediately attacked with passion and fervor.

I have tried to approach OS development with an open mind, analyzing OS design decisions that were made 50 years ago, and re-analyzing them to see if they were the best decision then and now. I'm currently focused on the underlying structures, the hardware devices, and the command line interface. To this end, I have created my own tools, my own programming language, and my own proof-of-concept OS from scratch. But ultimately, I am going to need a real User Interface that most people will be able to use, regardless of their level of expertise with a PC. And for that part, I am still looking for the ideal representation -- 2D, 3D, mouse, keyboard, microphone, voice recognition, AI, etc. Or some combination.

There are a lot of good ideas floating around this forum, each followed by an avalanche of negative comments. If you can extract the good ideas, and "pretend" that the inevitable negative feedback is actually constructive, helpful warnings, perhaps we can all work together to come up with some new, fresh ideas. I would also recommend not spending too much time responding to individual criticisms on a post-by-post basis. It will quickly consume any time that you may have, that could be used to actually create something tangible.

Anything is possible, and sometimes it just takes someone who believes in something so much that they simply ignore all of the nay-sayers and proceed to change the world. With that in mind, if there is anything I can do to help, just let me know.

Good Luck!

[iansjack]

I still maintain that the Classic Macintosh operating system is still the best designed operating system ever

Every program runs in kernel mode. There are kilobytes of global variables, at fixed memory locations. Are you sure it's the best design ever?

To be fair, Classic Mac OS was so hardware independent that it would run on 68000 family processors and the PowerPC family. The designers of Unix/Linux/BSD must be green with envy at such versatility.

[Brendan]

Hi,

So, Brendan seems to be advocating the Waterfall Model, which was originally conceived not as a legitimate methodology but as an example of a bad methodology. Software development and design requires iteration or you get locked into bad decisions- nobody can know all the problems they will encounter as part of a project, let alone solve them all perfectly without any experience using their design. Even Brendan has changed his design over time because of the experience he's gained working on previous versions and using other software.

I'm mostly advocating the separation of design and implementation; and the inclusion of research, consultation, feedback and a standardisation process into the design phase. This need not be waterfall - it can be iterative where each iteration has a design phase and an implementation phase. Note that what I'm suggesting is not new, or radical, or uncommon - it's the process that virtually every standardisation body uses (and virtually every standardisation body uses it iteratively).

For my project; consultation and feedback haven't been part of my design phase; and I have changed my design over time. Perhaps if consultation and feedback were part of my design phase the number of design changes (and effort spent on deprecated implementations) could've been greatly reduced. Of course consultation and feedback isn't very effective when you only have 1 stakeholder (yourself); but Linux doesn't have that problem.

Of course you should think through what you're going to do before you start coding away, but there's a middle ground. You have to be willing to change your design over time, even if for no other reason than to keep up with changing requirements from the real world. This is one reason Linux has "no official 'standard graphics interface'"- not because the people who made the older designs were idiots, but because they had different requirements, and because nobody has time to stop the whole world while they recreate the entire software stack and upgrade everyone's hardware to use the new interface every time it changes.

Consider Wayland - have they "stopped the world" to begin the Wayland project? Will they be expecting anyone to upgrade to "Wayland compatible" hardware?

More importantly; after Wayland is implemented will they need to change the design and break everything every month, or does the design process they're using adequately minimise the risk of future breakage? My guess is that the answer is "No", and that Wayland is destined to become a major cause of breakage (just like X is/was), until someone decides to replace Wayland with something else (which will also fail to minimise the risk of future breakage and cause just as much pain for end-users and developers as its predecessors).

Another reason is that there are many sets of requirements in use even at a single point in time. There is no silver bullet, there is no single interface that will satisfy all uses cases optimally. A design that works on a supercomputer would be insane to use on a cell phone, and a design for a cell phone would be insane to use on a workstation. You cannot build a system that will satisfy everyone all at once now, let alone into the future.

Wrong. If there are many sets of requirements in use even at a single point in time, then you can find out what all of the requirements are and attempt to design something suitable for all of the requirements rather than failing to try. If some of the requirements are mutually exclusive (e.g. the requirements for cell phones are insane for a workstation) then you split it into 2 standards (one for cell phone and one for workstation).

Now; see if you can predict where you will be in 5 minutes time. My guess is that you will be able to predict the future with better than 95% accuracy; because 5 minutes isn't very far into the future. As the "distance into the future" increases, your predictions will get less accurate.

For something like graphics APIs, the requirements haven't really changed much over the last 20 years. Mostly; people failed to identify the requirements properly 20 years ago, and that caused major changes in the implementations without any change in requirements. How accurately can you predict changes in "graphics API requirements" that will occur in the next 5 minutes? How about the next 2 years? For the next 20 years I'd predict that the only change in "graphics API requirements" will be an increase in the popularity of "head tracking" (e.g. where the display is fixed to your face and moving your head causes the graphics to pan); but it's trivial to build some insurance into the graphics API to cover that possibility; and possible to design a graphics API today that is unlikely to need any significant changes in the next 20+ years.

One example is the insanity on zero-install in this thread. If an application has to be usable directly from a USB drive, and cannot store any data elsewhere, then all the following use use cases are impossible:

• You can't store an application separately from its dependencies, so you duplicate them all everywhere- maybe not a problem on your typical desktop with TBs of storage, but it still wastes RAM and thus destroys the cache

Let's split dependencies into 2 categories: things that are a standard part of the OS and are therefore never missing; and things that should be considered part of the application rather than a dependency of the application. I see no problem here; only the opportunity to optimise the application better, and reduce wasted RAM and improve cache efficiency far more than "slightly shared by very few apps shared libraries" ever will.

• You can't store your apps in read-only locations- useful for security, for running off CDs (as mentioned here!), for running off the network

Yes you can. The only thing that would need to be modified is end-user configuration/settings, which needn't be stored with the application itself and shouldn't be stored with the application itself. Each user would have a "/home/username/settings/myApp" directory containing the settings for "myApp"; and if they use the application on a different computer they just get the default settings until/unless they customise the application's settings on that computer. Of course you'd also be able to put your "/home/username/settings/" on a USB flash stick or network or whatever; possibly including the ability to synchronise multiple copies of your "/home/username/settings/" in different places; and possibly including the ability to have the app and your "/home/username/settings/" on the same USB flash stick.

• You can't separate config and data from the app itself (yes, this is a feature)- useful when you want to store the app and config on different media (fast/small vs slow/large; central app db on a network with user profiles on their devices; user profile on a USB drive with app on machine; app on USB drive with profile on machine; use case-specific set of config profiles rather than one per user; etc.)
• You can't keep your config around when you un/re-install/update the app, or easily move it to another install- one of many reasons for a separate /home partition

Why can't use separate config and data from the app itself?


Cheers,

Brendan

[Brendan]

Hi,

If you design a system for idiots, only idiots will use it. And more importantly, the idiots will never learn not to be idiots.

Your most important problem is your desire. System without idiot users will straggle to get any attention and most probably die. You have mentioned improper marketing and other Apple's mistakes - it is the consequence of ignoring idiots. Another ignorant systems are unixes - they keep idiots far from understanding themselves. But idiots use Linux distributions - is it a joke? One noticeable distribution is named Android and very soon will reach 2 billion users.

In my opinion, for user interfaces the best solution is to use multiple modes where applicable, to "hide" the advanced stuff so that beginners don't get confused and don't screw things up, but so that advanced users can find/enable the advanced options if/when they want to.

I good example of this is KCalc. It has 4 modes (simple, science mode, statistics mode and numerical systems mode). The default setting is "simple". I mostly use "numerical systems mode". Some of the buttons in "statistics mode" are a little confusing to me (I'm too lazy to figure out what they do), but I never see them because I never enable that mode.

Of course there's a massive difference between "advanced features" (that are useful/needed in certain situations) and "complications that benefit nobody that the designer failed to avoid" (that provide no benefits compared to superior/simpler alternatives). A lot of the complexities that end-users are expected to deal with for "desktop GNU/Linux distributions" are the latter. Note: I've explicitly excluded Android here, as the "user space" that Android users see is significantly different.


Cheers,

Brendan

[Brendan]

Hi,

Getting anyone on this site to agree with you about anything would be a true miracle. The problem is that everyone here already has their "perfect" OS design in their head, and everything that doesn't happen to line up with it is immediately attacked with passion and fervor.

I'm not sure about other people; but when I post some sort of proposal on these forums I want people to find all the problems that I've failed to notice (and save me from making mistakes I'll regret). If people do nothing more than agree with me, then no problems get identified, and no mistakes that I'll regret get avoided - it's an extremely disappointing outcome.

100% agreement is a worst case scenario - it indicates that nobody cared enough to try to find problems.


Cheers,

Brendan