Modern Operating Systems considered Harmful

[CEBBAS]

Since the dawn of computing, the main goal was to run a program. This was originally a simple goal - maybe something as simple as calculating a formula. However, as time's gone on, computers have become more and more complex. The entire concept behind software is that you don't necessarily know what you want to do yet, so you need to perform late-binding. Unfortunately, software (and operating systems) have gotten far ahead of themselves, and I think this is a problem.

In the ideal operating system, files are made the main focus. Rather than trying to find the best program, the user just needs to open the file. This style is seen in things like Xerox's GlobalView and the original Macintosh. Most operating systems work differently, though. They create an environment with endless amounts of imported shared libraries. On Windows, you have to deal with installers that copy DLLs to GOD knows where on your hard drive. On Linux, you have package managers that install millions of dependancies. What? You don't like ImageMagick? Sux. MacOS is actually the closest I can think of to the ideal, which is horrifying to say. They have it set up to just drag and drop a program in, which is admittedly closer to what I want. Unfortunately, that's not the only issue.

Other issues come with the heritage of the operating systems. With MacOS, you immediately have flaws from its background in Unix. Similarly, Linux is flawed because it's a copy of Unix with none of the actual Unix. With Windows, you have decades of backwards compatibility layered on top of itself, making it impossible to have any consistency.

My suggested solution is to go backwards. I believe that if we start at Smalltalk-80 and build machines that run Smalltalk bytecode, we can have a starting point for an actually well made operating system.

[Maddie]

Unfortunately, modern operating systems are either hopelessly bloated or are broken and aren't very useful.

[nexos]

IMO, OS design culminated at 386BSD and all OSes sense won't ever be as good.
My OS will be more complex then 386BSD, not because of tons of features, but simply because of hardware (i.e., ACPI)

[Korona]

No, modern OSes are not bloated.

Modern OSes have a lot of features, but not for the sake of having features but because users actually use them.

[iansjack]

Operating systems evolve to do what the user wants of them. OSs based on Smalltalk, Oberon, Lisp, etc. have come and gone. They didn't fulfill the users' needs.

What we demand of computers nowadays is extremely complicated; it's hardly surprising that the OSs to support them are also complcated

[nexos]

Modern OSes have a lot of features, but not for the sake of having features but because users actually use them.

True, but somethings (Windows and GNU mainly) go a bit overboard. Linux straddles the line of bloated. I used to think Linux was extremely bloated, but recently, I reversed that judgement . I just like the fact that I can read 386BSD's code easily.

[Ethin]

I mean hey, if you want an OS that can't do machine learning, GPGPU computing, high-end gaming, general computational tasks, virtualization, cloud computing, cryptography, audio processing... I could go on, then sure, a smalltalk-80 OS would work well. But try doing any of the above in an OS that uses smalltalk-80 bytecode and you will quickly realize that it is either impossible or requires far more work than necessary. (It also will, most likely, be far slower since you'll be interpreting code instead of simply loading an executable image.)
Some of the above tasks could probably be done via smalltalk-80/LISP/insert-your-bytecode-language-here. But what about the more complex things: machine learning, GPGPU computing, high-end gaming (where timing is critical), real-time scheduling, or NUMA computing? Your OS could probably do real-time scheduling but machine learning? No way. Try doing even the simplest ML-based tasks in BASH or any limited scripting language like that and you will quickly realize that it becomes an utter mess (if you can even manage it to begin with).
The "modern" operating system has evolved so that it can allow the user to do all of these things. It is general-purpose enough that a user can do pretty much whatever they please. A bytecode-limited language would probably be more limited in what it would be capable of within a practical time limit, and so would be more strictly targeted and would be used by less people.

[vvaltchev]

Modern operating systems appear "bloated" because they solve an incredible amount of complex problems by exposing a ton of features. For sure there are various degrees of "bloatness" in software: two applications or operating system might solve the almost the same problem and have in one case a compact implementation and in another some extra bloat. It all boils down to the level of abstraction used, how often a custom solution is preferred to a generic one, how valuable is the code to be easily extensible and maintainable, how valuable the performance is etc.

With time and experience I realized that often people (including sometimes myself) judge a piece of software as "bad" simply because it doesn't fit their own system of values and trade-offs while, the same software looks good in the eyes of its creators. It's fine to continue to subjectively dislike a piece of software, but it's necessary to realize that we really dislike the system of values of its creators and a concrete software is just a product of that.

Back to the operating systems: the only way to have a simpler and more "elegant" OS is to solve fewer problems and offer fewer features, both in terms of hardware support and in terms of "software features". No way to avoid that.

Also, one interesting question to think about for the OP, but it applies for everybody as well:

Do you really know in details all what a kernel like Linux offers? I.e. do you know what all of its syscalls do?

There are a ton of advanced syscalls and flags for syscalls that are rarely used, but they are important and required an incredible amount of effort to implement by the community.

I've been programming at system level and using Linux for many years, but still I didn't know about an embarrassing amount of details, until I started to implement an OS partially compatible with Linux that was supposed to just run a shell and bunch of other simple console apps. How hard would that be? Well, harder than expected. After 100k lines of code (whole project), my OS can run shells, editors like VIM, framebuffer games, supports ACPI, but that's only a small fraction of what any general-purpose OS like Linux or Windows can do. The hardware support is also minimal. It's pretty obvious to me at this point why just the kernel of a mainstream OS is made by millions of lines of code. Just thinking about that makes my head spin. Therefore, my approach is to write something small that offers fewer features but because of that is: simpler, faster, easier to test, more predictable etc.

In conclusion, just spend a few years writing an operating system on your own like many people here, port 3rd-party apps to it, and then, probably, you will re-evaluate the existing operating systems

Note: I'm not saying that it's not possible to make a better OS than the current ones. I totally recognize the complexity of the current "mess", partially caused by legacy code. I just believe that one way or another, if you try to support all of the uses cases of modern OSs, your OS will become big and complex like the current ones. Nice, simple & elegant pieces of software are typically very limited.

[vvaltchev]

Most operating systems work differently, though. They create an environment with endless amounts of imported shared libraries. On Windows, you have to deal with installers that copy DLLs to GOD knows where on your hard drive. On Linux, you have package managers that install millions of dependancies. What? You don't like ImageMagick? Sux. MacOS is actually the closest I can think of to the ideal, which is horrifying to say. They have it set up to just drag and drop a program in, which is admittedly closer to what I want. Unfortunately, that's not the only issue.

That's has nothing (or very little) to do with OS per se, but with the way programs are distributed. On Linux you can have the same drag-and-drop thing, just use something like AppImage. For windows, same story. MacOS' apps, are in reality just a directory with an executable and plenty of shared libraries. On both Linux and Windows you can do the same. Just, we don't have the GUI that treats .app directories as applications.

Also, it's worth noting that installing the dependencies "the Linux way" saves a ton of disk space. In the Windows and MacOS model, each application bring its own set of DLLs. So.. we can discuss about what is the best, but.. that's not related with the core of the operating system. It's mostly related with the fact that Linux apps are typically open source. Indeed, Linux closed-source binary applications like Telegram for example, use AppImage or similar technologies.

[nexos]

I agree with vvaltchev. By having a package manager install the shared libraries, it can figure out if they are installed or not, meaning that the Linux (actually, modern Unix by extension) is better.

One thing to mention is that if OSes had a standard program installation database, then many of these issues could be avoided. In my OS, I will remove the shared library nightmare (improving on it), instead implementing a better mechanism at the OS level. Don't ask what this looks like yet

[rdos]

Most of the bloat of Windows and Linux are not useful. For a power user, it would be far better to be able to configure stuff like you want it and build your own applications (in C++). Instead, most of the bloat is to support script & interpreted languages, windowing systems, and complicated general solutions that are not needed.

For instance, you certainly don't need Java, PHP, C# or Apache to create a web-server. It works very well to do your own app in C++. What you need are powerful libraries that you can link, and that are open and not tied to specific compilers or OSes.

Besides, I always build static applications, and only use DLLs for internationalization and required interface isolation.

[nexos]

For instance, you certainly don't need Java, PHP, C# or Apache to create a web-server. It works very well to do your own app in C++.

That would not be very secure (i.e., buffer overruns). In a web context, buffer overruns are the last security vulnerability you want to deal with!

[vvaltchev]

Most of the bloat of Windows and Linux are not useful. For a power user, it would be far better to be able to configure stuff like you want it and build your own applications (in C++). Instead, most of the bloat is to support script & interpreted languages, windowing systems, and complicated general solutions that are not needed.

Well, I'd say that A LOT of complexity in modern kernels comes from the need to support containers. It's not only about supporting chroot(). Modern Linux has a sophisticated support for namespaces that allows each container to have a completely different and independent view of the system, including independent PIDs, network stack, UIDs and even time management since 2020. Containers now are almost VMs running the same kernel, but faster.

Containers are not only used in the cloud, but on workstations by developers as well. I prefer for myself to not work inside a container, but for many developers, that's mandatory.

Also, security is much more complicated today. Back in old days ('90s), capability-oriented security wasn't a thing. Now, it's all about that. It's not anymore that you have a permission on some resource or you haven't: there is fine granularity of capabilities you might or might not have.

Finally, for the sake of performance kernels got more complicated as well. In the good old days, we had just read(), write(), open() etc., right? Now we have readv(), pread64(), preadv(), preadv2(). The vectored I/O really helps, but it's more complex to support. Not only the p* versions of read() allow us to specify the position inside the file on each call and that allows multiple threads to use the same file without having to move the current position at every single read and having to synchronize.

Also, high-performance software doesn't just call open(). Now there is openat(), mkdirat(), fchownat(), unlinkat() etc. That improves the lookup time.

Also, in the good old days (like in my OS today ) we had just select() and poll(). Later came epoll() which is a monster, but still simple compared to what we got in kernel 5.1: io_uring. That's all because big companies need software on Linux to run faster and faster.

Also, how do you copy a file? In the good old days, we just used read() and stored the content in a buffer and then used write() for the other file. Today we have fancy syscalls like splice(), vmsplice() etc. that allows direct copy, without userspace buffers.

Also, in the good old times, we had simple SYSV signals. Now we have real-time signals, which are A LOT more complicated to both support and use, but have their own advantages of course. Along with that, we got all the "real time Linux support" which complicates internally a lot of stuff, not only the scheduler.

Finally, older operating systems, not only Linux, had limited monitoring features compared to modern ones. Now we have ftrace, which is extremely powerful and useful to enterprise users of Linux and kernel developers, but introduced plenty of complexity.

I believe you got the point: people now demand much more from operating systems than they did in the past. And that does not apply only to Linux, just I know Linux much better than the other mainstream operating systems and I'm using it as an example.

Don't get me wrong: I'm still a big fan of simpler and smaller operating systems, like the one I'm working on myself but, it will never be "good" for server or desktop machines with today's demands. That's why I'm thinking about small embedded devices, even MMU-less CPUs like ARM Cortex-R series. Writing a small operating system for those machines is still possible.

[nexos]

Wow, @vvaltchev I didn't realize Linux was that complex! The goal of my operating system is to trim off as much as possible, while still having an OS useful for day to day activities, and even server capabilities. I also want to overcome problems of some OSes (e.g, package management) and try to "fix" them.

[vvaltchev]

Wow, @vvaltchev I didn't realize Linux was that complex!

And that was just the tip of the iceberg, man! I wrote this long post, because I noticed that too often people don't know all of what modern operating systems really offer and immediately jump to the conclusion that they are bad and bloated.

The goal of my operating system is to trim off as much as possible, while still having an OS useful for day to day activities, and even server capabilities. I also want to overcome problems of some OSes (e.g, package management) and try to "fix" them.

Allow me to say that here you're using the wider definition of "operating system": you're not strictly speaking about kernels. If you wanna solve those kind of problems (e.g. package management) in a different way, you can reuse an existing kernel like Linux or FreeBSD. You can make an "OS" (using your wider definition) to do and look like almost anyway you want without touching a single line of code in the kernel itself. If you prefer Windows, technically you can customize it as well to the point it looks and feels like completely different to what we're used to. You can replace the logon, the shell (explorer), the services, a ton of stuff. But that's significantly more complicated to do than with Linux, and also you cannot redistribute a whole "OS" with the NT kernel without having an agreement with Microsoft: that's the reason why nobody does that with Windows, while people do it all the time with Linux.

If you wanna solve kernel problems, that's another story. First of all, you need to get a serious experience with the "state of the art" and then trying to innovate. So, learn the Linux or FreeBSD kernels by seriously playing with them.

Mixing "kernel dev" and things like "package management issues" doesn't make sense to me. I cannot think of any package management problem/customization that would require a feature that Linux doesn't already have. And if you want a very lightweight Linux system, configure and build your custom kernel and then build your custom mini-distro with buildroot or following the https://www.linuxfromscratch.org docs. "Embedded" Linux configurations can run on x86 as well and can boot in 1-2 seconds. The Linux kernel itself can get down to ~1 MB in size or maybe even less.

Booting Linux faster (conference talk): https://www.youtube.com/watch?v=fTLsS_QZ8us
1 sec Linux boot (30-sec demo video): https://www.youtube.com/watch?v=Zvu-dJ3Z91o

My takeaway: don't try to solve non-kernel problems by writing a kernel. Write a kernel only if you're interested specifically in kernel problems