Modern Operating Systems considered Harmful

[nexos]

I'm interested in both . Kernel dev is something that I've always wanted to do from when I first started coding, and then package management isn't so great in all the OSes I've used, meaning that I want to to do both.

[vvaltchev]

I'm interested in both . Kernel dev is something that I've always wanted to do from when I first started coding, and then package management isn't so great in all the OSes I've used, meaning that I want to to do both.

That's great, but will you keep those projects separate? You could create a portable package manager that works on mainstream POSIX systems or at least on Linux, plus on your kernel of course. So, many more people could benefit from your package manager, not only the ones who want to use your kernel. Even if everybody wanted to use your kernel, it will take you an incredible amount of time to get to a point when that's practically possible and only then you'll be able to show your package manager. Why waiting so much? You could benefit from that smaller project earlier by making it work on existing systems

EDIT: btw, sorry that I went completely off-topic here. Just, from my perspective it's a bit weird seeing many people here wanting to develop everything that a desktop OS has, like a window environment, text editors, package managers, by making all of that working on just a single kernel. Why not trying to re-use software somehow? What's the point in writing a custom text editor that will work only for your OS? I understand why people don't wanna port pre-existing software to their OS: because doing that is hard. A custom text editor for MyCustomOS will be simpler to implement, I agree. But, doing the other way around should be simpler: make YourFancyEditor work on my Linux machine as well, so that we both can test it independently. Also, bugs in the editor can be fixed while it's running on Linux.

[nexos]

That's great, but will you keep those projects separate

Yes . Sense my OS will more or less be POSIX, that would make more sense to me.

Also, I am not going to reinvent everything. I mainly want to write a kernel, bootloader, and package manager, plus maybe (emphasize maybe) a shell. I plan on using BusyBox or NetBSD's base utilities in my OS, plus XFCE for graphics management. My end goal is binary compatibility with the most common Linux apps, running on an OS that is very small and with a low memory footprint, while still performing well and being reasonably powerful.

As for the original topic, Modern Operating System aren't harmful. The problem is that people are trying to create a general solution for a very broad topic. A sludge hammer is a little much to drive a nail in. For this reason, while an OS like Ubuntu works well, it has a little much for somebody who wants to do very basic computing tasks (i.e., web browsing and document editing), while for a server, Ubuntu Desktop may not be enough by default (without adding lots and lots of packages).

[vvaltchev]

As for the original topic, Modern Operating System aren't harmful. The problem is that people are trying to create a general solution for a very broad topic.

Agreed. Custom-tailored solutions are typically better.

A sludge hammer is a little much to drive a nail in.

Agreed about the principle.

For this reason, while an OS like Ubuntu works well, it has a little much for somebody who wants to do very basic computing tasks (i.e., web browsing and document editing), while for a server, Ubuntu Desktop may not be enough by default (without adding lots and lots of packages).

This is more tricky. Are you talking about the Linux kernel Ubuntu ships with or about anything else other than the kernel?

Let me first address the kernel point point of view: modern web browsers like Chrome or Firefox are one of most complicated and demanding types of software a kernel can possibly run. They require most of the advanced features kernels have to offer, use intensively IPC, multi-threading, networking, async I/O etc. In addition to all of that, for a decent level of web browsing today, support for 3d hardware acceleration is required and that comes with an incredible amount of complexity for the video drivers. So, from the kernel point of view, "simple" user tasks like web browsing and document editing require very powerful kernels. Also, it's never just web browsing and document editing. Don't you have an external monitor, audio, USB drives, USB keyboards and mice? All of that require a ton of drivers. Also, don't you want your battery to last longer? The kernel needs to support suspension, CPU frequency scaling etc. In other words, you need a sophisticated power management too. At most, for the desktop use-case you might get away without implementing some of the "container" features.


Now, to the core point: Ubuntu as an OS in the wider sense is "too much bloated": I agree. Really, I won't spent a word for defending the versions of Ubuntu released in the last few years. They are really bloated with a too many services and things like snap-based applications that require one or more mount points each. That's absurd. The thing that makes me most furious is that older versions of snap apps are not only kept on disk, but mounted as well! And that's not bug, I've checked. Look at what I have on my system right now:

Code: Select all

/var/lib/snapd/snaps/audacity_857.snap on /snap/audacity/857 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core_11420.snap on /snap/core/11420 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core18_2074.snap on /snap/core18/2074 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core18_2128.snap on /snap/core18/2128 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core20_1081.snap on /snap/core20/1081 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-3-34-1804_72.snap on /snap/gnome-3-34-1804/72 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-3-38-2004_39.snap on /snap/gnome-3-38-2004/39 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-3-28-1804_161.snap on /snap/gnome-3-28-1804/161 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gtk2-common-themes_13.snap on /snap/gtk2-common-themes/13 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gtk-common-themes_1515.snap on /snap/gtk-common-themes/1515 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/snap-store_547.snap on /snap/snap-store/547 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/snapd_12704.snap on /snap/snapd/12704 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/obs-studio_1258.snap on /snap/obs-studio/1258 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/xournalpp_48.snap on /snap/xournalpp/48 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/yq_1233.snap on /snap/yq/1233 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/yq_1251.snap on /snap/yq/1251 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/xournalpp_61.snap on /snap/xournalpp/61 type squashfs (ro,nodev,relatime,x-gdu.hide)

I even have a script for deleting the older (disabled) SNAP apps, but unfortunately, over time, after updates, Ubuntu gets to the same situation as before. It's terrible, no excuses.

So, if you're frustrated with this, believe me, nobody can understand you better than me. But, that has nothing to do with Linux, the OS in the strict sense (kernel). We both have just to switch to a different distro or, in the worst case, build a distro on our own. It's totally possible to have a system that could boot super quickly, consume very little RAM and offer all the Linux features, because it will still run the same kernel. The problem at that point will be keeping heavy user apps like GNOME working. We'll have to gave up on GNOME. I'm all for it. I've discovered recently that modern versions of KDE are much more lightweight than GNOME and support more features too. XFce is light, but lacks some features and it's not supported so well. I've had recently some bad experience with it.

The most drastic solution would be to fork an existing distro and an existing lightweight window environment and make it good, but it would require an insane amount of work and none of that will be in the kernel. Unfortunately, most of the effort spent on Linux goes into the server segment and Linux as a desktop system is almost not funded at all.

[nexos]

I was mainly speaking of the user space of Ubuntu. While some things of Linux might be a bit much in the desktop world (i.e. containers), Linux isn't the real problem. The excessive user space is. Why snap is neccesary, I don't know. APT is good . Also, the lack of Linux in the desktop market is a shame, as you can control the size of Linux by compiling stuff yourself, while your kind of locked into the bloat with Windows.

[vvaltchev]

I was mainly speaking of the user space of Ubuntu. While some things of Linux might be a bit much in the desktop world (i.e. containers), Linux isn't the real problem. The excessive user space is. Why snap is neccesary, I don't know. APT is good . Also, the lack of Linux in the desktop market is a shame, as you can control the size of Linux by compiling stuff yourself, while your kind of locked into the bloat with Windows.

Yeah, I totally agree. SNAP is not necessary, it's just packaging system that avoid cross-dependencies by putting all of apps' dependencies (shared libraries) in a single image. It's super-wasteful for open source, but acceptable for closed source binary Linux apps (corner case). I prefer AppImage as nothing is mounted until I run the app.

So, yeah. I'd say we to switch to Kubuntu as a first step. No SNAP because there's no GNOME
In alternative, Debian can be considered, but the problem is that stable releases are not so frequent as Ubuntu's LTS and there are no semi-stable releases every 6 months. There is Debian-testing, but AFAIK that is updated too often and it's not stable enough. Choosing a Linux distro is a nightmare

[nullplan]

You know, boot times in Linux can be really complicated. I had once spent a lot of time converting my system to booting with minit, and it did improve the time from power on to login prompt. Only to then find that after logging in, the start of the zsh would be really slow now. It would take something like 20 seconds to start. And I managed to run strace on the process in that state and found that it was calling stat() on all files in all directories listed in $PATH. zsh did that all the time, anyway, but when booting with a more "bloated" init system, I had never noticed, because presumably some other part of the boot process would do the same thing, and then the i-nodes would be in cache and quick to access. But by cutting out all that unnecessary work, now the cache would be empty, and zsh would be the first process filling it.

Oh, by the way, total boot time till first shell prompt was pretty much a wash between both ways of booting, all told.

[rdos]

I think part of many new "features" in those complex OSes are offset by the legacy & general complexity, which means that when compared to simpler solutions, it's mostly bloat with little improved performance. The file API is a good example of that. Asynchronous IO and all the rest would not be needed with a file API free of the "everything is a file" legacy. Then you can improve performance by moving read & write to userspace and do dynamic mapping of file data.

[vvaltchev]

Then you can improve performance by moving read & write to userspace and do dynamic mapping of file data.

That's a fair point specifically for files, I agree.

I think part of many new "features" in those complex OSes are offset by the legacy & general complexity, which means that when compared to simpler solutions, it's mostly bloat with little improved performance. The file API is a good example of that. Asynchronous IO and all the rest would not be needed with a file API free of the "everything is a file" legacy.

Mmmh.. I agree that, theoretically, if we're willing to throw away all the existing code and POSIX compatibility, we could have better software, but that's impractical and it would be also very hard to design from scratch an API that will be so much better than what we have today. Better for sure, not sure by how much. I agree that memory-mapping files is efficient, but it cannot be used in many other cases, for example for network streams. Async I/O allows to have tens of thousands of connections with just a few hundred threads. More threads means consuming for memory for their stack and more memory for the kernel objects they require. Also, it means a potentially slower scheduler and more context switches.

Instead of "re-writing everything", what I'm observing is that the industry is pushing new, more efficient, interfaces in the kernel and then is making software that needs to scale and run faster to use the newer, more advanced, interfaces. Classic (legacy) software can continue to use the simpler interface and everything continues to work.