OSDev.org

This post is a bit of a rant, viewer depression advised.

In the past week or so I've been wondering where I could port my Smaller C compiler to (already self-hosting in DOS, Windows and Linux, hosting in RetroBSD). And I took a closer look at two OSes that many of you have probably heard of: xv6 (educational OS designed after UNIX) and KolibriOS (fork of MenuetOS). Why these two? Well, a few years ago someone asked on stackoverflow if there was a small C compiler for xv6 and sometime ago I noticed an archive of my compiler uploaded to a pile of KolibriOS (or MenuetOS) related files on the interwebs and then several days ago someone asked a question on this forum, which made me take a closer look at xv6.

My findings are not very encouraging so far.

xv6 issues:
* file sizes are limited to a little over 64KB, so right off the bat you just can't easily port an existing C compiler without either breaking it into smaller parts (good luck with that!) or upping the file size limit in xv6
* no lseek(), which means that all tools that work with structured binary files (such as object files and execuctables) cannot even be compiled without either being rewritten to work with files sequentially or changing xv6 to provide lseek()
* no time or rather limited time (uptime since boot), which we can live with (in theory we could ask the time from the user at OS start)
* temporary files need to be invented
* no environment/PATH
* the FPU context is not saved/restored on context switches

KolibriOS issues:
* all apps are graphical, there's no console feature in the OS, so you can't run your C programs and do read() or fprintf() using STDIN_FILENO or stdout (there's a separate user mode library that does provide some of the console window functionality and frees you from rendering letters, but it's more of a hack than a usable solution)
* naturally there's no provision for nested apps sharing the same window, so, if your compiler reports an error, it must report it within its own window (or a special system debug/log window) and it must not quit automatically, so the user can see the message
* there's no way to distinguish graphical apps from console apps, although you need that in order to handle them differently (e.g. graphical apps can be just executed and run concurrently, whereas when executing a console app you need to block the parent (if they are to share the console) until the child terminates)
* heck, there's not even a concept of file handles/descriptors (nor lseek()) at the system call level (there's yet another hacky user mode library that implements handles and fseek()/ftell() around the ugly system calls)
* the heap gives you memory in chunks of 4KB, so either you have to know in advance how much you'll need or you'll be wasting memory with small allocations; the documentation does not guarantee contiguity of the allocated chunks nor does it seem tell if physical memory is mapped in lazily during actual reads and writes, so you don't know exactly the overheads and the possible workarounds without studying the code or seeking help from the developers
* their version of FASM is graphical and can't be used with my compiler unless I'm cross-compiling for KolibriOS or making a new version of FASM for console use (if and when the right kind of console is available)
* temporary files need to be invented
* no environment/PATH

There's probably more.

So, if all that you have is just a boot sector (perhaps, even a boot loader) and odd bits of this and that, don't dare calling it an OS for clearly even more advanced OSes such as xv6 and KolibriOS appear rather unripe for consumption simply because you can't implement the standard C library for them without going through the painful experience of adding functionality in the OS or hacks and workarounds in the library. And if you can't quite develop for the OS while using it interactively or all you've got is an assembler, is it really an OS?

End of rants.

Hi,

First; Xv6 is a "teaching OS" and not something intended to actually be used. All "teaching OSs" and tutorials and example code cut corners to make things easier to understand. Otherwise things get complicated by practical considerations and you end up with something difficult for beginners to understand, which defeats the purpose.

Second; KolibriOS is a modern OS (regardless of whether it's good/bad for whatever other reasons) and isn't some clunky pile of puss derived from ancient mainframes. The problem is that the standard library for C is and your compiler are both designed for some clunky pile of puss derived from ancient mainframes. This is not KolibriOS's fault - it's the C standard library's fault and it's your fault for failing to notice which century we're in now.


Cheers,

Brendan

Oh, I see. Progress is when you can't do today what you could do yesterday.
So, I guess, KolibriOS and the likes will remain uselessly modern until someone actually bridges the present with the past and unlocks the riches of existing software.

Hi,

alexfru wrote:Oh, I see. Progress is when you can't do today what you could do yesterday.

Have you ever tried to feed hay to a modern car (or fertilise a garden with petrol/diesel engine exhaust gases)?

alexfru wrote:So, I guess, KolibriOS and the likes will remain uselessly modern until someone actually bridges the present with the past and unlocks the riches of existing software.

That depends on your perspective. "Unlocking the riches of existing software" is the same as "getting infected by obsolete crud that died 20 years ago but continues to haunt our lives and stifle any hope of meaningful progress".


Cheers,

Brendan

Brendan wrote:

alexfru wrote:Oh, I see. Progress is when you can't do today what you could do yesterday.

Have you ever tried to feed hay to a modern car (or fertilise a garden with petrol/diesel engine exhaust gases)?

Probably not. I'm pretty sure I did try to feed a doll or a teddy bear when I was a kid.

I don't think the comparison is valid, though. Software is rather malleable and supporting a relatively small traditional API should not be a big problem, whereas not supporting it immediately cuts off a lot of existing software that is written around it.

Brendan wrote:

alexfru wrote:So, I guess, KolibriOS and the likes will remain uselessly modern until someone actually bridges the present with the past and unlocks the riches of existing software.

That depends on your perspective. "Unlocking the riches of existing software" is the same as "getting infected by obsolete crud that died 20 years ago but continues to haunt our lives and stifle any hope of meaningful progress".

If you want to take it that far, could you name distinct modern features of KolibriOS that are not "infected by obsolete crud that died 20 years ago..."? It still uses files. And PIDs. And bitmap fonts. And a code page (with limited Unicode support). And there's nothing revolutionary in the system calls (some are plain ugly and inconvenient). Or maybe that if I try too boot it in DOSBox it just hangs hard at mouse initialization, without any diagnostic or further progress? (DOSBox is probably a bad idea, but I think it reveals some rough edges) What are the novel ideas? An OS written in assembly? UNIX was that before it got rewritten in C. Small size? Been there. A flat executable file format which every OS dev newbie tries once in their life? ext2 or NTFS? FAT, for god sake? Made not by Microsoft/AT&T/etc? We have that too. If I took it apart, I would find lots of things (most, in fact) going back to 20 years ago and beyond with very very little being truly new (I don't think we should count APIC, ACPI and other hardware-specific alphabet soups here, 'cause that's how modern hardware is).

Hi,

alexfru wrote:

Brendan wrote:

alexfru wrote:Oh, I see. Progress is when you can't do today what you could do yesterday.

Have you ever tried to feed hay to a modern car (or fertilise a garden with petrol/diesel engine exhaust gases)?

Probably not. I'm pretty sure I did try to feed a doll or a teddy bear when I was a kid.

I don't think the comparison is valid, though. Software is rather malleable and supporting a relatively small traditional API should not be a big problem, whereas not supporting it immediately cuts off a lot of existing software that is written around it.

The comparison is valid - cars aren't capable of doing things that the horses they replaced were able to do (and this includes many things, like being able to side-step, turn on the spot and handle steps). The fact is that sometimes progress means something is better despite not being able to do some of the things that the old alternative used to.

Ironically, often the opposite is true - by trying to ensure the new thing is able to do everything the old thing did, you have constraints that limit how far you can progress.

alexfru wrote:

Brendan wrote:

alexfru wrote:So, I guess, KolibriOS and the likes will remain uselessly modern until someone actually bridges the present with the past and unlocks the riches of existing software.

That depends on your perspective. "Unlocking the riches of existing software" is the same as "getting infected by obsolete crud that died 20 years ago but continues to haunt our lives and stifle any hope of meaningful progress".

If you want to take it that far, could you name distinct modern features of KolibriOS that are not "infected by obsolete crud that died 20 years ago..."?

Why? I'm not saying if KolibriOS is good or bad, and am only saying that their goals are obviously different from the "console that emulates a teleprinter from 1850" nonsense that you expect, and that this is your fault for expecting antiquated rubbish and not their fault for failing to provide antiquated rubbish.


Cheers,

Brendan

alexfru wrote:So, if all that you have is just a boot sector (perhaps, even a boot loader) and odd bits of this and that, don't dare calling it an OS for clearly even more advanced OSes such as xv6 and KolibriOS appear rather unripe for consumption simply because you can't implement the standard C library for them without going through the painful experience of adding functionality in the OS or hacks and workarounds in the library. And if you can't quite develop for the OS while using it interactively or all you've got is an assembler, is it really an OS?

So suddenly having a standard C library implementation is a criteria for an OS? There are plenty of good OS designs that can't have a standard C library but are still very powerful and can still run many useful applications; they just can't have existing applications ported to them easily, which again is not a criteria for an OS.

onlyonemac wrote: So suddenly having a standard C library implementation is a criteria for an OS?

Yes and no. My compiler comes with a library (and there are other open source standard C libraries available, which are similar). The problem is that it can't easily be adapted to the system calls provided in the two OSes I mentioned. There's something important missing that has to be implemented either in the kernel or around its limited system calls. This gets in the way of development and is an impediment to enrichment of the ecosystem, which is deadly for an OS that's neither historically popular nor solves an important problem, for which we could forgive its limitations. And we are seeing many newbie "OSes" that are not anywhere near these two OSes and are therefore lesser and poorer OSes if OSes at all. This is the whole reason for my post, some rants and a warning for OS devers that they need a little bit more stuff to make their OS a usable platform for porting toolchains and software. And this stuff is really small and standardized, just a handful of basic system calls described in POSIX that can serve as a foundation for a standard C library, a C compiler and C programs.

onlyonemac wrote: There are plenty of good OS designs that can't have a standard C library but are still very powerful and can still run many useful applications;

Plenty examples of good and very powerful and yet unable to have a standard C library OSes, please?

onlyonemac wrote: they just can't have existing applications ported to them easily, which again is not a criteria for an OS.

That would be a criteria for a dead OS nowadays, unless we're talking about something that solves a unique problem in a good way, for which we can forgive its nonstandardness(?).

alexfru wrote:

onlyonemac wrote: There are plenty of good OS designs that can't have a standard C library but are still very powerful and can still run many useful applications;

Plenty examples of good and very powerful and yet unable to have a standard C library OSes, please?

Even the C compiler for OS400 - which is an operating system pretty distant from UNIX-type ones - supplies an implementation of the C standard library. I'd be fascinated to know which good OSs can't support the library.

Classic Mac OS for one had no real standard C library. There were a few versions supplied with compilers, but they all opened a window to act as a console and allowed the user to manually enter environment variables before running. There was no way to use them alongside the GUI toolkit of the operating system itself. In other words, it was a way to port UNIX applications to the system, not a way to develop new software. If that's what you call a standard C library then fine, but I thought we were talking about actual useful implementations that don't impose a whole bunch of other limitations on your application. (I should also mention that the OS API provided functions to allocate memory, work with strings, read files, and so on and when it came to application development there was no real need for a standard C library.)

It is also worth noting that I said "OS designs", not "OSes". In other words, an operating system doesn't have to support a standard C library to be of a good design. For example, in some applications an OS based around streams rather than traditional file I/O could be a good design, but would be unable to (easily) support a standard C library. Similarly as we've seen with operating systems that don't have any kind of shell or console, and an all-GUI operating system definitely has its advantages in some situations.

Brendan wrote:

alexfru wrote:I don't think the comparison is valid, though. Software is rather malleable and supporting a relatively small traditional API should not be a big problem, whereas not supporting it immediately cuts off a lot of existing software that is written around it.

The comparison is valid - cars aren't capable of doing things that the horses they replaced were able to do (and this includes many things, like being able to side-step, turn on the spot and handle steps). The fact is that sometimes progress means something is better despite not being able to do some of the things that the old alternative used to.

First player - there are some needs that should be addressed.
Second player - sometime some needs can be addressed without any dependence on the other needs.
First player - but there still are some needs that should be addressed.
Second player - here are some examples of some needs that can be addressed without any dependence on the other needs.
...

Both are right. But what value the discussion has? Why both players just can't notice they speak about different things?

First problem - there is a need for simplicity and it's fulfilled with the standard libraries which are well known to many developers. The simplicity is achieved by the fact that many developers just know the standard and should spend zero time to learn it.

Second problem - there are some libraries that allow a developer to do things that was impossible to do with old libraries, but new libraries don't provide some functions familiar to many developers. Because there is a need for new functions we are going to use new libraries despite of the lack of standard functions and the need to spend some time on learning how to implement some stuff with new functions.

The main problem - there is a need for simplicity and second approach leaves us alone with the unsatisfied need.

The solutions:
1. Forget about simplicity and spend some time on learning. Consequence - time and money and great dissatisfaction for library users.
2. Change the library in a way that can help us achieve the wanted simplicity (add old functions or rethink it and create new and better functions). Consequence - time and money and great dissatisfaction for library developers, or low probability of inventing some better way of doing things in case of better functions attempt.

The difference = conflict of interest of users vs developers.

How to solve the conflict of interest?

Brendan's attempt - I'll show you something and you'll be amazed to the degree required to keep you satisfied while you are learning new stuff.
Many others - it is possible to copy your amazing things and next to add the standard stuff to it. As a result we'll have a better system than your's.

onlyonemac wrote:Classic Mac OS for one had no real standard C library.

The Macintosh Programmers Workshop provided the C Standard Library, a command shell, and command-line utilities. Of course, Classic Mac OS was orginally based on Pascal, not C. And I would class it as a rather old-fashioned and limited OS compared to today's offerings.

onlyonemac wrote: Classic Mac OS for one had no real standard C library. There were a few versions supplied with compilers, but they all opened a window to act as a console and allowed the user to manually enter environment variables before running. There was no way to use them alongside the GUI toolkit of the operating system itself. In other words, it was a way to port UNIX applications to the system, not a way to develop new software.

That must've been a lousy experience to have the tools very loosely integrated into the system. Better than nothing, though.

onlyonemac wrote: If that's what you call a standard C library then fine, but I thought we were talking about actual useful implementations that don't impose a whole bunch of other limitations on your application. (I should also mention that the OS API provided functions to allocate memory, work with strings, read files, and so on and when it came to application development there was no real need for a standard C library.)

Hold on, how providing functionality to an app imposes limitations on the app compared to not providing said functionality or providing it in an inconvenient fashion? If the app ('s developer) is free to not use it or use an alternative, there's no limitation. There's choice. So, I'm not sure what are you talking about here.

onlyonemac wrote: It is also worth noting that I said "OS designs", not "OSes".

Let me quote you a bit less succintly:

onlyonemac wrote: There are plenty of good OS designs that can't have a standard C library but are still very powerful and can still run many useful applications...

A design cannot run any application. It's a concept, idea, plan, etc. An implementation (of the design) can. Further, power(fulness?) here can really only be judged in practice, that is, by comparing an implementation of the design to others, whether it catches on and lives. Again, I'm not sure what you're trying to say. What I'm saying is that it sucks when an implementation lacks this and that (whether by design or because of being v1).

onlyonemac wrote: In other words, an operating system doesn't have to support a standard C library to be of a good design. For example, in some applications an OS based around streams rather than traditional file I/O could be a good design, but would be unable to (easily) support a standard C library. Similarly as we've seen with operating systems that don't have any kind of shell or console, and an all-GUI operating system definitely has its advantages in some situations.

Yeah, my cellphone with its OS is not really designed to let me compile C code on it and run it natively. But that's OK because the device and the OS have a narrower purpose and use cases than a PC with Windows, Linux and the likes. KolibriOS, OTOH, has no such a special use as my cellphone and it comes with an assembler and libraries for developing software for it and on it and it's natural to have different expectations for it compared to my cellphone OS.

alexfru wrote:Yeah, my cellphone with its OS is not really designed to let me compile C code on it and run it natively. But that's OK because the device and the OS have a narrower purpose and use cases than a PC with Windows, Linux and the likes. KolibriOS, OTOH, has no such a special use as my cellphone and it comes with an assembler and libraries for developing software for it and on it and it's natural to have different expectations for it compared to my cellphone OS.

The Classic Macintosh OS had one not-so-specific use that it did well at - running user-friendly graphical applications. With no C standard library, there was not printf/scanf to tempt developers, no environment variables to configure, no weird filesystem concepts to understand, and so on. The operating system provided everything that the user needed to get their work done, and everything that developers needed to write code for the users to get their work done - it was built from the ground up as a user-oriented graphical system, and as it was not designed to run existing UNIX software or to give UNIX users/developers a familiar environment, there was no need for a C standard library, and having one would have only reduced the user-friendliness (if we think of user-friendliness in terms of "graphical, mouse/menu-driven, standardised interface, etc.") of the applications developed.