OSDev.org

Posted: **Thu Feb 11, 2021 7:36 am**

vvaltchev wrote:Thanks for the feedback. It feels to me that you and @nexos like Tilck for the same reason, but I'd like to understand better what you mean with "code bloat". I got that you consider Tilck "not bloated" while the Linux kernel "bloated". The question is for @nexos as well. Can you explain better that? For example, I cannot hide the fact the Tilck do not have support for most of the hardware and it works only on i686. If, in the future, the project started to support plenty of hardware devices and other architectures, will that make it "bloated" or not? If not, can you define how "bloat" differs from "size"? I have some sort of definition of mine, but I'd like to hear yours first, guys.

I haven't a definiton of "bloated" at hand right now. Hmm, maybe complicated and maybe unneccessary features?

vvaltchev wrote:
PeterX wrote:If Tilck gets more stable/production stage I might even use it for my day-to-day work (ok that's a bit in the future, if/when Linux applications actually can run on Tilck).
First, I'm flattered that you'd use it. Unfortunately, you know, the amount of things you'll need the kernel to be able to run for using it day-to-day is huge. The project is far from there. However, actual (simple) Linux applications already run there. If you install gcc-multilib on your host system, you'll be able to run 32-bit linux apps, so you can run the same app on Linux and Tilck. Just link statically with a libmusl i686 toolchain and drop the binary in the 'sysroot' directory.

Anyway, I never intended it to become a desktop OS, because that's a kind of a "mission impossible". But, even if I wanted, I don't have enough time to make it possible. Unless other people don't start contributing, it will remain a mid-sized project I'll have fun hacking with in my free time. Every successful (big) open source project starts with a guy who works initially on it to the point the project is something worth showing and, at that point, maybe even slowly, the project starts to gain contributors. Without other people, it's impossible to achieve "great things". For the moment, I had some minor contributions, but nobody took interested in seriously contributing, so far.

OK, I guess you are right about that.

vvaltchev wrote:
PeterX wrote:I'm planning on writing a simple graphics stack ontop of the UEFI-linux kernel (which is in most parts identical to Legacy-BIOS-Linux.)
Sorry, I didn't get that. What do you mean with "UEFI-linux kernel" ? The Linux kernel is the same, no matter if it boots via UEFI or not. I don't quite understand what you're working on.

Yes, I meant Linux on an UEFI PC. The kernel is nearly identical, I guess except for some small parts dealing with UEFI specific details. I simply meant I want to write a kernel module for GOP (or alternatively use the existing GOP linear framebuffer device, but probably not). And ontop of this driver I want to write a graphics stack. But as I said, this is a future plan, I have to do some stuff before I reach that project. I'm currently working on UEFI application barebones.

Greetings
Peter

Posted: **Thu Feb 11, 2021 8:36 am**

nullplan wrote:
rdos wrote:I think Linux is a very bad design based on an ancient "Unix" concept that should have died long ago. In fact, while Windows has shed it's poor DOS legacy, Linux still carries the much older and equally poorly designed "Unix" legacy. The POSIX concept is just as bad, and the whole environment is extremely complex & bloated.
Well, Unix can't be all that bad, given that all major operating systems are converging on it. Those that don't learn from Unix are doomed to re-implement it, badly.

I know too much about Unix, and I'd never, ever want my native API to converge on it. It's a hopeless mess where every object is a file and where IOCTL is used to control devices. Fork is used to create new processes, which is a hopelessly outdated concept that works very poorly with multicore systems. It's a mess much worse than DOS ever invented. Signals are used for "messaging" instead of a more decent API.

nullplan wrote: I mean, Windows moved from DOS to NT. What are NT's most distinguishing features? Oh, basically everything from Unix (multiuser support, symlink support in the FS, ...). MacOS moved from their bespoke kernel to XNU, which claims not to be a Unix, but is very much a Unix anyway.

Nobody needs multiuser support anymore. The era of mainframe servers is long gone (and so this is completely outdated).

nullplan wrote: And yes, POSIX is bloated. Some parts of it read like somebody documented their implementation and now everybody has to do the same things. Ever read the description of system()? Jesus! However, POSIX is a standard one can support, and once supported, you immediately gain access to thousands of software packages. Therefore, unless you plan on re-implementing everything (like, say, font renderer, GUI toolkit, web browser,...), POSIX is really the only way to go. However, full conformance is not necessary to gain access to most of those packages.

That's just a joke. Most open source projects are littered with ifdefs for different compilers & OS versions, and it is close to impossible to get patches into them unless you work on a major OS. Even worse, you need a functional autoconf for your operating system to even be able to generate the source. Do I need to mention that autoconf is an extremely complex & bloated tool?

Also, I run a web-server at home using my operating system, and I don't need a port of the bloated Apache server to do that. I don't need PHP or Java either. It works very well to create an http C++ class and then create new instances of it to deliver dynamic content.

Posted: **Thu Feb 11, 2021 9:50 am**

PeterX wrote:Yes, I meant Linux on an UEFI PC. The kernel is nearly identical, I guess except for some small parts dealing with UEFI specific details. I simply meant I want to write a kernel module for GOP (or alternatively use the existing GOP linear framebuffer device, but probably not). And ontop of this driver I want to write a graphics stack. But as I said, this is a future plan, I have to do some stuff before I reach that project. I'm currently working on UEFI application barebones.

I get it now. Let me just point out that the Linux kernel is 100% the same: as far as I know, there are no "UEFI" Linux kernel builds. The same bits run on both legacy machines and UEFI ones. If anything UEFI-related needs to be done inside the kernel, it's done with runtime checks. If that wasn't the case, to download a distro like Ubuntu, we'd need to download a separate image for UEFI and legacy, or have one image but two separate kernels. Never been the case. Still, I know there is code inside Linux dealing with UEFI stuff.

Now, about the framebuffer module. I've done this in Tilck, WITHOUT the kernel knowing it was booted with UEFI or not. What such a module needs is just the physical address of the framebuffer, which is provided by the bootloader via multiboot (or, in case of Linux, via it's own specific protocol) and specific info about the video mode like bpp, width, height, and other more advanced stuff like the color masks. That's it. Completely transparent from UEFI. I'm not saying that won't be fun for you to do it, I'm just saying that it has nothing to do with UEFI.

Posted: **Thu Feb 11, 2021 9:59 am**

nullplan wrote:
rdos wrote:I think Linux is a very bad design based on an ancient "Unix" concept that should have died long ago. In fact, while Windows has shed it's poor DOS legacy, Linux still carries the much older and equally poorly designed "Unix" legacy. The POSIX concept is just as bad, and the whole environment is extremely complex & bloated.
Well, Unix can't be all that bad, given that all major operating systems are converging on it. Those that don't learn from Unix are doomed to re-implement it, badly.

I mean, Windows moved from DOS to NT. What are NT's most distinguishing features? Oh, basically everything from Unix (multiuser support, symlink support in the FS, ...). MacOS moved from their bespoke kernel to XNU, which claims not to be a Unix, but is very much a Unix anyway.

Very true. Microsoft's marketing skills (and the Unix wars) allowed NT to become very popular.

rdos wrote:Nobody needs multiuser support anymore. The era of mainframe servers is long gone (and so this is completely outdated).

Multiuser support is still very relevant. If you give everybody root permissions, then we have big issues going on

. Or having separate accounts for parental controls. Or having users across a networks have different permissions then user on a local computer (to prevent remote hijacking). Features such as Remote Desktop would be a disaster if it wasn't for multiuser systems!

Posted: **Thu Feb 11, 2021 11:25 am**

There is a difference between multiple users and multiple concurrent users. But you are correct - even in the Windows world multiple concurrent users are very common in the commercial world.

And the idea that the days of the mainframe are over shows a misunderstanding of big business.

Posted: **Thu Feb 11, 2021 12:42 pm**

vvaltchev wrote:Now, about the framebuffer module. I've done this in Tilck, WITHOUT the kernel knowing it was booted with UEFI or not. What such a module needs is just the physical address of the framebuffer, which is provided by the bootloader via multiboot (or, in case of Linux, via it's own specific protocol) and specific info about the video mode like bpp, width, height, and other more advanced stuff like the color masks. That's it. Completely transparent from UEFI. I'm not saying that won't be fun for you to do it, I'm just saying that it has nothing to do with UEFI.

OK. I nearly don't believe it, but it makes sense, since a framebuffer is just a piece of (video) memory, isn't it? I thought I have to use UEFI runtime services/functions. I just don't know UEFI yet, only small parts of it. And I don't know the framebuffer yet. (One step after the other...)

Just out of curiosity: How is the memory map handled if a kernel works both for UEFI and Legacy BIOS? I guess, by checking which firmware and then different code, right? I don't believe that the old BIOS memory map is identical to the EFI memory map, or is it?

Greetings
Peter

Posted: **Thu Feb 11, 2021 1:57 pm**

nexos wrote:Very true. Microsoft's marketing skills (and the Unix wars) allowed NT to become very popular.

Linux & Unix no doubt have their success due to marketing skills too, and so no difference. I mean, Linux is certainly not a remarkable OS in the technical sense, more like a pretty poor design.

nexos wrote:Multiuser support is still very relevant. If you give everybody root permissions, then we have big issues going on . Or having separate accounts for parental controls. Or having users across a networks have different permissions then user on a local computer (to prevent remote hijacking). Features such as Remote Desktop would be a disaster if it wasn't for multiuser systems!

I have no users and no permissions. I use completely different methods for ensuring integrity of the system. And remote desktop is a complete disaster no doubt, as are all the trojans & viruses that can live in spite of user accounts & permissions.

Posted: **Thu Feb 11, 2021 4:31 pm**

PeterX wrote:OK. I nearly don't believe it, but it makes sense, since a framebuffer is just a piece of (video) memory, isn't it? I thought I have to use UEFI runtime services/functions. I just don't know UEFI yet, only small parts of it. And I don't know the framebuffer yet. (One step after the other...)

I have no reason to lie

Almost all of the UEFI services are boot services. They can be used only before the call to ExitBootServices(). Then, UEFI offers a few runtime services that the OS can use at any time, plus it's possible to implement UEFI runtime drivers, but your OS doesn't necessarily really need to care about that. The OS might be totally agnostic, as Tilck is.

The framebuffer gets mapped by the firmware into a physical address in a special kind of non-cacheable memory area.
The UEFI bootloader queries (and maybe also sets) the current video mode and passes the video mode info to the kernel via the multiboot protocol. Let me show that in Tilck, where everything is simple:

1. In the exit path of efi_main(), the bootloader calls SetupMultibootInfo(): https://github.com/vvaltchev/tilck/blob ... main.c#L77

2. SetupMultibootInfo() calls MbiSetFramebufferInfo(): https://github.com/vvaltchev/tilck/blob ... boot.c#L30
which does nothing more than saving info about the current video mode in a special MBI struct (defined by multiboot), pointed by gMbi.

3. The last thing that my UEFI bootloader does is calling JumpToKernel(): https://github.com/vvaltchev/tilck/blob ... ain.c#L104

4. Which, the 32-bit case, does nothing more than jumping to kernel's entry point:

Code: Select all

void JumpToKernel(void *entry_point)
{
   /* Jump to the kernel */
   asmVolatile("jmp *%%ecx"
               : /* no output */
               : "a" (MULTIBOOT_BOOTLOADER_MAGIC),
                 "b" (gMbi),
                 "c" (entry_point)
               : /* no clobber */);
}

Before doing the actual jump, the multiboot magic is stored in EAX and the gMbi pointer is stored in EBX. In the case of 64-bit machines, because my kernel is only 32-bit, the UEFI bootloader just calls an assembly function to switch from 'long mode' to 'protected mode 32' and then the same jump is performed.

5. In kernel's main, the read_multiboot_info() function is called: https://github.com/vvaltchev/tilck/blob ... ain.c#L174
This function checks, among other things, if there is any framebuffer info. If there is, set_framebuffer_info_from_mbi() is called.

6. set_framebuffer_info_from_mbi() just copies the info in some global variables: https://github.com/vvaltchev/tilck/blob ... raw.c#L147

7. Later, the init_fb_console() function: https://github.com/vvaltchev/tilck/blob ... ole.c#L533
Calls fb_map_in_kernel_space(), which is:

Code: Select all

void fb_map_in_kernel_space(void)
{
   fb_vaddr = (ulong) map_framebuffer(get_kernel_pdir(),
                                      fb_paddr,
                                      0,
                                      fb_size,
                                      false);
}

8. Finally, map_framebuffer(): https://github.com/vvaltchev/tilck/blob ... ng.c#L1109
Maps the framebuffer in the virtual memory and uses PAT to make the memory area "write-combining", which roughly is needed make writes on that area extremely fast.

That's it.

PeterX wrote:Just out of curiosity: How is the memory map handled if a kernel works both for UEFI and Legacy BIOS? I guess, by checking which firmware and then different code, right? I don't believe that the old BIOS memory map is identical to the EFI memory map, or is it?

Good question! The memory map is NOT the same and UEFI defines much many memory region types. The multiboot protocol defines just 5 memory region types, which are the same ones returned by the legacy BIOS E820 function. How to handle that? Well, no matter if you boot the legacy way or the UEFI way, the kernel needs the memory map from the bootloader. Actually, that's so important that in UEFI, ExitBootServices() fails if you didn't call GetMemoryMap() immediately before it. So, in Tilck's UEFI bootloader, near the end of efi_main(), I call:

MultibootSaveMemoryMap(): https://github.com/vvaltchev/tilck/blob ... oot.c#L118
which calls GetMemoryMap() and then translates the many UEFI memory types back to the classic 5 using the EfiToMultibootMemType() function (in the same file).

This way, in kernel's read_multiboot_info() function (that you already saw from the example above), I get the memory map in the SAME format as in the case of the legacy bootloader. Of course, the legacy bootloader does a ton of stuff to get the same result.

My point here is to have a boot protocol: a common set of structures shared between the boot loader (no matter how it works) and the kernel, in order to the bootloader to pass information to the kernel. Multiboot is good and widely supported. Multiboot 2 is even more powerful, but QEMU doesn't support it. Linux uses a custom boot protocol, because it needs extra features that multiboot 2 does not cover.

I hope you had some fun following my code.

Vlad

Posted: **Thu Feb 11, 2021 6:20 pm**

Thanks for the infos! I have to chew a bit on that. I also like the github feature to point someone directly to a certain line-number.

Greetings
Peter

Posted: **Fri Feb 12, 2021 12:23 am**

PeterX wrote:
vvaltchev wrote:Do you know this epic essay? https://www.dreamsongs.com/RiseOfWorseIsBetter.html
In my experience, developers kind of actually divide between those two groups. Do you dislike Linux because you prefer the "MIT approach"?
I knew this essay, but it's been a long time since I've read it last time. I don't fit in there well, but I think Linux fails for both approaches. They were fast in their development so they reached production level fast. But that was at a cost: The code is a badly-designed, over-complicated mess. Of course it proves the talent of the coders who made Linux. I have no problem admitting that most (probably all) of them are better coders than myself. But they exclude mediocre coders, probably on purpose as they look down on them.

I like the "suckless" philosophy, though it is unclear what precisely is a suckless kernel. Definitely Mach is NOT of my taste, too.
Here is a link
https://suckless.org/philosophy/

EDIT: vvaltchev's kernel (Tilck) comes close to my ideal kernel)
Greetings
Peter

Oh, it's these philosophical questions!

I've had a lot to say about these in the past. I was marginally involved in the suckless community at one point. It seemed relevant to my interests, but at the time I was irritated by Linux and POSIX systems in general, (I still am,) and I had high hopes for 9front. I had everything I needed on 9front; suckless had nothing to offer me until I started using their web browser (surf) as a convenient and speedy way to run a Flash game on a laptop with relatively little RAM. But that browser's scripting interface seemed a little too barebones to be real, and I learned of Uzbl which was much more complete. But I never got around to using Uzbl. Firefox had recently(?) developed history-based suggestions, and that was the perfect command line for my web browsing. This, and the fact that the suckless community never adopted Uzbl, planted a seed of change in my attitude. At that time, I thought it was possible to have simple code and simple interfaces for everything, an opinion I'm now a little bit ashamed of. I gradually started to realize that simple interfaces can in some cases require complex code. Later, cinap_lenrek (9front kernel dev) realised that in parts of Plan 9, "you have to be a genius to understand the simplicity." Understanding this extended my realization: developing simple code and simple interfaces for some cases can require extremely sophisticated knowledge -- a lot of learning.

I was also rejecting terminals around this time. ANSI standard terminals enshrine the poor practice of in-band signalling, resulting in unavoidably poor quality. Guess which standard the suckless community chose for their "high quality" terminal emulator? And why did they even start when there were numerous small, correctly-functioning terminal emulators existing as much as a decade before they started? This is the project which really killed my interest in the suckless community. Prior to that, they started on sbase, a bunch of minimalist -- or at that time disabled -- shell tools which gave me a huge case of "why?" They already had a decent minimal set in the form of 9base, which suckless's founder created perhaps before he founded the community. But then 9base itself is weird. There was already plan9port providing all the same tools and optionally Plan 9's amazingly simple and powerful GUI editors and terminal. The freaky thing is 9base's build system is more complex than plan9port's, having a recursive structure with one source file per directory for the simpler commands. I'm not sure how 9base and the config.mk scheme came from the same place. Perhaps I'm looking at a learning curve effect. In any case, I found the Plan 9 shell tools to be substantially more comfortable than POSIX, or as I thought at the time, better, so the mere existence of the sbase project was incomprehensible to me. (And now I've given up on shell scripting altogether in favour of more complete, consistent languages.)

And why did they invent a new image format when Plan 9's and netpbm's are already simple, clear, and excellently well-suited for their purposes?

Looking at the site now, I was trying to find some more positive things to say when I stumbled upon scroll, just released. Why in the [how can I POSSIBLY find a polite word to insert here] do they think scrollback shouldn't be implemented in the terminal itself? There were simple, correct terminal emulators with fully-functional scrollback in the 90s! The linked page; the very front page of the project, shows the mess they're getting into as they try. I'm pretty sure they got into similar messes as I watched in the early 2010s--My memory is vague because I don't want to recall clearly. Basically, I think the same now as I thought in 2015: suckless coders are idiots who can't write complex code, so they take pride in attributing the problems caused by careless coding to complexity itself. But many people who are capable get lost in a dream and never lift their heads to see the wider world. Whatever the reason, I now see this conflation of complexity with carelessness as a toxic attitute and I think I've written quite enough about that community.

I'm not playing around when I say it's toxic. I used to associate with someone who helped shape the suckless point of view, and I really don't want to talk about that part of my life any more. As I was reminded not 5 minutes before I saw this thread, I believed in "simple is better" for a long time. In the end, I was forced to the conclusion that such a belief is itself overly simplistic. I remain impressed by simple solutions which are themselves good, but recognize that they are always possible. In the choice between simple interface and simple implementation, I will always try for simple interface.

As for the paper on the rise of "worse is better"... the more I read it, the less I appreciate its arguments, although I think it's basically right overall. The writer clearly ignores the fact that most programmers prefer C syntax to Lisp syntax which makes one of his key arguments look a bit fake. But I can simplify the paper down to a single sentence: The so-called "right thing" is just like demanding a Rolls Royce, practical people know better.

It's interesting that the "diamond-like jewel" scenario described in the paper as an example of the "right thing" is pretty-much what the creators of Unix chose for Plan 9. They did make it fast in some ways,

but they couldn't make it fast in every way.

Posted: **Fri Feb 12, 2021 10:42 am**

rdos wrote:
nexos wrote:Multiuser support is still very relevant. If you give everybody root permissions, then we have big issues going on . Or having separate accounts for parental controls. Or having users across a networks have different permissions then user on a local computer (to prevent remote hijacking). Features such as Remote Desktop would be a disaster if it wasn't for multiuser systems!
I have no users and no permissions. I use completely different methods for ensuring integrity of the system.

This is an extremely disingenuous answer, given the embedded-system usage model of your OS. A general-purpose OS simply won't be practical working that way in any realistic scenario. You have computers used by different people over time, computers with varying hardware configurations which may change over time (and whose users won't want or be able to recompile the kernel on each and every configuration change), computers which need to share file systems, computers which need to share peripherals, peripherals which may be connected only part of the time, and so on. All of these present security concerns which a real-world GPOS has to address.

My impression is that your design really comes down to, "general-purpose computing is a mistake, make everything a kiosk or embedded system instead", which I really doubt was your intention. While it is true that a system which does nothing to prevent malicious intrusion is worse than useless, so is one which does so by locking down everything into a fixed system which the user can't subvert because there is no user (not that I think your system can't be subverted in other ways, anyway, as there is no way to build a system which can't, at the very least, be physically destroyed by an attacker).

rdos wrote:And remote desktop is a complete disaster no doubt

Disaster or not, it is a practical necessity for many forms of business use.

Posted: **Fri Feb 12, 2021 12:21 pm**

Schol-R-LEA wrote:You have computers used by different people over time

Are single-user general purpose operating systems as useless as you are implying?

Most end-user devices I've had have only a single user profile. My laptop, my desktop, my phone, my Xbox. Even when I use my wife's laptop, I don't create a new user account on it, I just log in as her. If she wants to use my phone, I just unlock my phone and hand it to her. Yes, she can send emails as me, but you don't hand your smartphone to people you don't trust. I think most household computers are personal devices, or if they are shared they share a single user profile for the sake of simplicity.

The most multi-user device we have is our Nintendo Switch, where before launching a game it asks who is playing, so we can keep separate saved games.

Of course this completely goes out of the window for servers, mainframes, cloud computing, etc. where resources and permissions are allocated per user, and there are production users that only code reviewed binaries run under that have access to user data, etc.

Posted: **Fri Feb 12, 2021 12:41 pm**

AndrewAPrice wrote:Are single-user general purpose operating systems as useless as you are implying?

In most personal cases, yes, but the parental controls scenario still remains. Implementing parental controls would require having a separate user. I mean, in theory, you could just get a separate computer, but not everyone would want to spend money for that

.

Posted: **Fri Feb 12, 2021 12:59 pm**

AndrewAPrice wrote:Most end-user devices I've had have only a single user profile.

That must mean that you log on to your desktop and laptop as the root user. I'm surprised that anyone on these forums is that foolish.

All my computers have ordinary accounts as well as the root account and I always use the less privileged account 99% of the time.

Posted: **Fri Feb 12, 2021 1:14 pm**

iansjack wrote:
AndrewAPrice wrote:Most end-user devices I've had have only a single user profile.
That must mean that you log on to your desktop and laptop as the root user. I'm surprised that anyone on these forums is that foolish.

All my computers have ordinary accounts as well as the root account and I always use the less privileged account 99% of the time.

That's the right way to do it. On my Linux machine, I don't use the root account (except for sudo). On my Windows laptop, that's a different story

.

OSDev.org

Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain

Re: Linux disdain