OdinOS: I'd love some design feedback

[bwat]

It was clear in the original that your OS won't do more than one major thing. It's also clear that an OS that won't do more than one major thing is only good for a niche that is so small it can be ignored.

You really need to widen your horizons. Some niches are both small and very lucrative. Not the sort of thing I would want to ignore.

The same applies to a pool of computers. For example, a load balancer and a pool of 5 HTTP servers that don't waste CPU time for no reason is cheaper to purchase, cheaper to run and cheaper to maintain; while a load balancer, a pool of 7 HTTP servers that do waste CPU time plus 3 more "warm spares" to cover the downtime problem is just plain stupid.

You don't know what the specification is so you're judging decisions based upon your own technical prejudices. What if he wants to guarantee a worst-case response time to server requests? What if his machine must run for 5 hours on a battery in case of power cuts? We just don't know his metrics. Suddenly his design is not automatically stupid.

Don't take this the wrong way, but you come across as a bit inexperienced. You have technical know-how but you're not thinking like a designer. I think your narrow, prejudiced views limit the range of problems you will be able to solve - Ross-Ashby's Law of Requisite Variety tells us why. You can't simplify a problem to suit your frame of reference, you have to expand your frame of reference to suit the problem.

[mrstobbe]

The idea of a micro-kernel is to isolate drivers from each other and other processes (and the kernel). With drivers running at CPL=1 or 2 they can trash a process' code and data, and (by loading each others segments) trash each other. What you've described is a monolithic kernel with drivers running at CPL=1 or 2 (and not a micro-kernel at all).

Well, the idea was to not give them access to each other's memory space, just as they wouldn't have access to the kernel's... you're saying that's not possible? I've never played around with ring1-2. I also was not planing to give them each a segment, but do the standard thing of managing cr3 and rely on paging (similar to ring3 but with slightly higher privileges to do certain things). Worst case I can make them ring3... after all, CPU0's sole job is to be the classic preemptive multitasker and manage stuff.

Ah - I thought you meant "drivers running in their own segments; and all mapped into the single process' address space" (e.g. to avoid switching virtual address spaces completely). If you mean "drivers running in their own virtual address spaces", then just run them at CPL=3 and don't bother with CPL = 1 or 2 (you can still give them access to all IO ports by changing IOPL during task switches in this case, or only give them access to their own IO ports).

Sounds like a plan. I was hoping I could gain something more from running them in a mid-ring, but you, and several other threads have indicated, "don't bother".

The keep alive process thing... why couldn't that be a wrapped launcher for the process (like mysqld_safe?)? It would be sleeping until the process ends and never context switch. When the process ends, it could easily figure out why and react accordingly.

If one process starts a second process and waits until the child process terminates, then that's 2 processes (where only one is given CPU time, but both share memory, have file handles, etc) and not a single process. Of course if you're planning to have drivers running in their own virtual address spaces (as processes) it's not really single process anyway; and you're effectively doing "multiple processes and multi-tasking, with different obscure limits on what different types of processes can do".

Pure semantics... if one process starts another but can't exist (in terms of processor... can't see the light of day again) until the other one exits, it's still a mono-process system. Again, pure semantics.

For IRQ handling; with all IRQs (and all device drivers) limited to a single CPU you won't be able to handle the load of high speed networking (e.g. two or more gigabit ethernet cards running near max. bandwidth); which will probably make the OS "undesirable" for certain things (e.g. HPC).

Hmmm... that could be a problem. Need to think about that more. The whole IRQ management part of this is still fuzzy in my head and I'd like to have a firmer grasp on it by the time the basics are done (I'm still working on simple/generic things like paging and clock management and the like).

I'd assume that you wanted to keep IRQs away from other CPUs (and concentrate them on a single CPU) so that you could do some hard real-time thing on those other CPUs (even though SMM and power management on 80x86 will screw that up more than IRQ handling will). Because of the "single application" nature of the OS, everyone is just going to run it inside virtual machines to avoid wasting hardware resources, and their virtual machine is going to completely destroy any hard real-time thing you attempt.

You're probably right about the point about IRQs. I'll focus on trying to figure out the best way for CPU1..n to handle interrupts. I still don't think you're right about you thinking that server environments expect that their servers be general purpose, but we'll agree to disagree on that point (I think we understand each other). The virtualization point is a great one actually because you're essentially already in an SMP and the last thing you need is another SMP under that SMP when all you want to do is something specific. I mean... way add more context switching and IPC stuff in a virtualized environment when you just want to do something specific? Another good use-case for this.

The end result is still cache misses. However, this situation may not happen (e.g. cache large enough to hold data from several tasks) and it may happen without any task switches involved (e.g. cache too small to hold one task's data).

In modern use (depending on use of course), this seems incredibly likely to happen frequently.

For modern use both are likely - several small processes that all fit in cache is likely (for some people, including you and your "device driver processes"); and a single process that uses so much data that it can't all fit in cache at once is also likely (for other people, including those people that need to handle so much data that it makes sense to have a dedicated computer for it).

I guess what I'm saying is that cache misses, TLB misses, etc. that are caused by task switching adds up to "negligible background noise" (irrelevant). Also note that if an OS is able to run multiple applications at once, you get the same "less task switching" benefit when the OS is only running one application; so the only thing you gain from having a "one application" limit is a "one application" limit (and no other advantage against a "one or more applications" OS).

I definitely disagree that such a thing is "negligible background noise" (irrelevant). If it were, no CPU designer would be bending over backwards for the last 10 years figuring out ways to try and make it more efficient. As for the "one application" thing... again, agree to disagree.

EDIT: And to clarify a bit more... I understood what you where saying... I'm saying that the point of this OS is to be as idle as possible when idle, and to be as active as possible when active. I don't consider "down-time" to be "wasting seconds". Last time I checked down time was far from a bad thing. The energy consumption alone is a serious concern of most major server environments, and those same environments strive to segment everything exactly as I'm describing... one major purpose per server. At the same time, they want the best they can get under full-load.

RAM that is not being put to good use (e.g. caching something to improve performance) is wasted RAM. The worst possible situation is "all RAM is free" (which means all RAM is wasted instead of being used for anything useful).

CPUs that is are not being put to good use (e.g. doing less important work when there's no "most important work" to do) are wasted CPUs. The worst possible situation is "all CPUs idle" (which means all CPUs are being wasted).

If the point of this OS is to be as idle as possible when there's no "most important work" to do, then the point of the OS is to waste CPU time for no sane reason whenever there's no "most important work" to do.

For an example, if the computer is being used as a dedicated HTTP server, then it might have 100% CPU load at peak times and probably drop back to 25% load often, and may never go above "a daily average of 50% of CPUs wasted". That wasted CPU time could be used to do backups, install the updated software and content, defragment file systems, pre-cache/pre-process/pre-compress data, etc; so that less CPU time is wasted, downtime is reduced and the server performs better during peak time.

The same applies to a pool of computers. For example, a load balancer and a pool of 5 HTTP servers that don't waste CPU time for no reason is cheaper to purchase, cheaper to run and cheaper to maintain; while a load balancer, a pool of 7 HTTP servers that do waste CPU time plus 3 more "warm spares" to cover the downtime problem is just plain stupid.

Well, definitely disagree. What you just described is (from experience) no where near the reality in a more complex infrastructure. You'd obviously want to do "utility" tasks on general purpose OSs. I mean, we have quite a few servers but only a couple are dedicated to "general purpose" utility tasks, the rest: static HTTP this, or dynamic HTTP that, or DB this, or load-balance that, or distributed filesystem this, or what-have-you. Everyone else seems to operate the exact same way (and for good reason). You think Google or Facebook operates differently than that? But, again, we'll just have to agree to disagree.

[mrstobbe]

Ross-Ashby's Law of Requisite Variety tells us why. You can't simplify a problem to suit your frame of reference, you have to expand your frame of reference to suit the problem.

I definitely have to remember that from now on (I mean, beyond just this project, in general, in my life). Excellent thing to always have at the forefront of your mind.

[bwat]

I definitely have to remember that from now on (I mean, beyond just this project, in general, in my life). Excellent thing to always have at the forefront of your mind.

The quote: "You can't simplify a problem to suit your frame of reference, you have to expand your frame of reference to suit the problem" is actually from a US Army field manual. Don't ask me how I came across it cause I can't remember how.

[Kevin]

If one process starts a second process and waits until the child process terminates, then that's 2 processes (where only one is given CPU time, but both share memory, have file handles, etc) and not a single process. Of course if you're planning to have drivers running in their own virtual address spaces (as processes) it's not really single process anyway; and you're effectively doing "multiple processes and multi-tasking, with different obscure limits on what different types of processes can do".

Pure semantics... if one process starts another but can't exist (in terms of processor... can't see the light of day again) until the other one exits, it's still a mono-process system. Again, pure semantics.

Yes, but I think Brendan still has a valid point: Your drivers are processes that exist all the time and run in parallel with the single application. So you already have to have some kind of multitasking in order to run the drivers. (And if Brendan and I agree on something, there are chances it is right - because it doesn't happen too often.)

For this reason, a microkernel without multitasking is probably a contradiction in itself. The difference that you can make compared to the "normal" OS is that you don't schedule based on a timer, but only on events like IRQs or explicit calls into a driver function.

[Brendan]

Hi,

The keep alive process thing... why couldn't that be a wrapped launcher for the process (like mysqld_safe?)? It would be sleeping until the process ends and never context switch. When the process ends, it could easily figure out why and react accordingly.

If one process starts a second process and waits until the child process terminates, then that's 2 processes (where only one is given CPU time, but both share memory, have file handles, etc) and not a single process. Of course if you're planning to have drivers running in their own virtual address spaces (as processes) it's not really single process anyway; and you're effectively doing "multiple processes and multi-tasking, with different obscure limits on what different types of processes can do".

Pure semantics... if one process starts another but can't exist (in terms of processor... can't see the light of day again) until the other one exits, it's still a mono-process system. Again, pure semantics.

If an elephant has ears, legs and a heartbeat, and you temporarily pause its heartbeat, does the elephant cease to exist? Of course not.

If a process consists of a virtual address space, one or more threads, and any number of other things (file handles, signal handlers, etc); and you pause its threads, does the process cease to exist?

You can call it pure semantics if you like; as long as you understand that the semantics you've been using to describe your ideas are extremely confusing because you've mangled the terminology every else uses.

Hmmm... that could be a problem. Need to think about that more. The whole IRQ management part of this is still fuzzy in my head and I'd like to have a firmer grasp on it by the time the basics are done (I'm still working on simple/generic things like paging and clock management and the like).

I'd assume that you wanted to keep IRQs away from other CPUs (and concentrate them on a single CPU) so that you could do some hard real-time thing on those other CPUs (even though SMM and power management on 80x86 will screw that up more than IRQ handling will). Because of the "single application" nature of the OS, everyone is just going to run it inside virtual machines to avoid wasting hardware resources, and their virtual machine is going to completely destroy any hard real-time thing you attempt.

You're probably right about the point about IRQs. I'll focus on trying to figure out the best way for CPU1..n to handle interrupts. I still don't think you're right about you thinking that server environments expect that their servers be general purpose, but we'll agree to disagree on that point (I think we understand each other).

I didn't say that I expect servers to be general purpose; only that you can have special purpose servers without pointlessly crippling a kernel for no reason.

The virtualization point is a great one actually because you're essentially already in an SMP and the last thing you need is another SMP under that SMP when all you want to do is something specific. I mean... way add more context switching and IPC stuff in a virtualized environment when you just want to do something specific? Another good use-case for this.

The problem with running multiple operating systems under virtual machines on the same physical hardware is that those OSs don't/can't cooperate to ensure that the most important work is done before less important work. To allow more important work to be done before less important work, it's far better to run both applications under the same OS. This has the additional benefit of avoiding the overhead of virtualisation.

For a simple example, imagine running an FTP server and a HTTP server on a physical machine; where the HTTP server is more important (response times) and the FTP server is less important (as FTP has always tolerated "varying upload/download speed" well). If you run both in their own little "single application" OS, and put both of the OSs inside virtual machines on the same physical machine; then you're screwed - they will share CPU and network bandwidth equally (even though it is less important, the FTP server is allowed to take CPU time and network bandwidth away from the HTTP server).

Well, definitely disagree. What you just described is (from experience) no where near the reality in a more complex infrastructure. You'd obviously want to do "utility" tasks on general purpose OSs. I mean, we have quite a few servers but only a couple are dedicated to "general purpose" utility tasks, the rest: static HTTP this, or dynamic HTTP that, or DB this, or load-balance that, or distributed filesystem this, or what-have-you. Everyone else seems to operate the exact same way (and for good reason). You think Google or Facebook operates differently than that? But, again, we'll just have to agree to disagree.

Do you honestly think that Google or Facebook use "single application only" OSs? Why would they bother when any "as many application's as you like" OS is capable of only running one application?

Maybe you should design a car that can only do left-hand turns. I'm sure there are plenty of people that only ever turn left; and these people won't care about all the cars that have always been able to turn left or right that have the same performance.


Cheers,

Brendan

[bwat]

Maybe you should design a car that can only do left-hand turns. I'm sure there are plenty of people that only ever turn left; and these people won't care about all the cars that have always been able to turn left or right that have the same performance.

Absurd and childish. Why can't you accept that the OP is designing this system and he/she just might have needs that you do not share. If the OP wants to build a dedicated system then he/she is free to do so, and very possibly justified in doing so.

It surely isn't beyond your imagination to envisage a scenario where a dedicated OS is going to be preferable to a general OS.

[Combuster]

Absurd and childish.

ad hominem

[bwat]

Absurd and childish.

ad hominem

Argument not person. Note the quote from Brendan I included in the post. It's there for a reason.

[Brendan]

Hi,

Maybe you should design a car that can only do left-hand turns. I'm sure there are plenty of people that only ever turn left; and these people won't care about all the cars that have always been able to turn left or right that have the same performance.

Absurd and childish. Why can't you accept that the OP is designing this system and he/she just might have needs that you do not share. If the OP wants to build a dedicated system then he/she is free to do so, and very possibly justified in doing so.

It surely isn't beyond your imagination to envisage a scenario where a dedicated OS is going to be preferable to a general OS.

It is beyond my imagination - I can't think of a single sane reason for "single application" for a modern OS (regardless of whether the OS is intended for a special purpose or not).

Why can't you accept that I'm responsible for my own opinion, and I just might have opinions that you do not share? If I want to have an opinion then I am free to do so. You are being absurd and childish.


Cheers,

Brendan

[OSwhatever]

Hypothetically they then could become almost perfectly CPU bound as work demands, but would be extremely energy efficient while more "idle". In a general purpose SMP design, there is tons of context switching going on even in a mostly "idle" scenario, which is expensive in terms of energy, and a process (or processes) never get the opportunity to fully utilize a CPU because they are constantly being preempted at that point.

Um no, when my microkernel is idle, the CPUs are really idle. There is no context switching going on at all and they are just waiting for some external event.

As you bring up that your goal is to reduce power consumption, there are systems where your model can be beneficial. For example a system with one less powerful but extremely power efficient CPU cores and then several bigger cores that are powerful but draws a lot of power. Having only the power efficient CPU handling the interrupts might be good in this case as that can prevent you from powering up the bigger ones and this is the one that wakes up the other ones if necessary.

This is somewhat similar to Big Little configuration that ARM can provide but yet different. I've seen systems where all the interrupts are routed to a power management CPU (micro controller) which then wakes up other systems and reroute the interrupts. Extending this to also handle scheduling is an interesting idea.

I'm now sure how efficient your idea is going to be in terms of HW utilization but I think you should definitely try it and in order see how it turns out.

[bwat]

It is beyond my imagination - I can't think of a single sane reason for "single application" for a modern OS (regardless of whether the OS is intended for a special purpose or not).

It is often the case that embedded systems contain operating systems which run a single application.

Why can't you accept that I'm responsible for my own opinion, and I just might have opinions that you do not share? If I want to have an opinion then I am free to do so. You are being absurd and childish.

Of course you're free to have an opinion. I never tried to deny you an opinion. What I pointed out was that you were opining from a blinkered world-view. There is a big difference there.

What I called absurd and childish was your left-hand turn only car analogy.

[Brendan]

Hi,

It is beyond my imagination - I can't think of a single sane reason for "single application" for a modern OS (regardless of whether the OS is intended for a special purpose or not).

It is often the case that embedded systems contain operating systems which run a single application.

Are you sure you're not talking about "embedded bootable applications"? I can't think of an OS intended for embedded use that is a modern OS but doesn't support multiple processes in some way.

Not that it actually matters - mrstobbe said "4 or more CPUs recommended" and I don't think his idea would work on single-CPU; so that rules out almost all embedded systems. He also implied 80x86, which would rule out almost all embedded systems again.

Why can't you accept that I'm responsible for my own opinion, and I just might have opinions that you do not share? If I want to have an opinion then I am free to do so. You are being absurd and childish.

Of course you're free to have an opinion. I never tried to deny you an opinion. What I pointed out was that you were opining from a blinkered world-view. There is a big difference there.

If I was wrong (or "opining from a blinkered world-view") someone would've provided a sane reason for "single application" by now. That someone could've been you, but I guess you weren't able.


Cheers,

Brendan

[bwat]

Are you sure you're not talking about "embedded bootable applications"? I can't think of an OS intended for embedded use that is a modern OS but doesn't support multiple processes in some way.

Multiple processes does not mean multiple application. There's no need to confuse the two. To give a specific example, the old Ericsson EMP modem which can be found in many smartphones was an ARM chip running Enea's OSE operating system and a single control application which was broken up into several processes.

Not that it actually matters - mrstobbe said "4 or more CPUs recommended" and I don't think his idea would work on single-CPU; so that rules out almost all embedded systems. He also implied 80x86, which would rule out almost all embedded systems again.

There's plenty of embedded applications that run on x86 hardware (I've worked on two different process control systems that ran on x86 hardware). There's also plenty of embedded systems that have multiple CPUs (I worked on an embedded system which ran on a twin DEC Alpha CPU board back in the late 1990s/early 2000s, and more recently several different platforms with multicore PowerPC chips).

If I was wrong (or "opining from a blinkered world-view") someone would've provided a sane reason for "single application" by now. That someone could've been you, but I guess you weren't able.

Embedded systems, hardware and software dedicated to a single use. This is the second time I've provided that demonstration. You can deny it all you want but you're just exposing your limited experience.

[OSwhatever]

Hi,

Are you sure you're not talking about "embedded bootable applications"? I can't think of an OS intended for embedded use that is a modern OS but doesn't support multiple processes in some way.

There are plenty of small embedded kernels that has no real process concept. As the software in small embedded systems are known, there is less need for protecting parts of the software from each other. These OSes do support what we usually know as threads and also loadable code (usually elf files) but no process isolation.