Proper order to implement different OS components

[songziming]

Many books divide OS into components like: process manager, memory manager, io controller, file system, etc.

When writing my toy OS, I found that these components have strong coupling, i.e. process manager needs memory manager to switch virtual address spaces to protect processes, and memory manager needs to run as a service (process, microkernel).

So I'm wondering what component should be implemented first, and what's next, ...

[alexfru]

A simple memory manager and a simple file system are quite isolated things. You can implement both as regular apps and test them before moving them into the kernel/OS. Define system-specific interfaces and use virtual functions or callbacks to make substitutions (e.g. use a regular big file as a disk, have the readsector() function/callback call fread()).

[bwat]

Many books divide OS into components like: process manager, memory manager, io controller, file system, etc.

When writing my toy OS, I found that these components have strong coupling, i.e. process manager needs memory manager to switch virtual address spaces to protect processes, and memory manager needs to run as a service (process, microkernel).

So I'm wondering what component should be implemented first, and what's next, ...

How Dijkstra did it: http://www.cs.virginia.edu/~zaher/class ... jkstra.pdf.

[Combuster]

Essentially they are separate (and a lot of the real dependencies like manipulating the processes and page tables are actually in the kernel for safety reasons), but in practice you'll find yourself with a bootstrap problem that needs to point each of the primal services at each other before they can operate autonomously. Probably, the same sort of glue knows best how to restore these processes if one fails - if you even want to in such a case.

[sortie]

The preferable approach (if even possible) is to do zero initialization. The components are online from the very first moment the kernel begins proper (thanks to zeroing the bss ans good coding). This entirely eliminates the problem in the first place. Note that this would depend on bootloader support, such that it sets up the idt and gdt of the kernel (elf notes can have pointers to those) and other basic state such as paging.

[Brendan]

Hi,

When writing my toy OS, I found that these components have strong coupling, i.e. process manager needs memory manager to switch virtual address spaces to protect processes, and memory manager needs to run as a service (process, microkernel).

I'm curious - which memory manager/s (physical, virtual and/or heap) need to run as a service and why; and what are the advantages/disadvantages?


Cheers,

Brendan

[alexfru]

I'm curious - which memory manager/s (physical, virtual and/or heap) need to run as a service and why; and what are the advantages/disadvantages?

Well, there are at least two things that can be done asynchronously:
- scanning page tables/whatever to find clean and dirty pages that haven't been accessed recently, possibly writing them to the swap file, freeing the underlying physical memory (and changing the page availability/presence bits) to satisfy further memory allocation requests when the memory pressure goes up
- lazily zeroing freed pages so no secrets can be shared between the kernel and the apps and between the apps themselves unintentionally

[AndrewAPrice]

Performing something asynchronously doesn't require it's own address space.

[alexfru]

Performing something asynchronously doesn't require it's own address space.

Right, but who said/implied the opposite? What/whom are you replying to with this statement?

[Brendan]

Hi,

I'm curious - which memory manager/s (physical, virtual and/or heap) need to run as a service and why; and what are the advantages/disadvantages?

Well, there are at least two things that can be done asynchronously:
- scanning page tables/whatever to find clean and dirty pages that haven't been accessed recently, possibly writing them to the swap file, freeing the underlying physical memory (and changing the page availability/presence bits) to satisfy further memory allocation requests when the memory pressure goes up
- lazily zeroing freed pages so no secrets can be shared between the kernel and the apps and between the apps themselves unintentionally

Performing something asynchronously doesn't require it's own address space.

Agreed. You can do "asynchronous things" without having them implemented in user-space as a service (and many OSs do).

Getting back to my original point; from my perspective there are several things (physical memory management, most of virtual memory management, IPC and scheduling) that all processes/services depend on. If you shift these into user-space, then:

• You don't get the normal stability advantages (if they crash you can't keep running without them, or terminate the service and start another)
• There isn't much "different people can use different services that implement it differently" advantage, because software expects certain behaviour and that expected behaviour can't change (unless you rewrite all the software to expect something different)
• There's no performance advantage (the additional overhead of IPC for extremely frequently used basic functionality is likely to cause severe performance difficulties)

For virtual memory management it's not so simple. There are parts of it that I think should be in user-space as a service (and parts I think shouldn't be). For example, I'd have a "swap space manager" as a service (that the kernel's virtual memory manager uses to store/retrieve pages from swap space), and use the VFS as a service (that the kernel's virtual memory manager uses to store/retrieve pages from memory mapped files).

However; this is all just my perspective (for my micro-kernels). I don't know what songziming's reasons are, which is why I was curious.


Cheers,

Brendan