Osmium OS, IPC, PIC, and the rest

[Warrior]

(Moderator's note: This was spawned from "Your OS Design" following the description of Osmium OS conceptually placing even IPC outside its minimal microkernel. -- Solar

So the process has to expect a message (by using GetMessage) to be able to communicate?

I'd rather have some sort of "messenger" service notifying it of a message from the other process.

I havn't thought much about this stuff yet since my OS is just now crawling out of it's shell (Memory Management done, floppy driver, starting FAT12, etc..).

[iammisc]

Well, that way, a process can choose when it wants to receive a message. If a "messenger" services sent it automatically, then it might interrupt an important time-consuming operation and if too many messages were received then it will slow it down too much.

My kernel is a micro kernel so it will not handle the interprocess communication a seperate application will manage that. Although if something were to happen to that application, my kernel would save the messages, restart the application, and resend those messages. This way the message is always gauranteed to be passed onto the receiving application.

I changed my mind about devices as files. Devices will not be files. But you can create a "link" to a device which is just a text file that contains the name of the driver, the type of device, and the device number. Using this, the user can choose which driver they want to use with a device.

You can also link drivers together. eg. you can tell the kernel to pass all the floppy drivers output to a specific filesystem driver or you can tell it to pass it to any driver that can handle filesystems.

The main goal of my OS is to make it simple for first time users but powerful enough for expert users and developers.

[AR]

My kernel is a micro kernel so it will not handle the interprocess communication a seperate application will manage that.

Here's a problem, how do the programs communicate with each other without IPC since obviously, how can the IPC get to the "IPC server" without using IPC to begin with?

In a minimal microkernel, usually the kernel contains scheduling, memory management and IPC.

[iammisc]

Uh, Maybe i overlooked that problem. Okay I will include IPC with the rest of my kernel.

[Solar]

My kernel is a micro kernel so it will not handle the interprocess communication a seperate application will manage that.

Here's a problem, how do the programs communicate with each other without IPC since obviously, how can the IPC get to the "IPC server" without using IPC to begin with?

You could use some kind of implicit "contract".

{take a breath}

{wince}

[sup]Listen, I can't help it that AmigaOS is such a good example for so many things...[/sup]

In AmigaOS, everything was a library, and all calls to a library had to be made through a BasePointer. The kernel was a library too, and you needed its BasePointer to make any kernel calls... but where to get the BasePointer?

So a "contract" was made that ExecBase would always be available at 0x0000 0004. (Not that this was a particularily smart place to put it. At least you had to increment your favourite uninitialized pointer before wiping the kernel from the system. )

Can be done. But it's probably not worth it. You have to move data from one address space to the other anyway, so why waste another userspace / kernelspace transition by having everything tunneled through an userspace IPC server.

[Crazed123]

Actually, he said there would be "shared commands", ie: Remote Procedure Calls between processes. If those were kernel controlled then a userspace IPC server could be called via its shared functions, which would be provided to processes by the kernel. This would still be rather efficient, as message passing and RPC solve two different sets of problems in many cases.

However, if you're using RPC to begin with it wouldn't it be a rather better idea to just implement message passing as a set of standard shared functions a given process can export?

[JoeKayzA]

However, if you're using RPC to begin with it wouldn't it be a rather better idea to just implement message passing as a set of standard shared functions a given process can export?

Most times it is quite the other way round: You implement RPC on top of a lightweight message passing system. The point is, in a OS that uses address spaces the 'typical' way (each app has it's own, starting from 0), and you want to do a 'call' to an exported procedure of a remote address space, you would have to name it somehow: you need a seperate naming system. And this is where messaging is probably better suited as the foundation - it has to deal with naming anyway.

This gave me an interesting idea however: You might know the so-called 'single address space operating systems'? They implement distinct address spaces in a way that they are isolated from each other, but every address space has it's own base address and size, and these never overlap (this is only really manageable on processors with 64 bit virtual addresses).

This way you could implement a basic RPC system by simply calling an entry point in a remote address space, by it's unique virtual address. The task will pagefault, and the kernel can check whether a jump is legal or not (whether the entry point was exported and should be accessible to the calling task). If it is legal, the kernel switches address spaces and lets the calling thread migrate. This is quite similar to the way AmigaOS handles it (@Solar), however provides mmu-powered protection.

cheers Joe

[Colonel Kernel]

This gave me an interesting idea however: You might know the so-called 'single address space operating systems'? They implement distinct address spaces in a way that they are isolated from each other, but every address space has it's own base address and size, and these never overlap (this is only really manageable on processors with 64 bit virtual addresses).

This sounds interesting... What OSes do you know of that use this scheme? You're not actually talking about segmentation, right? In other words, each "small address space" except possibly the lowest one has a non-zero base address, and all the code must be position-independent?

[Crazed123]

However, if you're using RPC to begin with it wouldn't it be a rather better idea to just implement message passing as a set of standard shared functions a given process can export?

Most times it is quite the other way round: You implement RPC on top of a lightweight message passing system. The point is, in a OS that uses address spaces the 'typical' way (each app has it's own, starting from 0), and you want to do a 'call' to an exported procedure of a remote address space, you would have to name it somehow: you need a seperate naming system. And this is where messaging is probably better suited as the foundation - it has to deal with naming anyway.

This gave me an interesting idea however: You might know the so-called 'single address space operating systems'? They implement distinct address spaces in a way that they are isolated from each other, but every address space has it's own base address and size, and these never overlap (this is only really manageable on processors with 64 bit virtual addresses).

This way you could implement a basic RPC system by simply calling an entry point in a remote address space, by it's unique virtual address. The task will pagefault, and the kernel can check whether a jump is legal or not (whether the entry point was exported and should be accessible to the calling task). If it is legal, the kernel switches address spaces and lets the calling thread migrate. This is quite similar to the way AmigaOS handles it (@Solar), however provides mmu-powered protection.

cheers Joe

I'd actually been thinking of doing it with position independent code libraries/processes that got loaded into the calling processes address space like any old shared library, or perhaps with a mimicry of classes like AROS does it, but that sounds good too.

[JoeKayzA]

This sounds interesting... What OSes do you know of that use this scheme? You're not actually talking about segmentation, right? In other words, each "small address space" except possibly the lowest one has a non-zero base address, and all the code must be position-independent?

I'm definitely not talking about segmentation , and yes, PIC is a requirement. IIRC, one of these systems is called 'Mungi' (the grand google will help you), it's based on the L4 microkernel. In principle it works the way I explained it above, if I got their docs right. The biggest advantage of this scheme is that you can globally name any object (be it code or data) with a virtual memory address, and access it simply over the address bus, instead of a higher level construct. And finally, it is a way how you can exhaust an amd64's virtual address space (when you can't exhaust it with one single application ).

cheers Joe

[Crazed123]

Wait, why in heck do we WANT to exhaust an AMD64's virtual address space? That's so huge we should never deliberately want a program or system to get that big. In fact, as a rule, user content should be the only thing that is just allowed to take up gratuitous space or CPU cycles.

[DennisCGc]

Just to add my 2 cents; I don't think PIC should be used in programs. Of course it is the right way in libraries, but not in programs. It'll give the Assembly programmers a hard time IMO. My OS is designed for simple programming, especially in Assembly. (but that's one my goals, I don't think it's your 'goal'). Doesn't mean it's cool to program, but still it's a lot of unneccesary code which CAN be avoided IMO.

DennisCGc.

[JoeKayzA]

Wait, why in heck do we WANT to exhaust an AMD64's virtual address space? That's so huge we should never deliberately want a program or system to get that big. In fact, as a rule, user content should be the only thing that is just allowed to take up gratuitous space or CPU cycles.

That was just kidding . I wanted to state that this would be almost impossible on a 32bit addressing machine. We could only really exhaust the amd64 address space when we really have hundreds of tasks running, and, of course, they carry huge amounts of user data.

I don't think PIC should be used in programs. Of course it is the right way in libraries, but not in programs.

Already in my current OS project, I'm running torwards the direction that _any_ executable is either a position independent or relocatable binary, that exports and imports symbols, so it behaves much like a library. The reason that lead to this is that I don't think it's a good practise to have all the 'real' program code in a library (like done in the IE), just because you want other programs to be able to use your functionality, and then have a small starter 'executable' that loads this library, just because the system cannot launch a library function directly.

How do you exactly define the term 'program'? After microsoft's definition, a library is not contrary to the term program, a lib is a type of program (I read this on their site). After my definition, a program is a construct that is plugged together of several binaries, that could even execute in several address spaces, but the 'play together' to do some task.

Btw, writing things in assembly is really not my goal. But I always respect that there are different opinions .


cheers Joe

[Colonel Kernel]

With this single-address-space thing... Does this also imply single-page-directory? I see big problems with that. How are you supposed to use the MMU for protection when everyone shares the same set of page tables? What's to stop one app from trashing another's memory?

Assuming you use multiple page directories that all show the same address space but with various parts being inaccessible depending on whose page directory is being used... I can see how that provides protection. But then, you still have gratuitous TLB flushes on every task switch (at least on x86). So where's the big advantage?

[DennisCGc]

I don't think PIC should be used in programs. Of course it is the right way in libraries, but not in programs.

How do you exactly define the term 'program'? After microsoft's definition, a library is not contrary to the term program, a lib is a type of program (I read this on their site). After my definition, a program is a construct that is plugged together of several binaries, that could even execute in several address spaces, but the 'play together' to do some task.

My definition of program; a binary which is designed with a goal, like a browser. (processes) Library; pieces of code which can be reused in programs. (like jpeg.dll f.e.)

Btw, writing things in assembly is really not my goal. But I always respect that there are different opinions .

Me too, PIC somehow simplifies the OS internals IMO, reduces the overhead needed to switch the CR3. (or am I just at the wrong end?)

OTOH, it might slowdown the programs a bit. (guess why?)

Btw, I have a question, which just popped in my mind; how do you achieve program security? (meaning; a virus f.e. isn't able to overwrite other processes?)