OSDev.org

I've been working on the design of my own OS project, and I'm trying to iron out the details of my message passing system. The plan is to make sort of a hybrid microkernel/modular kernel by having drivers loaded as processes but keeping them all simultaneously mapped in high kernel space. This is what I have for a message passing system:

• Every message is sent on a port.
• Ports are opened between threads (the sender) and processes (the receiver).
• The message is passed in registers and has a fixed size.
• Shared memory is always linked to a port; it can be used to help pass oversized messages.
• Messages are handled asynchronously.
• Instead of having an explicit respond() operation, messages are sent on certain port numbers. Normal messages travel on even-numbered ports; the responses travel on the next highest odd-numbered port.
• A sender can have only one outgoing message waiting on each port; receivers have a fixed-size queue. If it fills, processes are blocked.
• When a process is ready to receive a message, the OS starts a special thread that runs its message handler. Only one message handler thread can run in that process at once.
• When the message thread is started, it takes over the remainder of the calling thread's timeslice unless it ends early enough for the calling thread to do more work.

If you have any suggestions for additions/changes, I'd love to hear them. This is my first attempt at OS dev, and I can use all the help I can get .

How can you pass the message in registers if it is to be asynchronous? A second message will overwrite the first one, which may not have been processed yet. And if you use a fixed port for replies, how does a process know that a reply is meant for it?

blm768 wrote:When a process is ready to receive a message, the OS starts a special thread that runs its message handler. Only one message handler thread can run in that process at once.

Why spawn a new thread if there's only going to be one thread running in that process at a time? You might as well have that thread fetch the next message or sleep after it finishes instead of destroying it. And since the request is going to preempt the 'main' thread anyway, it might be simpler to implement it like a *nix signal instead of a thread.

You try to design asynchronously messages, however in many situation the system is blocking which defeat the purpose (at any given time on a process, one sender only, one handler only).

So before look into implementation details, you should list your design goal. Is that way make API easier to work with? Are you optimizing for something?

Thanks for the feedback. In response to the questions:

The registers are only used for storage as the message is passed into/out of the kernel. If it can't pass the message right away, it will store it in its internal queue.
I guess "port" might not be the right terminology for what I'm doing; it's just a connection between a single sending thread and a single receiving process. Each process has its own "local" connection IDs, so it knows which process corresponds with which ID. A thread can be sending multiple messages on different connections; it just can't send two or more messages on the _same_ connection at once.
The main reason to make it asynchronous is to allow the sender to be waiting on multiple messages at a time. There is no real benefit on the receiver process side.
Making the thread sleep is a good idea; I might end up doing that. My original plan was to have a special thread that would never really be "destroyed," just halted and re-initialized (basically woken with the stack and program counter reset to the start of the message handler). I guess it's a lot like a *nix signal except that it doesn't halt the receiver's main thread unless explicitly told to do so.
My main goal is to make the API as simple and as possible while keeping it robust and reasonably efficient. I plan on having just one syscall, "send message," but a message with connection ID #0 goes straight to the kernel and is handled like a traditional syscall rather than a message.

If you are going to transfer the message to memory, what's the point in passing it in registers in the first place? Wouldn't it be easier just to pass a pointer to the message in the first place?

iansjack wrote:If you are going to transfer the message to memory, what's the point in passing it in registers in the first place? Wouldn't it be easier just to pass a pointer to the message in the first place?

The message has to be copied into the kernel's queue anyway; putting it in registers saves a memory access. For longer messages, an offset into shared memory will be passed in one of the registers.

When a process is ready to receive a message, the OS starts a special thread that runs its message handler. Only one message handler thread can run in that process at once.

This seems a bit restrictive. For example, it means that a filesystem service will only be able to serve a single client at a time, even on a multi-core machine. Do you have a reason for this ?

blm768 wrote:A thread can be sending multiple messages on different connections; it just can't send two or more messages on the _same_ connection at once.

However, the more common case is two application(or server-client) communicates with many message within _same_ connection. This restriction basically disable any asynchronous benefits. Take GUI as an example, application can't sent or responds to second GUI event before one end has processed it...

For the case only one messages on the _same_ connection at once, the application is either highly synchronous or at very low stress, you don't need to optimize for it.

blm768 wrote:My main goal is to make the API as simple and as possible while keeping it robust and reasonably efficient. I plan on having just one syscall, "send message," but a message with connection ID #0 goes straight to the kernel and is handled like a traditional syscall rather than a message.

I see, but the way you optimize for machine level somehow restrict the use case and affect the design choice itself.

It does look like I should allow multiple messages from a sender on one connection. I'd overlooked some of the speed advantages, and, on analysis, there's not really a good reason for the restriction. I might also relax the restriction on the number of message handler threads; my main reasons for restricting them were that most handlers would run fairly quickly, multiple handlers would likely pile up around locks, and I want to avoid spawning tons of threads.