Musings of an IPC Starved OS...

[elderK]

Hey people!
Long time no see!

It is time to investigate Interprocess communication and after much research and hairpulling, Im still somewhat confused.

So! I shall ask you guys!.
I intend to implement Messagebased communication, mostly because it seems a nice and tidy way of communicating. A nice polite way to do it.

The idea in my head :
- Processes have their own Mailboxes. A mailbox simply holds X entries of Mail. Mail is a simple small structure, holding the PID of the process that sent it and an ID of a Message buffer.

Message Buffers:
Each process has a Messagebuffer tree. These buffers hold the data of the actual message. They are variable sized and are under Kernel control.

They track the data sent, the sender and the buffer size and the actual ID of the Buffer. The Key is simply the Buffer ID.

The idea:
Processes talk to eachother.
A process can send data, a process can retrieve data.

Example:
Process A wants to send a structure, 32bytes in size to Process B.

It does the following call:
send(process_b_pid, &structure, 32);

Im undecided on whether Sends are immediate or are queued, but regardless their function is the same.

The Kernel allocates a new BufferNode and adds it to the Destination processes BufferTree, it then stores the Buffer related data (Sender PID, Size, Buffer ID). It knows the size, so it allocates the buffer itself and copies the data from the current context (process A) to the buffer.

SSWITTCHCHH
We are in Process B now, with its context and addressspace.

It wants to check its inbox for this data.

structure* bla = (structure*)recieve(process_a_pid);

This comes in two peices. First of all, the usermode call checks the Process Mailbox, which is mapped low and viewable by the Process. It finds that t here is a message from process_a, it also retrieves the BufferID.

A systemcall is made to determine the size of the Buffer, once this is done, the usermode recieve call allocates a appropriately sized buffer for the data, another system call is made to copy it from Kernel buffer into the Usermode buffer. This call also deallocates the Buffer once the copy is complete.

The USermode call simply returns the address to its buffer. You can free the buffer easily since the usermode part allocates it like any other usermode allocation.

Mailboxes further:
Okay, so we have mailboxes, sure. That implies a zone of space used for storing Mail, what happens if we dont have anymore mailbox space left?

NO PROBLEM! The Messaging subsystem of the IPC Component maintains queues. Basically, Processes are added to the Messageable list. When the Messager sends a message, itll call something like ... add_message(pid, thismessage). If the message cant be added, it is queued. Everytime a message is sent, the Messager checks the queues and tries to send the waiting messages.

When Mail is read from a Processes Mailbox, it is automatically deleted to make room for further mail.

Ideas I have now:
- You could have the buffer size in the Mail. (pid, bufferid, buffersz).
- This is potentially slow, due to all the copying.

I will eventually do something like Ports or Shared Memory.

Mutex / Semaphores in my mind are mostly useful for Intraprocess synchronization. Multithreaded programming, etc, making sure something doesnt use a variable when its already being used. I can see this stuff would be handy in the Kernel world too, especially in ports.

This is at this point, crazy conceptual thoughts that popped into my brain while chatting with a good friend.

Ideas, suggestions, they are welcome.
~Zeii.

[elderK]

No feedback at all?

*sigh*

Can anyone point me to a good detailed reference on the Messaging concept?

Tanenbaums book has told me all it can.

Any books you can recommend would help out a lot too.

Thanks people?
~Zeii

[Daedalus]

I wrote my own IPC that seems similar to the way you do it.

If you like, I could send you my mostly standalone IPC code.

The way I do is:

Code: Select all

* A process registers itself for IPC specifying a handle name, and is returned a pointer that you can send messages to.
* Another process can use the search function to look up another process' handle, and send messages to that

* Each process would have to check for waiting messages (can be checked by retrieving a message or having the IPC service count waiting messages for a given handle)

* Each process can send data to another process, giving a data address and the number of bytes to send. Also, optionally a process can send a reply-to handle so the receiving process can send a message back.

If you would like to see my code, I'll reformat it so it compiles in Turbo C again and you can see how my version works (it has tests that demonstrate the IPC in action.)

EDIT: I'd just like to add, that in my kernel, processes call an interrupt and specify a number that refers to the function to call. This is how processes call my IPC functions.

[nick8325]

That all sounds good to me

If you want to avoid copying everything twice, you could keep the kernel buffer mapped into the receiver's address space. You'd need to make sure that each page only contained messages belonging to one process, though, otherwise the receiver'd be able to see other processes' messages. Perhaps you could allocate buffer pages for each process separately, and allocate buffers out of that.

I'm not sure that there'll be much overhead from copying for small messages, though. You could always map big messages into the receiver instead of copying them.

Also, you'll need to be a little bit careful about race conditions: you'll need to make sure that nothing will go wrong if the kernel pre-empts the receiver (and maybe changes the mailbox) while the receiver is checking the mailbox. It should be OK, I think, as long as the kernel only adds things to the mailbox, and doesn't change or remove them unless the receiver asks it to.

[Daedalus]

If you want to avoid copying everything twice, you could keep the kernel buffer mapped into the receiver's address space.

Was this aimed at me? Please forgive me if it was not ...

My kernel currently does not have paging, and thus does not yet make any protections in this way.

It is however, small (~200 lines with the example included) and can run both in my OS and in turbo c/djgpp with little modification.

Eventually, when paging goes in I will have to look at these things, but for now it meets the requirements and does what needs to be done.

[elderK]

Thanks guys!
I like the idea of directly mapping the data into a standalone kernel buffer.
Much faster.

Problem is you can only map sizes of a 4kb, 2mb or 4MB, far too large for most things.

Ive figured for LARGE data transfers, Processes will work together and share memory. That way there is very little copying, just careful synchronization.

MAN! I tell you, Finding books on IPC and Memory management is damn near impossible.

Thanks again!
~Zeii.

[nick8325]

If you want to avoid copying everything twice, you could keep the kernel buffer mapped into the receiver's address space.

Was this aimed at me? Please forgive me if it was not ...

No, it was aimed at Zeii

Problem is you can only map sizes of a 4kb, 2mb or 4MB, far too large for most things.

Well, you can still do things with only one copy for smaller messages...(copy from sender to kernel buffer, have kernel buffer already mapped into receiver)

For big messages, you could use copy-on-write mapping. This acts just like copying, but is as fast as mapping. If you want to send, say, a page of memory, you make it read-only (by changing the page tables in the sender), then map it read-only into the receiver. Now both processes can read that page. If they try to write to the page, you'll get a page fault, and then the kernel will need to give each process its own private copy of that page (otherwise, the sender could see changes made by the receiver and vice versa, which wouldn't look like copying).

As for books, maybe one of Tanenbaum's would be good. There's "Operating systems: design and implementation" and "Modern operating systems". I've ordered a copy of the second one, should be arriving in the post soon! Yay!

[elderK]

HEheh YAY! *jumps for joy*
Both of those books I own I ordered "The Indispensible PC Hardware Book" two days ago,I cant wait for it to arrive . *dances* Apparently its like the ultimate interfacing book ... with cherries on top!

So, its awesome. Im thinking of buying an uber book on Data structures, it has everything from all the lists to all the trees, to sets and matrices.

... if I can manage to afford it .
Damn working in the Black Arts er, I mean OS programming .

Thanks Nick, Youve given me some new ideas to ponder with .
~zeii

[carbonBased]

So, its awesome. Im thinking of buying an uber book on Data structures, it has everything from all the lists to all the trees, to sets and matrices.

... if I can manage to afford it .

This is a worthwhile investment for any programmer. While working on any project, a programmer should have an good toolbox of data structures that he/she could use for a specific task. One might argue these are the most useful tool of any programmer.

Being able to code up a solution to a problem is one thing... doing it correctly, using the right structures, that scale well under load, is another.

Damn working in the Black Arts er, I mean OS programming .

I think you'll find all of these concepts in any moderately intricate program, not just OSDev. It's just harder to ignore in OSDev

--Jeff