IPC implementation (pipes, event, and messages)

[AndrewAPrice]

My kernel implements four types of inter-process communication. I would like to know people's thought on these.

Events (Signals)
Events are primarily used by the kernel to communicate with programs. They can be related to Unix signals. Examples of events can be:
- a key was pressed
- a message was received
- the mouse was moved
- the mouse was clicked
- an interrupt was triggered (in a driver)

An event is represented by this structure:

Code: Select all

typedef struct
{
    unsigned int Type;
    unsigned int Action;
    unsigned int Data1;
    unsigned int Data2;
    unsigned int Data3;
} EVENT;

If a key was pressed, Type would equal KEYBOARD, Action would equal KEYDOWN, and Data1 would represent the scan code. Data2 and Data3 are currently not used for when a key is pressed.

There are two system calls related to events, one places the current thread into a sleep if there are no pending events, and the other gets the first pending event. Most event-based programs use these system calls together so that a program spends most of its time sleeping when there are no events to handle. Real time programs (for example, a game) omit the first system call so that they are constantly running. If the second system call is called when no events are pending, then the kernel returns an event where Type equals NOTHING (or 0).

I have reserved a special event type (USER-DEFINED) which I was going to use for fast interprocess communication (program specific user-defined events?), but I have decided against this idea, since some programs may abuse this system and send KEYBOARD/KEYDOWN events to random programs.

Messages
A message is a block of data of varying size that can be sent between processes.

A message is sent simply by finding the Process ID of the recipient, and telling the kernel where the message is in memory, and how big it is. The kernel automatically generates a Message ID, which the sending process need not to worry about.

To receive a message, the receiving program must allocate its own memory to store the message, and tell the kernel the Message ID of the message it wants to receive and where in memory to store it. If the Message ID does not match any pending messages, no error messages are generated, and the memory is left unchanged. If the program can't allocate enough room or doesn't want to accept a message (e.g. it only wanted messages from a specific program) then it can ask the kernel the scrap a message.

There are two ways of finding out the Message ID, the size of the message, and the Process ID of the sending message. The first way; when a message is sent an event will be sent to the receiving process that says this information. The second way; the receiving program can make a system call which returns this information and also states how many messages there are waiting on the queue.

This system is great and efficient for small messages, but may create some overhead for larger messages (more than a few megabytes).

Pipes
A pipe is the same as a Unix pipe (first-in-first-out stream of raw bytes), except they are represented by a number instead of a name. This number is specific to each program.

There are three modes a pipe can run in (from the individual program's point of view):
- Send: The program can send data on this pipe but not read from it.
- Receive: The program can receive data on this pipe but not send from it.
- Pool: The program can send and receive from the pipe. It's not really a two way pipe, but rather a pool of data each process can add to and read from.

There are system calls to write to a pipe, read from a pipe, and to find out how much data is in the pipe.

A program may create a pipe and write to it straight away without the other program acknowledging that this pipe exists. The other program will know a pipe exists because they will receive an event telling them a pipe was created along with the process ID of the program on the other side of the pipe and the mode the pipe is operating in (SEND, RECEIVE, POOL). The program will also receive an event if a pipe is closed.

If either program closes, then the pipe will close and all data within the pipe will be lost.

Shared memory
Probably the system I've given the least thought to, and the most insecure form of IPC. It'll mostly be used by drivers that need to transfer large amounts of memory from one address to another.

For sharing memory between programs, I might have some sort of acknowledgment system, or leave it out all together. Nothing has been finalised for now.

For sharing memory between drivers (programs marked as 'drivers' - drivers run in user-space in my micro kernel environment), they can map/unmap any part of physical memory and the kernel will automatically find somewhere in the driver's addressing space to allocate it.



Tell me what you all think. I'm welcome to any suggestions.

[01000101]

hi, I am particularly fond of the first system, the Events method. You are correct about the potential abuse factor though, but this could probably be removed with various validations like making sure a keyboard event was actually triggered by the cooresponding keyboard event and not a forged exploit.

That system seems more efficient as far as the information it provides to the other components, it is very thourough.

[AJ]

With regards to the program-exploitation thing, how about just allowing the user mode program to fill the 'Action' and 'Data' fields? Any message being sent from a user program automatically has the 'Type' field filled with USER-DEFINED (or allow the 'Type' field to be filled, but the IPC mechanism automatically changes it to USER-DEFINED)?

Cheers,
Adam

[AndrewAPrice]

@01000101 and AJ: Thanks. I'm reserving events purely for kernel to user messaging. Although if I did allow user <-> user events I would make sure they were only allowed to send USER-DEFINED events.

@01000101: I can't just have events, since you'll just be limited to sending 20 bytes at a time (4 bytes would be reserved for specifying the event Type is USER-DEFINED, another 4 for encoding a header (message part, total parts, etc) into Action, leaving 12 bytes for the actual message). If I did have dynamically sized events then I would use them instead of messages, but after considering the overhead in such a system it is probably best just to stick with messages. And it makes it easier to distinguish between kernel and user communication.

[01000101]

yea, that makes sense about the size limitation issues.

another option to designing a dynamically sized event system would be to keep the message numeric and have the message numerals coorespond to an array of pre-defined messages. Array size would be pretty lengthy, but it would allow for alot of different messages to be declared and with it being predefined, probably eliminate any exploitation possibilities.
just a rough idea.

[bewing]

You might want to expand the event message body to 16 or 24 bytes -- ie. 2 or 3 (maybe 4) 64-bit qwords. And I'm pretty sure in the end that you will want to add the user-defined event messages back in. You don't want people cursing your name in the future for making an IPC mechanism that they can't use, and that is artificially limited to a too-small packet size. When you port to 64-bit, you will (at least) want to xmit 64 bit data. I can't tell you the number of times I've cursed K&R for using INTs, instead of LONGs, for the inputs to many kernel functions. Artificial limitations suck.

[AndrewAPrice]

Should pipes be updated request-by-request or should I buffer requests together? For example, if writing less than 4kb to a pipe then the data should be store it in a buffer until the buffer is at least 4kb and send the entire buffer at once. The same can be done for reading (read 4kb in one request and keep it in memory until needed). This wouldn't work for the pipes running in the pool mode, unless you wanted to pool 4kb packets at a time (what was a flaw in design is now a feature!). My main goal of buffering data is to reduce the number of system calls.

I've given some thought to shared memory also. A program may request a chunk of memory of certain size, and the creating program also provides a 64-bit key. The system returns the ID of the shared memory. By sending the ID and key of the shared memory to another process, they can request the size of the shared memory and add it to their memory.

The area of shared memory will have a reference counter associated with it. Any process may request access to the memory if they have the correct ID and key, and the counter will increment. When a process releases the memory the counter will decrement. When the counter reaches 0 the shared memory will be free and all data in it will be lost.

[senaus]

...they can map/unmap any part of physical memory...

Doesn't this defeat the purpose of putting the drivers in user space? If a driver gets the wrong address and overwrites parts of the kernel, who knows what will happen?

I like the idea of events. You say they can be related to Unix signals, is that because you will interrupt the thread to run the 'event handler'? It reminds me of my design, which is based entirely on remote method invocation, callbacks, and pipes ('cos sometimes you just wanna stream data...).

Cheers,
Sean

[AndrewAPrice]

...they can map/unmap any part of physical memory...

Doesn't this defeat the purpose of putting the drivers in user space? If a driver gets the wrong address and overwrites parts of the kernel, who knows what will happen?

I want to avoid that too. That's why I'm discussing it in this thread, to see if people can find flaws and/or suggest improvements