Microkernel multithreading IPC

[itisiuk]

well, got a lot of work done on my microkernel recently but have run into a problem where
things are starting to slow down a lot because of event handling.

i managed to get this to speed up alot now by just sending a direct msg to the gui server instead of each process
and allowing the gui server to handle everything from there which it did before.
however this is still slow as i have to wait for blocking events.

i believe i can speed this up alot by introducing threading into my kernel.
my problem is that i dont know how to implement my IPC for this.
this is my first attempt at using threads and am running into quite a few problems but
this is the main one at present.

currently i unblock the reciever on sending a msh
block on recieve if there is no msg.
then i schedulal the next process if necessary (which has helped speed things up)

each process has a msgbox which is a queue of msgs basically,
struct msgbox
{
msg *head
msg *tail
}

and each msg consists basically of
stuct msg
{
int sender // process who sent the msg
int recever // process to receive msg
int number // what to do
union{
int data1 // data types
int data2 // data types
}
msg *next
}

basically i want to know if i should continue to pass messages between process like above when using threads or
give each thread a msgbox and change my message struct to
stuct msg
{
int senderp // process who sent the msg
int sendert // thread who sent the msg
int receverp // process to receive msg
int recevert // thread to receive msg
int number // what to do
union{
int data1 // data types
int data2 // data types
}
msg *next
}

then rescedual to the next thread if nessecary.

im not wanting any more info on event handling my current way of doing things fit nicely together even if its slow.

thanks in advance

[AndrewAPrice]

I'm not sure exactly what you're asking, but here's a rundown of how I tackled the IPC with threading:

In my kernel, messages and kernel events are sent to each process, not each thread. I have two system calls to get the next message:
- Get message or return NULL if no message is waiting (for real time applications)
- Get message or sleep until a message is available (for event based applications)

Messages are sent to the process, not the thread. So when I call the second syscall (sleep until a message is available) only the thread that called the syscall is put to sleep, and only that thread is woken.

I guess you could have multiple message stacks, one for the process and one for each thread.
I am guessing you want a thread for each process that communicates with that process, so you will need to know when the calling process doesn't want to communicate with the callee anymore (especially if the caller terminates uncleanly and the callee still has a thread sleeping waiting for it).

I have gotten around this somewhat because another IPC primitive my kernel supports are pipes. For something like transferring a file from my VFS into a pipe, a spawn a thread in the VFS and I also initiate a pipe between caller/callee. The inherit design of my pipes gets around two issues I would have had with per-thread messaging:
- Automatic syncronisation. A thread (only that thread, not the entire process) is blocked if it tries to read more than what is in the pipe (until the pipe is filled up), or it tries writing more than the buffer allows (until the pipe is emptied). I also provide the mechanism for a process to prevent blocking in this occurs (though it requires a little extra logic when you read/write).
- Automatic termination. An event is sent to the other process if one process decides to close the pipe. The pipe is automatically closed (alone with sending the other process an event telling it so) when a process is terminated (cleanly or uncleanly).

[itisiuk]

i havent implemented pipes, currently i only have a basic vfs and a ramdisk
as im having a problem getting my ATA code to recognise my DVD drive.
it works on my test PC tho which only has a cdrom. but that ill sort out later.

but from what i read,

In my kernel, messages and kernel events are sent to each process, not each thread. I have two system calls to get the next message:
- Get message or return NULL if no message is waiting (for real time applications)
- Get message or sleep until a message is available (for event based applications)

basically you pass you messages to the process with a thread id attached to it
so only that thread in that process can read the message?

so what i want to do is when the thread wants to receive a msg it will look into the process msg queue,
in my case the msgbox, and look for its id in the msg.
if it carnt find its id in the msg, then i would block the thread instead of the process.

when i want to send a msg i would want to send it to the process msg queue with the sender thread id attached to the msg.
then unblock the receiving thread.

but now i would have the problem of knowing which thread is receiving the msg,
unless i make my servers and drivers single threaded,
or use specific thread id's for specific functions such as receive mouse input as thread 1, fetch keyboard input as thread 2, etc.

however user processes would still not be able to send msgs between themselfs because they would not
know which thread to send to.
but i carnt think of a reason why they would need to at present anyway.

[AndrewAPrice]

basically you pass you messages to the process with a thread id attached to it
so only that thread in that process can read the message?

so what i want to do is when the thread wants to receive a msg it will look into the process msg queue,
in my case the msgbox, and look for its id in the msg.
if it carnt find its id in the msg, then i would block the thread instead of the process.

My messages are per-process, I don't attach any thread ID to it.
In your case, you could attach a parameter to your message being the Thread ID. If the TID is 0 (reserve this number so that no processes have it (or use it for your kernel or something)) then either any thread (or only the root thread) will receive this message, otherwise the message can only be received by that particular thread.

however user processes would still not be able to send msgs between themselfs because they would not
know which thread to send to.
but i carnt think of a reason why they would need to at present anyway.

Are you saying you want to send messages between threads of the same process that share the same addressing space? You would normally do that in a user-space library (which would be a lot more efficient that passing messages through system calls), though if you wanted to do it in the kernel (to achieve consistency) then that's your choice.

[itisiuk]

Think im going to have todo a more research before i try and implement threading.
the IPC is getting confusing.

Are there any good source code examples about?

[itisiuk]

Well ive given this some more thought
and done some more research.
i did have a diagram to show what i mean but i couldnt get
it to upload so ill try and explain how i would have a threaded IPC work


1st. The user process will create a new thread with its own msg postbox
2nd. The thread would send a msg to the server e.g. fs server
3rd. the fs server would create a new thread and attach the message sent by the user thread to its message box.
4th. the fs server thread would then be able to send a message back to the user process thread
5th. now the two threads should be able to send messages directly.

what does anyone think about this method?

other quick question about threads,
should drivers have multiple threads??

[itisiuk]

cheers.

and yes thats kinda where i found my info.
however im not using ports, trying to keep away from that as it slows the
whole of the IPC down too much for me.

i want to keep my IPC fast and simple, which is what ive got now.
i.e just send, recieve and reply
none of this create connection, wait for message, close connection stuff as i dont think that is
needed at all.

my main concern is if im heading allong the right lines as im still
not sure on the theory as this is my first real attempt at getting threads
to work.