Message passing OR how to keep things in right order...
-
- Member
- Posts: 1600
- Joined: Wed Oct 18, 2006 11:59 am
- Location: Vienna/Austria
- Contact:
Message passing OR how to keep things in right order...
Yeah, it's me again, radio vienna speaking from the summit of Kahlenberg.
What brings me hither is message passing, and several portfeuilles coming up to my mind. May be you can scan throu' it. Feel free to tell me too, if i am on the way to build a load of bullshit.
1:
An Interrupt occurs. the ISR KNOWS, there is a task waiting on the blocked queue. It puts a message on the assigned location. I want to queue the messages up. Therefore, the ISR has to fetch data from the device and store it somewhere for the task. Or shall I put only ONE message? and the Driver task does the data fetching?
2:
A Process needs data from an other process, say an acknowledge. Is it ok to put it on a dedicated queue in the assigned location (I gonna name it Postbox), and remove it from there as soon as the message it is longing for, has arrived?
3:
A Process sends a message but doesn't need any values from the other part,which waits on the postbox.It just puts the message in the postbox and continues its work.
4:
A Process checks his postbox, if his counterpart(s) has(ve) dispatched messages for him. If not, it continues it's usual housekeeping work, otherwise, it fethces and processes the messages one by one till the queue in the postbox is emptied. When the communication protocol determines that there has to be an answer sent to the others, it sends answers. this is of course needed, when the other process(es) is/are waiting on the postbox.
5:
Is it good to locate the pointer to the postbox in the process structure?
thank's if you've considered reading this and giving me some suggestions or critics.
stay safe.
What brings me hither is message passing, and several portfeuilles coming up to my mind. May be you can scan throu' it. Feel free to tell me too, if i am on the way to build a load of bullshit.
1:
An Interrupt occurs. the ISR KNOWS, there is a task waiting on the blocked queue. It puts a message on the assigned location. I want to queue the messages up. Therefore, the ISR has to fetch data from the device and store it somewhere for the task. Or shall I put only ONE message? and the Driver task does the data fetching?
2:
A Process needs data from an other process, say an acknowledge. Is it ok to put it on a dedicated queue in the assigned location (I gonna name it Postbox), and remove it from there as soon as the message it is longing for, has arrived?
3:
A Process sends a message but doesn't need any values from the other part,which waits on the postbox.It just puts the message in the postbox and continues its work.
4:
A Process checks his postbox, if his counterpart(s) has(ve) dispatched messages for him. If not, it continues it's usual housekeeping work, otherwise, it fethces and processes the messages one by one till the queue in the postbox is emptied. When the communication protocol determines that there has to be an answer sent to the others, it sends answers. this is of course needed, when the other process(es) is/are waiting on the postbox.
5:
Is it good to locate the pointer to the postbox in the process structure?
thank's if you've considered reading this and giving me some suggestions or critics.
stay safe.
... the osdever formerly known as beyond infinity ...
BlueillusionOS iso image
BlueillusionOS iso image
Re:Message passing OR how to keep things in right order...
This probably depends on the device. If you don't queue messages, and the driver can't handle messages quickly enough, what should happen? The device didn't interrupt for no reason; it probably needs to driver to read some data out of it, and quickly.beyond infinity wrote:1:
An Interrupt occurs. the ISR KNOWS, there is a task waiting on the blocked queue. It puts a message on the assigned location. I want to queue the messages up. Therefore, the ISR has to fetch data from the device and store it somewhere for the task. Or shall I put only ONE message? and the Driver task does the data fetching?
However, most devices won't interrupt again until you acknowledge their interrupt, which can be done from inside the driver. But that might slow things down unnecessarily.
I handle this as follows:
-- ISR reads the relevant data from the device (e.g. scan code from the keyboard, or status bytes from the floppy drive), stores them in the device object, and asks the kernel to queue a deferred procedure call. The keyboard ISR queues scancodes because they're easy to queue, and because the keyboard can issue another interrupt without being asked. Floppy status bytes don't get queued, because it won't issue another interrupt without being asked (i.e. without another command being issued).
-- Deferred procedure call routine executes some time later, with interrupts and scheduling enabled. It dequeues data from the device queue and processes it.
That sounds OK to me.2:
A Process needs data from an other process, say an acknowledge. Is it ok to put it on a dedicated queue in the assigned location (I gonna name it Postbox), and remove it from there as soon as the message it is longing for, has arrived?
OK.3:
A Process sends a message but doesn't need any values from the other part,which waits on the postbox.It just puts the message in the postbox and continues its work.
Still OK.4:
A Process checks his postbox, if his counterpart(s) has(ve) dispatched messages for him. If not, it continues it's usual housekeeping work, otherwise, it fethces and processes the messages one by one till the queue in the postbox is emptied. When the communication protocol determines that there has to be an answer sent to the others, it sends answers. this is of course needed, when the other process(es) is/are waiting on the postbox.
I don't see why not. It makes things easier, assuming that this message passing is a central part of the kernel.5:
Is it good to locate the pointer to the postbox in the process structure?
- Pype.Clicker
- Member
- Posts: 5964
- Joined: Wed Oct 18, 2006 2:31 am
- Location: In a galaxy, far, far away
- Contact:
Re:Message passing OR how to keep things in right order...
roughly agree with Tim's opinion.
Personally, i would try to have the ISR being able to extract the data from the device (for hard disks and network card) and store them in the location defined by the current IO request (thus preparing a kernel area to map the user-area where the data finally should go, if the data must go to a user space).
I would also try to have the ISR being able to read the next request and send it to the device, so that the sleep time of the device is minimized as long as there are requests for that device.
Personally, i would try to have the ISR being able to extract the data from the device (for hard disks and network card) and store them in the location defined by the current IO request (thus preparing a kernel area to map the user-area where the data finally should go, if the data must go to a user space).
I would also try to have the ISR being able to read the next request and send it to the device, so that the sleep time of the device is minimized as long as there are requests for that device.
Re:Message passing OR how to keep things in right order...
Right... I'm of the opinion that an ISR should do as little as possible, as quickly as possible (service the device and leave)...Pype.Clicker wrote:Personally, i would try to have the ISR being able to extract the data from the device (for hard disks and network card) and store them in the location defined by the current IO request.
...so queueing the next request (which might involve spinning on a spinlock, waiting on a semaphore, or doing lots of I/O) is done with interrupts and the scheduler enabled, in a DPC.I would also try to have the ISR being able to read the next request and send it to the device, so that the sleep time of the device is minimized as long as there are requests for that device.
Re:Message passing OR how to keep things in right order...
thanks for you replys, lads. Gonna implement message passing as a main feature of my kernel thingy. It will be the way to pass information amongst taskses in kernel space. I may provide usertasks also with a library for message passing thus easing IPC.
Also, I have noted that you two both say it is better to queue up data - stuff it in a buffer - fetched from a device and free the device to continue work as soon as possible (in the isr) and have the device driver (can be a task or whatsoever) do the rest of the work.
have a nice day lads
Also, I have noted that you two both say it is better to queue up data - stuff it in a buffer - fetched from a device and free the device to continue work as soon as possible (in the isr) and have the device driver (can be a task or whatsoever) do the rest of the work.
have a nice day lads
- Pype.Clicker
- Member
- Posts: 5964
- Joined: Wed Oct 18, 2006 2:31 am
- Location: In a galaxy, far, far away
- Contact:
Re:Message passing OR how to keep things in right order...
Err ... Tim, i'm confused here ...Tim Robinson wrote:
Right... I'm of the opinion that an ISR should do as little as possible, as quickly as possible (service the device and leave)...
...so queueing the next request (which might involve spinning on a spinlock, waiting on a semaphore, or doing lots of I/O) is done with interrupts and the scheduler enabled, in a DPC.
You say you would defer reading of the block data of an ATAREAD command in a Deferred Procedure Call ... and that you would also issue the next request from the DPC, am i right ?
But ... but if i get a look at your ATA driver, you're exactly doing what i suggest: insb16 and issue the next ATA command (if needed) -- including waitFor(controller, !BUSY) in your ISR (or maybe is
Code: Select all
bool AtaCtrlIsr(device_t *dev, uint8_t irq)
Re:Message passing OR how to keep things in right order...
Yes.Pype.Clicker wrote:Err ... Tim, i'm confused here ...
You say you would defer reading of the block data of an ATAREAD command in a Deferred Procedure Call ... and that you would also issue the next request from the DPC, am i right ?
I know.But ... but if i get a look at your ATA driver, you're exactly doing what i suggest: insb16 and issue the next ATA command (if needed)
I'm just a crazy mixed-up guy.in which case the name is confusing ...
I'm doing things the DPC way in the second kernel I'm writing. The Mobius sources on the web don't scale at all to SMP, or even to any decent multitasking on a single processor.
- Pype.Clicker
- Member
- Posts: 5964
- Joined: Wed Oct 18, 2006 2:31 am
- Location: In a galaxy, far, far away
- Contact:
Re:Message passing OR how to keep things in right order...
oow. no chance ... and do you have any idea of why it is so ? the isr/dpc stuff could be an explanation ?
btw, in what kind of context do the so-called "DPC" occur ? is it still at kernel level ? is it in the address space of the interrupted process or in the space of the requestor process ?
Maybe the best way to handle this would be to re-enable the interrupts even in the IRQ handler (for disks, etc)? The high-priority interrupts like keyboard, mouse, video retrace etc. could still be processed, but no other disk operation (which wouldn't occur in any way).
And, imho, the fetch of I/O data is prioritary over executing user program, so there's no problem if the interrupt is long to return.
So far, in my ATA driver design, i have met the need for locking the access to the request queue while the queue is being reordered (so that the ISR-driven controller don't mess with operations), but those locks can be made very short (a few instructions) and happen with interrupts disabled (thus the only case when the driver would lock is a multiprocessor system, in which case a busy-loop can satisfy us as we're certain the lock will be released very soon)
I still have to give a look at Linux ide-dma module to see how things happen there ...
btw, in what kind of context do the so-called "DPC" occur ? is it still at kernel level ? is it in the address space of the interrupted process or in the space of the requestor process ?
Maybe the best way to handle this would be to re-enable the interrupts even in the IRQ handler (for disks, etc)? The high-priority interrupts like keyboard, mouse, video retrace etc. could still be processed, but no other disk operation (which wouldn't occur in any way).
And, imho, the fetch of I/O data is prioritary over executing user program, so there's no problem if the interrupt is long to return.
So far, in my ATA driver design, i have met the need for locking the access to the request queue while the queue is being reordered (so that the ISR-driven controller don't mess with operations), but those locks can be made very short (a few instructions) and happen with interrupts disabled (thus the only case when the driver would lock is a multiprocessor system, in which case a busy-loop can satisfy us as we're certain the lock will be released very soon)
I still have to give a look at Linux ide-dma module to see how things happen there ...
Re:Message passing OR how to keep things in right order...
Currently, in the new kernel, DPCs get called from any CPU's idle thread.
The idle threads' pseudocode (one per CPU) looks like this:
This means that DPCs get run in kernel mode, but outside of the context of any interrupt. It also means that they have a lower priority than any other thread (because they are in an idle thread); this will need to change because any CPU-intensive thread will block any I/O. I think DPC code should have higher priority than any thread but lower than any interrupt.
However, I don't think it should matter if I/O code gets interrupted, as long as the driver doesn't try to access the device in the meantime. At the moment, the IDE driver ins16() code is surrounded by a spinlock, which disables interrupts in this CPU and stops any other CPUs from acquiring the same spinlock. I don't see any reason why this spinlock couldn't be replaced by a mutex that protects the IDE controller's registers, so that hardware interrupts could occur, yet no other commands could be issued to the hard driver, until the data had been read. "Hardware interrupts occur" = IRQ0 code runs = scheduler gets called = task switch.
The idle threads' pseudocode (one per CPU) looks like this:
Code: Select all
while (!dpc_queue.is_empty())
{
dpc_queue.first().run();
dpc_queue.remove_first();
}
__asm__("hlt");
However, I don't think it should matter if I/O code gets interrupted, as long as the driver doesn't try to access the device in the meantime. At the moment, the IDE driver ins16() code is surrounded by a spinlock, which disables interrupts in this CPU and stops any other CPUs from acquiring the same spinlock. I don't see any reason why this spinlock couldn't be replaced by a mutex that protects the IDE controller's registers, so that hardware interrupts could occur, yet no other commands could be issued to the hard driver, until the data had been read. "Hardware interrupts occur" = IRQ0 code runs = scheduler gets called = task switch.
- Pype.Clicker
- Member
- Posts: 5964
- Joined: Wed Oct 18, 2006 2:31 am
- Location: In a galaxy, far, far away
- Contact:
Re:Message passing OR how to keep things in right order...
indeed, the mutex approach sounds better.
Could i ask you for a review of my design-to-be ?
1. at each interrupt, the ataController picks up its current ataRequest and signals it that the interrupt occured by calling ataRequest->handle(). This method return TRUE if the request completed or FALSE if it should remain the active request (for instance in the case of a multi-sectors read command).
handle() calls should thus occur in response of ATA interrupt, with IRQ enabled, but ATA IRQ masked by the PIC (no "EOI" sent)
2. when a request finally completed at an IRQ, the controller locks the request queue and pick the head request, then unlock the queue. ataRequest->issue() is called in the context of the current interrupt to prepare the next operation.
3. when a new request must be submitted, the ataScheduler locks the request queue and gives the controller an empty queue instead, then it unlocks. The request scheduling can then occur without disabling interrupts for too long. The locking technique involved is
a) interrupts are disabled (so that the locking time is deterministic)
b) a spinlock variable is set (so that any other processor can see the controller is locked, and enters a busy-loop until it is released)
4. When the ataScheduler reinstalls the queue, it locks the controller and checks whether there is a current request. If there's none, it will call queue->head->issue() with interrupt disabled before it returns to normal operations.
5. a long-term synchronizing technique like a semaphore is used to enter the ataScheduler and give it a request, so that we can ensure only one thread will attempt to access the queue at once, but still not blocking the CPU during that wait time (which may be pretty long).
sounds good ?
Could i ask you for a review of my design-to-be ?
1. at each interrupt, the ataController picks up its current ataRequest and signals it that the interrupt occured by calling ataRequest->handle(). This method return TRUE if the request completed or FALSE if it should remain the active request (for instance in the case of a multi-sectors read command).
handle() calls should thus occur in response of ATA interrupt, with IRQ enabled, but ATA IRQ masked by the PIC (no "EOI" sent)
2. when a request finally completed at an IRQ, the controller locks the request queue and pick the head request, then unlock the queue. ataRequest->issue() is called in the context of the current interrupt to prepare the next operation.
3. when a new request must be submitted, the ataScheduler locks the request queue and gives the controller an empty queue instead, then it unlocks. The request scheduling can then occur without disabling interrupts for too long. The locking technique involved is
a) interrupts are disabled (so that the locking time is deterministic)
b) a spinlock variable is set (so that any other processor can see the controller is locked, and enters a busy-loop until it is released)
4. When the ataScheduler reinstalls the queue, it locks the controller and checks whether there is a current request. If there's none, it will call queue->head->issue() with interrupt disabled before it returns to normal operations.
5. a long-term synchronizing technique like a semaphore is used to enter the ataScheduler and give it a request, so that we can ensure only one thread will attempt to access the queue at once, but still not blocking the CPU during that wait time (which may be pretty long).
sounds good ?
Re:Message passing OR how to keep things in right order...
This sounds a lot like the way I've been doing it in my older kernel source, and it's worked well so far.