Hi,
online wrote:The only thing I don't fully understand is the IRQ autodetection in the 4th case.
Could you please describe this more deeply ? How can I send the IRQ ? To force the device I must know it before to know how to force it to send an IRQ, right ?
Let's start from the start. For PCI, several devices can be sharing the same IRQ line. This means that (even if you know exactly which IRQ line the devices use) you want to be able to tell multiple device drivers when the shared IRQ occurs, and you want those device drivers to tell the kernel if their device generated an IRQ or not (so that the kernel can optimise the order that it tells device drivers about IRQs, and detect problems like IRQ floods).
What would happen if the kernel didn't care much and just sent every IRQ to every device driver? This would add a lot of unnecessary overhead (lots of devices drivers checking their device and telling the kernel that the device didn't generate an IRQ), but it would work fine despite the unnecessary overhead. You only really want know which devices use which IRQs to reduce this unnecessary overhead. For example, if "IRQ 44" occurs and this IRQ is only shared by 2 PCI devices, then it'd be better to only tell those 2 device drivers that the IRQ occured (and avoid telling any other device drivers that the IRQ occurred).
Now imagine that there's 20 devices (and 20 device drivers) and the kernel doesn't know which IRQs any of the devices use. It could just send every IRQ to every device driver and that'd work. However, don't forget that the device drivers would still tell the kernel if their device generated an IRQ or not. For example; if "IRQ 44" occurs and the kernel tells every device driver that the IRQ occurred, and if all of the device drivers tell the kernel "this device didn't generated an IRQ" except for one of them, then it'd be obvious that the only device that generated an IRQ is using "IRQ 44".
What if there's 8 IRQs (that aren't used for legacy/ISA devices); and 3 of those IRQs occur at about the same time and 3 different device drivers say "this device generated an IRQ"? In this case you don't know exactly which IRQ each of those 3 devices are using; but you do know that those devices aren't using any of the 5 IRQs that didn't occur.
Basically; initially the kernel might not have any idea which PCI devices are using which IRQ (each device could be using any IRQ), but as IRQs occur the kernel can "learn" (reduce the number of IRQs that each device could be using), and eventually the kernel would figure out exactly which devices use which IRQs.
This makes it sound easy. Unfortunately it's not as easy as it sounds - it's very racey, and you'd need to be very careful to avoid problems caused by "unfortunate timing". For example, the kernel might tell a driver that an IRQ occurred and the driver might tell the kernel "this device generated an IRQ", even though that device's IRQ hasn't actually been sent from the IO APIC to the CPU yet (and a different device caused the first IRQ). You'd need something "more clever".
The "more clever" approach might go something like this:
- When an IRQ occurs check if there are devices that definitely do use that IRQ, and tell them about the IRQ first. If any of them reply with "this device caused an IRQ" then send EOI to the IO APIC and you're done.
- Otherwise:
- increase an "unknown IRQs in progress" counter
- set a flag corresponding to the IRQ number
- tell all device drivers that could be using the IRQ that the IRQ occured
- for each device driver that could be using the IRQ that replies with "this device caused an IRQ", set a flag corresponding to the driver
- when all device drivers that could be using the IRQ have replied:
- decrease the "unknown IRQs in progress" counter
- if the counter has become zero, wait a little while with interrupts enabled
- if the small delay expires (and no IRQs are received immediately after the "unknown IRQs in progress" counter became zero):
- create a mask from the flags corresponding to the IRQs that were received
- for each driver that has its "device caused an IRQ" flag set; AND the set of IRQs it could be using with your mask (hopefully reducing the set of possible IRQs that the device could be using)
- clear the flag corresponding to the IRQ numbers ready for the next group of IRQs
The basic idea here is to wait until there's no IRQs and then process all the results from the "previously received" IRQs; to make sure you don't get messed up by an IRQ that you haven't received yet. This should be immune to any race conditions.
Note: I am NOT saying this is the only way or that this is the best or most efficient method (it's only an example).
Also note that if a device never generates any IRQ you will never know which IRQ that device uses; but this doesn't actually matter.
The next step is to find a way to speed this up, so that the kernel can get rid of the extra "I don't know which devices actually do use the IRQ" overhead faster. This is where the idea of forcing the device to generate an IRQs comes from. For example, during boot the OS could start device drivers one at a time, and (if the driver supports it) ask the driver to force the device to generate IRQs until the kernel "learns" which IRQ the device does use (and then do the same for the next device driver, and the next, etc). In this way (for drivers that support "IRQ forcing") the kernel could figure out which devices use which IRQs during boot when other devices aren't generating unrelated IRQs and making it take longer.
How to force the device to send an interrupt depends on what sort of device it is. For some ethernet cards there's a "loopback" mode (where it receives packets that it sends; and you can configure it to generate an interrupt whenever it receives a minimal packet that you sent). For IDE/ATA/SATA (and probably SCSI) controllers you might use the "identify device" command. For a video card you might just enable an "interrupt on vertical refresh" feature. For a sound card you might setup some sort of "interrupt on buffer full" and configure it (with a tiny buffer) to record data from "microphone in".
Of course for some devices there isn't any way to force it to generate an IRQ; so you can't use this to detect all IRQs during boot and the kernel have to figure out which IRQs these devices use while the OS is running (and may never know if the device never generates an IRQ).
Finally, I should point out that regardless of how well you do this there are some assumptions and some risks. For example, if the motherboard uses "level triggered active high" for PCI (instead of "active low") then it's going to cause major problems (however this would be very unusual). You'd also have to worry about IO APIC inputs that aren't connected to anything at all (but happen to be tied low). This means that it's important for the kernel to be able to detect IRQ floods; which brings me back to the first paragraph I wrote - "
you want those device drivers to tell the kernel if their device generated an IRQ or not (so that the kernel can optimise the order that it tells device drivers about IRQs, and detect problems like IRQ floods)".
Cheers,
Brendan
