OSDev.org

Hello again
I've finally got some time to work on my OS again and I've decided to rewrite my code for IRQ handling to gain some benefits from the I/O APIC. I'm currently using some very basic I/O APIC functions to handle the legacy ISA interrupts and that part is working properly, but while I'm rewriting the code I want to add support for PCI interrupts too. Even though I've read the I/O APIC specification several times, I still don't understand how to manage the PCI interrupts. I already have code for handling PCI devices and I can read the Interrupt Pin and Interrupt Line from the PCI header. I know that I need to change the redir entries (like I do with the ISA interrupts), but not exactly how I should do it.

I know this topic has been discussed several times before and I've read the previous posts, but I still don't get it. Hopefully somebody can clarify this for me

Hi,

zity wrote:I've finally got some time to work on my OS again and I've decided to rewrite my code for IRQ handling to gain some benefits from the I/O APIC. I'm currently using some very basic I/O APIC functions to handle the legacy ISA interrupts and that part is working properly, but while I'm rewriting the code I want to add support for PCI interrupts too. Even though I've read the I/O APIC specification several times, I still don't understand how to manage the PCI interrupts. I already have code for handling PCI devices and I can read the Interrupt Pin and Interrupt Line from the PCI header. I know that I need to change the redir entries (like I do with the ISA interrupts), but not exactly how I should do it.

I'm confused.

How can you get the legacy/ISA IRQs working properly without reading the "IRQ redirection entries" in the MP specification table and/or the ACPI "APIC/MADT" table; and how can you read about the "IRQ redirection entries" without noticing that some of the entries are for PCI devices?

My guess is that you haven't read about the "IRQ redirection entries" in either (MP or ACPI) specification, and therefore you probably don't have the legacy/ISA IRQs working properly. For example, maybe some of legacy/ISA IRQs (the ones you currently use) work by accident on the computers that you tested it on.


Cheers,

Brendan

Hi Brendan

I'm pretty sure my ISA interrupts aren't working by coincidence. I parse the MADT table to get the Interrupts Source Overrides, e.g. that the PIT is connected to the I/O APIC input 2 (active high, egde-triggered). I use this info to initialize the proper entries in the I/O APICs redirection table (IOREDTBL0-23).

82093AA IOAPIC Specification wrote: Interrupt inputs 16 through 19: These signals are connected to the redirection table entries [16:19]. Typically, these signals are connected to the PCI interrupts (PIRQ[0:3]). The steering of the PCI IRQs to the ISA IRQs is accomplished in the IOAPIC by setting the PCI redirection table entry to the correct ISA interrupt vector.

The quote above is from my I/O APIC specification. I know that I've 4 PCI interrupts connected to the I/O APIC, but I have a lot more than 4 PCI devices. Does this mean that I have to share these 4 interrupts between all the PCI devices, or? and where do I find the mentioned "PCI redirection table"?

I'm sorry I can't explain it better, but I'm really confused about this..

zity wrote: I know that I've 4 PCI interrupts connected to the I/O APIC, but I have a lot more than 4 PCI devices. Does this mean that I have to share these 4 interrupts between all the PCI devices, or? and where do I find the mentioned "PCI redirection table"?

PCI has four interrupt lines, which are shared by all devices in a rotating manner (so if you have four one-function devices, they'll all use different ones, but if you have more or two-function devices then you'll have overlaps and will have to work out which one has sent the interrupt).

Also, many PCI devices support 'message-signalled' interrupts, which completely bypass the IOAPIC and instead send whatever interrupt you want directly to one core. Naturally, there are various advantages and disadvantages to this, but remember that it's the *only* way used by PCIe devices, which means all relatively new graphics cards.

(edit: fixed a thinko)

All right, I think I'm beginning to understand how it works now.

Every PCI device has an Interrupt Pin value and an Interrupt Line value. If the value of both Interrupt Pin and Interrupt Line is zero, the device doesn't use any IRQs. Interrupt Pin value 1,2,3,4 is normally called A,B,C,D which generates signal 16-19 in the I/O APIC, right?

Devices that support Message Signaled Interrupts, should of course not send interrupts though the I/O APIC, but directly to the CPU (I'll look into MSI support later).

This leaves me with two questions. What is the Interrupt Line value used for? and how do I determine which device sent the interrupt on one of the four lines?

Oh btw, can I assume that the PCI interrupts always are connected to the I/O APICs line 16-19?

Hi,

Selenic wrote:PCI has four interrupt lines, which are shared by all devices in a rotating manner (so if you have four one-function devices, they'll all use different ones, but if you have more or two-function devices then you'll have overlaps and will have to work out which one has sent the interrupt).

It's a little more complex than that - each PCI host controller has 4 interrupt lines (but some systems have more than one PCI host controller); and in some cases devices built into the chipset may be connected to the I/O APIC using "extra" interrupt line/s (which I think is a violation of the PCI spec, but it does happen). On top of that there's PCI devices (e.g. disk controllers) that are emulating legacy devices (and therefore, these PCI devices use ISA IRQ lines not PCI IRQ lines), and MSI.

zity wrote:Every PCI device has an Interrupt Pin value and an Interrupt Line value. If the value of both Interrupt Pin and Interrupt Line is zero, the device doesn't use any IRQs. Interrupt Pin value 1,2,3,4 is normally called A,B,C,D which generates signal 16-19 in the I/O APIC, right?

The "Interrupt line" field in PCI configuration space says which ISA IRQ the interrupt is routed to when you're using the PIC chips (it's useless when you're using I/O APICs).

The MP Specification tables include an "I/O Interrupt Assignment Entry" for each PCI device that can generate an IRQ - e.g. "bus0:device1:function2 is connected I/O APIC input #123". This is fairly easy to use, but it can't handle things that have changed after the firmware set it up (e.g. hotplug PCI).

For ACPI there's "Interrupt Source Overrides" in the MADT which tell you how legacy/ISA IRQs are mapped to I/O APIC inputs. Unlike the MP Specification tables there are no entries describing PCI interrupts. For PCI interrupts you need to use an AML interpretter to decode/execute the AML code (for e.g. the "_PRT" object). This is a massive pain in the neck, but can handle things that change after firmware sets them up.

However, for ACPI the entire AML thing is an ugly mess that takes a long time to understand properly and even longer to implement properly; and if/when your code is perfect it's vulnerable to bugs in the firmware's AML. Also, the same AML can behave differently depending on which "OS name" you tell it; so AML that works perfectly when it's told that the OS is "Windows NT" may be buggy when it's told the OS is something else.

For ISA IRQs it's easy (the MP and/or APCI tables should work). For newer devices that support MSI it's easy (you don't need to care what MP or ACPI tables say if you're using MSI). For older PCI devices you have 4 choices:

• Use the older MP tables for PCI IRQ information (even when ACPI is present)
• Use ACPI AML and learn to cope with an overcomplicated mess
• Implement some sort of IRQ auto-detection feature (e.g. have an IRQ handler for every possible I/O APIC interrupt, and do "IRQ detection" for one device at a time; so that when you get an IRQ that isn't handled by any "already configured" device driver you know it belongs to the device currently doing "IRQ detection").
• Have a motherboard/chipset driver (one for each motherboard/chipset) that can tell you which PCI devices are connected to which I/O APIC inputs.

zity wrote:Oh btw, can I assume that the PCI interrupts always are connected to the I/O APICs line 16-19?

Definitely not. Any ISA IRQ and any PCI IRQ can be connected to any I/O APIC input. Most motherboards tend to connect most ISA IRQs to I/O APIC inputs 0 to 15, but nothing says this must be the case.

For example, nothing prevents I/O APIC inputs 7, 11, 13, 17, 19, 23, 29 and 31 from being used for PCI IRQs (with 2 different PCI host controllers); while I/O APIC inputs 30, 28, 26, 24, 22, 20, 18, 16, 14, 12, 10, 8, 6, 4, 2 are used for ISA IRQs 0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 (in that "upside-down" order). This very messed up example is perfectly valid (as long as there's appropriate MP table entries and ACPI "Interrupt Source Overrides").


Cheers,

Brendan

There is Intel's ACPICA reference ACPI implementation, which, from what I've read, looks reasonably straightforward to implement. Incidentally, I believe it is also the implementation used by Linux and most BSDs (And perhaps even Solaris?)

I know about the ACPI AML feature and I know that I don't want to waste my time implementing an interpreter, I really don't want to use AML at all, so I try to avoid ACPICA too.

I've been looking at the MP specifications and implemented a simple parser to get the information from the I/O Interrupts Entries, but I find it a little strange that there isn't a field that specifies whether an entry represents an ISA or PCI interrupt.

I like the idea of an IRQ detection routine. I would be fairly easy for drivers to call a function in my irq module, telling it to expect a unkown irq, and then the driver forces the device to send an interrupt.
But my question still remains; if two devices share the same interrupt, how am I able to distinguish interrupt signals from each other? The driver could tell the irq module that it is waiting for an interrupt, but both drivers could do that simultaneously.

Devices that can share IRQs always implement a status register telling if an interrupt has happened. When an interrupt line has been asserted, the system should call each interrupt handler that is registered for the line, and each will check the device connected to the interrupt line and handle the interrupt if it has originiated from that device. If a device can't report if an interrupt has occurred, it should not share an interrupt line with another device.

Thanks for all the help so far, I'm getting a little closer every day

I decided to continue working with the MP Structure to see what information I could get from it. I'm currently able to find the redirection entries for the PCI devices, but I must admit that the information looks a little odd to me. I've extracted the information below from the MP Structure:
Offhand, the redirection entries look all right, but according to the table IRQ pin A is connected to I/O APIC input 16, 17 22, and 23 and IRQ pin B is connected to I/O APIC pin 17, 19, and 21. Obviously this cannot be true, unless I missed something important.

I have extracted similar information directly from the PCI devices.
This information is nearly consistent with the information from the MP Structure, except from device 31. I guess the IRQ pin numbers are correct, but I'm confused about their connection to the I/O APIC.

That's a valid configuration. Each device has its own line A, B, C and D which could be connected to different system interrupt lines. Lines from different devices could also be connected to the same system interrupt.

Oh, I see. I thought that pin A-D was the same for all PCI devices, but if every device has its own set of pins A-D, the configuration makes a lot more sense I think I've got all the information I need to start coding now.

Thanks a lot for all the helpful information!

I would say that it´s wrong that every device has its own set of interrupt lines (but I can´t prove it with the specs, which I don´t have at the moment), because why have you to care in which slot a pci card has to be so that it doesn´t give problems. As I remember every pci bus has its own set of interrupt lines, but all devices on the same bus have to share the interrupt lines (if it would be as you saying it, why would we need to share interrupts)!

The specification doesn't require that all devices share the same 4 interrupt lines, but it's often the case. A common configuration is where the pins A-D are rotated with increasing device numbers. But there could also be a system where each pin on each device is connected to a different system interrupt.

The configuration is from a laptop, which might have something to do with the way the interrupt lines are assigned. Unfortunately it's the only computer I have right now, so I'm unable to check configurations on other machines at the moment.