IO-APIC info

[Brendan]

Hi,

Just a quick question that I don't feel STFWing: if the interrupt gets sent to both processors, will they both start their ISR and is it up to the implementation to synchronize this?

I have it set to 'lowest priority' mode. In this mode, when multiple cpu's are allowed to respond to an interrupt, they choose the least used cpu at the time to take the interrupt. I'm still learning about all of this, so maybe that is wrong, but that is the idea that I have got from the specs.

Yes, but it's a little more complex than that...

The I/O APIC doesn't know which CPU is least used. An OS needs to use the "Task Priority Register" in the local APIC (which has nothing to do with multi-tasking) to tell the I/O APIC how important the work it's doing is. However, bits 4 to 7 of the TPR also block lower priority IRQs, which you probably don't want. To get around that I'd set the TPR to values between 0x00 and 0x0F to indicate how important the work it's doing is (e.g. 0x0F when it's running really important code, 0x00 when it's idle, etc), so that all CPUs are still willing to accept all IRQs.

Also, if a CPU is already handling an IRQ then bits 4 to 7 of the interrupt vector for that IRQ also effect the CPU's priority. For example, if a CPU's TPR = 0x00 and it's handling an IRQ that's mapped to interrupt 0x23, then the I/O APIC will assume the CPU is at "priority = 0x20". This means that IRQs tend to go to CPUs that aren't already handling an IRQ.

Finally, older CPUs have an (optional) feature called "focus checking". The idea here is that if a CPU is already handing an interrupt/IRQ and that same interrupt/IRQ occurs again, then the interrupt will be sent to the same CPU. This is probably a good thing, as it helps avoid cache misses, lock contention, etc.

Also note that for modern CPUs the TPR is aliased to "CR8" in long mode; and (at least for Intel CPUs) there's an MSR that can be used to disable "TPR messages" (where the CPU is prevented from telling the I/O APIC that it's TPR was changed).


Cheers,

Brendan

[01000101]

I have a new issue, and since it is very related, I'll skip creating an entirely new topic.

I redirected ISA IRQ 0 and 1 (timer and keyboard) to vectors 0x20 and 0x21. Those work flawlessly. I have even set it up so that CPU's alternate for each interrupt acknowledgement (I know it's not efficient, but it is for testing purposes).

The problem I'm having now, is that I cannot get PCI interrupts to reach the CPU. I realize that the MP tables describe PCI IO Redirections a bit differently, but I set checks for such and handle them by the book. I have PCI IRQ 0xA and 0xB (taken from the PCI configuration spaces), and I search through the IO Redirections for IRQ (x-0xA) and only through IO Redirections whom have the source ID as a PCI BUS. I have found the two IO Redirections that were parsed from the MP tables that I need. But when I map them in the IOAPIC, I get no interrupts from the PCI device (network cards).

Code: Select all

IOAPIC:
id: 0x04 | version: 0x20 | flags: 0x01 | address: 0xFEC00000
id: 0x05 | version: 0x20 | flags: 0x01 | address: 0xFEC80000

Buses:
id: 0x00 | string: PCI
id: 0x01 | string: PCI
id: 0x02 | string: PCI
id: 0x04 | string: PCI
id: 0x07 | string: PCI
id: 0x08 | string: ISA

PCI IO Redirections:
src_id: 0x01 | src_irq: 0x00 | dst_id: 0x04 | dst_irq: 0x10 | int_type: 0x00
src_id: 0x02 | src_irq: 0x00 | dst_id: 0x04 | dst_irq: 0x10 | int_type: 0x00
src_id: 0x02 | src_irq: 0x08 | dst_id: 0x04 | dst_irq: 0x12 | int_type: 0x00
src_id: 0x04 | src_irq: 0x00 | dst_id: 0x04 | dst_irq: 0x12 | int_type: 0x00
src_id: 0x04 | src_irq: 0x01 | dst_id: 0x04 | dst_irq: 0x13 | int_type: 0x00
src_id: 0x07 | src_irq: 0x04 | dst_id: 0x04 | dst_irq: 0x12 | int_type: 0x00

Installed Redirections:
INTIN(0x02) -> ISR0 (timer)
INTIN(0x01) -> ISR1 (keyboard)
INTIN(0x13) -> ISR2 (eth0)
INTIN(0x12) -> ISR3 (eth1)

NIC PCI Config Entries:
eth0: irq_line: 0x0B | irq_pin: 0x01 | bus: 0x04 | dev: 0x00 | func: 0x00
eth1: irq_line: 0x0A | irq_pin: 0x02 | bus: 0x04 | dev: 0x00 | func: 0x01

note: ISRx correlates to IDT[0x20 + x]

I excluded all the PCI bus 0 entries and non-PCI entries as they are not relevant here.

I have no idea why the interrupt does not happen. I have all the interrupts unmasked in the NICs themselves and know that they are configured property (as they work perfectly in polling mode). Any ideas? anything look wrong in the entries above?

[Brendan]

Hi,

I checked what I could, and it all looks mostly right.

This looked strange to me:
eth0: irq_line: 0x0B | irq_pin: 0x01 | bus: 0x04 | dev: 0x00 | func: 0x00
eth1: irq_line: 0x0A | irq_pin: 0x02 | bus: 0x04 | dev: 0x00 | func: 0x01

It's unusual for a multi-function device to use IRQ lines like that - normally function 0 uses IRQ line A, and function 1 uses IRQ line B.

I have no idea why the interrupt does not happen. I have all the interrupts unmasked in the NICs themselves and know that they are configured property (as they work perfectly in polling mode). Any ideas? anything look wrong in the entries above?

You know the NICS were configured properly for polling mode, but are you sure they're configured properly for interrupt mode now?

Also, there should be information (in the I/O redirection entry in the MP tables) on what sort of signal each device uses (edge/level trigger, active low/high). The MP tables might say "level triggered active low", but they probably say "conforms to the specifications for the bus" instead (in that case, for PCI devices, you'd need to program the I/O APIC input as "level triggered active low"). Are you sure you've got this programmed in the I/O APIC right?

Also, I noticed that you've got 3 different devices sharing I/O APIC input 0x12. One is eth1. The other 2 devices are "PCI bus 7, device 1" and "PCI bus 2, device 2". If either of these other devices generates an IRQ then you'd need to handle their IRQ or mask the I/O APIC input (and stop eth1 from getting its IRQ). You can't just check if the IRQ came from eth0 and do EOI/iret if it didn't (that will cause an IRQ flood). This shouldn't happen though (normally a device won't send an IRQ unless you do something to ask for it).

I can't think of anything else...


Cheers,

Brendan

[01000101]

I tested the NICs using the PIC and it works fine. I was able to recieve, eoi the card, and eoi the PIC.

You were correct (I think) about the level assert with active low. Now I get interrupts... a whole flood of them. I think it has to do with the local APIC not sending the ACK to the IOAPIC, but I'm not sure how to proceed towards handling that. Thus far, I have only programmed the CPU logical ID's into the local APIC's and nothing more. Is it normal for the default config of the APIC's to do this, or should I be looking at a different issue?

I have it setup so that:
ISR0 = timer
ISR1 = keyboard
ISR2 = eth0
ISR3 = eth1
ISR5 = everything else.

[Brendan]

Hi,

You were correct (I think) about the level assert with active low. Now I get interrupts... a whole flood of them.

Cool! Too many IRQs is more fun than no IRQs...

I think it has to do with the local APIC not sending the ACK to the IOAPIC, but I'm not sure how to proceed towards handling that.

If you don't send an EOI then you don't get the IRQ again. If you do send an EOI and the device that's causing the IRQ doesn't realise you've handled the IRQ then it's interrupt line will still be "active" and the I/O APIC will just send you another IRQ.

Basically, I'm guessing you send the EOI without telling the device you've handled it's IRQ. I'm not sure about your devices (or which IRQs are flooding) but usually there's some sort of status register that you need to read and/or write to clear the condition that the device is telling you about.

Note: Edge triggered interrupts and ISA devices work differently...

ISR5 = everything else.

Everything else should be masked in the I/O APIC (unless/until you've got IRQ handlers for each device)...


Cheers,

Brendan