OSDev.org

it's the responsibility of the interrupt controller(s) to not trigger the same irq simultaneously on different cpus, at least global, hardware interrupts.

The act of writing to the EOI register causes the local APIC to delete the interrupt from its ISR queue and (for level-triggered interrupts) send a message on the bus indicating that the interrupt handling has been completed.

Upon acceptance of an interrupt into the IRR, the corresponding TMR [Trigger Mode Register] bit is cleared for edge-triggered interrupts and set for level-triggered interrupts. If a TMR bit is set when an EOI cycle for its corresponding interrupt vector is generated, an EOI message is sent to all I/O APICs.

Darwish wrote:So in the time between we ACK a hardware device and send an EOI, how does the IOAPIC know that this edge-triggered handler is still in execution, and avoids forwarding the device next interrupt to another core?

Brendan wrote: As I understand things (and not necessarily how things actually are)...

If a second edge-triggered IRQ occurs (while the first is being handled), then the I/O APIC would send the IRQ to the local APIC/s. If the local APIC is already handling the same IRQ then it'll set the corresponding IRR bit, and if the IRR bit is set when you do the EOI for the first IRQ then the local APIC will send the second IRQ to the CPU (and you'd get 2 IRQs, one after the other).

For "send to lowest priority" the actual behaviour depends on a few things. P6 and older CPU's support the "focus processor" feature. If this is enabled, then the second IRQ will be sent to the same CPU as the first CPU (even if that CPU isn't the lowest priority CPU anymore), and you won't need to worry about re-entrancy (you'd get 2 IRQs on the same CPU, one after the other). Also, once the IRR is set then setting it again won't make any difference, which means that a third (or fourth, fifth, etc) IRQ will be ignored by the local APIC. To be honest, I like the "focus processor" idea (mostly because the IRQ handler and it's data will still be in the CPUs cache, and there's less chance of lock contention), so I'd recommend using this feature if it's present.

For newer CPUs (and older CPUs with the "focus processor" feature disabled) it's likely that the CPU that's handling the first IRQ will not be the lowest priority CPU anymore, and therefore it's likely that the second IRQ will be handled by a different CPU.

Cheers,

Darwish wrote:For the sake of avoiding useless locking complexities, I guess I'll setup the legacy edge-triggered ISA IOAPIC routing entries with 'Fixed' delivery mode to the bootstrap core. AFAIK, they're already very slow devices, and won't bias irq handling balance very much. Any further suggestion?

IRQ_count:  dd 0xFFFFFFFF

IRQ_handler:
    lock add dword [IRQ_count],1
    je .handle_IRQ
    iretd

.handle_IRQ:
    * stuff *
    call send_EOI
    lock sub dword [IRQ_count],1
    jnc .handle_IRQ
    iretd

FlashBurn wrote:Unfortunately I can´t test my code on a real smp machine at the moment. So the only result I have so far is, that it works on bochs if it has 8 cpus and it doesn´t work on qemu (I test till 12 cpus). Doesn´t work means my pit handler code is called only once. As it works with bochs and 8 cpus, I would say it´s not a problem with my io-apic code. So what could it be?

• * Don't depend on the BIOS setting up the PIT for you
  * Make sure you've programmed it correctly to monotonic mode (mode 3)
  * Remember that in mode 3, the OUT-2 pin actually starts high and stands high half the programmed period, go low the other half, then return high. Thus, it's necessary to setup it as edge-triggered
  * Test your IRQs setup with the PICs first since it's hard to program the PIC wrongly
  * Make sure you've read where the PIT is connected to the IOAPICs correctly from either MP or ACPI tables
  * Make sure you enable the local APIC using both, the apic_enable bit in the spurious interrupt vector, and using the APICBASE_ENABLE MSR
  * Just in case, set the destination core local APIC 'Task Priority Register' (TPR) to zero

vector = PIT_VECTOR;
delivery_mode = FIXED;
destination_mode = PHYSICAL;
polarity = HIGH;
trigger = EDGE_TRIGGERED;
mask = UNMASK;
destination = bootstrap core APIC ID

FlashBurn wrote:OK, I setup the PIT myself (the same way I´m setting it up for using in my bootloader), so it should be programmed right (ICW= 0x34). The local APIC is enable, because I use it for my scheduler it this is working.

(the thing is, that it works with Bochs and 8 cpus should mean it works under some conditions).

Darwish wrote: (Also try to organize your thoughts a bit before posting)

Darwish wrote: Are you sure the PIT is setup monotonically? You mention the PIT timer, then jump talking about the PIC's ICW register.