OSDev.org

Hello,

It is a bit silly question, but normally my IRQ handler sends EOI to PICs a few hundred instructions before iret. But isn't that too long? What happens if another interrupt happens after EOI but before iret? As interrupt requests are disabled by CPU when an interrupt occurs, won't interrupts after EOI and before iret be lost?

So where should I send EOI? Do I have to send EOI just before iret or can I send it wherever I want?

Thanks in advance.

Agola wrote:Hello,

It is a bit silly question, but normally my IRQ handler sends EOI to PICs a few hundred instructions before iret. But is that correct? What happens if another interrupt happens after EOI but before iret? As interrupt requests are disabled by CPU when an interrupt occurs, won't interrupts after EOI and before iret be lost?

So when I should send EOI? Do I have to send EOI just before iret or can I send it wherever I want?

Thanks in advance.

The EOI isn't about interrupts as seen by the CPU, it's about IRQs specifically. So for example, you do NOT send EOI in response to a page fault - you send it only for IRQs e.g. the keyboard IRQ.

When you send it depends on the kind of interrupt. For normal ISA "edge-triggered" interrupts, you can send it pretty much when you want - it just tells the PIC that you are ready to receive more IRQs.

If it's a level-triggered interrupt (like PCI interrupt usually are), then it has to go like this:

1. You receive an interrupt.
2. You detect the cause of the interrupt and stop it from arriving (e.g. you clear some "interrupt status" register of a device; it's all in the spec for the specific device you're working with).
3. You send EOI.

For level-triggered interrupts, if you send EOI before the interrupt is cleared (step 2), then it will continue occuring indefinitely and you'll get an IRQ flood.

I hope that helps, if you need me to clarify any of this just ask.

mariuszp wrote:

Agola wrote:Hello,

It is a bit silly question, but normally my IRQ handler sends EOI to PICs a few hundred instructions before iret. But is that correct? What happens if another interrupt happens after EOI but before iret? As interrupt requests are disabled by CPU when an interrupt occurs, won't interrupts after EOI and before iret be lost?

So when I should send EOI? Do I have to send EOI just before iret or can I send it wherever I want?

Thanks in advance.

The EOI isn't about interrupts as seen by the CPU, it's about IRQs specifically. So for example, you do NOT send EOI in response to a page fault - you send it only for IRQs e.g. the keyboard IRQ.

When you send it depends on the kind of interrupt. For normal ISA "edge-triggered" interrupts, you can send it pretty much when you want - it just tells the PIC that you are ready to receive more IRQs.

If it's a level-triggered interrupt (like PCI interrupt usually are), then it has to go like this:

1. You receive an interrupt.
2. You detect the cause of the interrupt and stop it from arriving (e.g. you clear some "interrupt status" register of a device; it's all in the spec for the specific device you're working with).
3. You send EOI.

For level-triggered interrupts, if you send EOI before the interrupt is cleared (step 2), then it will continue occuring indefinitely and you'll get an IRQ flood.

I hope that helps, if you need me to clarify any of this just ask.

Ah, my bad. This is not actually what I asked, because the title is wrong, it should be actually "Where should I send EOI in IRQ handler".

You send the EOI once you're finished handling the interrupt. Usually this is before the iret, but it can be after (although this is very complicated and I can't think of any reason why you would want to do this). The CPU won't receive any more interrupts (interrupts remain disabled) until after the iret, no matter when you send the EOI (of course, you might receive another interrupt immediately after the iret).

The definition of "finished handling the interrupt" can be a bit fuzzy at times depending on how complex your kernel/operating system is. For a simple kernel handling a keyboard interrupt, you would probably read the key from the keyboard controller and put it in the keyboard buffer straight away, then send the EOI and return from the ISR. For a more complex kernel with separate drivers, it's up to the driver to handle the interrupt and decide when it's done. In a multitasking or multicore system you should aim to keep your interrupt handlers as short as possible and use idle CPU time for executing driver tasks, so you might just grab the relevant data during the interrupt handler and send it off to the driver before returning, while the driver can process the data at a later time.

The important part is that EOI tells the PIC that you've finished with that interrupt and you want to receive the next one. The PIC won't send any more interrupts until you've handled the current one, and the EOI is your way of telling the PIC that you've handled it. The PIC doesn't care if the CPU actually handles interrupts straight away, queues up the data for later, or leaves interrupts disabled and never processes any interrupts at all. The PIC can't force the CPU to handle another interrupt before the CPU is ready. The only purpose of the EOI is because the PIC can only send interrupts to the CPU one at a time, and it needs to know when the CPU/kernel is ready for the next one.

I expect the following behaviour. Please correct me, if I am wrong.

If you sent the EOI to the PIC and the PIC gets an IRQ before you executed your iret,
then the PIC will set the interrupt line to active (either low or high, don't know that exactly) to the CPU to request an interrupt.
The CPU checks the interrupt flag is clear and does nothing, so the interrupt line still remains active.
As soon as the execution of your iret finishes and the interrupt flag is set again, the CPU notices the interrupt line.
It now gets the data from the PIC, for example which interrupt number etc. and calls your interrupt again.
The PIC will hold the interrupt line active, until you send the EOI.
If you do not send the EOI, then, after the iret, the CPU calls your handler immediately again, because the interrupt line is still active.

The interrupt line is the copper line from one pin of the PIC to one pin of the CPU. I just can't remember the right word for that. *shame*

sebihepp wrote:If you sent the EOI to the PIC and the PIC gets an IRQ before you executed your iret,
then the PIC will set the interrupt line to active (either low or high, don't know that exactly) to the CPU to request an interrupt.

As another noob on these matters, your statement made me curious, about a possible race condition between the arrival of new edge-triggered interrupt and the signaling of the previous EOI. In particular, the new interrupt may be EOI-ed unintentionally as a side effect from acknowledging the previous one (and thus have its handling postponed indefinitely). And lo and behold, the issue is real.

What is further interesting, is that since the kernel usually is designed to issue EOI after some handler confirms the interrupt, the above race also forces it to call the handlers a second time just to cater for new arrivals. This may not be very clean from the point of view of the handler, since it appears to be called redundantly. So what AIX does as per the article I linked above, is allow the handler to EOI by itself (using an API call), and thus is also responsible for checking for work from the device afterwards.

I'm assuming a typical setup, given that the PIC can be configured in a few different ways. I'm also working off of memory, so your best bet is to look at the spec itself.

sebihepp wrote:I expect the following behaviour. Please correct me, if I am wrong.

If you sent the EOI to the PIC and the PIC gets an IRQ before you executed your iret,
then the PIC will set the interrupt line to active (either low or high, don't know that exactly) to the CPU to request an interrupt.
The CPU checks the interrupt flag is clear and does nothing, so the interrupt line still remains active.
As soon as the execution of your iret finishes and the interrupt flag is set again, the CPU notices the interrupt line.
It now gets the data from the PIC, for example which interrupt number etc. and calls your interrupt again.

I think up to here you are correct, however..

sebihepp wrote: The PIC will hold the interrupt line active, until you send the EOI.

The PIC will hold the interrupt line active until the CPU (not you) acknowledges the interrupt. However the PIC won't send _new_ (more) interrupts until EOI, unless the new interrupts are of a higher priority. Whether or not the CPU acts on said interrupts (by jumping to IDT/IVT designated locations) is dependent on the CPU's IF flag.

sebihepp wrote: If you do not send the EOI, then, after the iret, the CPU calls your handler immediately again, because the interrupt line is still active.

If you don't call EOI then you should not receive further interrupts of the same or lower (numerically higher) priority.

However if you don't deal with the origin of the interrupt (the device that signaled PIC for an interrupt) then that device will still assert interrupt to the PIC and depending on whether it's edge/level triggered, if you do send an EOI you may get another interrupt immediately because you've told the PIC it's taken care of but the device still asserts the interrupt so the PIC thinks it's a new interrupt.

simeonz wrote:

sebihepp wrote:If you sent the EOI to the PIC and the PIC gets an IRQ before you executed your iret,
then the PIC will set the interrupt line to active (either low or high, don't know that exactly) to the CPU to request an interrupt.

As another noob on these matters, your statement made me curious, about a possible race condition between the arrival of new edge-triggered interrupt and the signaling of the previous EOI. In particular, the new interrupt may be EOI-ed unintentionally as a side effect from acknowledging the previous one (and thus have its handling postponed indefinitely). And lo and behold, the issue is real.

I'm not sure if there's a race condition as you describe. As with the above reply, this is also from memory.

Once the interrupt is sent to the CPU and the CPU acknowledges it, then the ISR (In-Service Register) is set and the IRR (Interrupt Request Register) is cleared and the latch is armed. So as I understand it, at this point the PIC should be ready to receive a new interrupt at the same priority level (into the IRR). Once you EOI the first interrupt that should clear the ISR, and at this point the new interrupt waiting in IRR should be the highest and you should immediately get a new interrupt to the CPU.

I don't remember if the spec specifically mentions the case of ISR and IRR having the same bit (priority level = IRQ number) set, but the above would be my expectation.

Did I understand your race condition? As for the link you provided, I didn't go too deeply into it, so I'm not sure if that's AIX specific or if there's really a race condition..

simeonz wrote: What is further interesting, is that since the kernel usually is designed to issue EOI after some handler confirms the interrupt, the above race also forces it to call the handlers a second time just to cater for new arrivals. This may not be very clean from the point of view of the handler, since it appears to be called redundantly. So what AIX does as per the article I linked above, is allow the handler to EOI by itself (using an API call), and thus is also responsible for checking for work from the device afterwards.

I probably wouldn't make it part of the general interrupt handling procedure to call interrupt handlers a second time, rather I'd make it the responsibility of the drivers to handle the situation. For instance for many devices the race probably couldn't happen and the driver is the only one who can realistically do that. Also once the driver confirms the interrupt (causing EOI to be sent) the driver can poll the device once more, if relevant to that specific piece of hardware.


One thing I never really got was the spurious interrupts in the spec, I understand the explanation, which was something along the lines of:
- Interrupt from device to PIC is asserted
- PIC IRR is set
- PIC sends interrupt to CPU
- Interrupt from device to PIC is DE-asserted
- CPU ack's
- PIC no longer knows which interrupt number to send to CPU == send "something", AKA the lowest priority level (highest numerically)

All of the above makes sense, except the fact that the PIC does have to ability to know which priority level (IRQ#) caused the PIC itself to "send" interrupt to CPU and should still have it in the IRR. Or is the IRR really not a register after all? If anyone could clarify that, I'd appreciate it.

Maybe I should find the time to re-read the spec in case there's something there...

LtG wrote:Did I understand your race condition? As for the link you provided, I didn't go too deeply into it, so I'm not sure if that's AIX specific or if there's really a race condition..

Yes, you interpreted my post correctly, and I was speaking in haste, without having even read the specification. I was fishing for responses or corroborative silence, since this kind of race should be a rather hot topic. Besides, I have not heard anywhere of say, Linux dealing with it in a generalized fashion. That said, after reading the relevant parts of the short 8259A spec, I find your expectations sensible, but may not be justified.

LtG wrote:All of the above makes sense, except the fact that the PIC does have to ability to know which priority level (IRQ#) caused the PIC itself to "send" interrupt to CPU and should still have it in the IRR. Or is the IRR really not a register after all? If anyone could clarify that, I'd appreciate it.

Maybe I should find the time to re-read the spec in case there's something there...

The good news is that the spec explicitly states that IRR is a register here:

Interrupt Request Register (IRR):
8-bit register which contains the levels requesting an interrupt to be acknowledged. The highest request level is reset from the IRR when an interrupt is acknowledged. (Not affected by IMR.)

However, there is a rather strange twist further down in the "Edge and Level Triggered Modes" section:

In both the edge and level triggered modes the IR inputs must remain high until after the falling edge of the first INTA. If the IR input goes low before this time a DEFAULT IR7 will occur when the CPU acknowledges the interrupt. This can be a useful safeguard for detecting interrupts caused by spurious noise glitches on the IR inputs.

To me, this section has rather unclear implications. One consequence, if I interpret it correctly, is that if the pulse of an edge triggered interrupt falls while the CPU is still processing the previous interrupt, the trigger will result in setting bit 7 of the IRR instead of the corresponding bit for that interrupt line. What's more, they talk about the first INTA here, but INTA wont be initiated by the CPU before the PIC asserts the CPU's INT line. And this will happen only after the EOI clears the highest bit in the ISR. But I am currently under the impression (although I have to confirm somewhere - with the x86 manuals probably) that INTA will not be sent at all while the IF bit is cleared, thus until it is set with sti or iret. Something along these lines is suggested by this comment in the 8259A spec:

While the IS bit is set, all further interrupts of the same or lower priority are inhibited, while higher levels will generate an interrupt (which will be acknowledged only if the microprocessor internal Interupt enable flip-flop has been re-enabled through software).

If this is the case, the race would be even nastier, not to mention that prioritization would be essentially useless in edge triggered mode. Alternatively, it could be that the specification means that the pulse of the next edge-triggered interrupt line must not reset before the previous interrupt is acknowledged by the CPU. This would make more sense.

I wanted to post a few more "findings" of dubious quality. If anyone can make a quick clarification about the mandated behavior for edge-triggered interrupts that arise concurrently to the processing of the current in-service interrupt, I ask you to come forward.

I stand by my previous post for the time being, with the slight correction that spurious interrupts do not set bit 7 of the IRR, but are rather recorded somewhere out of band (for later delivery, if I interpret the spec correctly.) I tried to look into the APIC related specs that Intel has for comparison. They are not as deep regarding the signal timings, but the most detailed treatment is in the chipset datasheets. I need to say, I did not read them from end to end, but they copy almost verbatim the information about the legacy PIC circuitry, and have certain differences for the APIC mode. For example, this is the datasheet for ICH3-S. I just stumbled on it arbitrarily, but it has the same 8259 paragraph:

In both the edge and level triggered modes, the IRQ inputs must remain active until after the falling edge of the first internal INTA#. If the IRQ input goes inactive before this time, a default IRQ7 vector will be returned.

Here is what I consider the relevant APIC paragraph from that document ("EOI Message for Level Triggered Interrupts"):

EOI messages are used by local APICs to send an EOI cycle occurring for a level triggered interrupt to an I/O APIC. This message is needed so that the I/O APIC can differentiate between a new interrupt on the interrupt line versus the same interrupt on the interrupt line. The target of the EOI is given by the local APIC through the transmission of the priority vector (V7 through V0) of the interrupt. Upon receiving this message, the I/O APIC resets the Remote IRR bit for that interrupt. If the interrupt signal is still active after the IRR bit is reset, the I/O APIC will treat it as a new interrupt.

Note that according to this spec, the EOI directly affects the IRR on the IO APIC, and the possible condition of losing is stated explicitly. Here is the datasheet for the more recent PCH chipset. From randomly searching through it, here is another relevant snippet (in "REDIR_TBL—Redirection Table Register"..):

The APIC will respond to an edge triggered interrupt as long as the interrupt is held until after the acknowledge cycle has begun. Once the interrupt is detected, a delivery status bit internally to the I/O APIC is set. The state machine will step ahead and wait for an acknowledgment from the APIC unit that the interrupt message was sent. Only then will the I/O APIC be able to recognize a new edge on that interrupt pin. That new edge will only result in a new invocation of the handler if its acceptance by the destination APIC causes the Interrupt Request Register bit to go from 0 to 1. (In other words, if the interrupt was not already pending at the destination.)

Again, the circumstances of losing edge triggered interrupt are explicitly stated. Or so I think.

In other words, with 8259a, before the iret or sti, the edge-triggered interrupt becomes IR7 (because INTA will be possible much after the IR pulse reaches its falling edge). APIC (IO APIC proper) straight ditches the new interrupt before EOI, but at least records it between EOI and iret/sti. In my opinion, the situation on legacy PIC is more severe.

Edit: In retrospect, the ICH3-S paragraph about level-triggered interrupts may not be relevant, because they are quite different. Also, at my current knowledge, considering the matrix of chipsets and interrupt mechanics, my interpretations may be significantly off.

Edge-triggered interrupts should work properly with ISA devices and 8259a without much hassle. From this surviving scan of the ISA specification, there are two relevant excerpts:

6.3 INTERRUPT STRUCTURE
The interrupt lines of the connectors are directly connected to the Intel 8259A Interrupt Controller. The Interrupt Controller will react to the interrupt on a transition of low to high. There are no interrupt acknowledge lines on the ISA bus. The resource must use a memory or I/O access by the bus owner for an acknowledgement of the interrupt.

7.1.4 INTERRUPT SIGNAL GROUP
An interrupt may be requested by a platform resource or add-on card by enabling an IRQ line. The line must remain enabled until the interrupt is acknowledged by accessing the interrupting resource on the add-on card by the Primary CPU.

ISA conforming devices must hold the interrupt line asserted until the driver resets the condition by acknowledging the event to the device through PIO or MMIO. In fact, from the 8259a and ISA specs together, resetting the interrupt a second time in a single ISR should be exactly the kind of action that provokes IR7, because the second signal has not been maintained in the asserted state until the CPU could acknowledge it.

Lets consider the applicability historically and today. My impression is that for PC hardware, actual 8259a circuits should only exist in systems that support ISA channels. From what I have read, including older posts on this forum, it appears that PCI-circa systems would use an actual 8259a circuit and a PCI-to-ISA bridge. They would configure interrupt lines individually as level triggered for PCI and edge triggered for ISA, using additional configuration registers (ELCR). The acknowledgement interface would be mediated by the bridge, converted from the native PIC interface into a PCI transaction, which would deliver the interrupt vector to the CPU on the PCI bus. The edge-triggered ISA mechanism is electrically applicable in those configurations.

In newer systems, PCI uses level-triggered interrupts or message signaled interrupts. PCIe uses message signaled interrupts exclusively and emulates level-triggered interrupts through messages for the purposes of bridging to PCI. Message signaled interrupts are described as edge-triggered by Intel, but I don't know what relevance this could have. Finally, the LPC block uses serial (what I would call time-modulated) signaling to communicate ISA interrupts. So, although the Intel chipset datasheets (such the ones I posted in the previous message) state that they implement the equivalent of two cascaded 8259a, the only ISA devices that connect to those PICs are in the LPC block, and the 8259a signaling specification doesn't make much sense to me for the serial protocol used by the LPC SERIRQ line. I don't know if the interrupt request is converted to a native PIC interface, or the PIC implementation at that point is only conceptual.

I will conclude here. As I said, legacy 8259a should work correctly with ISA edge triggered interrupts and IR7 should manifest from either noise, faulty device, or faulty driver. And the interrupt condition of the device should not be reset a second time, because this is exactly the faulty driver behavior that will provoke IR7. I have nothing to say about the AIX fix. It relates to the PCIe message signaled interface, but I have not investigated that. I also have also skipped the APIC differences for the time being.

Corrections are welcome.

ARISTOS wrote:

Code: Select all

keaboar_int:
cli     ; <-stop external interrupts
...     ; code
EOI    ; says to APIC that the interrupt end, if the APIC reise another interrupt they will not rich the processor beacure the interrupt flag isn't set (cli)
iretd  ; return the interupt (also pops the EFLAGS from stack which the interrupt flag is set and only at the end of 'iretd' an external interrupt will handled!)

This doesn't make sense and is missing context.
Also, 'cli' should not be inside your interrupt handler, you should specify that in your IDT entry for that specific interrupt.