Re-firing existing IRQs

[JohnBurger]

I am writing an x86 kernel, and am trying to handle exceptions in Fault conditions.

Not "correctly" (necessarily), but certainly "nicely". My current problem is that when I get a Stack Fault (for example), it may be in the middle of an exception handler - but this is deliberate. Let me explain:

I am exploring the capabilities of the CPU, and how to handle various "exceptional" conditions such as Low Stack conditions. For example: what happens if an INT 12 (Stack Fault) happens during a Supervisor (Ring0) Task - and that Stack Fault was during an IRQ handler?

The scenario goes something like this:
* The (Ring0) Stack is (deliberately) nearly exhausted;
* A Timer interrupt (for example) occurs;
* The CPU pushes EFlags, CS and EIP - and now the Stack is literally FULL!
* The Timer handler, first thing, does a PUSH EAX - which underflows the Stack;
* The CPU starts the Stack Fault exception handler - which is in its own TSS, with its own Ring0 Stack.

At this point, the Timer interrupt is un-acknowledged (even in the PIC), so all interrupts are effectively disabled - the Timer's maximum priority will mask every other interrupt until it is acknowledged. Meanwhile, the CPU isn't ready to do that. True: if the Stack Fault exception handler organises a bigger stack for the underflowed Task, then an IRET should help the situation to acknowledge the interrupt - but what about until then? Is there a simple thing that I can do (probably to the PIC(s)) that will re-fire the interrupt so that the new Stack can support the IRQ? Something like (for example) re-programming the PIC's Interrupt Mask Register with its current value, making it re-fire the existing ISRs?

At the moment I'm interrogating the PICs' In-Service Registers, and simply manually calling the INTs to service them (manually emulating the PICs' process) - and then going on to the Stack Fault exception handler proper. Is there a better way to get the PIC to "re-fire" the previous outstanding interrupt, especially when the "traditional" interrupts (such as Timer and Keyboard) are edge-sensitive, rather than level-sensitive?

Oh, and needless to say - if there wasn't an outstanding interrupt, the effective result should be a NOP.

With thanks for any suggestions,

John Burger

[JAAman]

most systems will never get a stack fault because they use a basic flat mode, and therefore will never receive stack faults (though they might get a page fault on the kernel stack, which is basically the same thing)

the way I see it, there are 2 possibilities:
#1: you increase the size of the kernel stack immediately, and iret -- this (as you said) will fix everything (if you cannot, then see option #2)
#2: if you run out of kernel stack space, you assume that the kernel must be corrupt (obviously something went seriously wrong) and BSOD (or your equivalent)

your "until then" is rather irrelevant (imho), if your stack is still invalid, you shouldn't be calling anything, and if it is fixed, you should be returning, either fix the stack and resume, or abort with a critical error

[JohnBurger]

JAAman,

Thank you for your reply. While I agree a Flat model is common, like I said, I'm exploring the CPU and (especially) Segmentation, which I happen to prefer! But your Page Fault example is equivalent (although hopefully not during the invocation of an IRQ handler...)

Anyway, I finally got to the point where the Faulting scenario was fixed and did the IRET, expecting the code to resume and (hopefully) service the interrupt. And in the specific scenario I gave, it does - but that's not good enough.

If the circumstance is a little different, then I get the original problem. The new scenario is that the Stack is a little fuller, and the IRQ doesn't even have enough room to push the EFlags, CS and EIP. The Stack Fault occurs, the handler grows the stack, but the IRET returns to the interrupted instruction, NOT the interrupt handler: since the interrupt handler never started, all the CPU context is still in the code that was executing at the time of the interrupt, not the interrupt handler itself. And now the unacknowledged PIC has disabled all lower-priority interrupts - and if it was a Timer IRQ...

I'm guessing that the PUSHed Fault code for the Stack Fault would have the Ext bit (0x0001) set (I'll check this), but I'd still like to know what I could do about it. The code to poll the PICs' ISRs and manually invoke the indicated INTs is not very nice.

[bluemoon]

If the circumstance is a little different, then I get the original problem. The new scenario is that the Stack is a little fuller, and the IRQ doesn't even have enough room to push the EFlags, CS and EIP. The Stack Fault occurs, the handler grows the stack, but the IRET returns to the interrupted instruction, NOT the interrupt handler: since the interrupt handler never started, all the CPU context is still in the code that was executing at the time of the interrupt, not the interrupt handler itself. And now the unacknowledged PIC has disabled all lower-priority interrupts - and if it was a Timer IRQ...

In case you recovered and resume to faulty instruction, the previously interrupted routine will not notice the stack fault occurred, as if it did not happened.
What's the problem?

[alexfru]

The problem is probably that the IRQ ISR never starts when the CPU fails to push Eflags, CS, EIP when attempting to handle the IRQ. And the exception handler returns to the instruction following the one after which the IRQ fired, not the ISR. There is no queue in the CPU for hardware interrupts that fail to reach their ISRs in such situations, and so they get lost, leaving the interrupt requestig devices to think they're still being handled, but to no avail.

I think, if IRQs are expected to occur while ring 0 stack is completely exhausted, it may make sense to use interrupt tasks, that is, an IRQ ISR would get to run in its own task with its own stack, never touching the stack of the interrupted ring 0 code.

[bluemoon]

That's why you should control and limit the usage of kernel stack. For me the kernel stack always present (when thread is active) and won't growth (never have stack fault, or die!) so the above scenarios do not affect me.

[JohnBurger]

The problem is probably that the IRQ ISR never starts when the CPU fails to push Eflags, CS, EIP when attempting to handle the IRQ. And the exception handler returns to the instruction following the one after which the IRQ fired, not the ISR. There is no queue in the CPU for hardware interrupts that fail to reach their ISRs in such situations, and so they get lost, leaving the interrupt requestig devices to think they're still being handled, but to no avail.

This is exactly the problem, yes.

I think, if IRQs are expected to occur while ring 0 stack is completely exhausted, it may make sense to use interrupt tasks, that is, an IRQ ISR would get to run in its own task with its own stack, never touching the stack of the interrupted ring 0 code.

I'm not sure I'd trust a TaskGate in the IDT - what if a new interrupt came in before the old one had finished? Instant Invalid TSS (busy), and you'd be in further trouble. True, you could leave interrupts disabled for the duration, but that impacts interrupt latency.

That's why you should control and limit the usage of kernel stack. For me the kernel stack always present (when thread is active) and won't growth (never have stack fault, or die!) so the above scenarios do not affect me.

That may be, but a full stack was merely one example of many correctible faults that could occur, like Not Present or Page Fault.

Many faults can be "repaired" and the faulting instruction retried, to continue execution as if nothing had happened - indeed, my OS design relies on these notifications to delay processing until the event is triggered. It just means that I cannot allow "unfulfilled" conditions anywhere near the triggering of interrupt handlers, at the risk of locking up the whole machine.

And this means these limitations extend all through the full Kernel: for example, I can't have any unmapped blank pages in Stack0 for filling by the Page Fault handler. Which is why I wanted an easy way to make the PIC(s) re-trigger their pending interrupt(s) - for example, I tried re-programming the IMR with its existing value. It doesn't look like this is possible...

[bluemoon]

The idea is that, the "simplest" design is to ensure kernel stack is always ready, pre-allocated (and mapped), so it (hopefully) never generate fault by design.
This is indeed not expensive as the kernel stack can be keep at small size, everything the kernel require big memory should request a heap.

Also note that some kernel design only require one stack per core, that further reduce the memory footprint.

As a side note, you may also use the "stack index" feature on AMD64, this may solve the stack fault problem, but however it may introduce re-entrance problem.

[alexfru]

I think, if IRQs are expected to occur while ring 0 stack is completely exhausted, it may make sense to use interrupt tasks, that is, an IRQ ISR would get to run in its own task with its own stack, never touching the stack of the interrupted ring 0 code.

I'm not sure I'd trust a TaskGate in the IDT - what if a new interrupt came in before the old one had finished? Instant Invalid TSS (busy), and you'd be in further trouble. True, you could leave interrupts disabled for the duration, but that impacts interrupt latency.

First of all, the same IRQ should not arrive until you explicitly acknowledge the one you've already got and started handling.

Secondly, you should have per-IRQ interrupt tasks, so they can link/nest.

But in terms of latency you may be out of luck here anyway. Task switches are a rarely used mechanism and as such may not be optimized for the best performance. Choose what is of more importance to you.