[SOLVED] Issues programming the local/IO APIC

[briansmith]

Hey,

I've been working on implementing the local APIC and IO APIC in my kernel recently, as I was previously using the 8259 PIC but felt like it'd be nice to support both. However, despite setting things up properly, when I enable interrupts with sti, I don't get any interrupts. My setup process is as follows:

(the whole process can be seen in kmain here)
- Load and initialize a 256 gate IDT
- Initialize ACPI tables and parse them.
- If an IO APIC is found in the ACPI tables then map it into virtual memory, and mask all the redirection entries
- Setup the master and slave PIC
- If ACPI parsing was successful and the APIC was supported then disable PIC, then enable the local APIC by getting it's base address from the MSR, loading it back in with the enable bit to enable it, mapping it into memory, creating a spurious interrupt in the IDT and enabling it in the local APIC, setting the TPR to 0, setting destination format to flat mode, and the logical destination to CPU0, then I enable 16 IRQs in the IO APIC, mapping IRQ0 -> interrupt 0x20, IRQ1 -> interrupt 0x21, etc..
- I then proceed to enable the PIT which is connected to my clock system which is connected to my scheduler. I don't link these all because they work under the PIC, just not under the APIC.

What should I do to continue debugging this? I've been hitting my head against the wall trying to figure out why I'm not getting any interrupts when I call sti (as verified through enabling QEMU's interrupt logging). Are there any common mistakes made while implementing the local APIC?

Thanks.

edit: some additional notes, interrupts are definitely enabled since my NMI handler gets triggered on a divide by zero. I tried to build qemu with IOAPIC debugging enabled but wasn't able to completely compile it due to macOS being weird (stdlib.h not found when compiling an objc file?). Also, the complete output of running the kernel can be found here. I've also tried this under VirtualBox, it still doesn't work.

[xenos]

Have you checked whether there is an interrupt source override in the ACPI tables (in the MADT), such that ISA IRQ0 is redirected to global IRQ2? I have seen this quite often, so one should listen to IRQ2 instead of IRQ0 to get the timer working.

[briansmith]

Have you checked whether there is an interrupt source override in the ACPI tables (in the MADT), such that ISA IRQ0 is redirected to global IRQ2? I have seen this quite often, so one should listen to IRQ2 instead of IRQ0 to get the timer working.

I have these parsed but they're not used yet. Even still I map all 16 IRQs to a common handler which never gets called, and I've confirmed with qemu's interrupt logger that no interrupt ever even occurs in the first place.

[Korona]

Do you set the interrupt polarity and trigger mode to the correct values?

On real hardware I would blame not calling _PIC but it don't think that is necessary on qemu.

[Octocontrabass]

interrupts are definitely enabled since my NMI handler gets triggered on a divide by zero.

Divide by zero causes an exception. Exceptions are not maskable, so this doesn't prove that interrupts are set up correctly. (And it shouldn't cause NMI, so maybe you should make sure you can receive exceptions properly before you worry about hardware interrupts.)

[briansmith]

Do you set the interrupt polarity and trigger mode to the correct values?

On real hardware I would blame not calling _PIC but it don't think that is necessary on qemu.

What are the correct interrupt polarity/trigger mode? I've been using the default values (high is active & edge sensitive).

[briansmith]

interrupts are definitely enabled since my NMI handler gets triggered on a divide by zero.

Divide by zero causes an exception. Exceptions are not maskable, so this doesn't prove that interrupts are set up correctly. (And it shouldn't cause NMI, so maybe you should make sure you can receive exceptions properly before you worry about hardware interrupts.)

Ah thank you for the heads up. For some reason I was calling exceptions NMIs in my code, and I've fixed that. I've also fixed my APIC problems. It was an incredibly silly mistake: my IOREDTBL offset for my I/O APIC handling code was 3, instead of 0x10. I've been banging my head against the wall trying to solve this for months . Receiving interrupts now. Thank you everyone for your help.

[Brendan]

Hi,

Do you set the interrupt polarity and trigger mode to the correct values?

What are the correct interrupt polarity/trigger mode? I've been using the default values (high is active & edge sensitive).

The polarity and trigger mode are "whatever the motherboard manufacturer felt like for that specific motherboard". For a silly example; nothing says that a motherboard can't do "PIT timer/ISA IRQ 0 is connected to IO APIC input #29 of the third IO APIC, and is level triggered active low".

You need to use ACPI (or MultiProcessor Specification tables if there's no ACPI) to be able to figure out what is connected to each IO APIC input, and the correct polarity and trigger mode to use for each IO APIC input. This is easy for the legacy IRQs (just parse ACPI's MADT/APIC table) and hard for PCI IRQs (need an AML interpreter). Fortunately PCI IRQs are being deprecated (being replaced by MSI/"Message Signalled Interrupts" which bypass the IO APIC and can be setup without ACPI).


Cheers,

Brendan

[briansmith]

Hi,

Do you set the interrupt polarity and trigger mode to the correct values?

What are the correct interrupt polarity/trigger mode? I've been using the default values (high is active & edge sensitive).

The polarity and trigger mode are "whatever the motherboard manufacturer felt like for that specific motherboard". For a silly example; nothing says that a motherboard can't do "PIT timer/ISA IRQ 0 is connected to IO APIC input #29 of the third IO APIC, and is level triggered active low".

You need to use ACPI (or MultiProcessor Specification tables if there's no ACPI) to be able to figure out what is connected to each IO APIC input, and the correct polarity and trigger mode to use for each IO APIC input. This is easy for the legacy IRQs (just parse ACPI's MADT/APIC table) and hard for PCI IRQs (need an AML interpreter). Fortunately PCI IRQs are being deprecated (being replaced by MSI/"Message Signalled Interrupts" which bypass the IO APIC and can be setup without ACPI).


Cheers,

Brendan

How would you use the MADT/APIC to find out which legacy IRQs are setup to which I/O APIC input? Would you just use the interrupt source override entries in the MADT? Even then, how would you know which ISO refers to which device? Or is that just things you'd find out in the AML?

[Korona]

Which ISA IRQ belongs to which device is somewhat fixed: PIT is always IRQ 0, the PS/2 controller always uses IRQ 1 and so on. Of course, this is only true if the PC in question actually has a PIC and PS/2 controller, that is, it is not "legacy free". In practice most, but not all PCs are not legacy free. For other devices like the UARTs, the mapping is not fixed (most BIOSes allow you to change it) and can be determined by reading the _CRS object (which requires AML). For PCI devices, you have to evaluate the _PRT object instead and combine that with the _CRS of "link devices". It's not that complicated when you use ACPICA but it's not convenient either.

[Brendan]

Hi,

You need to use ACPI (or MultiProcessor Specification tables if there's no ACPI) to be able to figure out what is connected to each IO APIC input, and the correct polarity and trigger mode to use for each IO APIC input. This is easy for the legacy IRQs (just parse ACPI's MADT/APIC table) and hard for PCI IRQs (need an AML interpreter). Fortunately PCI IRQs are being deprecated (being replaced by MSI/"Message Signalled Interrupts" which bypass the IO APIC and can be setup without ACPI).

How would you use the MADT/APIC to find out which legacy IRQs are setup to which I/O APIC input? Would you just use the interrupt source override entries in the MADT? Even then, how would you know which ISO refers to which device? Or is that just things you'd find out in the AML?

For legacy IRQs in the MADT/APIC table; you start with the assumption that legacy IRQs are connected to IO APIC inputs in the same order as they're numbered (e.g. legacy IRQ #0 connected to IO APIC Input #0, legacy IRQ #1 connected to IO APIC Input #1, legacy IRQ #3 connected to IO APIC Input #3, ..., legacy IRQ #15 connected to IO APIC Input #15) and that they use ISA signalling ("edge triggered, active high", or triggered on the rising edge). Then the interrupt source overrides in the MADT tell you were this initial assumption is wrong. For example, there might be an interrupt source override saying that ISA IRQ #0 is actually connected to IO APIC Input #2, or that ISA IRQ #9 is actually level triggered and not edge triggered, or...

For "which device uses which legacy IRQ" it's the same as it is with PIC chips (without PCI), which comes from "historical use of IRQs on ISA bus" (and can come from "actual use of ISA IRQs" if the computer is old enough to have an actual ISA bus with ISA slots, etc). Some devices use fixed resources and fixed IRQs (e.g. PIT uses ISA IRQ 0, the PS/2 controller uses ISA IRQ 1 for 1st PS/2 port and ISA IRQ 12 for 2nd PS/2 port, etc) and these aren't much problem (and cover all the legacy devices that are still around today). However some devices don't (e.g. ISA sound cards).

For ISA devices that don't use fixed resources; ISA originally didn't have any method for software to autodetect which devices are present, or autodetect and autoconfigure the resources each device uses (IRQs, DMA channels, IO ports, memory mapped IO areas). Instead, typically there's little jumpers (or switches) on the card that allow someone to manually configure the resources the card uses, and the OS has to be told how the card was configured (e.g. by command line options to device drivers started in "config.sys" in MS-DOS or something). Eventually Microsoft tried to add autodetect/autoconfiguration to ISA cards by introducing an "ISA Plug and Play" standard; but it was too little too late - very few manufacturers bothered to implement it, and then everything switched to PCI anyway.

Fortunately there's very little reason to support ISA cards now (they're all over 20 years old and quite obsolete); which means that you only really need to worry about legacy devices with fixed resources (and PCI), and for all of these you can figure out the resources from the corresponding pages in the OSdev wiki. Specifically; there's:

• PIT (IRQ 0), possibly emulated via firmware/HPET
• RTC (IRQ 8 ), possibly partially emulated via firmware/HPET
• ISA DMA controller (doesn't have an IRQ)
• PS/2 controller (IRQ 1 for first PS/2 port, IRQ 12 for second PS/2 port), possibly emulated by firmware/USB, possibly doesn't exist at all
• Serial ports (the first 2 only, IRQs 3 and 4). May not exist (mostly replaced by USB).
• Floppy disk controller (the first one only, IRQ 5). May not exist (mostly replaced by USB).
• Parallel port (the first one only, IRQ 7). May not exist (mostly replaced by USB).
• ATA hard disk controllers (the first two only, IRQs 14 and 15). These are PCI devices emulating legacy ISA devices and can be detected via. PCI. (and have mostly been replaced by AHCI controllers)

There's also some ugly backward compatibility hacks involving the FPU and IRQ13; where (assuming "80486 or later") it's always better to enable the "native FPU exceptions" mechanism instead (and treat IRQ13 as "never used"), partly because this is required for multi-CPU anyway (and partly because it's faster and avoids a potential race condition).

Finally, ISA IRQ 2 doesn't exist - it's consumed by the "cascade line" between the slave PIC chip and master PIC chip.


Cheers,

Brendan