OSDev.org

Goal: Getting keyboard working and later other devices...

I'm bit stuck at programming device interrupts, any pointers welcomed

For now my OS is:

• Setting up paging
• Switching to Long Mode
• Creating interrupt handlers and loading IDT (Exceptions are working nice )
• Locating RSDP from ACPI specification
• Locating MADT (RSDP->XSDT->MADT)
• Parsing MADT and getting APIC and I/O APIC address
• If PCAT_COMPAT is set in MADT Flags then masking all IRQ in PIC and writing 01h to IMCR
• ???

I've read:

• 82093AA I/O ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (IOAPIC)
• Chapter 6 (INTERRUPT AND EXCEPTION HANDLING) and 10 (ADVANCED PROGRAMMABLE
  INTERRUPT CONTROLLER (APIC)) from Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3A: System Programming Guide, Part 1
• Chapter 5.2.12 (Multiple APIC Description Table (MADT)) from Advanced Configuration and Power Interface Specification (Revision 4.0a)

as I understand I've to program IOREDTBL[23:0]—I/O REDIRECTION TABLE REGISTERS
but I've no idea how exactly and what should contain each entry

PS. currently I don't care other cores and I'm using just BSP, also I'm discarding any compatibility and my OS will use only native modes, no PIC etc. I'm trying to make it as every modern OS should be, that's why ACPI
any other suggestions also would be nice


Thanks

Hi,

davispuh wrote:

• If PCAT_COMPAT is set in MADT Flags then masking all IRQ in PIC and writing 01h to IMCR

Only very old systems have the IMCR - newer systems don't, and all systems that support ACPI won't. The "PCAT_COMPAT" flag in the MADT only says that PIC chips are present, and doesn't mean that there's also an IMCR.

davispuh wrote:

• ???

The next step would be parsing the MADT properly - using the "Interrupt Source Override" structures to determine how legacy ISA IRQs are connected to IO APIC inputs. Once you know this you can setup the IO APIC to recognise/handle the legacy ISA IRQs (including the PS/2 controller).

You should probably also learn how the APICs handle interrupt priorities and decide what should use which interrupt vectors; and setup the local APIC properly - e.g. setup a "spurious interrupt", set the TPR, LDR, etc.

davispuh wrote:PS. currently I don't care other cores and I'm using just BSP, also I'm discarding any compatibility and my OS will use only native modes, no PIC etc. I'm trying to make it as every modern OS should be, that's why ACPI
any other suggestions also would be nice

It's far too easy to make design flaws that make it hard to add support for multi-CPU later (without rewriting lots of code). Things like locking and reentrancy (and scalability) need to be considered during the design, and something designed for single-CPU is rarely usable. I wouldn't start doing anything with IRQs (or IO APICs) until after you've proven that the local APICs are configured correctly by sending IPIs between different CPUs.

Also, I wouldn't bother with any device drivers until boot/initialisation code, CPU feature detection, physical memory management, virtual memory management (including the "multi-CPU TLB shootdown" stuff that requires IPIs), scheduling (including using the local APIC timer, and the ability to start and terminate processes if it's a micro-kernel) and IPC are all "working". Then I'd start doing some sort of "device manager", which would be responsible for deciding (and tracking) which devices use which resources (including which IRQs and which interrupt vector/s).


Cheers,

Brendan

Brendan wrote: Only very old systems have the IMCR - newer systems don't, and all systems that support ACPI won't. The "PCAT_COMPAT" flag in the MADT only says that PIC chips are present, and doesn't mean that there's also an IMCR.

ok thanks, IMCR was mentioned only in MP spec and I'm not parsing MP structures...

Brendan wrote: The next step would be parsing the MADT properly - using the "Interrupt Source Override" structures to determine how legacy ISA IRQs are connected to IO APIC inputs. Once you know this you can setup the IO APIC to recognise/handle the legacy ISA IRQs (including the PS/2 controller).

in MADT there was only one "Interrupt Source Override" with Source 00h and Global System Interrupt 02h
it looks like exactly from ACPI manual

if your machine has the ISA Programmable Interrupt Timer (PIT) connected to ISA IRQ 0,
but in APIC mode, it is connected to I/O APIC interrupt input 2, then you would need an Interrupt Source
Override where the source entry is ‘0’ and the Global System Interrupt is '2'

I'm not even sure if I'll need PIT, because I can use APIC timer...
also as I said I don't know how to set up IO APIC, where should I look?

Brendan wrote: You should probably also learn how the APICs handle interrupt priorities and decide what should use which interrupt vectors; and setup the local APIC properly - e.g. setup a "spurious interrupt", set the TPR, LDR, etc.

I've read chapter 10 of Intel manual 3A, but I still don't really understand how it all works together, I think I need more information on how to do programming... maybe I don't know how to read it or there isn't enough information for me

Brendan wrote: It's far too easy to make design flaws that make it hard to add support for multi-CPU later (without rewriting lots of code). Things like locking and reentrancy (and scalability) need to be considered during the design, and something designed for single-CPU is rarely usable. I wouldn't start doing anything with IRQs (or IO APICs) until after you've proven that the local APICs are configured correctly by sending IPIs between different CPUs.

Also, I wouldn't bother with any device drivers until boot/initialisation code, CPU feature detection, physical memory management, virtual memory management (including the "multi-CPU TLB shootdown" stuff that requires IPIs), scheduling (including using the local APIC timer, and the ability to start and terminate processes if it's a micro-kernel) and IPC are all "working". Then I'd start doing some sort of "device manager", which would be responsible for deciding (and tracking) which devices use which resources (including which IRQs and which interrupt vector/s).

I understand that, but currently I don't want to work with it, I'll start implementing it after keyboard and probably hdd. it's pretty possible that some parts will need to rewrite but I'm not worried about it and every big change requires some code rewriting

davispuh wrote:I don't know how to set up IO APIC, where should I look?

The IOAPIC registers are accessed through the use of two memory mapped locations - an index register and a data register. The index register is at base + 0x0, the data register at base + 0x10, where base is given in the IOAPIC structure at the end of the MADT table in the ACPI tables. To read a register you do a 32 bit write to the index register with the number of the actual register you want, followed by a 32 bit read from the data register. To do a write, do a 32 bit write to the index register, followed by a 32 bit write to the data register.

Essentially an IOAPIC has inputs from various devices, and outputs to any interrupt pin on any cpu. The ACPI specification defines a 'global system interrupt' - all interrupts given in the ACPI namespace are GSIs. Each IOAPIC in the system is responsible for a certain range of GSIs, defined by the global system interrupt base field in the IOAPIC structure in the MADT table. For a system with a single IOAPIC this is usually 0. To get the IOAPIC input pin from the GSI you do IOAPIC pin = GSI - gsi_base. Non-standard mappings from the normal ISA IRQs (e.g. PIT = 0, keyboard = 1 and so on) to GSIs are given by the Interrupt Source Overrides (as you've already spotted). PCI device -> GSI mappings are defined in the PCI configuration space.

The next mapping is from IOAPIC input pin to an interrupt pin on a particular CPU - this CPU interrupt pin is the interrupt which is raised within the CPU (i.e. the index in the IDT). The mappings are defined in the IOAPIC redirection table registers. IOREDTBL0 is for IOAPIC input pin 0 (normally, but not always GSI 0 - depending on the value of gsi_base as mentioned above) and so on. The redirection table entries are 64 bit values, therefore to set them you need to do two 32 bit writes - one to the low address and one to the high address. For IOREDTBL0, you would write 0x10 to the index register then the low 32 bits of the register value to the data register, then 0x11 to the index register then the high 32 bits of the register value to the data register.

The actual contents of the IOAPIC redirection table register fields are given in the IOAPIC specification. To send an interrupt to a particular processor, you can use physical mode. For this mode, set bits 0-7 to be the interrupt vector (i.e. which entry in the IDT to execute), bits 8-10 to 0 (for fixed delivery mode - only if this is a device, things like the NMI should be set up differently), bit 11 to 0 (for physical mode), bit 16 to 0 to unmask the interrupt, and bits 56-59 to the LAPIC ID of the processor you want to send the interrupt to. The triggering mode should hopefully be set up by the BIOS, so read the registers first, adjust what you need, then write them back. If not, you may need to parse the ACPI tables to find this information for ISA IRQs (Interrupt Source Override structure) or the PCI specification for PCI devices.

Please note this is the simplest way to set up the APIC and that you can acheive more than this (i.e. delivering interrupts to all a group of CPUs, delivering to only one of a predefined group, which changes dynamically - read up on interrupt priorities), but for starters should be enough.

As a side note, the LAPIC attached to each CPU also generates interrupts which are routed directly to the CPU they are attached to - again you can change the interrupt pin, but the registers are accessed in a different way - these are described in the CPU manuals. In particular the LAPICs provide a timer per-processor which is a good source to use for scheduler interrupts.

Regards,
John.

big thanks mostly everything you said I already know because I read IOAPIC manual before, but I didn't knew what to do with other flags (Delivery Mode, Destination Mode, Interrupt Input Pin Polarity, Remote IRR, Trigger Mode)

in my case all 24 IOREDTBL entries are 00010000h (16 bit set -> masked)
for now I read IOREDTBL1 (IRQ1, keyboard) and set INTVEC to 32, Destination Field to BSP APIC ID
keyboard interrupt handler got called on key press



basically it's really easy, only IOAPIC manual lack few explanations about things which I didn't understand

most useful sentence

jnc100 wrote: The triggering mode should hopefully be set up by the BIOS, so read the registers first, adjust what you need, then write them back.

one more thing I don't know is which devices are mapped to IOAPIC pins above 15 (all below are identity mapped same as PIC with exceptions from MADT "Interrupt Source Override")

anyway I need some more reading about these things, devices, Trigger Modes and Polarity... what would you recommend ?

Thanks

Hi,

davispuh wrote:most useful sentence

jnc100 wrote: The triggering mode should hopefully be set up by the BIOS, so read the registers first, adjust what you need, then write them back.

I've always assumed that the BIOS doesn't bother setting up anything in the IO APIC, and I've always done everything myself (e.g. rather than reading old values, finding out that the BIOS didn't do it, then writing new values; I'd just write new values without reading the old values at all).

For ISA IRQs, you assume the IRQ is "edge trigger, active high", unless interrupt source overrides in the MADT tell you to use something else.

I'd probably construct a table that describes what each IO APIC input is used for, from information in the MADT (if possible) or from information in the MultiProcessor Specification tables (if there's no ACPI). When initialising the IO APIC/s I'd just disable everything. Then I'd have a set of functions to install interrupts/IRQ handlers; that create the IDT entry and program the correct IO APIC input according to the information you put in your table earlier.

davispuh wrote:one more thing I don't know is which devices are mapped to IOAPIC pins above 15 (all below are identity mapped same as PIC with exceptions from MADT "Interrupt Source Override")

The remaining IO APIC inputs are used by PCI devices (or possibly nothing). The ACPI MADT won't tell you which PCI IRQs are connected to which IO APIC inputs - you have to use ACPI AML for that (which is excessively painful, but is theoretically better because it is able to cope with things like hot-plug PCI). The "MultiProcessor Specification" tables will tell you which PCI IRQs are connected to which IO APIC inputs (but it can't handle things like hot-plug PCI where devices aren't plugged in until after boot).

In general, it'd be a good idea to get ISA IRQs working before worrying about PCI.


Cheers,

Brendan

I have a related IO apic question:

Do I need to turn on IO apic from somewhere? If yes, is there a universal way of doing it?

Neither 82093AA IOAPIC manual nor Intel's Volume Software Developer’s Manual 3A are very specific about this.

regards
Mac2004

It seems that a few people here are working on getting the I/O APIC up and running. Good to see!

I'm been working on it as well with Pure64. The only ISA interrupt that Pure64 uses is the RealTime Clock. In my testing it works under Bochs and QEMU but not VirtualBox or VMware. Perhaps my ACPI parsing is the cause but I'm not sure.

For the Interrupt Source Override Structure entries... how many would you expect to see? Under QEMU there are only five (Source, GSI; 0x00, 0x02; 0x05, 0x05; 0x09, 0x09; 0x0A, 0x0A; and 0x0B, 0x0B) and Bochs has none. I notice that Source 0x08 (The RTC) is missing from both.

Any ideas?

Hi,

mac2004 wrote:Do I need to turn on IO apic from somewhere? If yes, is there a universal way of doing it?

No - just setup (and unmask) each input in the IO APIC's "I/O Redirection Table". There is no "master enable" or anything else to worry about.

ReturnInfinity wrote:I'm been working on it as well with Pure64. The only ISA interrupt that Pure64 uses is the RealTime Clock. In my testing it works under Bochs and QEMU but not VirtualBox or VMware. Perhaps my ACPI parsing is the cause but I'm not sure.

For the Interrupt Source Override Structure entries... how many would you expect to see? Under QEMU there are only five (Source, GSI; 0x00, 0x02; 0x05, 0x05; 0x09, 0x09; 0x0A, 0x0A; and 0x0B, 0x0B) and Bochs has none. I notice that Source 0x08 (The RTC) is missing from both.

What Source Override Structure entries do you get from VirtualBox or VMware (and do they have Source 0x08)?

If your OS also supports the use of the PIC chips (e.g. for old single-CPU systems without any APICs), then it would be easy to test if the problem on VirtualBox or VMware is the RTC (and not the IO APIC)...


Cheers,

Brendan

ReturnInfinity wrote:It seems that a few people here are working on getting the I/O APIC up and running. Good to see!

I have it running, but I support both modes. If I have an APIC, I'll use IO APIC, otherwise I default to PIC. I don't do full parsing of ACPI yet, but I will eventually do this. I have no support for MP tables, and I don't even think any of my test-machines have MP tables.

I will eventually read-out the PCI mappings as well, since I have a functional AML parser.

ReturnInfinity wrote:For the Interrupt Source Override Structure entries... how many would you expect to see? Under QEMU there are only five (Source, GSI; 0x00, 0x02; 0x05, 0x05; 0x09, 0x09; 0x0A, 0x0A; and 0x0B, 0x0B) and Bochs has none. I notice that Source 0x08 (The RTC) is missing from both.

Often only 1 source override to map the timer to GSI 2. I notice under Qemu, the only override that remaps is 0 mapped to GSI 2. That seems quite normal. A missing entry means that interrupt is identity mapped with default ISA polarity and trigger.

it would be really good if someone with good knowledge about these things could improve wiki about this topic, because in http://wiki.osdev.org/IOAPIC there isn't enough information and just reading manuals isn't enough to understand whole thing (at least for me it wasn't enough...)

VirtualBox has two entries (Source, GSI; 0x00, 0x02; 0x09, 0x09) and VMware has one (Source, GSI; 0x00, 0x02).

I'm not sure what I changed but the RealTime Clock is firing in QEMU, Bochs, and VirtualBox. Not under VMware (I'm using Workstation 8.0.1). Any ideas? Is there something I could have missed? The ACPI references makes note of I/O SAPIC entries and says:

The I/O SAPIC structure is very similar to the I/O APIC structure. If both I/O APIC and I/O SAPIC
structures exist for a specific APIC ID, the information in the I/O SAPIC structure must be used.

Also is it possible that some computers do not have a PIC? I tried booting an older version of Pure64 (That only used the PIC) a few months ago but got no interrupts firing on an IBM server I tried. All other PCs I tested were fine.

Hi,

ReturnInfinity wrote:VirtualBox has two entries (Source, GSI; 0x00, 0x02; 0x09, 0x09) and VMware has one (Source, GSI; 0x00, 0x02).

I'm not sure what I changed but the RealTime Clock is firing in QEMU, Bochs, and VirtualBox. Not under VMware (I'm using Workstation 8.0.1). Any ideas? Is there something I could have missed?

Could be something missing in your RTC initialisation code...

ReturnInfinity wrote:The ACPI references makes note of I/O SAPIC entries and says:

The I/O SAPIC structure is very similar to the I/O APIC structure. If both I/O APIC and I/O SAPIC
structures exist for a specific APIC ID, the information in the I/O SAPIC structure must be used.

As far as I know SAPICs only ever exist on Itanium; and for 80x86 you always get the normal IO APIC entries, and either the normal local APIC entries or both normal local APIC and local x2APIC entries. If you ignore the local x2APIC entries (if present), then your OS will probably be limited to 255 CPUs (not likely to be a big problem).

ReturnInfinity wrote:Also is it possible that some computers do not have a PIC? I tried booting an older version of Pure64 (That only used the PIC) a few months ago but got no interrupts firing on an IBM server I tried. All other PCs I tested were fine.

It is possible, but not very likely (yet) as there's little reason to break compatibility when everyone is mostly using the same chipsets made by the same 3 companies (Intel, NVidia and VIA) as everyone else. If it was a thin client or tablet (designed for specific software only) or a huge high-end server, then you might find some "not very PC compatible" things.


Cheers,

Brendan

to beginning... I'm stuck again...

I tested my OS with VMware and interrupts are working there, but now I tried on VirtualBox and on laptop (real hardware ), but none of interrupts were called
on QEMU interrupts works, that's pretty weird, maybe there's some configuration which I haven't done?

everything else looks like works and there isn't any errors or exceptions

I'm getting I/O APIC address from MADT (ACPI)
I/O APIC works, I can successfully get it's version and ID (using memory mapped registers)

but when I set up IRQ 1 and enable interrupts nothing happens when I press any key on keyboard
(PIT interrupt also doesn't get called, on VMware it does)

this is code, I'm not doing anything more and it works in VMware
(rdi is location to I/O APIC memory-mapped registers) EDIT: I think it's something with local APIC because I'm not touching it I don't understand how it should be programmed...