am I accessing the I/O APIC incorrectly?

[mariuszp]

I tried programming the I/O APIC for the first time, I was enumerating the I/O APICs from ACPI and the base was always 0xFEC0. The IOAPIC data sheet (from Intel) says that the base is 0xFEC0 xy00 ("xy = see APICBASE in PIIX3"). I'm not sure exactly what they meant by "see APICBASE in PIIX3", I assumed they would be 0, and therefore mapped the physical page 0xFEC00xxx to virtual address 0xFFFF808000002xxx. Next, I tried remapping the IRQs such that they are mapped to interrupt vectors 32+. Here's the code which actually does that (note the page has the page-level cache disable bit set):

Code: Select all

	uint32_t *regsel = (uint32_t*) 0xFFFF808000002000;
	uint32_t *iowin = (uint32_t*) 0xFFFF808000002010;

	kprintf("APIC ID: %d\n", apic->id);
	int i;
	for (i=0; i<16; i++)
	{
		*regsel = (0x10+2*i);
		__sync_synchronize();
		uint64_t entry = (uint64_t)(i+32) | ((uint64_t)(apic->id) << 56);
		*iowin = (uint32_t)(entry);
		__sync_synchronize();
		*regsel = (0x10+2*i+1);
		__sync_synchronize();
		*iowin = (uint32_t)(entry >> 32);
		__sync_synchronize();
	};

The "APIC ID" prints as 0 (currently only using one CPU).

When I enable interrupts, I immediately get interrupt 34. When it returns, it goes back to an infinite while loop right after the "sti" instruction, executes "jmp $" twice, then, randomly, interrupt 8 is thrown. When returning from its interrupt handler, the "iretq" instruction throws a page fault (non-present, read, kernel) at address 0x240.

The interrupt handlers worked perfectly when I used the legacy PIC. I presume a possible cause is that I am misinterpreting the "ioapicbase" in the ACPI table, or just misunderstanding the datasheet?

Bochs does not show any diagnostic messages when I'm remapping the IRQs (I don't know if it should, but it does show a message when I initialize the local APIC). Also, Bochs is not showing any diagnostic message when the iretq instruction is executed.

Has this happened to anyone else?

[bace]

Code: Select all

	uint32_t *regsel = (uint32_t*) 0xFFFF808000002000;
	uint32_t *iowin = (uint32_t*) 0xFFFF808000002010;

Just from a quick glance, this should be marked as volatile so your compiler doesn't optimise the memory accesses away (see the wiki page on the APIC for an example)

[mariuszp]

Oops, that was a silly mistake. I've now marked the data pointed to by the pointers as volatile (uint32_t* volatile) but I still get the same problem.

[bace]

Did you read the wiki page I linked you to? This correct code should look like:

Code: Select all

uint32_t volatile *

[mariuszp]

Sorry for being an idiot, here's my code right now, still with the same problem:

Code: Select all

	uint32_t volatile* regsel = (uint32_t volatile*) /*(ioapicbasephys+0xFFFF800000000000)*/ 0xFFFF808000002000;
	uint32_t volatile* iowin = (uint32_t volatile*) /*(ioapicbasephys+0xFFFF800000000010)*/ 0xFFFF808000002010;

	kprintf("APIC ID: %d\n", apic->id);
	int i;
	for (i=0; i<16; i++)
	{
		*regsel = (0x10+2*i);
		__sync_synchronize();
		uint64_t entry = (uint64_t)(i+32) | ((uint64_t)(apic->id) << 56);
		*iowin = (uint32_t)(entry);
		__sync_synchronize();
		*regsel = (0x10+2*i+1);
		__sync_synchronize();
		*iowin = (uint32_t)(entry >> 32);
		__sync_synchronize();
	};

And I doubt "volatile" makes any difference since I'm explicitly setting hardware memory barriers (which imply software barriers), but that's a debate for another day.

[mariuszp]

It now looks like when I send EOI to the APIC, then I don't get interrupt 8 after interrupt 34...
EDIT: Now I'm getting int 34, then 33, then 34 again, and then 8.

[Brendan]

Hi,

Some questions...

What happens if there's 4 separate IO APICs that have 8 entries each?

What makes you think the first 16 entries are supposed to be "edge triggered active high"? Note that there is no guarantee that ISA IRQs are connected to IO APICs in a 1:1 way and IO APIC inputs 16 to 31 could be used for ISA IRQs instead. Also, on a lot of systems I've seen "IO APIC input #9" is used by ACPI's embedded controller and isn't "edge triggered active high".

Why are you enabling the IRQs in the IO APIC before you have device drivers for those devices ready to handle the IRQs?

Are you trying to set up 16 interrupts for ISA IRQs even though ISA only has 15 IRQs (one is used for the "cascade" from slave to master)?

Given that (for APICs in general) the "interrupt vector" determines the IRQs priority, are you sure you want IRQ priorities configured like that?

Have you masked all IRQs in the PIC chips? Have you remapped the PIC chips and installed IRQ handlers for the PIC chip's "spurious IRQs" that can't be disabled?

Are you sure the "random interrupt 8" is not a double fault?


Cheers,

Brendan

[mariuszp]

I dont understand the concept of edge/level trigger... how do i know which one to use?

Also, is there a way to determine whether it was a double fault? My interrupt handlers worked perfectly with the legacy PIC and unhandled IRQs are ignored (EOI is sent and the handler returns).

[Brendan]

Hi,

I dont understand the concept of edge/level trigger... how do i know which one to use?

Edge/level (and active high/low) relate to how the device signals that it wants an IRQ. You have to program the IO APIC to match, otherwise it causes problems (missed IRQs, IRQ floods, etc).

ACPI and the MP specifications both tell you what you need to use (and also tell you which devices are connected to which IO APIC inputs in the same place).

For the MP specification, it's just a parsing a table to determine what you need to know all IRQs (e.g. ISA and PCI). Because this table is generated by firmware at boot it can't handle things like hot-plug PCI (e.g. devices that didn't exist when the firmware was creating the table). Also, MP specification is obsolete now, and shouldn't be used if ACPI exists.

For ACPI, for some IRQs (e.g. motherboard and ISA IRQs) it's still just a parsing a table. However, for PCI IRQs, to avoid the "devices that didn't exist when the firmware was creating the table" problem, ACPI requires a hideously bloated/over-engineered AML interpreter to figure it out.

Also, is there a way to determine whether it was a double fault? My interrupt handlers worked perfectly with the legacy PIC and unhandled IRQs are ignored (EOI is sent and the handler returns).

The local APIC has an "in service register" that has a bit for each interrupt vector. You can test the bit corresponding to "interrupt vector 8" to determine if the interrupt came from the local APIC (or from the IO APIC via. the local APIC). The PIC chips also have an "in service register" that can be used the same way. You can also check (carefully) if the interrupt handler's "return CS:EIP" points to an "int 8" instruction. That only leaves 2 possible causes - an exception or a spurious IRQ; but it's impossible for PICs to generate a "spurious interrupt 8" and you'd have to seriously mis-configure that local APIC to get it to try to send "spurious interrupt 8".

Of course there's also a much easier hack - if the interrupt occurred while you were running CPL=0 code with a known CS; then the interrupt handler can check its stack to determine where the "return CS" is. If there's an error code on the stack then the "return CS" will be in one place, and if there is no error code then the "return CS" will be in a different place.


Cheers,

Brendan

[mariuszp]

I disabled the PIC as follows:

Code: Select all

	outb(0xA1, 0xFF);
	outb(0x21, 0xFF);

I got this code from the osdev wiki article on the PIC. This is correct, right?

And what ACPI tables tell me which IRQs are mapped where and whether they are edge triggered? Searching the ACPI spec for "edge triggered" only showed some AML code which is clearly too much to process at this point.

[bace]

You'll also need to remap the PIC's IRQs, so that you can handle spurious interrupts from the PIC correctly. Since you've talked about a PIC driver already, I'll assume this shouldn't be too hard for you.

[mariuszp]

You'll also need to remap the PIC's IRQs, so that you can handle spurious interrupts from the PIC correctly. Since you've talked about a PIC driver already, I'll assume this shouldn't be too hard for you.

Oops, now that I ignore spurious IRQs (7 and 15), it all works fine. Haha.
Eh, now I'm not receiving interrupts from the PIT (IRQ0). Not too worried because I'll start using the APIC timer soon, but it's pretty strange and I should probably look at that in case it causes something worse later... again, it worked perfectly with the PIC...
But I'm constantly getting interrupt 34 (IRQ2). Could it be that somehow it got mapped to a different IRQ? (But then again, I still get interrupt 34 even BEFORE programming the PIT at all).

[bace]

Yes, IRQ 2 is used for the PIT with the APIC. And, the BIOS probably uses the PIT for timings so you can expect it to be firing the IRQ.

[mariuszp]

Yes, IRQ 2 is used for the PIT with the APIC. And, the BIOS probably uses the PIT for timings so you can expect it to be firing the IRQ.

Thank you, that explains everything. Thanks for replying to this constant stream of questions

[Brendan]

Hi,

Yes, IRQ 2 is used for the PIT with the APIC. And, the BIOS probably uses the PIT for timings so you can expect it to be firing the IRQ.

Often PIT happens to be connected to IO APIC input #2. Sometimes it's connected to IO APIC input #0. Sometimes its connected somewhere else. Sometimes it's not connected to the IO APIC at all.

There is no guarantees, and it could be connected to any IO APIC input on any IO APIC (or not connected to any IO APIC). You must parse ACPI's MADT/APIC table or the MP Specification tables to determine if/where it is connected. You can't assume.


Cheers,

Brendan