OSDev.org

Hello,
I'm currently working on my I/O Apic code and ran into some problems as I'm not quite sure how to initialize the interrupt redirection entries. After parsing Intel's multiprocessor table and examinig the result I'm now even more stumbled:
[pre]o----o--------------o------o------o------o-------o
|irq | windows says | Type | Bus | Pol | Edge |
o----o--------------o------o------o------o-------o
| 00 | sysclock | EXT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 01 | kbd | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 02 | | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 03 | COM2 | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 04 | COM1 | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 05 | | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 06 | FDC | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 07 | | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 08 | RTC | INT | ISA | high | edge |
o----o--------------o------o------o------o-------o
| 09 | ACPI | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 10 | MIDI | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 11 | SMBus | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 12 | Mouse | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 13 | CoProcessor | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 14 | Prim ATA | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 15 | Sec ATA | INT | ISA | std | std |
o----o--------------o------o------o------o-------o
| 16 | 2*USB, VGA | INT | PCI | low | level |
o----o--------------o------o------o------o-------o
| 17 | Sound | INT | PCI | low | level |
o----o--------------o------o------o------o-------o
| 18 | USB | INT | PCI | low | level |
o----o--------------o------o------o------o-------o
| 19 | USB | INT | PCI | low | level |
o----o--------------o------o------o------o-------o
| 20 | | INT | PCI | low | level |
o----o--------------o------o------o------o-------o
| 21 | | INT | PCI | low | level |
o----o--------------o------o------o------o-------o
| 22 | Ethernet | INT | PCI | low | level |
o----o--------------o------o------o------o-------o
| 23 | PCI-USB | INT | PCI | low | level |
o----o--------------o------o------o------o-------o[/pre]
Obviously windows remapped some of the PCI irqs:
- IRQ #11 is ISA in the table but PCI in windows
- I found two entries for IRQ #16 but in windows there are 3 devices connected two it
- There are also 2 entries for IRQ #18 but only one windows device

As one of the IRQ #16 devices is definitly the graphic-card (I can tell because it has an PCI bus of its own), I somehow assume that windows must have remapped one of the USB controllers from IRQ #18 to IRQ #16.
Wouldn't it however be way more logical to give the AGP card an exclusive IRQ line and have two of the USB controllers (which are way less important anyway) share a single line ? Apart from that IRQs #20 and #21 are not used at all. Are there some hardware restrictions that keep PCI from using these lines or is my computer just poorly set-up ?
I'm also wondering why the SMBus Controller gets mapped to an interrupt that is normally used by ISA devices..

On an other computer (-that uses some rather cheap board-) the multiprocessor table isn't even complete. There are only 10 entries in it, while windows enumerates 15 IRQs in its hardware manager. What's missing are almost all legacy ISA devices except for PIT, keyboard and some mysterious IRQ #2 which doesn't correspond to any device window uses. I'm wondering how other operating systems actually detect these devices, after all they'll hardly be in the ACPI tables either..
While there are too few ISA enties, all PCI are enumerated correctly. Actually there's even one entry too much, but that hardly matters in that case.

With all this I was wondering how reliable the multiprocessor table really is. It's not only incomplete on some computers, but also only preliminary as a lot of IRQs obviously have to be remapped first. Wouldn't it be more resonable to dump the whole table in favor on some PCI bus specific detection routines ? I'm however not sure what to do with the ISA devices then..

regards,
gaf

Hi,

gaf wrote:I'm currently working on my I/O Apic code and ran into some problems as I'm not quite sure how to initialize the interrupt redirection entries. After parsing Intel's multiprocessor table and examinig the result I'm now even more stumbled:

First, what is Windows reporting? It could be reporting which I/O APIC input each device is connected to, but it could be reporting some sort of "global IRQ" number that it made up and it may have changed things from default anyway.

From your table, if the MP Specification tables are saying that I/O APIC input 0 is for EXTint (i.e. it's connected to the master PIC's interrupt output line), then there's no way that IRQ 0 would be the PIT (or sysclock). In this case you'll probably find that ISA IRQ0 is connected to the I/O APIC's input 2 (but it is possible for the PIT to be not connected at all).

gaf wrote:Wouldn't it however be way more logical to give the AGP card an exclusive IRQ line and have two of the USB controllers (which are way less important anyway) share a single line ? Apart from that IRQs #20 and #21 are not used at all. Are there some hardware restrictions that keep PCI from using these lines or is my computer just poorly set-up ?

There are restrictions - each "PCI host controller" can only use up to 4 IRQs. The MP Specification tables will tell you which bus each IRQ source is from. I'd assume you've got 2 PCI host controllers. This would make "PCI bus 0" the first host controller and "PCI bus 1" the second host controller, where any other PCI buses are behind "PCI to PCI" bridges.

In addition to this, newer motherboards (and some PCI devices) support "Message Signalled Interrupts", where a PCI device sends the IRQ directly to local APIC/s (without going through the PIC chips or the I/O APIC).

gaf wrote:I'm also wondering why the SMBus Controller gets mapped to an interrupt that is normally used by ISA devices..

Normal PCI interrupts are "hard-wired" to I/O APIC inputs. It is possible to leave the PIC enabled and use "mixed mode" though, and the SMBus is a twisted thing (it can generate an SMI instead of a PCI IRQ). I'd guess that as long as no ISA device is using the IRQ then it's free to use for other things.

BTW Windows XP says my SMBus controller actually uses IRQ 3, but without "mixed mode" I can't figure out how this is possible (although I haven't read the chipset documentation properly)...

gaf wrote:On an other computer (-that uses some rather cheap board-) the multiprocessor table isn't even complete. There are only 10 entries in it, while windows enumerates 15 IRQs in its hardware manager. What's missing are almost all legacy ISA devices except for PIT, keyboard and some mysterious IRQ #2 which doesn't correspond to any device window uses. I'm wondering how other operating systems actually detect these devices, after all they'll hardly be in the ACPI tables either..
While there are too few ISA enties, all PCI are enumerated correctly. Actually there's even one entry too much, but that hardly matters in that case.

That doesn't add up to me (2 ISA + 1 strange + 7 PCI = 10?). I'll bet the mysterious IRQ #2 is marked as "EXTint" too.

BTW I have seen computers where all of the "I/O APIC interrupt assignment entries" aren't in one group - for e.g. you could have 10 of them followed by a few "local APIC interrupt assignment entries", followed by more "I/O APIC interrupt assignment entries"...

gaf wrote:With all this I was wondering how reliable the multiprocessor table really is. It's not only incomplete on some computers, but also only preliminary as a lot of IRQs obviously have to be remapped first. Wouldn't it be more resonable to dump the whole table in favor on some PCI bus specific detection routines ? I'm however not sure what to do with the ISA devices then..

The motherboard/chipset can happily connect anything it likes to any I/O APIC input. If they get bored, I/O APIC input 22 might be the ISA PIT, input 17 might be the keyboard and input 8 might be your video card. Usually motherboard and chipset manufacturers aren't this chaotic, but in general there's only 4 ways to determine what is connected where. The first 2 ways are MP specification and ACPI tables.

The next way is chipset specific drivers. The last way is "probing" - make sure nothing can generate an IRQ, then trigger an IRQ on a specific device and see which IRQ is received. I wouldn't recommend these last 2 methods though..


Cheers,

Brendan

Hello Brendan,
thanks for your fast answer..

Brendan wrote:First, what is Windows reporting? It could be reporting which I/O APIC input each device is connected to, but it could be reporting some sort of "global IRQ" number that it made up and it may have changed things from default anyway.

Good point. Do you know any tool to get more reliable results ? I've already tried your kernel but it doesn't seem to print very much about the detected devices.

Brendan wrote:From your table, if the MP Specification tables are saying that I/O APIC input 0 is for EXTint (i.e. it's connected to the master PIC's interrupt output line), then there's no way that IRQ 0 would be the PIT (or sysclock). In this case you'll probably find that ISA IRQ0 is connected to the I/O APIC's input 2 (but it is possible for the PIT to be not connected at all).

If legacy devices are not always assigned their traditional IRQ number on the I/O APIC, but may occure in a completely different order, I'm in some trouble. How am I supposed to know which of the 16 ISA entries is the ps/2 mouse, which the keyboard and which the floppy controller ? The multiprocessor table doesn't tell me anything about that and, as the latter 2 methodes you mentioned (chipset driver, probing) aren't really an option for me either, I'll probably end up rolling dices or possibly asking a completly clueless user..

Brendan wrote:There are restrictions - each "PCI host controller" can only use up to 4 IRQs. The MP Specification tables will tell you which bus each IRQ source is from. I'd assume you've got 2 PCI host controllers. This would make "PCI bus 0" the first host controller and "PCI bus 1" the second host controller, where any other PCI buses are behind "PCI to PCI" bridges.

Hmm, I though that the PCI bus numbers weren't that important so that I didn't post them:
[pre]IRQ 16 -> PCI0, PCI1 (=>VGA)
IRQ 17 -> PCI0
IRQ 18 -> PCI0, PCI0
IRQ 19 -> PCI0
IRQ 22 -> PCI2 (=>Ethernet)
IRQ 23 -> PCI0[/pre]If I got it right almost all devices (including onboard sound, ATA/ATAPI, etc) are connected to pci bus 0, while the graphic-card and the onboard ethernet both have a bus of their own (pci1 = agp, pci2 = ethernet). All of the USB controllers (4*UHCI + EHCI 2.0) are on pci0, so that there's no need to move one of the inputs of IRQ #18 to IRQ #16. In fact this means that the pci0 needs an additional IRQ, so that it now uses a total of 4, which is the maximum as you said. Without remapping this IRQ, pci0 would only use 3 IRQs, and if IRQ #16 was used exclusivly by the graphic-card (remapping the USB controller from #16 to #19), the total amount of IRQs on the bus would only be 2. In that case ASUS wouldn't even have needed to add yet another pci bus bridge to their mainboard to get the ethernet working..

I've attached the output of a pci bus probing tool (Craig Hart's PCI32) to the post in case that you need some details.

--snip---

Brendan wrote:That doesn't add up to me (2 ISA + 1 strange + 7 PCI = 10?). I'll bet the mysterious IRQ #2 is marked as "EXTint" too.

Actually there are only 6 PCI entries. Two of the higher PCI IRQs aren't used and # 16 is mapped to two buses:[pre]o----o--------------o------o-------o------o-------o
|irq | windows says | Type | Bus | Pol | Edge |
o----o--------------o------o-------o------o-------o
| 00 | sysclock | EXT | ISA | std | std |
o----o--------------o------o-------o------o-------o
| 01 | kbd | INT | ISA | std | std |
o----o--------------o------o-------o------o-------o
| 02 | | INT | ISA | std | std |
o----o--------------o------o-------o------o-------o
| 03 | | | | | |
o----o--------------o------o-------o------o-------o
| 04 | COM1 | | | | |
o----o--------------o------o-------o------o-------o
| 05 | | | | | |
o----o--------------o------o-------o------o-------o
| 06 | FDC | | | | |
o----o--------------o------o-------o------o-------o
| 07 | | | | | |
o----o--------------o------o-------o------o-------o
| 08 | RTC | | | | |
o----o--------------o------o-------o------o-------o
| 09 | ACPI | | | | |
o----o--------------o------o-------o------o-------o
| 10 | | | | | |
o----o--------------o------o-------o------o-------o
| 11 | | | | | |
o----o--------------o------o-------o------o-------o
| 12 | Mouse | | | | |
o----o--------------o------o-------o------o-------o
| 13 | CoProcessor | | | | |
o----o--------------o------o-------o------o-------o
| 14 | Prim ATA | | | | |
o----o--------------o------o-------o------o-------o
| 15 | Sec ATA | | | | |
o----o--------------o------o-------o------o-------o
| 16 | VGA | INT |PCI 0,1| low | level |
o----o--------------o------o-------o------o-------o
| 17 | | | | | |
o----o--------------o------o-------o------o-------o
| 18 | Sound | INT | PCI 0 | low | level |
o----o--------------o------o-------o------o-------o
| 19 | | INT | PCI 0 | low | level |
o----o--------------o------o-------o------o-------o
| 20 | PCI-USB | INT | PCI 0 | low | level |
o----o--------------o------o-------o------o-------o
| 21 | PCI-USB | INT | PCI 0 | low | level |
o----o--------------o------o-------o------o-------o
| 22 | | | | | |
o----o--------------o------o-------o------o-------o
| 23 | USB Contr. | INT | PCI 0 | low | level |
o----o--------------o------o-------o------o-------o[/pre]
My guess is that #19 is the motherboard's onboard ethenet card, which I however disabled in windows as it's not needed. I right now don't remember whether it's activated in the BIOS though.
The other mysterious IRQ (apart from #2) is the second input to #16 from pci bus 0. At the moment I can't access this computer, but I'll run the PCI probing tool on it as soon as possible to find out more about it..

Brendan wrote:BTW I have seen computers where all of the "I/O APIC interrupt assignment entries" aren't in one group - for e.g. you could have 10 of them followed by a few "local APIC interrupt assignment entries", followed by more "I/O APIC interrupt assignment entries"...

That shouldn't be a problem as my code doesn't rely on the entries to be continuous. I'll nevertheless check the entry number declared in the mp header agains my counted entry number to make sure that I didn't miss anything.

Brendan wrote:The motherboard/chipset can happily connect anything it likes to any I/O APIC input. If they get bored, I/O APIC input 22 might be the ISA PIT, input 17 might be the keyboard and input 8 might be your video card.

Ok, so the PCI bus can only tell me that device X uses PCI IRQ Y ? I was just hoping that the bus would know to which I/O Apic pins its interrupts are connected.

regards,
gaf

If I may ask, where did you get all your IO apic information? I searched for it some time ago and I could find loads of APIC information but barely any IO APIC information...

Hi,

I've been analyzing the output from Craig Hart's PCI+AGP bus sniffer...

[pre]HOST (0:00:0)
|__PCI-to-AGP (0:01:0)
| |__Video (1:00:0, INT#A, IRQ 16)
| |__Video (1:00:1)
|__USB #1 (0:29:0, INT#A, IRQ 16)
| |__USB #2 (0:29:1, INT#B, IRQ 18)
| |__USB #3 (0:29:2, INT#C, IRQ 18)
| |__USB #4 (0:29:3, INT#A, IRQ 16)
| |__USB #5 (0:29:7, INT#D, IRQ 23???)
|__PCI-to-PCI (0:30:0)
| |__Ethernet (2:05:0, INT#A, IRQ 22)
|__PCI-to-LPC (0:31:0)
|__EIDE (0:31:1, no PCI INT - uses ISA IRQ)
|__SMBus (0:31:3, INT#B, IRQ 11???)
|__Sound (0:31:5, INT#B, IRQ 17)[/pre]

From the datasheets for this chipset (http://www.intel.com/design/chipsets/84 ... tation.htm), the I/O APIC pin mapping goes like this:

0 = Cascade from 8259/PIC (should be "EXTint")
2 = 8254 counter 0, HPET #0 (legacy mode)
8 = RTC, HPET #1 (legacy mode)
9 = option for SCI, TCO
10 = option for SCI, TCO
11 = HPET #2 (legacy mode), option for SCI, TCO
13 = FERR# (FPU error in legacy mode)
14 = IDE/SATA primary (legacy mode)
15 = IDE/SATA secondary (legacy mode)
16 = PIRQA# = USB controller #1 (0:29:0), USB controller #4 (0:29:3)
17 = PIRQB# = Sound (0:31:5), modem, option for SMBus
18 = PIRQC# = USB controller #3 (0:29:2), IDE/SATA (native mode)
19 = PIRQD# = USB EHCI controller #2 (0:29:7)
20 = PIRQE# = LAN, option for SCI, TCO, HPET #0,1,2
21 = PIRQF# = option for SCI, TCO, HPET #0,1,2
22 = PIRQG# = option for SCI, TCO, HPET #0,1,2
23 = PIRQH# = USB EHCI controller #5 (0:29:7)

There's 2 things that strike me as completely unusual here. The first thing is that the USB EHCI controller is using IRQ 23 when it's on the wrong bus for it. Intel's datasheet states:

Interrupt pin - RO. The value of 04h indicates that the EHCI function within the Intel ICH5's multi-function USB device will drive the fourth interrupt pin from the device- INTD# in PCI terms. The value of 04 in function 7 is required because the PCI Local Bus Specification, Revision 2.3 doesn't recognize more than 4 interrupts and older APM-based OSs require that each function within a multi-function device has a different Interrupt Pin Register value.

NOTE: Internally the EHCI controller uses PIRQH#.

IMHO this basically means that it doesn't comply with PCI specifications and Intel have internally connected it to something they shouldn't.

For the SMBus, I dug a little deeper. Intel's documentation states that it's optional on PIRQB# (or IRQ 17) which is where I'd expect it to be. Alternatively, it's possible to make the SMBus generate an SMI or a "TCO Interrupt" ("Total Cost of Ownership"). Your system has ACPI enabled so the SMI becomes an SCI which is connected to IRQ 9. This means Windows must be using the TCO interrupt (as SMBus isn't using SMI/SCI). The TCO interrupt is routed to the I/O APIC and/or PIC using completely different methods - the chipset datasheet has a "TCO_CNTL - TCO Control Register" which allows the TCO to be routed to IRQ9, IRQ10, IRQ11, IRQ20, IRQ21, IRQ22 or IRQ23. Basically, SMBus isn't using any PCI interrupt line...


Cheers,

Brendan

Hi,

Candy wrote:If I may ask, where did you get all your IO apic information? I searched for it some time ago and I could find loads of APIC information but barely any IO APIC information...

You can still get the I/O APIC datasheet/s from Intel (http://www.intel.com/design/chipsets/specupdt/290710.htm?iid=PCG+devside&). This is "old" though - for modern chipsets the I/O APIC is built into the "PCI to LPC bridge", and is not a seperate chip...


Cheers,

Brendan

Hello,
from your listing most of the ISA legacy devices obviously aren't connected to the I/O APIC. As I however do need access to (at least) the ps/2 devices, RTC (to initialize the apic timer), floppy and IDE, I will probably end up using mixed mode. This sure is ugly as I originally intended to avoid relying on the PIC, but as the only other option is to use polling for these devices, I don't see any other way.

I was also wondering if the FERR# IRQ is still used today. After all there's also the floating point exception and I don't really see why an internal FPU should use an IRQ.
The same goes for the two legacy IDE channels. According to your listing there's also a native IRQ (#18) but I'm not quite sure if this means that I can ignore the ISA interrupts or if the new interface only works for SATA.

thanks,
gaf

Hi,

gaf wrote:from your listing most of the ISA legacy devices obviously aren't connected to the I/O APIC. As I however do need access to (at least) the ps/2 devices, RTC (to initialize the apic timer), floppy and IDE, I will probably end up using mixed mode. This sure is ugly as I originally intended to avoid relying on the PIC, but as the only other option is to use polling for these devices, I don't see any other way.

Ah - no. AFAIK the "Intel 82801EB I/O Controller Hub 5 (ICH5)/Intel 82801ER I/O Controller Hub 5 R (ICH5R)" doesn't have a floppy drive controller or a PS/2 (keyboard/mouse) controller. I would assume they aren't included in this chip so that it's easy to make a computer with USB keyboard & mouse and no floppy drive.

I'd also assume these devices may be supported on a seperate chip (if necessary). If they are present they would/should be connected to both the PIC and I/O APIC and reported by the MP specification tables.

Basically, I didn't list them because the datasheet I was reading didn't show them (because they're not part of that chip)...

The RTC and PIT (or 8254 counter 0) were reported at I/O APIC inputs 0 and 8. It's also nice that your chipset has a built-in HPET (High Performance Event Timer)...

gaf wrote:I was also wondering if the FERR# IRQ is still used today. After all there's also the floating point exception and I don't really see why an internal FPU should use an IRQ.

FERR# is supported for legacy purposes (DOS?). All modern OS's set the "NE" flag in CR0 which disables the use of FERR# and enables exception #16. The IRQ line is still useless though...

gaf wrote:The same goes for the two legacy IDE channels. According to your listing there's also a native IRQ (#18) but I'm not quite sure if this means that I can ignore the ISA interrupts or if the new interface only works for SATA.

I'm not sure about the native IDE IRQ - I'd assume that if Windows isn't using it then it's either not worth using or Intel specific...


Cheers,

Brendan

<pypebot>
adding this thread to the "tell me about APIC" page of the FAQ.
</pypebot>