Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Octocontrabass wrote:I think you have missed something. MSI-X allows you to specify a different address for each message, and x86 uses 8 bits of the address to specify an APIC ID, so you usually can send each interrupt to a different processor without extra hardware.
Oh, I'll look into MSI-X again -- I probably did miss something. Oops? (And doesn't the APIC become 24 bits or something like that in... Cluster mode, I think? I don't remember, but I remember the Intel manual saying something about the APIC ID being larger in a certain mode.)
rdos wrote:many PCI interrupts are shared among multiple devices which makes them unattractive.
Your always going to have interrupt sharing, regardless of what method you use. The architecture supports far too few interrupts to support every possible device, and things like the IOAPIC are just hacky workarounds to try to solve this problem. This is especially aggravating given that we're starting to see more and more devices being able to allocate over a thousand MSI-X interrupt entries to themselves (NVMe is a good example, since every I/O queue can have its own interrupt allocated to it) and then we have to squish them into a single tiny set of interrupts. I think that RISC-V has (sort-of) solved this problem, but I don't know how much. And I suspect that the only reason Intel hasn't made the IDT support 2^32 interrupts is because it would break existing OSes. Even if doing so would be a very, very good idea, IMO, and would support the (probably impossible) case of 16777216 devices in PCIe, and would support practically every kind of computer setup for centuries to come, if not longer.
I think designs that require 1000s of interrupts are poor. Some 200 interrupts per CPU core should be enough. Just because some new PCI designs allow a large number of interrupts doesn't mean it is a good idea to use them. It's often better to read out the interrupt state from the device instead, unless it is highly loaded and the load is distributed to several cores.
nullplan wrote:You know, I'd like the exact opposite of that: Could we please take a page from PowerPC and get a way to receive all external interrupts at a single place, then maybe push an identifier to the stack?
Ethin wrote:And doesn't the APIC become 24 bits or something like that in... Cluster mode, I think? I don't remember, but I remember the Intel manual saying something about the APIC ID being larger in a certain mode.
APIC IDs are 32 bits in x2APIC mode. You need to use the IOMMU if you want 32-bit APIC IDs with MSI or MSI-X.
Ethin wrote:And I suspect that the only reason Intel hasn't made the IDT support 2^32 interrupts is because it would break existing OSes. Even if doing so would be a very, very good idea, IMO, and would support the (probably impossible) case of 16777216 devices in PCIe, and would support practically every kind of computer setup for centuries to come, if not longer.
You know, I'd like the exact opposite of that: Could we please take a page from PowerPC and get a way to receive all external interrupts at a single place, then maybe push an identifier to the stack? Because right now, I already have two-hundred twenty-four interrupt handler routines that are all doing basically that in software. This also can't be a big problem in the CPU as it already knows which interrupt is being invoked, and it already has a way to push things on stack. (It also can't be a big problem to make the new feature switchable with a bit in, say, EFER. Or make a new MSR if you must.)
Or to make it short, I want the interrupt number as data, not as a code address.
For those curious, on PowerPC, all exceptional conditions end up at a certain address defined in the architecture, and "external interrupt" is one such condition. All external interrupts go to address 0x700 (physical). To find out which device interrupted you, you have to ask the PIC, thus making the interrupt number available as data. The architecture only supports this one interrupt level, and a given machine could have any PIC implementation desired, supporting however many interrupts you want.
Now that's a beautiful idea. I imagine it would minimize code by quite a bit. My Rust kernel has all interrupts -- all 256 of them -- defined because I wasn't sure how to dynamically tweak the IDT back when I did that, and if x86 did this I could just do a single handler that dispatches to other functions depending on what the interrupt really is.
nullplan wrote:You know, I'd like the exact opposite of that: Could we please take a page from PowerPC and get a way to receive all external interrupts at a single place, then maybe push an identifier to the stack?
Was this actually ever implemented in any processors/microcode updates? I remember it being discussed on here a while back as a "possible architecture extension" (if I remember right) but I don't think I ever heard if Intel actually went through with it.
Octocontrabass wrote:
Ethin wrote:And doesn't the APIC become 24 bits or something like that in... Cluster mode, I think? I don't remember, but I remember the Intel manual saying something about the APIC ID being larger in a certain mode.
APIC IDs are 32 bits in x2APIC mode. You need to use the IOMMU if you want 32-bit APIC IDs with MSI or MSI-X.
