Question about virtual wire mode

[doxrobot]

If at any time during my question you find that my interpretation is flawed please point out my mistake so I can correct it, thank you.

On a certain machine my Test (hobby) OS runs on there is no IMCR (established via IMCRP/MP tables) so I assume either LINT0 or LINT1 is set to ExtInt for backwards compatibility and usage of 8259 PIC.

My question is that, isn't the local APIC disabled at the time bios maps and jumps to my bootloader thus if it were ExtInt interrupts are not accepted? Or is the APIC already enabled?

At which point since there is no IMCR, all one would need to do after parsing interrupt source overrides and getting interrupt inputs is unmask IOAPIC entries and good to go?

I'm just a little confused at this point, maybe I cant read, maybe I missed it, maybe I need better documentation... Someone nudge me in the right direction. Thanks

[Brendan]

Hi,

On a certain machine my Test (hobby) OS runs on there is no IMCR (established via IMCRP/MP tables) so I assume either LINT0 or LINT1 is set to ExtInt for backwards compatibility and usage of 8259 PIC.

An IO APIC input could also be configured as ExtInt; such that the master PIC's interrupt line is connected to an IO APIC input and the IO APIC forwards to the CPU.

My question is that, isn't the local APIC disabled at the time bios maps and jumps to my bootloader thus if it were ExtInt interrupts are not accepted? Or is the APIC already enabled?

The local APIC may be "software disabled" (where it still responds to things like NMI) or it may be enabled with almost everything masked.

At which point since there is no IMCR, all one would need to do after parsing interrupt source overrides and getting interrupt inputs is unmask IOAPIC entries and good to go?

You'd:

• determine if PIC chips exist using the ACPI tables (assume they do exist if there are no ACPI tables)
• if PIC chips exist:
  • remap the PIC chips (because it's impossible to prevent them from sending spurious IRQs, even when you aren't using them)
  • mask everything in the PIC chips
• determine if local APICs exist using ACPI tables (or MP Spec tables if there are no ACPI tables)
• if local APICs exist:
  • initialise them by setting TPR, logical APIC ID, spurious vector, etc; and masking normal local APIC interrupts (local APIC timer, thermal status, performance monitoring)
  • if ACPI exists do what any "local APIC input override structures" in the ACPI tables tell you to do. If the ACPI tables don't say anything about one or more local APIC inputs (e.g. LINTIN1) then don't touch it. If there isn't any ACPI then don't touch LINTIN0 or LINTIN1.
  • determine if IO APIC/s exist using ACPI tables, or MP Spec tables if there are no ACPI tables
  • if there are no IO APIC/s you have to use PIC chips (but can still use local APIC)
  • if there are IO APIC/s:
    • do the IMCR thing if MP Specs say there is one (on systems that support ACPI there won't be an IMCR)
    • mask all of the IO APIC's inputs (for each IO APIC)
    • parse the ACPI (if present) or MP Spec (if ACPI isn't present) tables and figure out how to setup IO APIC inputs. You'd just keep this information in a table in RAM (there's no need to configure each IO APIC input)
    • setup an IDT with exception handlers and handlers for the PIC chip's IRQ 7 and IRQ 15 (the PIC chip's spurious IRQs). Note that the PIC chip's IRQ 15 handler still needs to send EOI to master PIC; and you might need different IRQ handlers (one for the IO APIC case where you're only handling spurious IRQs from PIC, and one for the PIC case where you need to determine if the IRQ is spurious or not).

Note: I'd recommend doing all the parsing/checking first; then do all the masking/configuring after.

Once all this is done all IRQs are masked in the PIC chips (if present), IO APIC/s (if present) and local APIC (if present), and everything is ready to go. You can enable IRQs in the CPU now ("STI").

After all this; when you're installing device drivers you'd initialise/reset the device; then (based on the information you gathered earlier) install the corresponding IRQ handler in the IDT, and finally either unmask the IRQ in the PIC, or configure and unmask the IRQ in the IO APIC, or configure the device's MSI in PCI configuration space.


Cheers,

Brendan

[doxrobot]

This is very helpful Brendan thank you. I was a little unsure how to handle the 8259 PIC. Also, when I am initializing the local APIC on my AP's do you know if an AP can receive a SIPI after it executes a CLI then a HLT?

From my interpretation after a hard reset this is how the AP's wait. I've also read in the MP specification that it's also possible to use the warm reset vector (assuming BIOS obviously, not EFI/UEFI) and just issue an INIT?

Will an AP wake up to a SIPI if it has it's IF flag clear?

Thanks again.

[Brendan]

Hi,

This is very helpful Brendan thank you. I was a little unsure how to handle the 8259 PIC. Also, when I am initializing the local APIC on my AP's do you know if an AP can receive a SIPI after it executes a CLI then a HLT?

I don't know what effect a SIPI would have on a normally running CPU - I'd assume "undefined behaviour" (likely that the CPU simply ignores the SIPI but different CPUs might do different things).

"STI; HLT" puts a CPU into a wait state where any interrupt can wake it up. I don't know what effect a SIPI would have on the CPU in this state either; but I'd assume it'd have the same effect as it would on a normally running CPU. There's also a "MWAIT" state (which is similar to the "HLT" wait state except a write to the specified cache line can wake it up too).

"CLI; HLT" puts the CPU into a halted state where it can only respond to INIT, SMI and NMI. In this state the CPU will ignore a SIPI. Note that because of the possibility of SMI and NMI I'd want a loop (e.g. "CLI; HLT; jmp do_it_forever") and I'd want to make sure nothing moved these instructions (e.g. you don't want an SMI to occur when the "CLI; HLT" code has been paged to disk or something). Note: If your OS is taking a CPU offline, it's wise to disable caches and then clear the caches before doing the "CLI;HLT").

The INIT is like a CPU reset. It's capable of taking the CPU out of any state, and puts the CPU into a special "wait for SIPI" state. This is the only state where a SIPI makes sense. If a CPU is in the "wait for SIPI" state (and possible only if it's in the "wait for SIPI" state) the SIPI will tell it where to start executing.

Note: All of the above only applies to CPUs with a built-in local APIC (e.g. Pentium or later).

From my interpretation after a hard reset this is how the AP's wait. I've also read in the MP specification that it's also possible to use the warm reset vector (assuming BIOS obviously, not EFI/UEFI) and just issue an INIT?

For CPUs with an external local APIC (e.g. 80486) there's an "INIT_assert" and an "INIT_deassert" that make the CPU reset, but there isn't any SIPI - the CPU just starts running at "0xFFFF000:0xFFF0" instead. In this case the firmware's code (at 0xFFFFFFF0) has to check if it's the BSP during power on/reset or if it's an AP being started (either by the BIOS or by an OS). For this reason there's a warm reset vector that tells the firmware's code what the recently started CPU should do. Of course this isn't worth caring about anymore - SMP was rare 20+ years ago when 80486 was new and almost all of those few rare machines have died and/or been replaced and thrown away now.


Cheers,

Brendan

[doxrobot]

So in most newer processors an INIT essentially just causes it's target to enter a wait-for-SIPI state? The firmware/baseboard code at the reset vector (0xFFFFFFF0) is not executed?

[Brendan]

Hi,

So in most newer processors an INIT essentially just causes it's target to enter a wait-for-SIPI state? The firmware/baseboard code at the reset vector (0xFFFFFFF0) is not executed?

In all processors with inbuilt local APICs (e.g. Pentium or later); INIT resets most of the CPU's state (e.g. registers but not MTRRs) and puts the CPU into the "wait for SIPI" state; and the firmware/baseboard code at the reset vector (0xFFFFFFF0) is only executed during power-on or hard reset (and even then it might only be executed by the BSP and not executed by AP CPUs).


Cheers,

Brendan