Enable local-apic interrupts?

[Brendan]

Hi,

Is this fault really because of HLT instructions and the dealing of Bochs with them? I don't think so because there is also the instructions-fault, which I can not comment out.
And this is only a warning and should work even if Bochs doesn't like HLT with IF=0...

To be honest I'm not sure what the problem is - I never get the "unknown instruction" warning (I do get the "HLT with IF=0" problem though).

For "HLT with IF=0", if a CPU disables interrupts and does HLT then the only things that can make it continue are a reset, INIT IPI, NMI or SMI. For single-CPU systems Bochs detects this and (correctly) does a warning, as it's impossible for anything to continue. For multi-CPU this isn't the case - one CPU can do HLT with IF=0 and another CPU can send an INIT IPI to restart the halted CPU.

My BIOS auto-detects the number and type of CPUs that are present. To do this it starts all CPUs, checks how many started, etc, but then needs to stop them running (that's why my BIOS uses HLT with IF=0). If the OS supports SMP it'll issue an INIT IPI to start the halted CPUs when it's ready (as part of the INIT-SIPI-SIPI startup sequence).

The normal Bochs BIOS doesn't work like this - it doesn't know anything about the MPS tables (the emulator generates them before the BIOS is started), which is why the fields for CPU features and CPU signature in the default MPS tables are wrong, and also why you don't get the "HLT with IF-0" problem with the normal Bochs BIOS.

I'm currently downloading the pre-compiled "bochs-2.3.5.exe" from the Bochs web site to see where the invalid opcode problem is coming from. I'll post again after doing some testing...

[EDIT]I downloaded the pre-compiled version of Bochs and installed it on a Windows XP machine here. My BIOS ran without any problems for single-CPU but I couldn't test if SMP worked or not because the pre-compiled version doesn't support SMP at all - I get a ">>PANIC<< numerical parameter 'n_processors' was set to 2, which is out of range 1 to 1".[/EDIT]


Cheers,

Brendan

[nooooooooos]

Yeah, but there is another file of this version, compiled for SMP, called "bochs-smp-2.3.5-win32.zip" at the bottom of the List in sourceforge...

EDIT: There is the hole Bochs-Log-part of the unkown instruction fault:

Code: Select all

 BxError: instruction with opcode=0xff
 mod was c0, nnn was 7, rm was 7
 WARNING: Encountered an unknown instruction (signalling illegal instruction)

Thx
Nooooooooooos

[Brendan]

Hi,

Yeah, but there is another file of this version, compiled for SMP, called "bochs-smp-2.3.5-win32.zip" at the bottom of the List in sourceforge...

Doh - I missed that..

I've downloaded the SMP version of Bochs for Windows and tried it out - everything worked fine (no invalid opcode or HLT with IF=0). There was a "SMI still not implemented" error for each additional CPU core, but I know about that (just click "continue" - it is implemented if the compile time option is enabled, and my BIOS autodetects whether or not SMI support was enabled or not).

EDIT: There is the hole Bochs-Log-part of the unkown instruction fault:

Code: Select all

 BxError: instruction with opcode=0xff
 mod was c0, nnn was 7, rm was 7
 WARNING: Encountered an unknown instruction (signalling illegal instruction)

Could you post the entire Bochs log and your "bochsrc.txt", or email them to me ([email protected])?



Cheers,

Brendan

[nooooooooos]

How, I know what the reason of this strange problem is...

It's Code from my OS which changes the Shutdown-Byte in the CMOS (to start the APs). And something is wrong at this Code, so that the Shutdown-Byte is also wrong at the next Start of Bochs...(which makes it very complicate to debug especially with a Debugger )

That is my Code before the setting of the AP...

Code: Select all

	mov al,0xF
	out 0x70,al
	nop
	nop
	nop
	in al,0x71
	mov [Statusbyte],al

	nop
	nop
	nop

	mov al,0xF
	out 0x70,al
	nop
	nop
	nop
	mov al,0xA
	out 0x71,al

...and here is the CMOS-Code after my AP-enable code, which should set the Statusbyte to the value, that it had before...

Code: Select all

	mov al,0xF
	out 0x70,al
	nop
	nop
	nop
	mov al,[Statusbyte]
	out 0x71,al

Is there anywhere a problem with this code?


Cheers
Noooooooooos

[Brendan]

Hi,

How, I know what the reason of this strange problem is...

Ahh - good

It's Code from my OS which changes the Shutdown-Byte in the CMOS (to start the APs). And something is wrong at this Code, so that the Shutdown-Byte is also wrong at the next Start of Bochs...(which makes it very complicate to debug especially with a Debugger )

First, there's nothing really wrong with the code you've posted.

For my BIOS, the BIOS itself will restore the CMOS reset byte if it's used. I'm not sure if this is what a BIOS should do, but it prevents infinite reset loops (for e.g. if a CPU triple faults before the OS changes the CMOS reset byte back, then it can endlessly triple fault). This means that (at least for my BIOS, in theory) you'd need to set the CMOS reset byte to 0x0A for each AP CPU you start, but...

Unless the local APIC is an old 82489DX chip you don't need to change the CMOS reset byte - the address that the AP CPU starts executing at is determined by the SIPI you send and the BIOS (and CMOS reset byte) aren't used. Bochs can't emulate an old 82489DX chip, so the CMOS reset code is never needed for AP startup on Bochs.

BTW a previous version of my OS tried to support the 82489DX (and multi-CPU 80486), but I couldn't find any way to test the code. I spent about 6 months hoping that a multi-CPU 80486 would show up on eBay before I gave up. Now I just display a warning (e.g. "This computer uses 82489DX chips - booting as a single CPU system").


Cheers,

Brendan

[nooooooooos]

OK...Is the 82489DX only used for 80486 multicore cpus?

[Brendan]

Hi,

OK...Is the 82489DX only used for 80486 multicore cpus?

Mostly yes, however I do have a strange feeling that they may have also been used for a few (very early/rare) Pentium systems.

For Pentium and later CPUs the local APIC is built into the CPUs (and disabled by the BIOS in some older single-CPU systems). I'm not sure what 80386 did (or if anyone ever made a multi-CPU 80386 system), but they came before Intel's Multiprocessor specification and would have been very non-standard.


Cheers,

Brendan

[nooooooooos]

Thx...Are there any single-processor-systems that are configured without an IO-Apic? Or are all single-processor-systems configured without an IO-Apic?

Cheers
Nooooooooooos

[Brendan]

Hi,

Thx...Are there any single-processor-systems that are configured without an IO-Apic? Or are all single-processor-systems configured without an IO-Apic?

Most Pentium III and later single-CPU systems don't have I/O APICs, but some might (it's up to the motherboard manufacturer to decide if they want to include it or not).

Most Pentium 4 and later single-CPU systems do have I/O APICs. This is partly because most Pentium 4 motherboards are designed to support hyper-threading (and more recently, multi-core), so it makes sense for the BIOS to allow the I/O APIC to be used when the CPU itself doesn't support hyper-threading or multi-core. However, some older Pentium 4 motherboards (and motherboards for non-Intel CPUs) might not have I/O APICs.

In general, if a single-CPU system does have an I/O APIC then it will also have MPS tables and/or ACPI. If the BIOS doesn't provide either of these then an OS can't detect if an I/O APIC is present or not, and there'd be no reason for motherboard manufacturer to include the I/O APIC.

The same generally applies to local APICs. Even though Pentium and later CPUs have them, the BIOS can permanently disable the local APIC, and on most Pentium III and older single-CPU systems the BIOS does permanently disable the local APIC. This is a seperate issue though - for example a BIOS might leave the local APIC enabled and still not have an I/O APIC.

Lastly, for some CPUs the local APIC can't be permanently disabled. In this case an OS might be able to re-enabled the local APIC (even though the BIOS is trying to pretend it doesn't exist), but you'd need to be very careful in this case to avoid conflicts with memory mapped devices (e.g. video display memory) and to ensure that the area isn't cached. Re-enabling the local APIC (where possible) may or may not be worth the hassle - the only advantage would be access to the local APIC timer.


Cheers,

Brendan

[nooooooooos]

And when I want to support Hyper-Threading? Do I use the Shutdown Code then? Because now I can find the other part of the processor, but it wan't get enabled...

Cheers
Noooooooooooos

[Brendan]

Hi,

And when I want to support Hyper-Threading? Do I use the Shutdown Code then? Because now I can find the other part of the processor, but it wan't get enabled...

For starting CPUs, hyper-threading works the same (always "INIT, SIPI, SIPI" as the CPU is always "Pentium or newer").

The only difference is that the MPS tables won't list additional logical CPUs (e.g. for a dual core chip with hyper-threading you'd get 2 CPUs listed by MPS), while the ACPI tables will (e.g. for a dual core chip with hyper-threading you'd get 4 CPUs listed by ACPI).


Cheers,

Brendan

[nooooooooos]

The only difference is that the MPS tables won't list additional logical CPUs (e.g. for a dual core chip with hyper-threading you'd get 2 CPUs listed by MPS), while the ACPI tables will (e.g. for a dual core chip with hyper-threading you'd get 4 CPUs listed by ACPI).

But then how I know the ID of the other part of the processor with? Do both processors have the same ID and a IPI reaches always both "parts of a processor"?

How can I find out with MPS Tables that there is HT? With CPUID?

Thx
Noooooooooos

[Brendan]

Hi,

The only difference is that the MPS tables won't list additional logical CPUs (e.g. for a dual core chip with hyper-threading you'd get 2 CPUs listed by MPS), while the ACPI tables will (e.g. for a dual core chip with hyper-threading you'd get 4 CPUs listed by ACPI).

But then how I know the ID of the other part of the processor with? Do both processors have the same ID and a IPI reaches always both "parts of a processor"?

The other logical CPU in the core has it's own seperate local APIC and it's own local APIC ID.

How can I find out with MPS Tables that there is HT? With CPUID?

You could use MPS to find one logical CPU in each core, and then use CPUID (on the logical CPU you did find) to see if it supports hyperthreading and if there are other logical CPUs in the core. Then you could assume that the additional logical CPUs (if any) have sequentially numbered local APIC IDs.

For example, if MPS says that one CPU has the local APIC ID 0x04, and if CPUID says that this CPU has one additional logical CPU, then you can assume the additional logical CPU has the local APIC ID 0x05.

In general, you should use ACPI tables (if present) to detect the local APIC IDs for other CPUs (if any), even if you use MPS tables for everything else.

There's other things you probably should know here...

For hyper-threading, for some software it makes performance worse, and for some OSs (that aren't designed for hyper-threading) it could cause timing problems because the performance of one logical CPU depends on what work the other logical CPU is doing.

For these reasons most BIOSs have an option to enable/disable hyperthreading, and the end user may have deliberately disabled hyperthreading. If hyper-threading is disabled, then the BIOS won't report additional logical CPUs in the ACPI tables (but that's the only difference). In this case your MPS code would start logical CPUs anyway - the BIOS setting will be ignored, and the end user won't be able to disable hyperthreading if it hurts performance.

Also, if the OS isn't optimized for hyperthreading then it's more likely that using the additional logical CPUs will reduce performance. AFAIK this is why the MPS tables never report the other logical CPUs - they assume that old OSs (that don't use ACPI tables) aren't optimized for hyperthreading and may have timing problems.


Cheers,

Brendan

[nooooooooos]

Thx...now everything works fine!!!
Do I have to optimize my OS especially for HT or is it the same as for Dual-Core?
When I don't want to use the shutdown code, is it possible to skip the sending of the Init-IPI?
My last question....: Does it make sense to support APIC even when there aren't any SMP or ACPI tables?

Cheers Noooooooooos

[TomTom]

Thx...now everything works fine!!!
Do I have to optimize my OS especially for HT or is it the same as for Dual-Core?

HT and Dual-Core from a SMP point of view are the same. However, your scheduler needs to know about HT so it can use it more efficiently. Usually on a SMP system the scheduler should avoid moving tasks between processors (unless the penalty in speed is worth it because one cpu is occupied too much). On HT you should detect how many physical processors exist, and how many logical processors each physical processor has (usually 2). To take advantage of HT you basically allow the scheduler to move processes between logical cpus as this doesn't result in as much overhead as if they were moved between physical processors. The dispatcher shouldn't need any modifications.