Hyper-threading

[kataklinger]

I was thinking about implementing HT support and I realised that I don't need to scan ACPI table to find logical processors, I can look at CPUID flag (all physical processors must be the same in SMP, so if one support HT then all others must support it as well). And now my question: how does BIOS/MP init. protocol assignes local APIC ID?

PP - physical processor, LP - logical processor, ID - local APIC ID

a. PP0 PP1 ....
. LP0 LP1 LP0 LP1 ....
. ID0 ID1 ID2 ID3 ....

or

b. PP1 PP2 PP1 PP2
. LP0 LP0 LP1 LP1
. ID0 ID1 ID2 ID3

or some other way?

This won't be final solution (final code will relay on ACPI), but for beginning.

[Brendan]

Hi,

I was thinking about implementing HT support and I realised that I don't need to scan ACPI table to find logical processors, I can look at CPUID flag (all physical processors must be the same in SMP, so if one support HT then all others must support it as well).

Some notes...

For most (decent) SMP motherboards the BIOS won't report faulty CPUs in the ACPI and/or MP specification tables so that an OS won't try to use these faulty CPUs.

If you don't use ACPI or MP specification tables you won't know how IRQs have been assigned to the I/O APIC/s, the addresses of the local APICs and I/O APICs, if you need to mess with an IMCR to disable the PIC chips, etc. If you assume that the BIOS hasn't changed the address of the local APIC (it'd be unusual if the BIOS did) then you could get it working in "PIC mode" though.

For dual core chips without hyper-threading, the "HT present" feature flag will be set (the "number of logical CPUs" reports the total number of logical CPUs in the chip rather than the number of logical CPUs per core).

There's no guarantee that APIC ID's are sane - for e.g. a system with 4 CPUs and hyperthreading could have APIC IDs that go 0/1, 4/5, 8/9, 12/13 rather than 0/1, 2/3, 4/5, 6/7.

If you don't care about trying to start faulty CPUs, you can send the INIT/SIPI/SIPI sequence as "broadcast to all except self" so that you don't need to know what the APIC IDs are to begin with. Once the other CPUs are running your code you can make them store their APIC IDs in memory somewhere.

And now my question: how does BIOS/MP init. protocol assignes local APIC ID?

AFAIK, it doesn't. When the computer is turned on the CPUs do handshaking to determine their "fixed local APIC IDs". This is the ID you can read from CPUID, which can't be changed by software (unlike the version from the local APIC).

a. PP0 PP1 ....
. LP0 LP1 LP0 LP1 ....
. ID0 ID1 ID2 ID3 ....

APIC ID's inside of a single chip get APIC IDs in order, like you wrote above. For example, for a (made up) chip with 2 cores and 4 logical CPUs per core you'd get:

APIC ID 00: chip0, core0, logical0
APIC ID 01: chip0, core0, logical1
APIC ID 02: chip0, core0, logical2
APIC ID 03: chip0, core0, logical3
APIC ID 04: chip0, core1, logical0
APIC ID 05: chip0, core1, logical1
APIC ID 06: chip0, core1, logical2
APIC ID 07: chip0, core1, logical3

This won't be final solution (final code will relay on ACPI), but for beginning.

IMHO it'd be better to it with ACPI and/or MP specification tables the first time, rather than doing it twice....


Cheers,

Brendan

[durand]

Fortunately, the start up sequence for SMP, dual(or more) core CPU's and hyper-threading is all the same so if you implement HT, you implement the rest. And they all interact using their local APIC - whether they are physical or logical.

According to the Intel docs, the BIOS is supposed to number physical first, down into logical second.. so, according to your diagram:

PP0 PP1
LP0 LP1 LP2 LP3 ..

And that's what I've seen.

I don't think you'll get repeats of logical APIC ID's in a normal system. That would hurt (and wouldn't be logical, haha). However, I could be wrong because it's really up to the BIOS.

I've just recently added MP support to my kernel and it's simple. You just need to allocate resources for each CPU and make sure your locking is good. The boot up sequence and scheduling algorithm change, of course, but that's to be expected.

I don't use the ACPI tables to detect the number of CPU's in a system. I just send a global startup signal and each CPU then boots itself and starts allocating itself its own resources before they start running any scheduled threads.

Be aware that the AP processors all start up in real mode and you have to give them 16 bit code to start executing. So I include a 16 bit trampoline sequence in my kernel which I copy over the old boot sector location (0x7c000) and start each AP processor there. The trampoline code switches each AP processor to protected mode, ensures valid stacks, segments, etc and then jumps into the real kernel code for the rest.

According to the Intel documents (again), a CPU could possibly miss the global APIC start up signal. They have detailed an algorithm in their docs to ensure boot up correctly but I haven't implemented it. I haven't had any problems.

Also, the CPUID is serializing instruction. That means that if your processor issues a CPUID, all of it's pipelines will complete (empty) and they will all wait together for CPUID to be executed. This could impact on performance because you're nullifying the effect of multiple pipelines. So, you should probably only issue CPUID once during boot up and cache the value somewhere instead of using it very often.

Otherwise, it's cool and not as complicated as it sounds if you start with implementing it at the very beginning of your kernel development.

[kataklinger]

My kernel is already SMP aware, and it use MP table to detect physical processors (dual core processors has two enteries in the tabel, but HT has only one). So I'm interested if MP table says that local APIC of CPU has ID 0, that it means logical procesor 1 has ID 0 and logical processor 2 has ID 1. I need to know this because I copy MP table in RAM or if it is default configuration I make the table and want to fix it if CPUs have HT features.

For dual core chips without hyper-threading, the "HT present" feature flag will be set (the "number of logical CPUs" reports the total number of logical CPUs in the chip rather than the number of logical CPUs per core).

But there's number of cores in CPU, so it's easy to know how many logical processors per core you have.

All CPU execut CPUID as a part of init. stage, so if I need some CPUID flag I just get it from memory.

[Brendan]

Hi,

My kernel is already SMP aware, and it use MP table to detect physical processors (dual core processors has two enteries in the tabel, but HT has only one).

Sorry - wasn't sure how much you already supported..

In this case it should be easy - the MP specification tables will give you local APIC ID for the first logical CPU in each core, and any other logical CPUs in the same core will be numbered sequentially starting from the first. For e.g. "first + 1" for the second, "first + 2" for the third, etc (although Intel only make CPUs with 2 logical CPUs per core at the moment).


Cheers,

Brendan

[kataklinger]

Thanks. No need to sey sorry