OSDev.org

Brendan wrote: If the HTT flag is clear then it's an old single-core CPU without hyper-threading. In that case "logical_CPU_bits = 0" and "core_bits = 0" and you're done.

Brendan wrote: decided that Logical APIC ID (which shouldn't be confused with the "APIC ID") and the "Destination Format Register" would be hardwired; and that "Logical Destination

Brendan wrote: For complete CPU topology, there's 4 levels:
Which logical CPU within a core
Which core within a chip
Which chip within a NUMA domain
Which NUMA domain

limp wrote:

Brendan wrote: If the HTT flag is clear then it's an old single-core CPU without hyper-threading. In that case "logical_CPU_bits = 0" and "core_bits = 0" and you're done.

I am afraid; it’s not that simple There are some multi-core processors with no HT support (Intel Pentium D, Intel Pentium Dual-core, Intel Core 2 Duo, Intel Core 2 Quad).

limp wrote:

Brendan wrote: decided that Logical APIC ID (which shouldn't be confused with the "APIC ID") and the "Destination Format Register" would be hardwired; and that "Logical Destination

I guess you meant that LAPIC ID shouldn’t be confused with “ACPI ID” and not APIC ID (as LAPIC ID and APIC ID is the same thing).

limp wrote:

Brendan wrote: For complete CPU topology, there's 4 levels:
Which logical CPU within a core
Which core within a chip
Which chip within a NUMA domain
Which NUMA domain

On the 2 last levels, you’re mentioning NUMA domains. On an SMP system, there is not possibility of having NUMA domains, so I guess what have you described applies only on NUMA system and not on SMP ones.

limp wrote:On an SMP system, there is not possibility of having NUMA domains.

Brendan wrote:

limp wrote:

Brendan wrote: If the HTT flag is clear then it's an old single-core CPU without hyper-threading. In that case "logical_CPU_bits = 0" and "core_bits = 0" and you're done.

I am afraid; it’s not that simple There are some multi-core processors with no HT support (Intel Pentium D, Intel Pentium Dual-core, Intel Core 2 Duo, Intel Core 2 Quad).

It is that simple. All Intel multi-core CPUs set the "HTT" flag, regardless of whether they support hyper-threading or not. Basically that "HTT" flag should be called a "more than one logical CPU in chip" flag.

Brendan wrote:

limp wrote:

Brendan wrote: decided that Logical APIC ID (which shouldn't be confused with the "APIC ID") and the "Destination Format Register" would be hardwired; and that "Logical Destination

I guess you meant that LAPIC ID shouldn’t be confused with “ACPI ID” and not APIC ID (as LAPIC ID and APIC ID is the same thing).

I meant, "logical APIC ID" is extremely different to "local APIC ID". For example, you can set the logical APIC ID to 0x00000001 on all CPUs, and broadcast an IPI to all CPUs that have bit 0 set (using logical destination mode).

Brendan wrote:

limp wrote:

Brendan wrote: For complete CPU topology, there's 4 levels:
Which logical CPU within a core
Which core within a chip
Which chip within a NUMA domain
Which NUMA domain

On the 2 last levels, you’re mentioning NUMA domains. On an SMP system, there is not possibility of having NUMA domains, so I guess what have you described applies only on NUMA system and not on SMP ones.

For an OS designed to support NUMA, it's useful to have a different perspective: all computers are NUMA (just some only have one NUMA domain containing everything).

Combuster wrote:

limp wrote: On an SMP system, there is not possibility of having NUMA domains.

Symmetric multiprocessing essentially comes down to each processor seeing the same memory at the same place. NUMA means that access to memory differs in speed and latency between processing units. Those are not mutually exclusive: if CPU1 sees memory unit 1 at 0G and memory unit 2 at 1G, and CPU 2 sees them at the exact same addresses, SMP is satisfied. If CPU1 controls memory 1 and asks CPU2 for anything in memory 2, whereas CPU2 is connected to memory 2 and asks CPU1 for anything in memory 1, NUMA is satisfied.

In other words: get a multisocket Opteron system

limp wrote:

Combuster wrote:Symmetric multiprocessing essentially comes down to each processor seeing the same memory at the same place. NUMA means that access to memory differs in speed and latency between processing units. Those are not mutually exclusive: if CPU1 sees memory unit 1 at 0G and memory unit 2 at 1G, and CPU 2 sees them at the exact same addresses, SMP is satisfied. If CPU1 controls memory 1 and asks CPU2 for anything in memory 2, whereas CPU2 is connected to memory 2 and asks CPU1 for anything in memory 1, NUMA is satisfied.

In other words: get a multisocket Opteron system

I guess what you're saying is that an NUMA system (like Opteron) satisfies SMP is certain case however, an SMP system (like Intel's multicores) is not possible to satisfy NUMA in any case.

Brendan wrote:

limp wrote:

Brendan wrote: If the HTT flag is clear then it's an old single-core CPU without hyper-threading. In that case "logical_CPU_bits = 0" and "core_bits = 0" and you're done.

I am afraid; it’s not that simple There are some multi-core processors with no HT support (Intel Pentium D, Intel Pentium Dual-core, Intel Core 2 Duo, Intel Core 2 Quad).

It is that simple. All Intel multi-core CPUs set the "HTT" flag, regardless of whether they support hyper-threading or not. Basically that "HTT" flag should be called a "more than one logical CPU in chip" flag.

limp wrote:

Brendan wrote:

limp wrote:I am afraid; it’s not that simple There are some multi-core processors with no HT support (Intel Pentium D, Intel Pentium Dual-core, Intel Core 2 Duo, Intel Core 2 Quad).

It is that simple. All Intel multi-core CPUs set the "HTT" flag, regardless of whether they support hyper-threading or not. Basically that "HTT" flag should be called a "more than one logical CPU in chip" flag.

I did a quick check on my Intel Celeron M 440 and ACPI reports 2 LAPICs although HTT flag is not set (the processor should not support HT according to the specs). Specifically, ACPI founds a LAPIC entry with APIC ID 0, ACPI ID 1 and one with APIC ID 0x81 and ACPI ID 2. Does this makes any sense to you?

Brendan wrote: The first part (HTT flag clear) makes perfect sense - according to Intel's site, Celeron M 440 is a single-core chip without hyper-threading.

The second part (ACPI results) look wrong. Even if there's 4 separate physical packages/chips (which doesn't make much sense for mobile parts), it'd be very rare for the local APIC IDs to be in such a strange order.

that max CORE_ID and max SMT_ID should be equal to the nearest power-of-2 integer that is not smaller than (1 + EAX[25:14]) and (1 + EAX[31:26]) respectively)

“One should not assume that the number of available threads or cores will be a power of two.”

limp wrote:

Brendan wrote: The first part (HTT flag clear) makes perfect sense - according to Intel's site, Celeron M 440 is a single-core chip without hyper-threading.

The second part (ACPI results) look wrong. Even if there's 4 separate physical packages/chips (which doesn't make much sense for mobile parts), it'd be very rare for the local APIC IDs to be in such a strange order.

Thanks for the response. Apparently I was parsing the MADT correctly (I have verified that there are indeed 2 APIC entries) but I forgot the importance of the "Enable" field (I know, quite a big mistake). Specifically, the "APIC ID 0x81, ACPI ID 2" entry is marked as “disabled” (you were right that it looked wrong).

I remember you mentioning in other posts that Intel tends to disable/enable certain features in their processors, either because they have some defects or just because they seek of selling them under different series eliminating the need/cost of making new chips. I think that applies in my case. Either my processor is a defected HT processor (with 1 of its APIC not working) or a processor from a more expensive series (so they just disabled 1 APIC).

limp wrote:Now, a question I have regarding the CPU topology detection. When I run my topology detection code on my Celeron 440 M (which is a single-core processor without HT), I get (using CPUID leaf 4), max PACKAGE_ID = 1, max CORE_ID = 1 and max SMT_ID = 1.

I get max CORE_ID = 1 and max SMT_ID = 1 because CPUID says that max CORE_ID and max SMT_ID should be equal to the nearest power-of-2 integer that is not smaller than (1 + EAX[25:14]) and (1 + EAX[31:26]) respectively). So, although I get “0” from EAX[25:14] and EAX[31:26], I add “1” because that’s what it says to do. Also, I get PACKAGE_ID = 1 because although I get count=0 from EBX bits 16 to 23, I add 1 to the result as the manual says.

So, I end up with max PACKAGE_ID = 1, max CORE_ID = 1 and max SMT_ID = 1 which is certainly wrong. Do I have to assign all max PACKAGE_ID, max CORE_ID and max SMT_ID to 0 without reading the CPUID flags at all in this case (single-core processor without HT) ?

limp wrote:Something else that confuses me is that Intel’s manual 2A and “Intel’s Detecting Multi-Core Processor Topology in an IA-32 Platform” document seem to have a different opinion upon certain things.

While the former says as mentioned that:

that max CORE_ID and max SMT_ID should be equal to the nearest power-of-2 integer that is not smaller than (1 + EAX[25:14]) and (1 + EAX[31:26]) respectively)

the latter says that:

“One should not assume that the number of available threads or cores will be a power of two.”

Brendan wrote: If the HTT flag is clear then it's a single-core CPU without hyper-threading, and looking further is a waste of time.

Brendan wrote: That doesn't look contradictory at all.

If the max CORE ID is 4, then it doesn't imply that there's actually 4 cores and you could have 1 core with 3 unused IDs.

max PACKAGE_ID = 65, max CORE_ID = 2, max SMT_ID = 1

LAPIC (acpi id[0x01]  lapic id [0x00] enabled)
LAPIC (acpi id[0x02]  lapic id [0x01] enabled)
LAPIC (acpi id[0x03]  lapic id [0x82] disabled)
LAPIC (acpi id[0x04]  lapic id [0x83] disabled)

max PACKAGE_ID = 65, max CORE_ID = 2, max SMT_ID = 1

LAPIC (lapic id [0x00]  => Package 0, Core 0, Log. CPU 0 enabled)
LAPIC (lapic id [0x01]  => Package 0, Core 0, Log. CPU 1 enabled)
LAPIC (lapic id [0x82]  => Package 0x20, Core 1, Log. CPU 0 disabled)
LAPIC (lapic id [0x83]  => Package 0x20, Core 1, Log. CPU 1 disabled)

limp wrote:One last question: In my Intel Atom 330 target, when I disable HT from the BIOS, I get the following:

Code: Select all

max PACKAGE_ID = 65, max CORE_ID = 2, max SMT_ID = 1

LAPIC (acpi id[0x01]  lapic id [0x00] enabled)
LAPIC (acpi id[0x02]  lapic id [0x01] enabled)
LAPIC (acpi id[0x03]  lapic id [0x82] disabled)
LAPIC (acpi id[0x04]  lapic id [0x83] disabled)

So, for HT-disabled, we have:

Code: Select all

max PACKAGE_ID = 65, max CORE_ID = 2, max SMT_ID = 1

LAPIC (lapic id [0x00]  => Package 0, Core 0, Log. CPU 0 enabled)
LAPIC (lapic id [0x01]  => Package 0, Core 0, Log. CPU 1 enabled)
LAPIC (lapic id [0x82]  => Package 0x20, Core 1, Log. CPU 0 disabled)
LAPIC (lapic id [0x83]  => Package 0x20, Core 1, Log. CPU 1 disabled)

This indicates (according to the topology masking bits/width) that when HT is disabled, we don’t have a single APIC per CPU core as I would expect but two APICs on a single CPU core and none on the other (which suggests that our CPU acts like a single-core HT CPU).

Could you (or anyone else) please help me understand why is that?

Brendan wrote: I probably should've mentioned this somewhere..

For local APIC ID's, there's the ID that happens to be in the local APIC's "Local APIC ID register". This can be changed by software at any time and therefore can't be relied on for topology detection. This also means that the firmware could've changed the "Local APIC ID register" to whatever it likes before creating the ACPI tables; and the local APIC IDs listed in the ACPI tables (and/or MP specification tables) can't be relied on for topology detection either.

"CPUID = 0x00000001" returns the "The 8-bit value assigned to the local APIC physical ID register at power-up" in EBX bits 24 to 31. Because this can't be changed by any software, it can be relied on for topology detection (and is the only "ID" that can be used reliably for topology detection).

limp wrote:

Brendan wrote:I see...thanks for clarifying this. So, for determining which LAPIC is in which Core/Package, we need to get the “Initial LAPIC” of each enabled LAPIC unit. But how can I do that without booting all the CPUs first (for instance, in the case in which HT is enabled, I want to be able to boot only one LAPIC / core)?

Brendan wrote:Hi,

limp wrote:

Brendan wrote:I see...thanks for clarifying this. So, for determining which LAPIC is in which Core/Package, we need to get the “Initial LAPIC” of each enabled LAPIC unit. But how can I do that without booting all the CPUs first (for instance, in the case in which HT is enabled, I want to be able to boot only one LAPIC / core)?

I don't think there's any way to do it without starting the other (enabled/present) CPUs.

If you want some (enabled/present) CPUs to remain "unstarted", then you could either skip them entirely (and maybe do the topology detection on them later on if/when you do want to start them), or start them and shut them down again afterwards.