CPUID - how to determine number of physical cores?

[RayeR]

I'm updating my cpuid utility and tunning it on intel core i5-750 and I little bit wonder how many cores I get.
I know the CPU has 4 physical cores, also cpu-z and coretemp reports that.
But how they calculate this value?
I can read
1) CPUID level 1 EBX[23:16] - this returns 16 logical cores
2) CPUID level 4 EAX[31:26] - this returns 7 as max APIC ID (also named as number of cores-1)
How do I get the number 4?
I display temperatures from DTS so it wouldn't make sense print it for a bunch of logical cores but only physical ones.

[Cognition]

Use CPUID Function EAX = 0x0B. It's covered in the Intel SDM Volume 2A. You can still potentially get junk values as to the active number of logical processors your system has though. For instance your processor has Hyperthreading disabled by manufacturer, which would likely explain the max APIC ID of 7 from CPUID Leaf 4. SMBIOS might also give you a description of the installed processor. Keep in mind though that this information still won't necessarily tell you the number of active processors. Ideally you should read the ACPI MADT table or MP Tables for a list of enabled processors then build a topology for each of them.

[RayeR]

Hm, so I will need iterate through cores...? I also found this document
http://software.intel.com/en-us/article ... umeration/, it doesn't look trivial. I will study it but I hoped there there's direct value somewhere I could easily read...
On other forum I read advice to read number from level 4 and then divide it by 2 if HTT feature flag is set. But on older CPUs (like my C2D at home) it reports same number in level 1 and 4 and HTT is also set so I then got half of cores. Maybe it would be acceptable to limit this approach only for core i3/5/7.
Also for AMD is needed to read it different:
get_cpuid_info(0x80000008, &eax, &ebx, &ecx, &edx);
cpu_phys_cores=(ecx&0xFF)+1; // last CPU core APIC number ECX[7:0]+1

[Cognition]

Leaf 11/0xB is a newer extension, it won't be supported by older processors. For those you can use Leaf 0x4 and assume a processor running HT will have only 2 threads a core. Basically use 0xB when it's available, it will give you the most specific information. Once again though this will just give you the configuration of APIC id's on the processor package, it doesn't guarantee that all the potential cores or threads are active. For that you need to rely on a specification which actually enumerates logical processors like the ACPI or MP tables.

[RayeR]

Well, I try understand and use level 11 if avail. Currently I don't know how to handle ACPI tables.

[Brendan]

Hi,

I'm splitting this into 2 parts because it was getting long and it sort of made sense to split it up. The first part is general CPU topology detection. The second part is CPUID usage.

PART 1

For complete CPU topology, there's 4 levels:

• Which logical CPU within a core
• Which core within a chip
• Which chip within a NUMA domain
• Which NUMA domain

For convenience, I normally express this as a colon separated ID. For example, "CPU 3:2:1:0" would be logical CPU #0 in core #1 of chip/package number 2, in NUMA domain #3.

To begin with, use the ACPI MADT table or the MP Specification table to determine the APIC IDs for all CPUs. I normally create a table of structures that contains the information that is different for each CPU (e.g. APIC ID, topology information, etc.). This "CPU info" structure includes a pointer to a separate "CPU description" structure that stores information that can be the same for different CPUs (e.g. vendor, family/model/stepping, brand string, feature flags, cache sizes, etc). For a quad core CPU you'd have 4 of the "CPU info" structures and only one "CPU description" structure.

To detect NUMA domain information you have to use the ACPI 'SRAT' (System Resource Affinity Table). First, preset the "NUMA domain" field for each CPU to "unknown" (in case the CPU isn't mentioned in the SRAT table). Then, for each entry in the SRAT table you determine if it's a CPU or memory, and if the entry is a CPU you use the 'APIC ID' field to determine which CPU and the 'proximity domain' field to determine which NUMA domain that CPU is in; and set the NUMA domain for that CPU. Note that there's 2 different structures in the SRAT table - one for 8-bit local APIC IDs and one for 32-bit APIC IDs, and if you support x2APIC you need to use both.

To get nice clean results in the later steps, you should sort your list of CPUs now - sort them in order of NUMA domain, and sort each NUMA domain in order of APIC ID.

For cores and logical CPUs; for each CPU, you want to use CPUID to determine:

• Number of bits in the APIC ID that are used to identify which logical CPU
• Number of bits in the APIC ID that are used to identify which core

How to use CPUID to determine these values is described in "Part 2".

The values are used for a set of calculations:

• logical_CPU_number_within_core = APIC_ID & ( (1 << logical_CPU_bits) -1)
• core_number_within_chip = (APIC_ID >> logical_CPU_bits) & ( (1 << core_bits) -1)
• chip_ID = APIC_ID & ~( (1 << (logical_CPU_bits+core_bits) ) -1)

The "chip ID" is a temporary value that you could discard later.

Note: I do these calculations for each logical CPU separately, because it's theoretically possible for different physical chips in the same computer to use APIC ID bits differently.

Once you've done these calculations you can determine the "chip number within NUMA domain". All CPUs that have the same "chip ID" and the same NUMA domain are in the same chip.

For example:

Code: Select all

    for(domain = 0; domain <= highest_domain; domain++) {
        chip_number = 0;
        for(firstCPU = 0; firstCPU < total_CPUs; firstCPU++) {
            if( CPU_info_table[firstCPU].domain == domain) {
                if( CPU_info_table[firstCPU].chipNumber == -1) {
                    CPU_info_table[firstCPU].chipNumber = chip_number;
                    chipID = CPU_info_table[firstCPU].chipID;
                    for(i = firstCPU; i < total_CPUs; i++) {
                        if( CPU_info_table[i].domain == domain) {
                            if( CPU_info_table[i].chipID == chipID) {
                                CPU_info_table[i].chipNumber = chip_number;
                            }
                        }
                    }
                }
                chipNumber++;
            }
        }
    }

There's a bit of strangeness I should explain here. In general it doesn't make too much sense for the same chip to be in different NUMA domains. However, when Intel designed x2APIC they decided that Logical APIC ID (which shouldn't be confused with the "APIC ID") and the "Destination Format Register" would be hardwired; and that "Logical Destination Mode" would only support "cluster model". They decided that the highest bits of the Logical APIC ID would determine which "group of CPUs" and the lowest 4 bits of the Logical APIC ID would determine which CPU within the group. The end result is that, if a chip has more than 16 logical CPUs it will be split into different NUMA domains. For the algorithm above, this case is treated as separate chips in different NUMA domains (even though it is technically one physical chip spread across multiple NUMA domains). This is deliberate, as it avoids complications everywhere else (in the APIC/interrupt handling code, in the scheduler, etc).


Continued next post

[Brendan]

Continued from previous post


PART 2

Part 1 describes everything, except how to use CPUID to determine:

• logical_CPU_bits = number of bits in the APIC ID that are used to identify which logical CPU
• core_bits = number of bits in the APIC ID that are used to identify which core

CPUID eax=0x00000001

This is the first step. Check if the "HTT" flag is set. If the HTT flag is clear then it's an old single-core CPU without hyper-threading. In that case "logical_CPU_bits = 0" and "core_bits = 0" and you're done.

Otherwise (if the "HTT" flag is set), you want to get the "count" field from EBX bits 16 to 23. You (might) need it later.


INTEL: CPUID eax=0x0000000B

For Intel CPUs (and not AMD), this CPUID function tells you "Number of bits to shift right APIC ID to get next level APIC ID", and needs to be used twice. The first time (with ECX=0) it tells you how many bits of the APIC ID are used to identify the logical CPU within each core (logical_CPU_bits). The second time (with ECX=1) it tells you how many bits of the APIC ID are used to identify the core and logical CPU within the chip; and to get "core_bits" from this value you subtract "logical_CPU_bits" from it.

INTEL: CPUID eax=0x00000004

For Intel CPUs (and not AMD), if "CPUID eax=0x0000000B" wasn't supported then this is the next CPUID function to try. You're only interested in EAX bits 26 to 31, which tells you the number of APIC IDs reserved for this chip. Use the "count" field you got earlier (from CPUID eax=0x00000001) rounded up to the next power of 2 to determine "logical_CPU_bits", then do "temp = EAX_bits_26_to_31 >> logical_CPU_bits" and use the result rounded up to the next power of 2 to determine "core_bits".

If "CPUID eax=0x00000004" isn't supported, then it's a single-core CPU with hyper-threading. Use the "count" field you got earlier (from CPUID eax=0x00000001) rounded up to the next power of 2 to determine "logical_CPU_bits", then set "core_bits = 0".



AMD: CPUID eax=0x80000008

For AMD CPUs (and not Intel), use this function to determine "core_bits". First check ECX bits 12 to 15, if it is not zero, then it contains "core_bits". Otherwise, use ECX bits 0 to 7 to determine the number of cores, round it up to the next power of 2 and use it to determine "core_bits".

Now that you've got "core_bits"; use the "count" field you got earlier (from CPUID eax=0x00000001) and shift it right by "core_bits" bits. Then round the result up to the next power of 2 and use it to determine "logical_CPU_bits".


AMD: CPUID eax=0x00000001

For AMD CPUs (and not Intel), if CPUID eax=0x80000008 isn't supported just assume that the "count" field you got earlier (from CPUID eax=0x00000001) is the number of cores; and round it up to the next power of 2 and use it to determine "core_bits". In this case set "logical_CPU_bits" to zero.


That's everything I can think of..


Cheers,

Brendan

[RayeR]

Brendan, thanks for detailed info.
I did some experiments with CPUID level 11, here's my output of core i5-750 - EAX, EBX, ECX, EDX for sublevels 0-2:

00000001 00000001 00000100 00000000
APIC ID bit shift=1, fatcory conf proc=1, levelt=1
00000004 00000004 00000201 00000000
APIC ID bit shift=4, fatcory conf proc=4, levelt=2
00000000 00000000 00000002 00000000
APIC ID bit shift=0, fatcory conf proc=0, levelt=0

So it means I got 2 sublevels 0 = thread level, 1 = core level ( 2 = end of reading - EAX & EBX = 0)
I found that EBX contains an interesting bit field [15:0] - number of factory configured processors at this sublevel. So if I call this with ECX=1, sublevel type ECX[15:0] is core, then EBX[15:0] should give me the numbers of cores. This can be 1-8 on Nehalem family. There's note that this value NOT change based on intel HT technology disable and core disables. 1 log. cpu per core if HT factory-disabled and 2 log. cpu if HT factory-enabled. I'm confused by this. Does it mean that I have HT disabled? I got 4 factory configured cores in EBX[15:0]. When HT would be enabled (how can I do it? get better CPU or some tool?) will I read EBX[15:0] = 4 or 8?

EDIT:
I found an example using EBX[15:0]:

Code: Select all

/*
 * LeafBHWMTConsts
 *
 * Query CPUID leaf B to report the processor's hardware multithreading capabilities
 *
 * Arguments:     None
 *     
 * Return:        None, sets glbl_ptr->error if tables or values can not be calculated.
 */
int LeafBHWMTConsts()
{
CPUIDinfo infoB;
int subLeaf = 0;
	do
	{
		int levelType;
		int ThreadCnt;
			
		_CPUID(&infoB,0xB,subLeaf);
		if (infoB.EBX == 0)  
		{	// we already tested leaf B provide valid info, if EBX[15:0] of this subleaf is not valid, we can exit the loop
			break;
		}

		levelType = getBitsFromDWORD(infoB.ECX,8,15);
		ThreadCnt = getBitsFromDWORD(infoB.EBX,0,16);
		switch (levelType) 
		{
		case 1:		//level type is SMT, so ThreadCnt is the # of logical processor within the parent level, i.e. a core
			
			glbl_ptr->HWMT_SMTperCore = ThreadCnt;
			break;
		case 2:	//level type is Core, so ThreadCnt is the # of logical processor within the parent level, i.e. a package		
			glbl_ptr->HWMT_SMTperPkg =  ThreadCnt;
			break;
		default:
			// handle in the future
			break;
		}				
		subLeaf++;

	} while (1);
	if( !glbl_ptr->HWMT_SMTperCore || !glbl_ptr->HWMT_SMTperPkg) return -1;
	return 0;
}

[Brendan]

Hi,

I did some experiments with CPUID level 11, here's my output of core i7 750 - EAX, EBX, ECX, EDX for sublevels 0-2:

00000001 00000001 00000100 00000000
APIC ID bit shift=1, fatcory conf proc=1, levelt=1
00000004 00000004 00000201 00000000
APIC ID bit shift=4, fatcory conf proc=4, levelt=2
00000000 00000000 00000002 00000000
APIC ID bit shift=0, fatcory conf proc=0, levelt=0

Doh. Thanks. I just realised there's a mistake in my previous post - I forgot to subtract. I've edited/fixed my post now (to avoid confusing people in the future).

Your data says that the lowest 1 bit of the APIC ID is used to determine which logical CPU within the core, and the next "4-1" bits are used to identify which core in the chip.

I found that EBX contains an interesting bit field [15:0] - number of factory configured processors at this sublevel. So if I call this with ECX=1, sublevel type ECX[15:0] is core, then EBX[15:0] should give me the numbers of cores.

No - it'll give you the number of factory configured cores, and not the number of cores that are actually usable. For an example, if a core fails it's BIST (Built-In Startup Test) during boot, then the BIOS/firmware will detect that and won't list the faulty core in the ACPI MADT table (or Multiprocessor Spec table) to make sure that the OS doesn't rely on faulty CPU/s. In this case you might have 4 factory configured cores and only 3 usable cores.

The wording in Intel's application note ("core disables") makes me wonder if there's other reasons for a factory configured core to be unusable. For an example, it might be possible for Intel to make a quad core CPU (with 4 factory configured cores) and then sell it as a budget "triple-core" CPU by disabling a core. Another example might be an embedded system where the manufacturer deliberately disable core/s - maybe their software is all single-threaded and needs fast single core performance, but Intel don't make a fast single-core CPU anymore so the manufacturer buys dual-core CPUs and disables the unnecessary core to save power.

This can be 1-8 on Nehalem family. There's note that this value NOT change based on intel HT technology disable and core disables. 1 log. cpu per core if HT factory-disabled and 2 log. cpu if HT factory-enabled. I'm confused by this. Does it mean that I have HT disabled? I got 4 factory configured cores in EBX[15:0]. When HT would be enabled (how can I do it? get better CPU or some tool?) will I read EBX[15:0] = 4 or 8?

It looks like you've got 4 factory configured cores with 1 factory configured logical CPU per core; or a total of 4 logical CPUs. This is entirely possible, as Intel have a habit of disabling features for marketing reasons (product differentiation). For example, they might sell a quad-core Nehalem with hyper-threading and turbo-boost enabled for $234, and also sell a quad-core Nehalem with hyper-threading and turbo-boost disabled for $123 (even though the CPU's design, silicon, etc is probably identical in both cases).

Also note that there's a difference between "permanently disabled by Intel" and "enabled by Intel but disabled by the BIOS". For the "disabled by BIOS" case you'd expect CPUID to report "2 factory configured threads/logical CPUs per core" even though the OS can only use 1 thread/logical CPU per core.


Cheers,

Brendan

[RayeR]

So if I understand it well, the CPUID info level 1 EBX[23:16] and level 4 EAX[31:26]+1 gives me only possible maximum count of log. CPUs and cores for the whole CPU family (e.g. Nehalem) and not for specific CPU (e.g. i5-750 - I checked intel specs and it has 4 cores and HT disabled). Also level 11 tells only the maximum values. BTW why differs level 4 EAX[31:26]+1 from level 11/sublevel1 EBX[15:0] (8 vs 4 max cores)? So I can use this info for construction of APIC ID bit-mask that is in my case:
PPPPCCCL (4bit for packages, 3 bits for cores, 1 bits for log cpus in core)

Then I must apply this mask to all collected APIC IDs. But this is OS platform specific stuff - beyond CPUID instruction? I could do it under windows by setting affinity winapi function. But how can I do it under older OSes that doesn't support SMP? Or how do you do it in your own OS? I really have no experiences with reading and parsing ACPI tables Can I read something usefull from MSRs? I hoped that counting the cores would be simpler task... EDIT: I checked CPU-Z utility under win98 and it reports cores: 1, threads: 1 so it seems also relying on winapi.

BTW are intel cores/HT really hard locked (e.g. by final laser trimming)? I remember that on ATI and nVidia GPUs it was possible to reenable some processing units but in some cases they was defective.

[Brendan]

Hi,

Also level 11 tells only the maximum values. BTW why differs level 4 EAX[31:26]+1 from level 11/sublevel1 EBX[15:0] (8 vs 4 max cores)?

You're comparing "number of APIC IDs reserved for the chip/package" to "number of factory configured logical CPUs". For a quad-core CPU with hyper-threading, Intel would reserve 8 APIC IDs and have 8 factory configured logical CPUs. For the same silicon with hyper-threading factory-disabled, the circuits for routing interrupts to APICs (and determining which range of APIC IDs to use for the chip/package in multi-chip situations) is probably exactly the same, and it's too much hassle for Intel to modify it just to make things a little cleaner, so you end up with the same number of APIC IDs reserved for the chip/package and less factory configured logical CPUs.

So I can use this info for construction of APIC ID bit-mask that is in my case:
PPPPCCCL (4bit for packages, 3 bits for cores, 1 bits for log cpus in core)

Then I must apply this mask to all collected APIC IDs. But this is OS platform specific stuff - beyond CPUID instruction? I could do it under windows by setting affinity winapi function. But how can I do it under older OSes that doesn't support SMP?

For an application/process running on an OS that doesn't support SMP, there's no real reason to care and you probably shouldn't bother doing any of it.

Or how do you do it in your own OS?

I do it exactly as I described above in my 2 part explanation. There's a few details I skipped though - I start all the CPUs running beforehand, and each CPU executes CPUID and does topology detection separately (I don't assume all CPUs are the same); and I also gather other information intended to be used for optimising performance in the memory manager and scheduler (like using the ACPI "SLIT" table to detect "inter-domain" NUMA performance penalties, detecting which L1/L2/L3 caches are shared by which logical CPUs, etc).

I really have no experiences with reading and parsing ACPI tables

There's only one way to get experience with reading and parsing ACPI tables...

The RSDT/XSDT, the MADT and the SRAT are all relatively easy to understand and parse; and you can ignore all the other ACPI tables for now.

Can I read something usefull from MSRs?

Not really. There's 2 problems with MSRs: they're model specific, and none of the MSRs are intended to be used for getting information about CPUs.

I hoped that counting the cores would be simpler task...

The CPUID instruction is an ugly mess - lack of foresight by CPU designers and no cooperation between competing CPU manufacturers has lead to several different ways of doing the same thing (for CPU topology detection and cache detection). Then you throw errata on top of that (e.g. different CPUs reporting wrong information via. CPUID) and it gets even worse.

However, even though it's an ugly mess it's the "least worst" method.

Note: A good OS would provide a clean API that processes/applications can use to get information about CPUs; so that processes/applications never need to use CPUID and don't need to deal with any of the mess.

BTW are intel cores/HT really hard locked (e.g. by final laser trimming)? I remember that on ATI and nVidia GPUs it was possible to reenable some processing units but in some cases they was defective.

I doubt it's possible for an end-user to tamper with a recent CPU to unlock "factory disabled" features.


Cheers,

Brendan

[jal]

The first part is general CPU topology detection. The second part is CPUID usage.

Though I lack the time to even check, is this in the wiki and if not, can someone put this in there?


JAL

[Brendan]

Hi,

The first part is general CPU topology detection. The second part is CPUID usage.

Though I lack the time to even check, is this in the wiki and if not, can someone put this in there?

If I was was smart (instead of lazy) I would've added it to the wiki and provided a link to it in the first place....

I'll have a go at adding it now.


Cheers,

Brendan

[jal]

I'll have a go at adding it now.

Great, thanks.


JAL

[RayeR]

Hi,
You're comparing "number of APIC IDs reserved for the chip/package" to "number of factory configured logical CPUs". For a quad-core CPU with hyper-threading, Intel would reserve 8 APIC IDs and have 8 factory configured logical CPUs. For the same silicon with hyper-threading factory-disabled, the circuits for routing interrupts to APICs (and determining which range of APIC IDs to use for the chip/package in multi-chip situations) is probably exactly the same, and it's too much hassle for Intel to modify it just to make things a little cleaner, so you end up with the same number of APIC IDs reserved for the chip/package and less factory configured logical CPUs.

Still confused... If level 4 EAX[31:26] + 1 is "number of APIC IDs reserved for the chip/package" including possible HT variant then why level 1 EBX[23:16] "number of logical processors per physical processor package" shows 16.
I would expect that if Lynnfield hax maximum 4 cores + HT option I should get
level 1 EBX[23:16] = 8
level 4 EAX[31:26] = 7
and if Lynnfield has maximum 8 cores + HT option I should get
level 1 EBX[23:16] = 16
level 4 EAX[31:26] = 15

In other intel doc the level 4 EAX[31:26] is described as "maximum number of addresable IDs (core_id) that can be used to enumerate differentprocessor cores in a physical package." So it doesn't count if has HT.
then if I would expect that if Lynnfield hax maximum 4 cores + HT option I should get
level 1 EBX[23:16] = 8
level 4 EAX[31:26] = 3
and if Lynnfield has maximum 8 cores + HT option I should get
level 1 EBX[23:16] = 16
level 4 EAX[31:26] = 7

this case match my reading but I don't know about 8-core lynnfileld CPU

Level 11 has two sublevels that changes meaning of EBX[15:0]
ECX=0 for threads and ECX=1 for cores. So I expect that in 1st case
EBX[15:0] is correctly named as "number of factory-configured logical processors" (per core) - returned 1 because I don't have HT. If I have HT I should see 2 here.
In 2nd case it should be called "number of factory-configured cores" (per chip) because it's executed at core level. As it return 4 it seems to be right. I would need to collect CPUID data from some HT enabled CPU to see the difference but unfortunatelly I don't know anybody with such CPU. Within a month one of my friend will upgrade to Sandybridge so I'll got some new data...

For an application/process running on an OS that doesn't support SMP, there's no real reason to care and you probably shouldn't bother doing any of it.

I'm doing just CPU info tool not the OS. I wish it could display accurate info about core count regardless the OS it actually is running. For older CPUs I can recognize if it's P4HT or C2D without HT so I can display cores cound even if e.g. cpu-z display only 1. Of course then I will not try to read DTS from all cores but only one.

I do it exactly as I described above in my 2 part explanation. There's a few details I skipped though - I start all the CPUs running beforehand, and each CPU executes CPUID and does topology detection separately

That's I missed before...

The RSDT/XSDT, the MADT and the SRAT are all relatively easy to understand and parse; and you can ignore all the other ACPI tables for now.

Well it's my task for future to learn read ACPI. In case of no OS services how can I get this data? Is there some BIOS call or do I need to grab it somewhere from physical memory?

Not really. There's 2 problems with MSRs: they're model specific, and none of the MSRs are intended to be used for getting information about CPUs.

Of course they are model specific, I'm handling it to get multiplier, FSB, DTS and other info and it must be separated for different CPU families and vendors. Otherwise improper MSR reading cause crash...
But if there's nothing interesting about cores then I don't care...

The CPUID instruction is an ugly mess - lack of foresight by CPU designers and no cooperation between competing CPU manufacturers has lead to several different ways of doing the same thing (for CPU topology detection and cache detection). Then you throw errata on top of that (e.g. different CPUs reporting wrong information via. CPUID) and it gets even worse.

Unfortunatelly it happen to most of new hardware. IBM was set wide standards of PC that was respected for years but now everyone make his proprietary HW and closed sourced drivers only for windows (with some light exceptions). In last years it seems that AMD supply better docs. One of my fried is working on coreboot and he appreciated what AMD did for them...