Calibrating apic timer?

[AlexHully]

Hi,

I understood that:
One local apic == one core.

I understand that a cpu cycle is a very abstract way to talk about time.

100 cpu cycles on a core running at 1Ghz are not the same as on another one running at 2Ghz in term of time.
(The time spent dealing with those 100 cycles on the 1Ghz will be greater than on the 2Ghz core)

So, what is the matter ? We can know the cpu speed (cpuid..). Cpus have 4/6 different functional speeds (P-states). So we know how many cycles it can do in, say, one second, since we know its speed.

May a P-state change occur, we can recalibrate the cpu cycles according to the difference.

So why do we need an external source (PIT, whatever) to calibrate the apic timer ?
What did I miss ?

thanks, bye

[alexfru]

SpeedStep?

[Brendan]

Hi,

I understood that:
One local apic == one core.

Unless there's something like hyper-threading, where one local APIC = one logical CPU (and there's 2 or more logical CPUs in a core).

So, what is the matter ? We can know the cpu speed (cpuid..). Cpus have 4/6 different functional speeds (P-states). So we know how many cycles it can do in, say, one second, since we know its speed.

May a P-state change occur, we can recalibrate the cpu cycles according to the difference.

So why do we need an external source (PIT, whatever) to calibrate the apic timer ?
What did I miss ?

In "roughly chronological" order...

Originally the local APIC was a separate chip (not built into the CPU at all) and its timer ran at bus speed not CPU speed. For these ancient systems (80486) there was no CPUID.

Then (first for Pentium Pro, but then back-ported into later "Pentium with MMX" chips) Intel shifted the local APIC into the same chip as the CPU and changed a few things in the local APIC. The local APIC still ran at bus speed and not CPU speed. These CPUs did have CPUID, but CPUID didn't tell you how fast the CPU is so that doesn't help.

Much later (Pentium 4?), Intel started adding "nominal clock speed" to their CPU brand name strings. Nominal clock speed has nothing to do with actual clock speed and (when a chip is overclocked) the "nominal clock speed" reported doesn't tell you the actual nominal clock speed either. Also; the local APIC still uses bus frequency and not any of the CPU's frequencies, and its insanely difficult to determine bus speed from the CPU speed (if you could know the CPU speed you'd still need the ratio of bus speed to CPU speed).

Even later Intel added "TSC deadline mode" to the local APIC that does use "CPU speed sort of maybe" (in addition to the traditional/older modes that still use bus speed). However; for modern CPUs (e.g. probably all CPUs that are new enough to support TSC deadline mode) the local APIC's TSC deadline mode (and the TSC in general) has nothing to do with CPU's (variable) frequency and is actually a fixed frequency clock (that runs at "faster than nominal clock speed" so that its incremented at least once per CPU cycle even when the CPU is running faster than its nominal clock speed due things like turbo-boost, etc). Also, on some CPUs (Atom) it works differently (to avoid "yet another clock"); and the TSC runs at bus speed and they pretend it runs at a faster frequency by adding a value (e.g. 16) every bus cycle.

Basically; you need to calibrate the local APIC timer (which may mean measuring bus speed or measuring TSC speed), and CPUID won't tell you anything useful regardless of how new/old the CPU is.

Note: For the sake of completeness; with massive quantities of model specific voodoo (lookup tables, etc), for "Intel family 6 and later" CPUs I think it's possible to determine bus speed by combining information from both CPUID and MSRs; but if this is possible it won't work for any overclocked CPU or future CPUs (e.g. until you figure out the model specific bus multiplier and update your tables), and probably won't work for AMD or VIA CPUs either.


Cheers,

Brendan

[AlexHully]

Thank you Brendan,

I missed the bit where some apic timers (Core 2 duo, etc) are not really on the cpu but close, hence the bus speed.

The calibration for those cpu families is very approximate/fuzzy.

So, to get a serious timer which is very low overhead (tsc) and precise, one has to buy at least very recent cpus (that support it), right ?

[Brendan]

Hi,

So, to get a serious timer which is very low overhead (tsc) and precise, one has to buy at least very recent cpus (that support it), right ?

That depends what you want, and how you implement it.

Typically there's 3 different things to worry about - precision (e.g. how small the smallest amount of time the timer can measure is), accuracy (e.g. how close "1 second according to the timer" is to 1 second and long term drift) and whether or not it will generate IRQs.

For implementation; you can work around accuracy problems by periodically using a higher accuracy timer to synchronise a lower accuracy timer. This can be done in a "cascading" way; where something with extremely high accuracy (e.g. NTP) is used to synchronise something with high accuracy (e.g. RTC or HPET), which is then used to synchronise something with lower accuracy (e.g. TSC, local APIC timer, PIT). For example; if you've got an older CPU where TSC speed varies (due to power management, etc), then TSC still has very good precision and just has bad accuracy; but you can synchronise it periodically with another timer (e.g. HPET) to end up with "very good precision and good accuracy" from the variable speed TSC.

Also note that typically you want 3 different things:

• Something scheduler uses to determine when the currently running task has had enough CPU time; where you don't care too much about precision or accuracy but do need an IRQ (and you really want one per CPU). Local APIC timer is extremely good for this.
• Something the OS uses to keep track of "real time" (for time stamps, etc); where you care about accuracy, but very high precision isn't needed and you don't need an IRQ. Modern "fixed frequency TSC" is really good for this, but HPET's main counter is too. Older "variable frequency TSC" isn't good on its own (it lacks the accuracy you want, and only has precision you don't need). There may also be an ACPI timer that's probably good enough for this.
• Something the OS uses to manage delays (e.g. "sleep()", "nanosleep()"); where you care about accuracy, but very high precision isn't really needed and you do need an IRQ. For this; modern "fixed frequency TSC" combined with "TSC deadline mode" in the local APIC is extremely good (but has precision you don't really need, and using the same timer for 2 or more different things can get messy), and HPET is very good too; but PIT and RTC can also be good enough.

Mostly, when an OS boots you want to determine which timers are present (local APIC, TSC, HPET, APIC timer, PIT, RTC) and figure out their capabilities (precision, accuracy, ability to generate IRQs); then have code to choose which timer is the best timer to use for which purpose/s. Then you'd use some abstraction so that the rest of the kernel/OS doesn't need to care which timer was chosen for what.

Note: There is another thing you probably need to be aware of. In some systems, when a CPU is put into some "low power" modes (where the CPU "stops" to save power) the TSC and/or the local APIC timer stops ticking. In this case if you're using that timer to determine when the currently running task has used enough CPU time, then it doesn't matter because you don't use these low power modes when the CPU has work to do anyway; but if you're using that timer for other things ("sleep()", TCP/IP time outs, etc) then it won't work and you'll need to use some other timer (either "always", or just when the CPU is in the low power mode).


Cheers,

Brendan