Correct setup of performance-monitoring counter's handler

[limp]

Hi all,

I've managed to setup a performance monitoring counter (PMC) and make it generate an APIC interrupt when it overflows.

In the Intel manuals is stated that when interrupted by a counter overflow, the (PMC) interrupt handler needs to perform the following actions:

• • Save the instruction pointer (EIP register), code-segment selector, TSS segment selector, counter values and other relevant information at the time of the interrupt.
  • Reset the counter to its initial setting and return from the interrupt.

Can anyone tell me how they actually mean the EIP register save? Do they really mean that we need to save the stack segment when the interrupts occurs and then restore it? Also, they mention that it is required to save the TSS segment selector. In my OS, the TSS is not used at all (the OS doesn’t support multitasking, etc). In this case do I really need to do that? Does this mean that I need to add TSS support to my OS (add a TSS descriptor entry in the GDT, etc.) for being able to reload the PMC?

My PMC handler is actually executed when the counter overflows (I've verified this by a printf) but the processor generates a "General Protection Fault" when it returns from the PMC handler.

A second question is referred to the stub exception handler. In Intel manuals is stated that the steps required (by the OS) for being able to generate a local APIC interrupt when a performance-monitoring counter overflows are the following:

• 1) Provide an interrupt vector for handling the counter-overflow interrupt.
  2) Initialize the APIC PERF local vector entry to enable handling of performance monitor counter overflow events.
  3) Provide an entry in the IDT that points to a stub exception handler that returns without executing any instructions.
  4) Provide an event monitor driver that provides the actual interrupt handler and modifies the reserved IDT entry to point to its interrupt routine.

I am having some difficulty trying to understand the third step. What is actually a stub exception handler? Is it an error handler? Can I assign the PMC handler (from step 2) to also be the stub exception handler?

Thank you all in advance.

[Brendan]

Hi,

In the Intel manuals is stated that when interrupted by a counter overflow, the (PMC) interrupt handler needs to perform the following actions:

I couldn't figure out which section in which version of the manual you're talking about.

The minimum the PMC interrupt handler would need to do is:

• reset the counter's count or disable the counter
• send the EOI to the local APIC
• return from the interrupt

Of course if you do nothing else, then there'd be no point generating the interrupt in the first place. Usually you'd want to actually do something.

For example, you might want to gather some information (like the EIP at the time the interrupt occurred) and then store that information somewhere (in an array or something?) so that the information can be processed/analyzed later.

A second question is referred to the stub exception handler. In Intel manuals is stated that the steps required (by the OS) for being able to generate a local APIC interrupt when a performance-monitoring counter overflows are the following:

I couldn't figure out which section in which version of the manual you're talking about here either.

I am having some difficulty trying to understand the third step. What is actually a stub exception handler? Is it an error handler? Can I assign the PMC handler (from step 2) to also be the stub exception handler?

I think what Intel are saying is that: Normally you'd only enable the performance monitoring counters when you actually need them (e.g. when the user starts something like Intel's "VTune analyzer"). Therefore, normally (when performance monitoring counters aren't being used) you'd have a "dummy" interrupt handler that does nothing, just in case the performance monitoring counters weren't disabled properly or something. When the user starts the profiler you'd have to replace the normal/dummy interrupt handler in the IDT with something else that gathers information and passes it to the profiler.

I'd do it differently - e.g. have one interrupt handler that gathers performance monitoring info, and then have a "where to send info" variable. At the beginning of the interrupt handler I'd do something like "if(where_to_send_info != NOWHERE") { gather_info(); }"). Of course I'd also reconfigure the performance monitoring stuff during task switches to get "per thread" performance monitoring (so you can have 123 threads all with their own independent performance monitoring, rather than global performance monitoring).


Cheers,

Brendan

[limp]

Hi Brendan,

In the Intel manuals is stated that when interrupted by a counter overflow, the (PMC) interrupt handler needs to perform the following actions:

I couldn't figure out which section in which version of the manual you're talking about.

A second question is referred to the stub exception handler. In Intel manuals is stated that the steps required (by the OS) for being able to generate a local APIC interrupt when a performance-monitoring counter overflows are the following:

I couldn't figure out which section in which version of the manual you're talking about here either.

Both of them are under the “Monitoring Counter Overflow” section in Vol. 3B (I’ve checked the March 2009 and May 2011 manual versions).

The minimum the PMC interrupt handler would need to do is:

• reset the counter's count or disable the counter
• send the EOI to the local APIC
• return from the interrupt

What I do is stopping the counter, send an EOI to LAPIC and then return. However, I get a “General Protection Fault” when I return.

I am having some difficulty trying to understand the third step. What is actually a stub exception handler? Is it an error handler? Can I assign the PMC handler (from step 2) to also be the stub exception handler?

I think what Intel are saying is that: Normally you'd only enable the performance monitoring counters when you actually need them (e.g. when the user starts something like Intel's "VTune analyzer"). Therefore, normally (when performance monitoring counters aren't being used) you'd have a "dummy" interrupt handler that does nothing, just in case the performance monitoring counters weren't disabled properly or something. When the user starts the profiler you'd have to replace the normal/dummy interrupt handler in the IDT with something else that gathers information and passes it to the profiler.

Hmm…your interpretation makes sense, thanks. I initially thought that this “stub handler” they’re saying would be something like an error handler (like the one LAPIC has). It’s really frustrating when you faced up with terms like this for which there isn’t any definition.

I'd do it differently - e.g. have one interrupt handler that gathers performance monitoring info, and then have a "where to send info" variable. At the beginning of the interrupt handler I'd do something like "if(where_to_send_info != NOWHERE") { gather_info(); }"). Of course I'd also reconfigure the performance monitoring stuff during task switches to get "per thread" performance monitoring (so you can have 123 threads all with their own independent performance monitoring, rather than global performance monitoring).

Cool stuff!

Cheers.

[Brendan]

Hi,

What I do is stopping the counter, send an EOI to LAPIC and then return. However, I get a “General Protection Fault” when I return.

You're definitely not the first person to get a general protection fault on IRET. Common causes include mismatched pushes and pops (e.g. pushing more on the stack than you pop off it), failing to save/restore used registers and trashing the stack.

You could post the code (and/or a disassembly of the code) for the interrupt handler if you can't find the problem. I can understand how it'd be hard to debug when it can't be tested under an emulator like Bochs.


Cheers,

Brendan

[limp]

Hi Brendan,

What I do is stopping the counter, send an EOI to LAPIC and then return. However, I get a “General Protection Fault” when I return.

You're definitely not the first person to get a general protection fault on IRET. Common causes include mismatched pushes and pops (e.g. pushing more on the stack than you pop off it), failing to save/restore used registers and trashing the stack.

You were right! I don’t know how but I was accidentally doing some extra pushes on the stack than I should. I now fixed that and I don’t get any errors.

I am now doing the following in the PMC interrupt handler:

• • reset the counter's count
  • print a message using printf
  • send the EOI to the local APIC
  • return from the interrupt

The problem is that although I get the first PMC interrupt, I don’t get any further ones.

I did some checks by putting some printfs after resetting the counter’s count and I verified that this is done correctly and I also verified that the counter continues to count up from the reset value. I also verified that the counter overflows after resetting it but for some reason, there are no interrupt generated. Do you happen to know if there is another int flag that I should clear apart from LAPIC’s one (which is cleared with an EOI)?

I can’t think of something else that I missed really; any suggestion will be really appreciated.

Kind regards.

[kumarjat]

Hello,

I am trying to setup Performance Monitoring in a bare born os called PintOS. I have managed to start/stop/read/configure the performance counters using rdmsr and wrmsr instructions. I want to generate the PMI on counter overflow but it is not working as of now. I have ensured the following:

1. The counter (MSR_P6_PERFCTR0) is indeed overflowing
2. INT bit is set in MSR_P6_EVNTSEL0
3. LAPIC is enabled and I have set memory mapped address 0xFEE00340 to (~APIC_LVT_MASKED)&APIC_DM_NMI, which means unmaking the corresponding LVT entry and using NMI for passing on the interrupt. I have also tried without APIC_DM_NMI and setting LOCAL_PENDING_VECTOR in the interrupt vector address field.

It would be great if you could please guide me if I am missing something or could provide me some useful pointers on this.

Thanks in advance

[parfait]

The problem is that although I get the first PMC interrupt, I don’t get any further ones.

I did some checks by putting some printfs after resetting the counter’s count and I verified that this is done correctly and I also verified that the counter continues to count up from the reset value. I also verified that the counter overflows after resetting it but for some reason, there are no interrupt generated. Do you happen to know if there is another int flag that I should clear apart from LAPIC’s one (which is cleared with an EOI)?

I can’t think of something else that I missed really; any suggestion will be really appreciated.

Kind regards.

Hello Limp
Have you succeeded in solving this problem?
I am having exactly the same problem now, I receive the first interrupt, reset the counter to the initial count, clear the overflow flag but none of the following overflow interrupt are received...

[Boris]

Hi,
If you do not receive some interrupt:
- Pause your VM (or bochs ), and Check your interrupt flag.
- if it is set, be sure you have sent a EOI