How can I use INVLPG to flush TLB for a Linux process?

[wangt13]

Hi,
I am learning X86 and Linux.
Now I am trying to figure out how to flush TLB for a Linux process.
So here comes the first question, what is the source parameter for INVLPG instruction?

My codes consists of 2 parts, a Linux kernel module and a user space process.

The user space process malloc a buffer, then pass the buffer address (in the process's virtual address) to kernel module through an IOCTL.
Kernel module will call INVLPG with that virtual address, as follows,
static inline void invlpg(unsigned long addr)
{
asm volatile("invlpg (%0)" ::"r" (addr) : "memory");
}

Then, the 2nd question comes, how can I know this TLB flushing worked?

My understanding of the effect of TLB flushing is buffer access should be slowed down, since TLB is flushed, and memory page table walking should happen (and cost more cycles).
So that I can compare the access time spent before and after the TLB flush, is it correct?

Thanks,
-tao

[alexfru]

Why do you want to flush the TLB? And if you really need to, I bet the kernel already provides a function for that (perhaps, that can do that on all CPUs as well).

[wangt13]

Why do you want to flush the TLB? And if you really need to, I bet the kernel already provides a function for that (perhaps, that can do that on all CPUs as well).

What I want to do is just to learn the INVLPG could work in my way.

You are right, Linux kernel does provide this kind of API, I will test that later (in SMP).

Thanks,

[alexfru]

What I want to do is just to learn the INVLPG could work in my way.

That's a confusing statement (to me).

What is there to learn?
If you want to see INVLPG effects other than timing, you need to break some code by removing this instruction from it instead of adding it.

What's your way?

[fpissarra]

What I want to do is just to learn the INVLPG could work in my way.

INVLPG is a privileged instruction. Will not work on ring 3 (userland).

[wangt13]

What I want to do is just to learn the INVLPG could work in my way.

That's a confusing statement (to me).

What is there to learn?
If you want to see INVLPG effects other than timing, you need to break some code by removing this instruction from it instead of adding it.

What's your way?

Let me provide more details I want to learn.

INVLPG needs a parameter, I think it should be virtual address.
Now, I want to learn first is what is the virtual address?
It means, in which context the virtual address is, kernel space, or user space?

For example, tmp = malloc(0x1000) in user space, and to flush the TLB for that PFN translation, what should do?
INVLPG is a privileged inst. so it needs to tell kernel to do that, so is it OK to pass the tmp directly to the kernel (to the INVLPG).
If so, I think it should be in the process context, that is the CR3 rooted tree.

And my test showed this worked.

If I am wrong, please correct me.

Thanks,

[alexfru]

INVLPG needs a parameter, I think it should be virtual address.

Right. Did you know you could download CPU documentation for free? E.g. here.

Now, I want to learn first is what is the virtual address?

It's explained in the documentation and other places, e.g. Wikipedia.

It means, in which context the virtual address is, kernel space, or user space?

The context is the current address space. It may have only kernel-accessible pages or those and user-accessible ones. INVLPG doesn't care which of the two the address points to.

to flush the TLB for that PFN translation, what should do?
INVLPG is a privileged inst. so it needs to tell kernel to do that, so is it OK to pass the tmp directly to the kernel (to the INVLPG).

That's pretty much the only way to do it (reloading CR3 would flush the entire TLB (except the so-called global pages), not just a single page; there's also INVPCID in newer CPUs, but to keep things simple let's not go there).

If so, I think it should be in the process context, that is the CR3 rooted tree.

When you switch the process context, you switch the address space (and change CR3). INVLPG affects the current address space.

[wangt13]

[quote="alexfru"][quote="wangt13"]
Thank you for the detailed reply.

Thanks,