TLB and INVLPG

[8infy]

Hi everyone, I'm currently working on memory management in my OS and right now i'm implementing the PageDirectory class.

Currently it always flushes the entire TLB by moving into the cr3 register, however, I know that INVLPG is a better option for flushing single pages.
My questions if I map a new table in the page directory do I have to flush the entire TLB or do I not have to flush anything at all since it wasn't in the cache
in the first place?
Also, can you use INVLPG to flush a page directory entry?
Is my understanding correct that INVLPG takes a virtual address of the page that you want to flush?
And one last thing, should I disable interrupts when doing changes to the page directory/table?

Here's my current implementation of map_page(), which as you can see does flush_all(), however I tried without it and it doesn't cause a page fault.



Thanks.

[Octocontrabass]

Currently it always flushes the entire TLB by moving into the cr3 register, however, I know that INVLPG is a better option for flushing single pages.

Moving into CR3 does not flush global pages. You need INVLPG if you use global pages.

My questions if I map a new table in the page directory do I have to flush the entire TLB or do I not have to flush anything at all since it wasn't in the cache in the first place?

Intel and AMD CPUs don't cache entries marked not-present. As long as you flushed the TLB entry after you first changed the entry from present to not-present, you do not have to flush it again after you change the entry from not-present to present.

Also, can you use INVLPG to flush a page directory entry?

Yes. On all Intel CPUs, and on AMD CPUs with EFER[TCE]=0, INVLPG flushes all upper-level entries. On AMD CPUs with EFER[TCE]=1, INVLPG flushes only the upper-level entries associated with the TLB entry being flushed.

Is my understanding correct that INVLPG takes a virtual address of the page that you want to flush?

Yes. (It's actually the virtual address of one byte within the page you want to flush.)

And one last thing, should I disable interrupts when doing changes to the page directory/table?

Probably. What would happen if you didn't, and an interrupt happened while you were updating an entry but before you flushed the TLB?

Here's my current implementation of map_page(), which as you can see does flush_all(), however I tried without it and it doesn't cause a page fault.

Of course you see no page fault - Intel and AMD CPUs don't cache not-present page table entries.

Instead, you might see a much worse bug: if the page is already marked present, you update the entry without flushing the TLB, so the CPU might silently access the wrong page!

(Also, please don't post screenshots of code.)

[nexos]

To help you understand how the CPU searches for PTEs or PDEs, here is what it does.

First, some program will access memory.
The CPU now has a virtual address. The memory controller uses physical addresses, however, so the MMU will now translate.

The MMU now looks in the TLB for this entry. The TLB is a giant table containing paging entries. Lookups are very fast. The TLB, however isn't needed. It just speeds things up. It was added in the 486, while paging was added in the 386. We write entries to the TLB with INVLPG

If the MMU finds it, it is a TLB hit. That is it, it uses the information int the TLB to access the physical address.

Else, it is a TLB miss. In this case, the MMU now gets the page directory's physical address from CR3. I finds its index in here, and access it via memory addresses. This is slower then the TLB, but works still.

If it can't find it here, it doesn't exist, and a page fault occurs,

Else, the CPU accesses the physical location.

To summarize, the TLB isn't needed. It is just for performance's sake.

[Octocontrabass]

It was added in the 486, while paging was added in the 386.

The TLB was added in the 386, INVLPG was added in the 486. (Imagine a page walk for every memory access!)

We write entries to the TLB with INVLPG

We remove entries from the TLB with INVLPG.

[8infy]

Thanks guys @Octocontrabass @nexos.

Why is it bad to post code screenshots?
The code block formatting on this website doesn't support highlighting and stuff and just looks bland.

[Octocontrabass]

Some visitors to this forum have vision impairments.

I'm colorblind, so syntax highlighting doesn't always work as intended.

[8infy]

Some visitors to this forum have vision impairments.

I'm colorblind, so syntax highlighting doesn't always work as intended.

Ah, this makes sense, sorry. Won't upload code screenshots anymore.

[8infy]

@Octocontrabass a few more questions if you don't mind:

So to flush a page directory entry I would flush any virtual address in the 4MB range of that table?
What if I replace the table completely with a new one that has all different mappings, would INVLPG work then? or do I have to flush the entire thing?

[Octocontrabass]

So to flush a page directory entry I would flush any virtual address in the 4MB range of that table?

If you're using large pages and you're flushing the large page's TLB entry, yes.

If you're not using large pages, it depends on EFER[TCE]. If you've set EFER[TCE], you have to flush a virtual address in the 4MB range of that table. If you haven't set EFER[TCE], flushing any address will flush upper-level entries.

What if I replace the table completely with a new one that has all different mappings, would INVLPG work then? or do I have to flush the entire thing?

You would need to use INVLPG for each page. Possibly all 1024 of them, if all were marked present before the change. You must use INVLPG at least once to flush the page directory entry (even if none of the page table entries need to be flushed).

AMD says you should mark the page directory entry not-present and flush the TLB entries before you update the page directory entry with its new value. I haven't worked out exactly what could go wrong if you update it without marking it not present, but I trust AMD knows what they're talking about.

[8infy]

So to flush a page directory entry I would flush any virtual address in the 4MB range of that table?

If you're using large pages and you're flushing the large page's TLB entry, yes.

If you're not using large pages, it depends on EFER[TCE]. If you've set EFER[TCE], you have to flush a virtual address in the 4MB range of that table. If you haven't set EFER[TCE], flushing any address will flush upper-level entries.

What if I replace the table completely with a new one that has all different mappings, would INVLPG work then? or do I have to flush the entire thing?

You would need to use INVLPG for each page. Possibly all 1024 of them, if all were marked present before the change. You must use INVLPG at least once to flush the page directory entry (even if none of the page table entries need to be flushed).

AMD says you should mark the page directory entry not-present and flush the TLB entries before you update the page directory entry with its new value. I haven't worked out exactly what could go wrong if you update it without marking it not present, but I trust AMD knows what they're talking about.

Thank you!

[linguofreak]

Some visitors to this forum have vision impairments.

I'm colorblind, so syntax highlighting doesn't always work as intended.

I'll add that for completely blind users, images won't work *at all*, even if the text in the image is readable to colorblind users. Using an image to render text more prettily is *never* appropriate.