Self-referencing PML4 and when to flush the tlb or invlpg

[gerryg400]

The first thing to be careful of is TLB invalidations. Basically, if you change a page directory entry, page directory pointer table entry or PML4 entry, then you need to invalidate anything in the area you've changed (like you normally would/should) but you *also* need to invalidate the area in the "master map of everything". Of course in most of these cases it's usually faster to flush the entire TLB anyway. For example, if you change a page directory entry then you'd need to do "INVLPG" up to 512 times in the address space plus once in the mapping (or flush the entire TLB), and if you change a page directory pointer table entry then you'd need to do "INVLPG" up to 262144 times in the address space plus 512 times in the mapping (or flush the entire TLB).

I found this in an old thread from last year. I have the self-referencing PML4 trick working well. Haven't seen any problems for a few weeks. When mapping a new page my code loops from the top level tables down to the bottom modifying entries and adding tables as required. But I do a flush_tlb on ANY change. Possibly up to 4 times when mapping in a single page frame?

Code: Select all

int kpage_map(uint64_t vaddr, uint64_t paddr) {

	/* Map pages at the 3 upper levels if necessary */
	p = pml4t
	for (level=0; level<3; ++level) {
		if ((there_is_no_pt_at_this_level) {
			p = kphys_alloc();
			kpage_flush_tlb();
		}
		p = next_level
	}
	/* Map the phys address at the lowest level */
	p = paddr;
	kpage_flush_tlb();

	return 0;
}

(NOTE that I number my levels from 0 at top to 3 at bottom, the opposite of the Intel manual.)

Obviously this is overkill. Do I need to flush the tlb when I make a change at any level ? Because I am adding a new page could I just do an invlpg on the linear address of the new page ? Any advice would be appreciated. I'm keen to understand how this should work but there is precious little information on the internet (except osdev.org).
Thanks.