Recursive page mapping causes PDE movement

[computafreak]

I've decided to scrap my old kernel (monolithic, horrible design) and try my hand at creating a microkernel. So far, I've been doing fairly well - I can bootstrap multitasking, and do a jump into ring 3; I've learned a lot. However, I've noticed a few 'phantom' page mappings in my page directory. I've traced it down to the allocation of kernel heap page directory entries (so that it doesn't lose state on context switches.)

I use a recursive page mapping, which puts the page directory at 0xFFBFF000, and the page tables in one sequential block starting at 0xFFC00000. I have a higher-half kernel, so also have a mapping at 0xFFF00000 to 0x01070000 (the physical address of my higher-half mapping page table). Apart from that, my page directory is zeroed.

This all works out relatively simple, until I allocate page directory entries for my kernel heap. The code I use to do so is below:

Code: Select all

for(unsigned int i = KernelHeapStart >> 22; i <= (KernelHeapEnd >> 22); i++)
	{
		if((properKernel[i] & x86::PageDirectoryFlags::Present) == 0)
			properKernel[i] = (unsigned int)Physical::PageAllocator::AllocatePage(false) |
				x86::PageDirectoryFlags::Present | x86::PageDirectoryFlags::ReadWrite;
		else
			asm volatile ("cli;hlt" : : "a"(0xBAD));
	}

(properKernel is just the virtual address of my page directory table, 0xFFBFF000)
This should create page directory entries between 0xD0000000 and 0xEFFFF000, the range of my kernel heap. It does this, but it also creates page directory entries which have the correct physical addresses, but have virtual addresses between 0xFFF40000 and 0xFFFBFFFF. I'm aware of the cause of this - the recursive page mapping means that my page directory entries are also viewed as page tables, putting them in that range. But I don't have any idea of a solution.

I've verified that the page directory entries are actually present, and they are. By adding a page table entry to them, I make them show up in Bochs' paging structure viewer. My main concern is the massive loss of virtual address space that I'm seeing.

I would appreciate any help on how to make my recursive page mappings 'right', and how to make my page directory allocations stay where I tell them to.

For reference, I created my page directory layout in assembly; the code I used to do so is listed below.

Code: Select all

section .data
align 0x1000
BootPageTable:	; Contains the initial identity mapping of the first 4 MiB
	%assign i 0
	%rep 1024
		dd i | 111b
		%assign i i+4096
	%endrep

align 0x1000
; This page table is the same as the one above, but gets placed at a different page directory index
KernelPageTable:	; Contains the main kernel page table, mapping the first 4 MiB to 0xC000000
	%assign i 0
	%rep 1024
		dd i | 111b	; Note: at present this is set to be user-accessible. I don't want this when I load a program
		%assign i i+4096
	%endrep

align 0x1000
ReverseMappingTable:
	%assign i 0
	%rep 1023	; The last entry points to the page directory
		dd 0
		%assign i i+1
	%endrep
	dd (PageDirectory - VirtualAddressBase + 11b)

align 0x1000
PageDirectory:
	dd (BootPageTable - VirtualAddressBase + 11b)
	times (KernelPageNumber - 1) dd 0
	dd (KernelPageTable - VirtualAddressBase + 111b)
	; There's a -2 because the reverse mappings go at the end
	times (1024 - KernelPageNumber - 1 - 2) dd 0
	dd (ReverseMappingTable - VirtualAddressBase + 11b)
	dd (PageDirectory - VirtualAddressBase + 11b)

When I boot the kernel, I map it to 0xC0100000 and erase the identity mapping of the first 4 MiB in this way:

Code: Select all

mov dword [PageDirectory], 0
	invlpg [0]

[Gigasoft]

0xFFBFF000 doesn't sound like a possible page directory address with recursive mapping, unless you have multiple mappings. With the page table as at 0xFFC00000, the page directory would be at 0xFFFFF000.

Do you intend to use large pages for the kernel heap? If so, set bit 7 (PS) when setting the PDEs and make sure that the allocated physical memory is actually large. The extra mapping at 0xFFF40000 - 0xFFFBFFFF is not a problem, it can just be ignored. There's no loss of virtual address space beyond the 0x20000000 bytes you have reserved. However, I think that reserving 512MB of physical memory for the kernel heap is very wasteful.

invlpg [0] will only invalidate the first 4K page. You should reload CR3 to invalidate the entire TLB.

[computafreak]

0xFFBFF000 does point to the page directory - the last entry of the second-last page table sets up that mapping. However, I've looked at my code again, and simplified it so that the setup you described is working. When I was looking at implementing a recursive page mapping, I thought it would require two mappings - the last PDE, and the last PTE of the second last page table. That was partially because I looked at some tutorial code to learn about it, and simply thought that was the way it was supposed to be.

No, I'm not using large pages for the kernel heap (although the way it's designed wouldn't make that a problem), just a large number of 4 KiB pages. I had used them previously, but I prefer to have a large amount of granularity, and from what I remember about the layout of the paging structures, it'd be difficult to keep the kernel heap consistent across address spaces.

I've just experimented with my code, and if you're correct about the large mapping between 0xFFF40000 and 0xFFFBFFFF - it's simply me getting confused by the Bochs debugger. For future reference, 0xFFF40000 is simply 0xD0000000, but represented as a page table entry. It doesn't actually use a page table

When I was deciding upon a memory map, I decided that the kernel heap should be 512 MiB large because I felt it was a fairly permissive maximum. I won't always be using that 512 MiB - I've just reserved page tables which dictate how large it can get, so that when I clone a page directory, they get linked. As it is, I've only wasted about half a MiB, as a result of the page directories needing one page each.

As for invlpg, I didn't know about that. I'll update my code accordingly - I though it invalidated a page table, not a page. Thanks

[Gigasoft]

Well, in that case, the pages you've allocated are page tables which are accessed at 0xFFF40000. You should then fill these with zeroes. There's no mystery there.