Access page tables after paging is enabled

[CJ901]

I have managed to enable 32-bit paging and identity map the kernel and video memory. This works fine and I can run my kernel, without generating a triple fault. Since this memory mapping is done before paging is enabled I can access page tables with their physical address.

I'm gonna make a heap manager for the kernel, on top of this virtual memory manager, and therefore need a way to access the page tables after I have enabled paging.

I have read about the self referencing page directory trick but I don't understand how that will give me access to the page tables. With this trick I can only access PDE's and the PDE's only contain the physical address of the table.

As a side note: If I try to access a random unmapped address I will get a page fault. However, if I try to access the page directory, after paging is enabled, the kernel will crash. Why am I not getting a page fault in this case?

[NickJohnson]

There are basically two (good) ways to do this.

The first is what Linux does, and is both the most straightforward to understand and the nicest to the TLB when doing inter-address-space stuff. Its downside is that you waste a ton of virtual memory space and if you want more than ~2GB of memory on 32-bit, things get hairy. For 64-bit it is therefore the obvious choice. Technique 1 simply involves mapping as much of physical memory as you can linearly into the higher half of the address space. The translation from virtual addresses to physical addresses and vice versa is trivial, so manipulating the paging structures is the same after translating all the pointers by a constant. If physical memory is mapped at 0xC0000000, then you would compute *(page_dir_physical + 0xC0000000 + 4 * page_table_number) - 0xC0000000 + 4 * page_offset_in_page_table to get the virtual address of a page table entry (this is simplified and needs to have stuff for masking off flag bits).

The second, known as the "recursive page directory trick" around here, takes advantage of certain properties of the x86/x86_64 paging structure formats to reduce the virtual memory usage of the paging system to exactly 4 MB while also making programming simpler (you no longer even have to walk the page table tree). The downside is that it is arcane, tricky to use for inter-address-space stuff, and probably a bit slower. Technique 2 involves putting the physical address of the page directory into the last entry of the page directory, thereby making it a valid page table as well. When a normal page table is added to the page directory, it then "magically appears" at a corresponding location in the top 4 MB of virtual memory (because it was mapped by the page table that is actually the page directory). This results in all page table entries being located consecutively in the upper 4 MB of memory, including the PTEs that are actually PDEs. This is very convenient, because instead of finding the page directory, translating an entry, finding the page table, then translating that entry, you simply compute 0xFFC00000 + 4 * (page number) to get the virtual address of any page table entry. The page directory entries themselves are just the page table entries from 0xFFFFF000 to 0xFFFFFFFC.

[bluemoon]

Technique 2 involves putting the physical address of the page directory into the last entry of the page directory, thereby making it a valid page table as well.

You can indeed extend the tricky to support mapping page directory into any entry, for instant, I map last entry on 32-bit flat paging, and entry 510 (-512GB ~ -256GB) on 64-bit build, furthermore I use flexible macro instead of hard-code those magic number so that it can be easily changed.

Then, you can also adopt an hybrid mix of the two methods - access local pages with recursive, while have method (1) to case-by-case specially handle situations for cross address-space access, like having a small section of each process mapping globally.

[CJ901]

There are basically two (good) ways to do this.

The first is what Linux does, and is both the most straightforward to understand and the nicest to the TLB when doing inter-address-space stuff. Its downside is that you waste a ton of virtual memory space and if you want more than ~2GB of memory on 32-bit, things get hairy. For 64-bit it is therefore the obvious choice. Technique 1 simply involves mapping as much of physical memory as you can linearly into the higher half of the address space. The translation from virtual addresses to physical addresses and vice versa is trivial, so manipulating the paging structures is the same after translating all the pointers by a constant. If physical memory is mapped at 0xC0000000, then you would compute *(page_dir_physical + 0xC0000000 + 4 * page_table_number) - 0xC0000000 + 4 * page_offset_in_page_table to get the virtual address of a page table entry (this is simplified and needs to have stuff for masking off flag bits).

How much of the physical address space should be mapped? Also, the way I store the tables in physical memory is I ask my physical memory manager to provide the first free block. This means my tables can be anywhere in memory and could mean they're not mapped, unless I map all the physical address space.