kernel address space mapping

[slide_rule]

I'm having trouble understanding a couple design descisions: I just read Tim Robinson's memory management tutorial, and his idea of mapping the page directory into itself raised a couple questions. As I understand it, the idea is so the kernel can get at the page directory/tables via virtual addresses as well as physical. He mentions that an alternative is to map the physical memory linearly into the kernel address space (linear address n+c maps physical address n), but mentions the 1GB limitation on this. I know that Linux opts for the linear mapping (with the highmem shuffle to get to pages that it can't directly map), but I haven't figured out precisely why.

Mr. Robinson seems to be saying that mapping the pgdir into itself obviates the need for that linear mapping, which (to me) implies that getting to page tables/directories is the only time the kernel would need to address a physical location in RAM. This seems too good to be true.

I guess my questions are these: Why does Linux directly map physical ram into the kernel address space? If there is need for the kernel to address ram physically (not just for the sake of page allocation, but actual dereferencing), how does Mr. Robinson's method address that? (no pun intended)

My plan is to find out the hard way by adopting the self-referential PD method and seeing where I get into trouble (my favorite way of learning ) but I thought I'd ask anyway and give people wiser than I a chance to yell "No! Don't do it!" first.

[iammisc]

Uhmmm... well it really depends on your kernel. I believe actually that you could completely avoid mapping the page directory into itself(if you really want to). However, I would recommend mapping it into itself. But I have other ways of accessing physical ram. I reserve a 1 page area in virtual memory, where I could map any physical page I wanted and do operations on it. I have a page for accessing stacks of other processes(For process creation) and for IPC.

But that's about it. At least you would have to map the page directory.

[hailstorm]

The way I see it, somebody correct me if I'm wrong, is that you are confusing two things.

First of all, identity mapping (physical addr = virtual addr) has nothing to with a pagedirectory containing an entry with a physical address to itself, allthough using pgdir mapping can help perfectly when you don't want to use identity mapping all of the time.
I believe that the linux kernel uses identity mapping when booting and I also believe many others here use the same functionality when booting up. But when paging has been initialised, and pgdir points to itself, you can drop identity mapping. That is, when you initialised your pagetables some way that your kernel also has a virtual address space.

For example; my kernel's virtual address is the kernel's physical address + 3 gigs (0xc0000000), meaning that all virtual memory below this border can be used by the user process. This also means that the formula phys = virt - 0xc0000000 or virt = phys + 0xc0000000 can be used.
At the same time, I let the last entry of each pagedirectory point to the pagedirectory itself. This way, I can use a virtual address for the pagedirectory AND its pagetables.

I hope this helps somehow, because I somehow have the slightest feeling I didn't answer your question...

[JamesM]

I have written two kernels, one a rewrite of the other. In the first I used the recursive page directory method, in the second (currently in development) I use a different method.

Mr. Robinson seems to be saying that mapping the pgdir into itself obviates the need for that linear mapping, which (to me) implies that getting to page tables/directories is the only time the kernel would need to address a physical location in RAM. This seems too good to be true.

The only time that I can see you wanting to access physical RAM is (a) When copying a page across virtual address spaces (for example using copy-on-write, and one process does a write), and (b) when using DMA.

However, accessing physical RAM directly (when it is not mapped into the current virtual address space) is costly: it normally involves disabling paging and interrupts and doing a manual copy, then reenabling them again. It involves wasted CPU time, delayed interrupts and a flushed TLB, so should be avoided at all costs. That is why page tables and entries are mapped into each virtual address space.

The page directory -> itself method DOES work. Definately. However it does have it's drawbacks. Firstly you lose one pgdir entry, which is of course 256MB of address space. Secondly the bit twiddling involved in accessing it is a little complex and easy to get horribly horribly wrong.

I use a different method. My kernel heap is mapped in across all address spaces at 0xc0000000. My kernel heap allocator can be made to give me a chunk of virtual memory on a page boundary, and also return the physical address of the chunk allocated.
To make a page directory or table, I tell it to allocate on a page boundary and give me the physical address. My page directory has an array of pointers to page tables (virtual addresses) and also an array of the physical locations of those page tables (also page aligned). When changing address spaces, I set CR3 to the address of the array of physical locations of page tables (wow, that was a mouthful). So now I can access any page table or page table entry from any address space with the minor implication that each directory uses 4096 bytes more memory. That's something I'm able to live with!

I hope those explanations helped a little bit...

[AJ]

Firstly you lose one pgdir entry, which is of course 256MB of address space.

Isn't it 4MiB of address space that you would lose in 32-bit virtual memory space?

I've just been playing with the same mechanism in 64 bit address space - here you lose 512GiB of virtual address space from your PML4, but when you have 256TiB of nemory to play with, who cares? This system seems to work nicely in practice (it gets a little confusing - pml4 == pdpt == pd == pt) on the emulators I have but I still need to try it on real hardware.

Cheers,
Adam

[slide_rule]

I have written two kernels, one a rewrite of the other. In the first I used the recursive page directory method, in the second (currently in development) I use a different method.

The only time that I can see you wanting to access physical RAM is (a) When copying a page across virtual address spaces (for example using copy-on-write, and one process does a write), and (b) when using DMA.

However, accessing physical RAM directly (when it is not mapped into the current virtual address space) is costly: it normally involves disabling paging and interrupts and doing a manual copy, then reenabling them again.

Let me just make sure I understand: Linux maps physical addresses into the kernel address space so it doesn't have to disable paging to access physical memory, while the recursive method would have to, correct?

Here's what occurs to me:

a) DMA: find 128K of memory below 16MB and identity map it, so you can just give the virtual address to the DMA controller and forget about it. This is problematic if you don't have useable memory down there, but that's going to make problems with DMA anyway, right?

b) COW: allocate a new page and map it somewhere in your current address space. Copy the old page into it using virtual addresses, then unmap the old page from the "current" process (the one that threw the exception) and remap the fresh page into the old page's virtual address. Mark old page as read-write so the other process wont throw an exception. Would that work?

[JamesM]

Firstly you lose one pgdir entry, which is of course 256MB of address space.

Isn't it 4MiB of address space that you would lose in 32-bit virtual memory space?

Cheers,
Adam

You're quite right. Sorry.