Several vital functions require initialization of physical memory before the page being mapped into the linear address space. For instance, page directory entries should be zeroed before being inserted into the paging system. Another example is demand loading of pages in a user level application. The context must be read from disc and cannot be inserted into the user mode linear address space until this is done. There probably are other examples too, but these are the most obvious.
One solution is to map all physical memory in the kernel linear address space, but this won't work for 32-bit x86 where there is typically many GBs of memory. Another solution would be to temporarily allocate a page frame in kernel, map the physical address, and initialize it. After initialization, it must be unmapped and mapped to the destination address. This causes a TLB shoot-down on all processor cores, which is pretty nasty for handling user level demand loading of pages.
Perhaps there could be a 4k page frame preallocated per core? This would only require a local TLB invalidation, not a global. Additionally, the kernel would not need to allocate a page frame. Still, it would create other problems, like the need to lock the thread from being scheduled.
How to initialize pages in x86 SMP kernel
Re: How to initialize pages in x86 SMP kernel
There are ways to make it work there as well, but it gets very complicated. Look through the Linux source code for CONFIG_HIGHMEM. And keep a barf bag handy.
I did ponder about using a known temporary address. You also have to preallocate the page directory (and possibly page directory pointer), and then you just remember where the current CPU's temporary page table entry is mapped. Then you can always map any physical page if need be. You cannot share that address with other cores, but it is enough to implement paging.
You do have to have some way to locate CPU-local data, but then you of all people are no stranger to segmentation.
Carpe diem!