OSDev.org

i think you're doing a melting pot between virtual and physical addresses. PCI memory-mapped are of course mapped to some PHYSICAL memory. once paging is enabled, you can make the video frame buffer (for instance) appear wherever you want in the process address space, just by filling PTEs with physical addresses of the memory array that VBE will have returned to you.