Cannot add new page tables

[~]

I have basic paging set up and a page fault handler that prints the faulting address.

I have mapped the first Megabyte and up to the size of my kernel.

When I try to add a new page table, right after my kernel, and add addresses to it, a page fault occurs.

It seems that page directory and page table structures themselves need to be page-mapped, before mapping data pages themselves. If I try to build a page table at 32 Megabytes but I have only mapped 1.5 Megabytes, a page fault occurs when I set entries of this new page table at 32MB. The CPU doesn't seem to allow access not even to paging structures that aren't mapped themselves inside virtual addresses.

Is paging supposed to be fault-driven to add protection by indirect events?

Does the CPU disable the paging bit for being able to directly access physical addresses while the page fault doesn't execute IRET to return control?

How can I add new page tables if I no longer have free available memory that is in the already mapped pages once I map addresses 0-kernel_end? Anything new I do will simply trigger a page fault.

----------------------------------------------------------------------------
----------------------------------------------------------------------------
----------------------------------------------------------------------------
----------------------------------------------------------------------------

The solution I've implemented so far is including a static empty page in the kernel used solely for things like adding new page tables and data pages in currently unmapped memory, and a function to change the physical address of that page (or others). That page will always exist with a valid static virtual address in ring 0 and an ever-changing physical address so no page faults will occur while I set up physical addresses with that function. I just manipulate any physical page through that special page as if it was a data view to any other physical address:

Code: Select all

;Inputs:
;      WIDEDX -- CR3 array offset address
;      WIDESI -- Virtual address to modify
;      WIDEDI -- New physical address to point to
;;
align wideword_sz
OPCODE__CPU_x86_32_remap_physical_page_address:
 pushfwide
 push widebx
 push widecx
 push widedx
 push widesi


 ;Get the base address of the specified
 ;page directory in WIDEDX:
 ;;
  mov widedx,[widedx*wideword_sz+CR3_array]


 ;Convert bits 22-31 to page directory entry index offset:
 ;;
  mov widebx,widesi   ;Put virtual address in WIDEBX
  shr widebx,22       ;Only get page directory index bits
  shl widebx,2        ;Multiply index by 4
  add widebx,widedx   ;Go to page directory base address


 ;Read the physical address in the entry
 ;in WIDEBX:
 ;;
  mov widebx,[widebx]  ;Get page directory entry value
  and widebx,11111111111111111111000000000000b        ;Get rid of attribute bits


 ;Convert bits 12-21 to page table base address:
 ;;
  mov widecx,widesi     ;Load virtual address in DX
  shl widecx,10         ;Get rid of bits 22-31
  shr widecx,10+12      ;Get only page table index bits
  shl widecx,2          ;Multiply by 4 to go to the actual entry
  add widecx,widebx     ;Go to the page table base address


 ;Read the physical address of the virtual page table
 ;and only preserve type/attribu bits:
 ;;
  mov widesi,[widecx]
  shl widesi,20
  shr widesi,20


 ;Place the new address instead:
 ;;
  or widesi,widedi
  mov [widecx],widesi


 pop widesi
 pop widedx
 pop widecx
 pop widebx
 popfwide
retwide

[iansjack]

When paging is turned on (always in 64-bit mode) you can only access physical memory (from the CPU) through mapped pages. (A bit of a simplification, as some devices access physical memory directly, but good enough for your purposes.) So, to manipulate page tables, the physical pages used by those page tables must be mapped. The CPU doesn't, of its own accord, disable paging at any point.

[songziming]

Page tables don't need to mapped, but you still have to write page entries into those tables right? Writing page entries to those page tables means you have to access the memory of those tables, thus requiring those page tables to be mapped.

Normally, kernel should map all available physical memory into kernel space (usually identity-mapping), not just the memory used by kernel image. So the kernel can access and manipulate any physical memory on this computer.

[Octocontrabass]

Normally, kernel should map all available physical memory into kernel space (usually identity-mapping), not just the memory used by kernel image. So the kernel can access and manipulate any physical memory on this computer.

I think that depends on your kernel design. If your kernel doesn't need to access all physical memory, why bother mapping it?

It's also impossible on some computers. For example, PAE allows physical addresses with at least 36 bits, and there's no way to map all of that in a 32-bit virtual address space.

[nullplan]

It's also impossible on some computers. For example, PAE allows physical addresses with at least 36 bits, and there's no way to map all of that in a 32-bit virtual address space.

Linux's approach (not saying they're sacrosanct, only that they appear to be quite successful) had been to map all physical memory to the 3GB line. Therefore, they could only use a little less than 1GB of RAM. That was fine until RAM started getting larger than that. So then they implemented a swapping scheme, where certain memory (page tables among it) was mapped at all times, as before, and with the same relationship between physical and virtual address (vaddr = paddr + 3GB), but other allocations could be marked to be process local, and would only be visible to the current process. Since the important allocations usually take up less than 1GB (kernel text and data, most kernel heap, and page tables), this worked very well, even into the time of PAE.

Then AMD64 rolled around, and now they're back to mapping all physical memory into kernel space.

So I guess, the lesson is, apparently you do need to map some memory at all times, while other memory mappings can be swapped out. However, the easiest solution is to go with the direct mapping. Since that is impossible in 32-bit mode, go 64-bit!