OSDev.org

Hi!

I'm testing my kernel heap by kmallocing in a loop. The heap asks the low level memory manager for a page everytime the heap runs out. Everything works great while in PDE 0 with PTE's 0-1023 but when fetching a page from PDE 1 and page fault is raised with the present flag.

I figured that in the page fault handler I need to set the present flag of the page in question to 1, correct? After that I'm supposed to restart the instruction that caused the page fault but how to do that?

A strange thing I noticed that if I start allocation from physical address 0x400000 the RW flag is also set... By default I start allocating from physical address that grub gives me in the multiboot header, which happens to be 0x100000

The #PF is generated with an error code pushed on stack.
I would do something like:
The handler would do:

* If page not present
- decide if a page should be allocated, I do this by marking a special bit on the PTE entry after page_alloc, so the handler know if the address has called page_alloc or is it really a fault.
- I specially handle the topmost 4MB so that a page will always be allocated for new PTE entry.
- do other validations and accounting
At the end if the handler decide to allocate page(s) for the fault, it request the page allocator, update the page table and invlpg(); otherwise terminate the application or panic.

* You may also implement other leet features like CoW in the handler

The special bit sounds like a good idea. The rest is pretty much what I have...

Just having a hard time googling/understanding how do I re execute the instruction that caused the fault.

The fault will be resume upon IRET.
If you need to terminate the faulting process it become a bit tricky.

Ah. Yeah.

There may be something else strange going on though...

Lets say I begin to allocate 0x1000 chuncks. Now the manager successfully maps pages on PDE 0 for PTE's 0-16 but when trying to get the page 17 it faults...

Virtual addresses:
Page 15 <-- 0xF810
Page 16 <-- 0x10810
Page 17 <-- Should be: 0x11810 but this is where it faults and the address stored in CR2 is: 0xF000C661

Shouldn't the faulting address in CR2 be 0x11810 which was the virtual address that I tried to allocate a frame for?

The address in CR2 maps back to PDE 960 PTE 12.

I'm confused...

Yo:

I didn't really read thoroughly, but I think more information is needed if you want better guidance.

First, have you implemented demand paging for your per-process VMM? If not, then there should be no reason for you to get a page fault when allocating memory or when accessing memory that has been previously allocated. Demand paging allows you to "fake-map" a page and set it as "not present". An application asks for a large amount of memory, so you cull it and commit a small amount of physical memory, while returning the full quantum of virtual memory, fake-mapping the rest of it that is uncommitted. If you don't support this feature or something similar like swapping, etc., then you should not be getting page faults for kernel heap allocations.

Apart from that, a few vaguely targeted repsonses to your questions:

(1) You shouldn't unquestioningly set the present bit on a page that has been faulted on. Applications, including the kernel should only access pages that they have been allowed to by the kernel. If a process (including the kernel) accesses a page that has not been given to it by its VMM, it should be terminated. In the case of the kernel, you will have to halt execution due to address space corruption or rogue pointers and print debug information.

(2) As a rule, unless you deliberately set a page to fault (fake-mapped, swapped out, or other reason) it should never generate a fault. Outside of deliberate provision, a page fault equals a process being terminated for illegal memory accesses.

(3) I remember getting a similar error with the kernel faulting on pages beyond the first PDE given to the heap. It had something to do with me not properly flushing entries IIRC, but I can't remember.

--Good luck
gravaera

Hi,

eino wrote:There may be something else strange going on though...

Some general advice:

• Before you implement a virtual memory manager, test the daylights out of your physical memory manager. If there's a bug in your physical memory manager that allows the same physical page to be allocated a second time (including pages that were in use when the physical memory manager was initialised), or allows something that isn't RAM to be allocated (e.g. a page that is half RAM and half EBDA or something) then it's going to seriously confuse things when you start implementing the virtual memory manager. For example, your virtual memory manager could have no bugs at all, but have problems due to bugs in the physical memory manager; and these bugs may not make any sense (e.g. a page table somewhere that was working fine, that starts being corrupted after the physical memory manager allows the underlying physical page to be allocated a second time). This testing should include testing for race conditions on multi-CPU systems (continually allocate/free pages with as many CPUs as you can at the same time); and testing on systems with different physical memory maps (maybe there's a bug that only occurs on systems with 1 GiB or more RAM, or maybe there's a bug that only occurs when the start address of a RAM area isn't aligned on a page boundary, or..).
• When implementing a virtual memory manager, not invalidating TLB entries correctly is a major cause of confusion - you can end up with code that works fine most of the time, but occasionally has strange behaviour on some CPUs under some conditions; and it can be very hard to figure out what the problem actually is. To help with that, use something like "#define STRICT_TLB_FLUSH" to enable a "don't use global pages and reload CR3 every time any page table or page directory is modified for any reason" mode. Then if you have strange problems you can see if it's related to TLBs by checking if the problems disappear when you enable "STRICT_TLB_FLUSH" mode.

eino wrote:Lets say I begin to allocate 0x1000 chuncks. Now the manager successfully maps pages on PDE 0 for PTE's 0-16 but when trying to get the page 17 it faults...

Virtual addresses:
Page 15 <-- 0xF810
Page 16 <-- 0x10810
Page 17 <-- Should be: 0x11810 but this is where it faults and the address stored in CR2 is: 0xF000C661

Shouldn't the faulting address in CR2 be 0x11810 which was the virtual address that I tried to allocate a frame for?

Depends - maybe an IRQ handler crashed, and the page fault has nothing to do with allocating the page.

The number 0xF000C661 looks suspiciously like a (real mode) IVT entry (e.g. "0xF000:0xC661") to me, which could indicate that you've accidentally replaced something important (e.g. kernel data?) with the page at physical address 0x0000000 (which still happens to contain data from the real mode IVT). It could also just be a coincidence.


Cheers,

Brendan

Allright! Thanks guys.

This is solved for now. The physical memory manager was indeed returning physical addresses that were already in use. This was an easy fix when I finally figured it out with your help. There was also an issue with the heap function that asks for a new page if the heap runs out of memory. The problem was that when the address returned + the size asked for got over the page boundary the next page asked for was not aligned. I have a temp solution for this now but I'm not ofcourse satisfied with it because it causes huge fragmentation by skipping the page. fortunately I do have an idea how to make it better thou.

Thanks!