Chicken-and-egg dilemma in PMem manager!

[zhiayang]

Wow, I'm pissed. Apparently I just got logged out, and the long post I just typed vanished.
I'm in no mood to write another essay, so here's the short and sweet version.

1. I identity map 4mb of memory at boot, them jump to long mode.
2. I have in place a table of page entry descriptors that 'describe' each page of memory.
3. I have a kmalloc function, that basically finds and hands out free pages.
4. It knows that once it hands out memory past 4mb, it needs to map() the memory or else it gets a #PF (duh)
5. So, I have a test routine that basically allocates memory in a while() loop.
6. Eventually, I reach 4mb. The allocator knows this, and calls map() to 1-1 map the memory.
7. Because I only created mappings for 4mb at bootstrap, these new mappings past 4mb don't exist.
8. The map() routine needs to put these page tables somewhere, so it calls kmalloc() to allocate memory for them.
9. It's past the 4mb limit so... See the problem?


If I don't map the memory in kmalloc(), I get a #PF after 4mb (duh).
If I don't allocate memory for the page tables in map(), they end up somewhere in memory (I'm guessing somewhere in my kernel), I get a #GPF (duh)

So. The question is; is it something wrong with my algorithm/approach to the pmem manager, or is it something obvious I'm not getting?

Thanks, guys.

[thepowersgang]

Quick question - why does the PMM need to care about where the page goes?

I'd suggest writing an "AllocatePhys" function that hands out a physical page address (but doesn't map it), then implimenting "AllocateVirt" (hands out mapped pages) using that. (and then kmalloc atop that - don't call it *malloc unless it acts like C's malloc())

[bluemoon]

Wow, I'm pissed. Apparently I just got logged out, and the long post I just typed vanished.
I'm in no mood to write another essay, so here's the short and sweet version.

If you use Chrome, the input content will be restored if you back.

I can't think of how to resolve your dependency problem right now, but this is what I do in my OS:

@Boot:
- I craft a page table with an entry of specially designated address mapped with 4KB of memory from bss, let's call this page stack: size_t page_stack[];
- The 4MB zone (actually a bit less, I scan the memory map to calculate the amount needed) following page_stack are marked as "allocate on demand" with a flag in the PTE
- Process memory map and push entry into page_stack
* Note that @ 1024-th entry (or 512-th entry for 64-bit kernel), a #PF is generated and the #PF handle allocate memory from the page_stack itself, which already contain some free pages.
At this point, I got page_stack for whole memory map for page frame allocation.

Logical memory management is implemented to manipulate the page directory, and call PMM_alloc / PMM_free if necessary.

k_sbrk is implemented using the concept of regular sbrk, I mark a zone in the address space as allocate on demand, have a heap pointer, and expand it upon allocate.

kmalloc / kfree maintain the "free'd memory, grouped by allocation size into multiple lists; if it run out of memory in the pool it request from k_sbrk.

The #PF handler is a bit tricky to handle allocate on demand.

[mrvn]

There are 4 solutions that I know of:

• create identity mapping for all memory. Don't wait till you hit the 4MB limit before mapping the rest. Map everything first chance you get. Then you can still allocate the new page tables in the first 4MB to map the rest.
• keep a few pages in reserve. Instead of mapping new pages when you exceed 4MB (or whatever the currently last maped page is) start mapping new pages before that. 12k before that in long mode so you always have 3 reserved pages to create a new L3, L2 and L1 page table as needed.
• reserve some virtual addresses for temporary pages. Then when you need to alter a page table you temporary map the page table there, alter it and unmap it again. You need an address that is already coverted by existing pagen tables, so somewhere in your first 4mb.
• use fractal mapping (search for that in the forums). It will cost you one top level slot in the page tables (so 4MB address space in 32bit and 512G in 64bit mode) but no physical memory and all page tables map themself automatically. it truely is a beatifull solution.

MfG
Mrvn