memory mapping with paging

[MTermi]

As it seems, when paging is enabled, and one allocates memory from some allocator to fill some page table with entries and put the address of that page table in one of the page directory's slots, there is a problem.

The address of the page table is a virtual address, wheter the page dir needs to be loaded with a real address for the cpu to understand. Or am i wrong?

Also, the same problem happens when u create a new process, create a page dir for it ( in virtual memory). But now u need to load the real addres of the page dir in register CR3.

So some kind of mapping is needed. As i read, a possibility is recursive mapping of pages.

I have organized the physical memory in pages. For each page, using tis page nr, its physical address can be computed as page nr * PAGZE_SIZE.

i have a function/allocator (vm_request_mem), which allocates real pages. Now, i wrote a function which returns a page table/direcotry:

uint32 *alloc_pagetable()
{
vm_page_frame *page = vm_request_mem();

// lock page
page->flags = (VM_READ | VM_WRITE | VM_LOCKED);

uint32 *real_addr = (uint32 *) PAGE_SIZE*page->p-nr;
uint32 i;

for( i =0; i++; i<1023)
real_addr = 0 | 2; //initialize all entries as unused

/* sets its last item, item 1023, to its real address */
real_addr[1023] = real_addr | 3;

return real_addr;

}

// when u need the real address of some page table/directory
uint32 *getRealAddrPagetable(vm_page_frame *p)
{
return (uint32 *) p[1023];
}

So the last item (1023) of a page table stores its real address.
Is this what i meant by recursive mapping?

But there is a problem then, what if the process to which this page table belongs, tries to access item 1023?
e.g. for page dir entry 0, page table entry 1023, this is virtual address : 0*4MB + 1023 * 4096B = 0x003FF000
This will map to the physical address of this pagetable!

SDo i do this wrong?
Is there a simpler/better way to do this mapping?
I think memory management is the most complex part of a kernel

[ces_mohab]

But there is a problem then, what if the process to which this page table belongs, tries to access item 1023?

I think that only kernel process have this kind of mapping.

I think memory management is the most complex part of a kernel

Really When I was working on memory manager I thought that, when I worked on multitasking I also thought that, also hardware drivers are very complex.
;D

[DynatOS]

Simply do not allow non-privileged access to that last Page Directory Entry. Make sure it is Ring-0/Read-Only privileged to prevent subversion by mailicious Ring-3 code.

If you are having issues in the same protection Ring (i.e. Ring-0 overwrites), then you need to exercise a little more care and control in your programming.

[MTermi]

right, don't allow access. But if the tables are of some user process, and the linker links some user code to that adress then i can restrict it to access that address but still there will be a problem, unless i write a linker for my os? :-\

Also, for the page directory, this means that the upper 4 MB of the 4 GB virtual space is not usable, ans of every possible 1024 page tables, their upper 4 KB.

Isn't there a better solution for mapping physical addresses and virtual ones? someone? i realy don' like this solution.

[DynatOS]

You are talking Virtual Address space here, not Physical RAM. Where is it exactly that you have your kernel mapped? The most common method these days is to take up the last GB of Virtual Memory and reserve it for the Kernel only... very few OSes have any problem with that. Any user process that takes up more than 3GB of Virtual Address space (thus solely hogging any RAM/VM) is, without any question *whatsoever*, a poorly designed program and should not be supported in the first place.

It is good that you keep such detail in mind, but once you start to get used to the nuiances of the Paging mechanism you can easily find yourself allocating a 4MB block (PDE) for something much less than 100KB of Physical RAM in some instances. It is hard to explain now, but once you get deeper into Paging and kernel design, you will understand why.

Just keep chopping away at your OS development with this quote in mind... "Make it work, then make it work better"... and you will find yourself answering more of your own questions when faced with how the *actual* design needs to work. Believe me, most questions you will find at this place are idealistic, and some with blatent disregard to the reality of things altogether.

[Habbit]

Any user process that takes up more than 3GB of Virtual Address space (thus solely hogging any RAM/VM) is, without any question *whatsoever*, a poorly designed program and should not be supported in the first place.

So I take you think DB servers are "poorly designed and should not be supported". There is a kind of programs, of which databases are the most known example, for which every single MiB of additional memory is a gift of heaven. That's why Windoze has the /3GB kernel option, and is a very big part of the reason to move to 64 bit architectures.

Just keep chopping away at your OS development with this quote in mind... "Make it work, then make it work better"... and you will find yourself answering more of your own questions when faced with how the *actual* design needs to work.

That is indeed a good quote as long as you keep your "final, ideal design" in mind and ensure the interface will support it from the beggining. If you just throw quick and dirty functions at first (so it works) and then try to replace that API with a better designed one, you'll soon find yourself in a compatibility nigthmare inside your own OS, not to talk of application compatibility problems when modifying APIs in deployed, production OSes.

[Kevin McGuire]

A solution with disregard to complexity:

Write a structur:
struct tmminterface{
unsigned long (*pAlloc)(unsigned long *result);
unsigned long (*pFree)(unsigned long);
}; typedef struct tmminterface MMINTERFACE; typedef struct tmminterface* PMMINTERFACE;

Write a function to create a page directory:
void vmCreate(PAGEDIR *forward, PAGEDIR *backward, PMMINTERFACE *pmm, unsigned long flags);

The idea goes like this:
1. Use pmm->pAlloc() to get a free page, clear it for the directory.
2. Do the same for backward.
3. Create a table for both at pre-choosen offset.
3. Shadow both directories and tables in forward.
4. Add a entry for both directories and tables in backward.

The forward is a page directory that holds a map for virtual to physical, while backward holds physical to virtual. The processor never reads or uses backward, but does forward -- cr3 is loaded with forward. PMMINTERFACE allows you to switch into a different physical page memory manager once you set bit 31 in cr0.

void vmAlloc(PAGEDIR *forward, PAGEDIR *backward, ADDRESS phy, ADDRESS virt, PMMINTERFACE *pmm, unsigned long flags);

This is the base function, most functions will link right into this one. What it does is calculate the directory and table index for "ADDRESS virt" and places "ADDRESS phy" or'ed with flags there. Nothing speacil. The trick is making the function work in non-paged mode and paged mode. This is done by checking if cr3 has a value. if(getcr3() != 0). If so you tranverse into the backward map and lookup the virtual address eqivilent -- since in your code you should be shadowing all tables you create.

If a user mode call is made to vmCreate then have it fill the first X number of table links in the directory page to the kernel's copies. Do not allow the kernel to allocate higher than X number of table links in it's directory page. You could go high making 3G(user)/1G(kernel) or whatever you like.

Once, you have done all the mapping in non-paged kernel mode switch the functions in PMMINTERFACE to something more appropriate like a linked list of 1022 entry lists of free physical pages. The original functions that are placed in PMMINTERFACE can be ones that are initialized with a range or more of memory addresses.

mmSimpleConf(ADDRESS offset, PAGECOUNT count);
mmAlloc();
mmFree(ADDRESS phy);

Then switch the pointers to something else after you get pagging turned on and dump the remaining pages from the old interface to the new, "mmDumpPages" or such other wierd concoction of sphegetti code.

The summary is use two page directories; one for the virtual map and the other for the reverse lookup of tables and directories.