mapping memory in paging mode

[scarab]

Hi

I've been trying to improve my memory manager but am having problems. This is what I am trying to do:

1. I initialize a stack containing addresses to all the free physical pages
2. I make a page directory and map the kernel memory into it (and the memory for the stack.)

So far the memory has been mapped 1 - 1 so, I have mapped 0x0 - 0xend of kernel and stack

3. i enter paging mode.

now everything works as it should until i call my malloc function, this tries to allocate memory that is marked "not present" in the page directory, so i get a page fault. when i get the page fault i then am trying to get a new page from the stack of free physical pages and map it into the page that caused the page fault.

what happens is, i map in the physical memory address and then get a page fault for the physical memory address (I guess because the location for that address is also not present in the page directory)

how do i put the physical address into the page directory when in paging mode? Is it possible?

thanks
andrew

[mystran]

You don't get page faults for physical addresses, you get them for virtual addresses instead. Once you map a table to the directory, and a page to the table, you don't need to do anything else, except make sure no TLB's are wrong, so you have to invalide those, before you return from the first page-fault.

If you are invalidating TLB's correctly, then you probably have a so called "bug" in your code. Make sure (debug prints?) that the page you get from the stack is "sane", and make sure your page mapping functions are correct. It's relatively easy to get a bit or two wrong.

Anyway, failing that, what addresses are you using where? I mean, on x86 you can be dealing with 3 types of addresses: logical (pre-segmentation), linear (after segmentation, pre-paging) and physical (after paging). Page directory/table entries take physical addresses, but obviously you need to access the page directories/tables through linear addresses. How do you map your page directories into memory? Did you remember to invalidate TLB also when you change mappings of the tables themselves?

[Brendan]

Hi,

how do i put the physical address into the page directory when in paging mode? Is it possible?

Yes it's possible . Assuming you're not using PAE, that you've got a routine to allocate a physical page that never returns an error, and that you insert the page directory into itself as a page table which makes page directory entries accessible (via "PDmap") and page table entries accessible (via "PTmap"):

Code: Select all

allocLinearPage:
    ; ebx = linear address to map an allocated page

    mov ecx,ebx              ;ecx = linear address for page
    mov edx,ebx              ;edx = linear address for page
    shr ecx,12               ;ecx = page table entry number
    shr edx,22               ;edx = page directory entry number

    test [PDmap+edx*4],1     ;Is the page table present?
    jne .hasPageTable        ; yes

    call allocPhysicalPage   ;eax = physical address of new page table
    or eax,7                 ;Set page type flags
    mov [PDmap+edx*4],eax    ;Put the new page table into the page directory
    invlpg [PTmap+ecx*4]     ;Invalidate the page table mapping

.hasPageTable:
    call allocPhysicalPage   ;eax = physical address of new page
    or eax,7                 ;Set page type flags
    mov [PTmap+ecx*4],eax    ;Put the new page into the page table
    invlpg [ebx]             ;Invalidate the page table mapping
    ret

Cheers,

Brendan

[scarab]

Thanks for the reply.

I think I am doing all you have said, I've attached my code for my page mapping function.

void map_page(unsigned long *pgdir, unsigned long virt, unsigned long phy, unsigned short priv) {
   unsigned long dir_ent;
   unsigned long tab_ent;

unsigned long *pde;

   dir_ent = (virt >> 22) & 0x3FF;
   tab_ent = (virt >> 12) & 0x3FF;

// first we need to work out where the pte is and if the pt has been mapped
if ((pgdir[dir_ent] & 0x1) == 0) {
// need to allocate the pt
pgdir[dir_ent] = (unsigned long)kern_heap_alloc() | 3;
reload_tlb();
}
// need to enter pte
pde = (unsigned long *)(pgdir[dir_ent] & 0xFFFFF000);
pde[tab_ent] = phy | priv; // Fails here
}

where I wrote the "Fails here" I think is where the problem is. The first page fault I get it maps a page using this function, firstly it realizes that it needs to allocate a Page Table then reloads the tlb, next it tries to map the page into the newly allocated page table, then it page faults trying to put the page into the page table. The page fault it returns is the address of the newly allocated page table.

To invalidate the tlb i use this function:

mov eax, cr3
mov cr3, eax

any ideas?

thanks
andrew

[scarab]

Hi Brendan

thanks for the reply

Yes it's possible . Assuming you're not using PAE, that you've got a routine to allocate a physical page that never returns an error, and that you insert the page directory into itself as a page table which makes page directory entries accessible (via "PDmap") and page table entries accessible (via "PTmap"):

when you say PDmap is this a pointer to say page_dir[1023] (if this PDE contains a pointer to the location of the page dir? What then is PTmap a pointer to? do I need to point say page_table[1023] to the location of the page table as well? or can I access this through the original mapping (like page_dir[1023] + something

thanks andrew.

[Brendan]

Hi,

Here's your problem:

Code: Select all

    pde = (unsigned long *)(pgdir[dir_ent] & 0xFFFFF000); 
    pde[tab_ent] = phy | priv; // Fails here

The variable "pde" contains the physical address of the page table, which can't be accessed (unless you turn paging off first, which may be slow or might not make any difference - more on this later).

Generally (as long as the page directory is inserted as a page table, and "unsigned long *PDmap" is the linear address where the page directory has been mapped, and "unsigned long *PTmap" is the linear address of the 4 Mb area where the page tables are mapped), then you'd want something like:

Code: Select all

void map_page(unsigned long virt, unsigned long phy, unsigned short priv) {
   unsigned long dir_ent;
   unsigned long tab_ent;

   dir_ent = (virt >> 22);
   tab_ent = (virt >> 12);

    // first we need to work out where the pte is and if the pt has been mapped
    if ((PDmap[dir_ent] & 0x1) == 0) {
        // need to allocate the pt
        PDmap[dir_ent] = (unsigned long)kern_heap_alloc() | 3;
        reload_tlb();
    }
    // need to enter pte
    PTmap[tab_ent] = phy | priv;
    reload_tlb();
}

To invalidate the tlb i use this function:

mov eax, cr3
mov cr3, eax

any ideas?

It depends which CPU's you support. The 80386 does not support global pages or the INVLPG instruction, so for this CPU there is no choice.

For 80486 and above it is much better to use the INVLPG instruction, which will only invalidate the TLB for the area that needs to be invalidated rather than invalidating everything in the TLB. By reloading CR3 you flush everything, so the CPU has to load it's page directory and page table data from RAM even though most of it hasn't changed (and RAM is slow).

Also if your OS uses (or will ever use) global pages, then reloading CR3 will not invalidate pages that are marked as global, which is usually any pages that are mapped into all address spaces (e.g. all pages used by the kernel). Global pages reduce the cost of process/task/thread/whatever switches by keeping any pages marked as global in the TLB when CR3 is changed. The only way to invalidate a global page is to use INVLPG.

Anyway, if your kernel is identity mapped, and if everything your "mappage()" function uses is also identity mapped (e.g. the kernel's stack/s), and if you invalidate the TLB by reloading CR3, then you could disable paging, map the new page and then re-enable paging. Under these conditions it won't be any slower (but I wouldn't recommend the excessive TLB flushing required by these conditions for any 80486 or above)....


Cheers,

Brendan

[scarab]

Generally (as long as the page directory is inserted as a page table, and "unsigned long *PDmap" is the linear address where the page directory has been mapped, and "unsigned long *PTmap" is the linear address of the 4 Mb area where the page tables are mapped), then you'd want something like:

ok... so if for example my page directory was mapped into itself so : page_dir[1023] -> page_dir, then PDmap would equal : 0xFFC00000 and PTmap would equal PDmap[dir_ent]

is that right?

thanks again
andrew

[Brendan]

Hi,

ok... so if for example my page directory was mapped into itself so : page_dir[1023] -> page_dir, then PDmap would equal : 0xFFC00000 and PTmap would equal PDmap[dir_ent]

PDmap would equal 0xFFFFF000 as the page directory becomes mapped as the highest 4 Kb of linear memory. PTmap would equal 0xFFC00000 as all 4 Mb of page tables become mapped as the highest 4 Mb of linear memory.

This means the first page table entry can be accessed via "PDmap[0x0]", and the highest page table entry can be accessed as "PTmap[0x000FFFFF]". For page directory entries the lowest would be "PDmap[0x0]" and the highest would be "PDmap[0x000003FF]".

To make sure you understand what's going on, see if these 2 sentences confuse you. "PTmap[0x000FFFFF]" and "PDmap[0x000003FF]" both refer to the self-reference. All page directory entries can be referenced using PTmap (instead of PDmap) - the lowest page directory entry would be "PTmap[0x000FFC00]" or any page directory entry can be found with "PTmap[0x000FFC00+page_directory_entry_number]".


Cheers,

Brendan

[scarab]

Thanks for all your help, I think I understand it all now. I got the page fault handler working (I think) just getting general protection faults and LDTR=invalid stuff, but I think thats to do with my exception handling. Anyway thanks.

Andrew