Paging again

[Whodoo]

I?ve search and searched the net but this damn paging is hunting me. I know you use page directories to "map" page tables that maps the memory.. but in an other topic I read that you "deal out a page dir to each process"

But how do I "give" a process a special page dir? (so it will use that dir an not the other). I dont really understand that, since I need to trick two different processes so that One logical adress is Two different physical adresses (one for each one of the processes) like

[Proc 1] [Proc 2] [Proc 1 part 2]
0x... 0xFFFF 0xAnother

Cause both Proc 1 and Proc 2 will use logical 0xFFFF but when Proc 1 does so, I want it to refer to 0xAnother. Really dont understand how this works.

[Whodoo]

Thought that it would be something like.. Each Page can store 4 kb.. Each Page table can then store 1024 Pages. in 1024 page Directories(holding 1024 page tables each), you can then store about 4 gig.. and I make a complete own page directory for every task/process..
and then I switch between different directories using CR3?

[Candy]

Thought that it would be something like.. Each Page can store 4 kb.. Each Page table can then store 1024 Pages. in 1024 page Directories(holding 1024 page tables each), you can then store about 4 gig.. and I make a complete own page directory for every task/process..
and then I switch between different directories using CR3?

It's EXACTLY 4GB. It's defined as 4 * 1024 * 1024 * 1024.

(Note: some people want you to call it a GiB or a Go or even Gio, to avoid physics people misunderstanding it as 4*1000^3. The o comes from french afaik, and stands for Octet, since they don't want to call them bytes...)

You make a page directory for each process (just decide on a number and make it his). You put this number in a process structure in your process table (you need a process logic stuff, try looking at my ProcessManager module (kernel/core/pm)). This page directory is like giving out a roadmap to somebody telling where each street is. You put on the streets he needs to know, and leave out the ones he shouldn't know. He can then ask a cab (or something) to go to his fourth street, which is god knows where, and he'd end up at a place he'd call 4th street. On somebody elses map it might be somewhere else entirely, or even at the same place. Nobody has the maps of anyone else, so they can't visit each other in most cases. They can if you make it identical (say, point a few PD entries at the same page tables).

You have the master map, telling you who has what map (pointers again), plus where you yourself can live. Your home is mapped into each process' map, so they can all visit you.

This leaves the obvious logic that if you want them to share a page (shmem), you map it on both their maps, and they end up at the same street. If you give it the same number on both, they can switch out pointers (addresses, in the example such as 10 42th street, top floor, third door on the left, or in reality as 0x01042023... do we have a warped view of reality or what? ).

You control the maps so you can even tell them what they can and cannot change. You can make maps on normal paper with a pencil, which means they can change any street location they want to (not a good idea security wise). You can make the map not a part of their address space, or put it in pen-written glossy paper, which they can then not change. You can even protect a few pages against writing (libraries shouldn't be changed), and protect a few against execution (warning, road ends in 15 mile?).

HTH, Candy

[WhoDoo]

Thank you! So each directory is covering a whole segment? so that each process has its own map over the entire memory?

But how does it work if a programmer wanna make a direct pointer like:
char *p = 0xC000
*p = ...;

he probably wants to write to the physical address 0xC000 but he cant because that location is mapped somewhere else? Or it shouldnt be possible for applications to do so? (good protection?

[Solar]

Exactly: User-space applications should not be able to write to a given physical address (since any such kind of hardware-hitting is reserved to the OS).

I get a bit of a deja-vu here... didn't we cover this already?

[Whodoo]

Hee hee we did but as I coded I got them crazy ideas

[Dreamsmith]

he probably wants to write to the physical address 0xC000 but he cant because that location is mapped somewhere else? Or it shouldnt be possible for applications to do so?

Depends on your OS and how it interacts with applications. You CAN allow applications to write directly to physical memory by "identity mapping" certain pages in their page tables, for example, you could make an application's page table point logical address 0xB8000 to physical addres 0xB8000 -- then it could write directly to that address.

Would you want to? Depends. An OS might hand out specific physical memory blocks to whoever wants it first, assuming that if it needs to be multiplexed, the bootup sequence will ensure that memory that does need to be multiplexed will be first claimed at bootup by the application designated to multiplex it, and never relinquished. Anyone else want access, tough, go talk to the app that has it. An exokernel might take this approach to memory management. In this case, you'd probably also give the app some control over its own page tables (but not unlimited control, obviously), perhaps syscalls for altering the page table (with appropriate checks to ensure it doesn't gain access to things it shouldn't this way). That and user-space page fault handling could give any kernel the kind of speed benefits Engler and crew rave about.

Another idea is to do identity mapping selectively and transparently. In my own OS, each application has direct access to screen memory when it has "focus", but when another app gets focus, the first app has its pages redirected to an offscreen buffer. It never knows whether it's writing to real video memory or not, nor should it need to (although it can check, since there might be things it can just not bother doing if it knows no one can see it, such as stop updating the time counter it's displaying, or even forgoing any double-buffering it might otherwise be doing, since it's getting it for free when it's unfocused).

[Whodoo]

I see sounds good.. but spose I only map the memory that the process has access to (the program and mabye some run time allocated memory).. and then the program tries to reach some memory that isn?t mapped (e.g. by using malloc)... does it trigger an exception or what? So the kernel knows that a process?s asking for memory it doesnt have, so the kernel can deal out some more.. (by changing the page directory)

[Dreamsmith]

You got it. When the program tries to access unmapped memory, it generates a Page Fault exception (0x0E), and the CR2 register contains the address that the program was trying to access. The kernel can then load that page into memory, allocate new memory for it, kill the process, or whatever else it wants.

In my own kernel, if the address points to a once valid location that was written to disk, I pull it back. Otherwise, I kill the process -- I don't let processes grow their own address space by page faulting. You could conceivably do this, but it makes debugging harder, as you can't then tell an invalid pointer reference from a request for more memory.

One exception -- I only give processes a 4K stack at start, and I grow it as needed to a limit (if it was unlimited, a program with a bug in a recursive function would claim all available memory before crashing).

[Dreamsmith]

and then the program tries to reach some memory that isn?t mapped (e.g. by using malloc)

I should add, malloc should never attempt to access unmapped memory. When malloc exhausts the pool of memory it already has, it calls the OS to request more (via a syscall to sbrk, morecore, or something to that effect -- depends on the OS).

[Whodoo]

thank you very much, startin to get it now..must write a memory manager for my little OS ;D but I get a strange error:

I?ve written a function for the paging, mapping lots and lots of memory:

void *currentAdress;    // Points to a place where there is free memory
void paging()
{
   currentPage = 0x401000;
   unsigned long *page_directory = (unsigned long *) 0x100000;
   unsigned long *page_table = (unsigned long *) 0x400000;
   unsigned long address = 0;
   unsigned int i;

   for(i=0; i<1024*1024; i++)
   {
      page_table = address | 3;
      address = address + 4096;
   }

   for(i=0; i<1024; i++)
   {
   page_directory = &page_table[i*1024];
page_directory = page_directory | 3;

   }

   write_cr3(page_directory);
   write_cr0(read_cr0() | 0x80000000);
   currentAddress = 0x401000;
}

void *malloc(int size) {
   currentAddress += size;       
   return (void *)currentAddress - size;
}

I call the function from my main-method..All this could be done without paging, but I get a Page-exception so I continue.. I also got this, a simple linked list:

struct nummer
{
   int nbr;
   struct nummer *next;
} __attribute__ ((packed));
typedef struct nummer nummer;

struct nummer *first;
struct nummer *last;

void addNbr(int nbr) {
   if(first == 0) {
      first = (nummer*)malloc(sizeof(nummer));
      first->nbr = nbr;
    last = first;
   }
   else {
      nummer *n = (nummer*)malloc(sizeof(nummer));
      n->nbr = nbr;
      last->next = n;
      last = n;
   }
}

when Adding some numbers I get an exception; Page Fault..dont really understand why

[Whodoo]

Cause I can add one or two elements..and as far as I can see I?ve mapped the entire segment (4 GB)

[Dreamsmith]

...
unsigned long *page_table = (unsigned long *) 0x400000;
...
currentAddress = 0x401000;
...

void *malloc(int size) {
currentAddress += size;
return (void *)currentAddress - size;
}

Your page table starts at 0x400000 and ends at 0x7FFFFF. Malloc is handing out addresses starting at 0x401000. You're clobbering your page table.

[Whodoo]

hmm I dont really understand what you mean..dont I map the entire memory? I wanna (just for fun or something) map the entire memory, but the logical address should be the same as the physical

[Whodoo]

Be being just as stupid as allways, sorry man I get it thanks.. my calculations where horrible