Paging, Memory Management and Processor Independant Code

[Therx]

Ok, so not all processors support virtual addresses but lets assume they do just in different ways. I want the main CPU independant kernel to do the complicated memory management but the 'processor layer' (PAL) code todo the mapping. The problem is I can't see a way of doing this which doesn't sacrafice speed or portability.

Because each task will have different matched areas I want to have each task with its own tables which can be initialised/deinitialised just by changing the page directory.

Option 1 - Speed
The main kernel understands how to use the i386 page entries and creates its own tables for each task and only calls the PAL to change the table used for certain areas of memory.

Advantages : Speed
Disadvantages : Specific to i386

Option 2 - Portable
The main kernel has to call the PAL to map every single page. The page directory always points to the same tables and the entries are changed on a one by one basis.

Advantages : Portable
Disadvantages : Very slow when tasks use lots of memory

Option 3 - Halfway House
The main kernel uses a function provided by the PAL to create page tables and then just changes the one used with another call to the PAL to change the page directory.

Comment : This presumes that all processors with memory mapping support will use a directory/table hierachy but allows for changes in the format of the entries. The speed will be slow to begin with while the tables are being created but will speed up once later.

Questions
1. Which would you use?
2. Are there any other options?
3. Should I make it so that the whole memory manager is in the PAL?

Thanks for any help

Pete

[Tim]

Any sensible processor will support paging similar to the 386, with a multilevel page directory/page table structure. Most RISC processors support paging through manual TLB updates (i.e. no PD/PT, you get an exception on a TLB miss and have to fill in the entry yourself) so you can do what you like there.

IMHO it's not worthwhile targetting processors with and without an MMU in the same OS. Each architecture requires very different techniques, so you'd be looking at having two versions of the kernel to do this.

I would put the following functions in the PAL:
[x] Allocate physical page (with options for < 16MB for DMA, etc. as needed)
[x] Free physical page
[x] Allocate page directory
[x] Free page directory (could be the same as "free physical page")
[x] Update mapping for the current page directory
[x] Switch page directories

Clearly any processor without an MMU is going to have trouble with the "update mapping" and "switch PD" functions, with little hope of emulating them (although I suppose a 286 could emulate this, using segments).

[Therx]

Why does physical memory allocation have to be controlled by the PAL? As it is also unworthwile to attempt to write an OS capable of running to 16bit, 32bit and 64bit enviroments, surely the addressing scheme will be the same for all processors (32bit pointers).

From what I gather you're recommending Option 3 in my list. This was what I was probably going to do I was just a bit unsure about whether the ammount of calls needed to initialize a table would be excessive and make the OS run too slow.

Thanks

Pete

[Tim]

OK, maybe the HAL -- it's not processor dependent, but it is dependent on the environment in which the processor is running. Determination of the memory size and layout is ultra platform-dependent -- there are several ways of doing it on the PC architecture alone. The relative layout of RAM and adapter memory changes on different architectures. And the physical memory manager might want to implement page colouring, which requires good knowledge of the cache architecture of the platform.

[Therx]

The assembly stub in my OS is in the PAL and it is called by a multiboot compatible boot loader so I don't need to detect memory myself. The PAL then passes to the kernel the base and limit of the main memory and also passes the base and limit of where the temporary paging tables are. The last peice of info passed is the top of memory.

Pete

[Therx]

If the page directory says read, user, present and then the page table says write, supervisor, present. Which of the two overpowers?

Pete

[Candy]

If the page directory says read, user, present and then the page table says write, supervisor, present. Which of the two overpowers?

Pete

quoting the AMD manual on system programming, order # 24593

"This bit controls read/write access to all physical pages mapped by the page table entry"
and for the U/S bit:
"This bit controls user (CPL3) access to all physical pages mapped by the table entry"

quotes are from sep/2002 version

This in short means that your page will be accessible from supervisor space only, and only for reading.

The present bit indicates about the page table page however, but I suppose you know that

--- update

Present bit also cascades of course, if you cannot lookup any page you can implicitly assume it's not mapped... still... you save 4k because the page table page is also never checked...

[Tim]

In other words, if you can't access the page table, you can't access any of the pages within it.

[Therx]

So which ever is more restrictive applies.

[Candy]

what if you put the cutoff point somewhat further in the code so you potentially duplicate more code, but do get optimal code for all? For instance, making a function such as mem_fork_dup() that would duplicate the entire page table hierarchy for the current process into a new hierarchy for the new process, and thus be optimized for the current platform.

Another option would be to code the OS-independant parts using things seeming like templates in C++, ie, using a tss_t for each task, a pageoffset_t for each page offset. That would allow you to code a alloc_page(pageoffset_t target) that would pass a 32-bit int (example) to the 32-bit memory allocator, when compiled with that one as target, and pass a 64-bit int (example) to the 64-bit memory allocator when compiled with that machine as target.

[Therx]

I suppose I could support IA64 but I doubt my OS will ever need more than 4GB of RAM

I've tried to implement paging and it doesn't seem to do anything. the system doesn't crash but also my attempted mapping of 4mb to the videomemory for testing doesn't work it just seems as though nothing happens.

Here's my code:-

Code: Select all

unsigned long *page_dir = 0x9E000;

void init_vm(unsigned long mem_limit)
{
   unsigned long *page_table = 0x200000;
   unsigned long address = 0;
   unsigned long i;

   // Set up initial page directory
   vm_setarea(page_table, 0, mem_limit, 1);
   vm_setarea(0, mem_limit, 0xFFFFFFFF, 0);

   // Set up directly mapped page table (supervisor, write, present)
   for(i = 0; i < (mem_limit / 4096); i++){
      vm_setentry(page_table, i, address, 1, 0, 1);
      address += 4096;
   }
   
   vm_setentry(page_table, 1024, 0xB8000, 1, 0, 1);

   // Store location of page directory
   asm("mov %0, %%cr3" : : "a" (page_dir));

   // Enable paging
   asm("mov %cr0, %eax          \n"
       "or %eax, 0x80000000     \n"                       // Bit 31 = paging enable
       "mov %eax, %cr0          \n");
   return;
}

void vm_setarea(unsigned long table, unsigned long base, unsigned long limit, char present)
{
   unsigned long i, address;
   unsigned long i_base = (int)(base / 0x400000);
   unsigned long i_limit = i_base + (int)(limit / 0x400000);
   for(i = i_base; i < i_limit; i++){
      page_dir[i] = (address & 0xFFFFF000) + 2 + present;
      address += 4096;
   }
}

void vm_setentry(unsigned long *table, long index, unsigned long address,
                 char write, char user, char present)
{
   table[index] = (address & 0xFFFFF000) + (user << 2) + (write << 1) + present;
}

I suspect the problem is with my inline asm code. I'm not too familiar with it but what I want them to do (I think) is this:- (this is intel format but above is in AT&T)

mov cr3, page_table (0x200000)

mov eax, cr0
or eax, 0x80000000
mov cr0, eax

Thanks for any help

Pete