Me and paging don't go togheter :(

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
User avatar
matthias
Member
Member
Posts: 158
Joined: Fri Oct 22, 2004 11:00 pm
Location: Vlaardingen, Holland
Contact:

Me and paging don't go togheter :(

Post by matthias »

Ok I'm at the point of writing a memory manager from scratch, yes for the sixth time now :lol: But again as always I have problems with paging :evil:
The problem lies in the pde's and pte's. When I try to load my pagetable to cr3 my kernel somehow crashes. As a guide I used the following tutorial, looks easy on the eye, but it turns out otherwise. (http://www.osdever.net/tutorials/paging.php?the_id=43) After reading this tutorial and implementing some functions in the assembly part of my kernel I decided to map all physical memory 1:1 into kernel space. But when I get to the point of loading the pagetable it fails. I made sure my pagetable is page aligned (see code) :arrow:
Setting up the page directory is rather easy. We just need to find some free memory that is 4k aligned(if the address can be devided by 4096 and not have a remainder, the address is 4kb aligned). In the case, I have choosen 0x9C000 for the address(this may not work though depending on where your kernel is located at). Now we need to set up a pointer so that we can access that memory
:arrow: This I read in the tutorial ;) Beneath here I posted my code. Don't pay attention to my bit setting functions they work but they are not optimal, I know, but this is just for the time being, first implement basic stuff ie enabling paging, then optimize ;)

functions:
- void clear_bit(unsigned long* field, unsigned long index)
- void set_bit(unsigned long* field, unsigned long index)
- unsigned long is_bit_set(unsigned long field, unsigned long index)
- addr_t AllocPage()
- void FreePage(addr_t address)

are OK and have no bugs; all tested through and through, not yet optimized.

function: void mm_install(unsigned long mem)

is NOT ok has only one bug and that is setting up the paging.

types:

addr_t - > typedef unsigned long addr_t

Troubling piece of code:

Code: Select all

asm("movl %0,%%eax\n" 
	"movl %%eax,%%cr3\n"
	"movl %%cr0,%%eax\n"
	"orl $0x80000000,%%eax\n"
	"movl %%eax,%%cr0\n"
	"jmp 1f\n"
	"1:\n"
	"movl $1f,%%eax\n"
	"jmp *%%eax\n"
	"1:":: "m" (page_directory));
Entire file:

Code: Select all

#include <libc\stdlib.h>
#include <libc\stdio.h>
#include <libc\string.h>
#include <mm.h>
#include <kernel.h>

extern size_t end;

/* lookup table */

unsigned long bit_pattern[] = 
{ 
	0x00000008, 0x00000004, 0x00000002, 0x00000001, 
	0x00000080, 0x00000040, 0x00000020, 0x00000010,  
	0x00000800, 0x00000400, 0x00000200, 0x00000100,  
	0x00008000, 0x00004000, 0x00002000, 0x00001000,    
	0x00080000, 0x00040000, 0x00020000, 0x00010000, 
	0x00800000, 0x00400000, 0x00200000, 0x00100000,      
	0x08000000, 0x04000000, 0x02000000, 0x01000000, 
	0x80000000, 0x40000000, 0x20000000, 0x10000000,
};

/* bit setting functions */

void clear_bit(unsigned long* field, unsigned long index)
{
	*field = ((*field) & (~bit_pattern[index]));
}

void set_bit(unsigned long* field, unsigned long index)
{
	*field = ((*field) | (bit_pattern[index]));
}

unsigned long is_bit_set(unsigned long field, unsigned long index)
{
	if((field | (~bit_pattern[index])) == ~bit_pattern[index])
	{
		return 0;
	}

	return 1;
}

/* some global data, info about paging memory */

addr_t* page_directory;

size_t num_pte = 0;
size_t num_pde = 0;

static addr_t MemoryStart = (addr_t)&end;
addr_t MemoryEnd = 0;
size_t MemorySize = 0;

size_t Pages = 0;
size_t PagesInUse = 0;

addr_t* Bitmap;
size_t BitmapSize;

size_t ref_ptr_one = 0;
size_t ref_ptr_two = 0;

/* some lookup values */

#define LOOKUP_VALUE_L2			131072

size_t lk1;

addr_t AllocPage()
{
	while(ref_ptr_one < lk1)
	{
		while(ref_ptr_two < 32)
		{
			if(!is_bit_set(Bitmap[ref_ptr_one], ref_ptr_two))
			{
				set_bit((unsigned long*)&Bitmap[ref_ptr_one], ref_ptr_two);
				PagesInUse++;
				return MemoryStart + (ref_ptr_one * LOOKUP_VALUE_L2) + (ref_ptr_two * PAGE_SIZE);
			}

			ref_ptr_two++;
		}

		ref_ptr_two = 0;
		ref_ptr_one++;
	}

	// try going to begin of bitmap
	// first chek if we have any pages left
	if(!(Pages - PagesInUse))
	{
		// out of memory
		return 0;
	}

	/* reset parameters */
	ref_ptr_two = 0;
	ref_ptr_one = 0;

	/* retry */ 
	return AllocPage();
}

void FreePage(addr_t address)
{
	// just clear a bit ;)
	unsigned long page = ((address - MemoryStart) / 4096);
	unsigned long entry = (page / 32);
	unsigned long index = (page % 32);

	clear_bit((unsigned long*)&Bitmap[entry], index);

	PagesInUse--;
}

void mm_install(unsigned long mem)
{
	puts("creating paging bitmap... ");

	MemoryStart = MemoryStart + (4096 - (MemoryStart % 4096)); // align
	MemoryEnd = mem * 1024;
	MemorySize = MemoryEnd - MemoryStart;

	Pages = MemorySize / PAGE_SIZE;

	Bitmap = (addr_t*)MemoryStart;
	BitmapSize = (Pages / 32) * 4; // in Bytes

	memset(Bitmap, 0, ((BitmapSize / 4096) + 1) * 4096); /* zero bitmap */

	while(PagesInUse < (BitmapSize / 4096) + 1)
	{
		set_bit((unsigned long*)Bitmap, PagesInUse);
		PagesInUse++;
	}

	lk1 = (BitmapSize / 4) + 1;

	puts("OK\n");

	puts("testing memory... "); // just 1 chek, slows boot, but better to prevent crashes ;)
	// the more the memory the more time it takes to boot ;)

	addr_t* i = (addr_t*)Bitmap + (PagesInUse * 4096);

	// first loop
	for(; i < (addr_t*)MemoryEnd; i = i + 4)
	{
		*i = (addr_t)i;

		if(*i != (addr_t)i)
		{
			printf("error @ 0x%x loop 1", (addr_t)i);
			asm("hlt");
		}
	}

	addr_t* j = (addr_t*)Bitmap + (PagesInUse * 4096);
	i = (addr_t*)Bitmap + (PagesInUse * 4096);
	// second loop
	for(; j < (addr_t*)MemoryEnd; j = j + 4, i = i + 4)
	{
		*i = ~*i;

		if(*i != ~(addr_t)j) // chek with real answer
		{
			printf("error @ 0x%x loop 2", (addr_t)i);
			asm("hlt");
		}
	}

	puts("OK\nfilling with 0... ");
	
	i = (addr_t*)Bitmap + (PagesInUse * 4096);

	// zero memory
	for(; i < (addr_t*)MemoryEnd; i = i + 4)
	{
		*i = 0x00000000;
	}

	puts("OK\nloading page tables... ");



/* **************************************************** */
/* PROBLEMS START HERE!!!!!!!!!!!!!!!!!!!!!!                                    */
/* **************************************************** */


	/* we map all physical memory into kernel virutal memory */
	/* 1:1 mapping */

	/* calculate number of pte's and pde's we need */
	num_pte = (mem * 1024) / PAGE_SIZE;
	num_pde = (num_pte / 1024) + 1; // round up

	size_t iterator_a = 0;
	size_t iterator_b = 0;

	addr_t base = 0x00000000;

	// allocate memory for pd
	page_directory = (addr_t*)AllocPage();

	if((addr_t)page_directory % 4096) // error if not aligned (debug function)
	{
		printf("Error: %d 0x%08x", (addr_t)page_directory % 4096, (addr_t)page_directory);
		asm("hlt");
	}

	memset(page_directory, 0, PAGE_SIZE); /* force empty table */

	addr_t* pde; // page table entry

	// fill pd with pde's and pte's
	while(iterator_a < num_pde && num_pte)
	{
		pde = (addr_t*)AllocPage();

		// fill it with pte's
		while(iterator_b < 1024 && num_pte)
		{
			pde[iterator_b] = base | 3; // attribute set to: supervisor level, read/write, present(011 in binary)
			base += PAGE_SIZE; // 4096 = 4kb

			iterator_b++;
			num_pte--;
		}

		iterator_a++;
		iterator_b = 0;

		page_directory[iterator_a] = (addr_t)pde;
		page_directory[iterator_a] = page_directory[iterator_a] | 3;
	}

	num_pte = (mem * 1024) / PAGE_SIZE; // restore

/* don't work commented out */
	// write_cr3, read_cr3, write_cr0, and read_cr0 all come from the assembly functions
	//write_cr3((addr_t)page_directory); // put that page directory address into CR3
	//write_cr0(read_cr0() | 0x80000000); // set the paging bit in CR0 to 1

/* above doesn't work let's try something else */

	asm("movl %0,%%eax\n" 
	"movl %%eax,%%cr3\n"
	"movl %%cr0,%%eax\n"
	"orl $0x80000000,%%eax\n"
	"movl %%eax,%%cr0\n"
	"jmp 1f\n"
	"1:\n"
	"movl $1f,%%eax\n"
	"jmp *%%eax\n"
	"1:":: "m" (page_directory));

/* reboot here -_- doesn't work either */

	puts("OK\n");
}
P.S sorry for my longest post ever :p
The source of my problems is in the source.
Mikae
Member
Member
Posts: 94
Joined: Sun Jul 30, 2006 1:08 pm

Post by Mikae »

What about interrupt handlers? Are they installed?
User avatar
matthias
Member
Member
Posts: 158
Joined: Fri Oct 22, 2004 11:00 pm
Location: Vlaardingen, Holland
Contact:

Post by matthias »

Mikae wrote:What about interrupt handlers? Are they installed?
Yes they are installed and tested, and tried this with and without interrupts on. Forgot to mention
The source of my problems is in the source.
Mikae
Member
Member
Posts: 94
Joined: Sun Jul 30, 2006 1:08 pm

Post by Mikae »

Well, this is very strange. I see no way for CPU to reboot the system without any notification.

Are you sure that linear and physical addresses for the code, that contains switching to page are equal? Also, what about paging for handlers?
User avatar
matthias
Member
Member
Posts: 158
Joined: Fri Oct 22, 2004 11:00 pm
Location: Vlaardingen, Holland
Contact:

Post by matthias »

Mikae wrote:Well, this is very strange. I see no way for CPU to reboot the system without any notification.

Are you sure that linear and physical addresses for the code, that contains switching to page are equal? Also, what about paging for handlers?
Well I think I'm sure, but I'll test it out. Build some extra code into my kernel and debug. Results will come soon. The handlers write data to the screen thus accesing memory so it's logical it will fail :o A well got to do more testing ;)

edit:

seems not ok:

Code: Select all

addr: 0x00000000 pde: 0 pte: 0
addr: 0x00001000 pde: 0 pte: 1
addr: 0x00002000 pde: 0 pte: 2
addr: 0x00003000 pde: 0 pte: 3
addr: 0x00004000 pde: 0 pte: 4
addr: 0x00005000 pde: 0 pte: 5
addr: 0x00006000 pde: 0 pte: 6
addr: 0x00007000 pde: 0 pte: 7
addr: 0x00008000 pde: 0 pte: 8
addr: 0x00009000 pde: 0 pte: 9
pde: 0 pte: 0 should be 0x00001000??

and after filling one pde:

Code: Select all

addr: 0x00400000 pde: 1 pte: 0
not Ok as well
Last edited by matthias on Sun Aug 13, 2006 6:43 am, edited 1 time in total.
The source of my problems is in the source.
User avatar
gaf
Member
Member
Posts: 349
Joined: Thu Oct 21, 2004 11:00 pm
Location: Munich, Germany

Post by gaf »

mikae wrote:Well, this is very strange. I see no way for CPU to reboot the system without any notification. (snip) Also, what about paging for handlers?
That exactly the point: If the memory system doesn't work the handlers are most likely not mapped either. The CPU thus triple-faults when it tries to switch to the page-fault handler.

I only had a peek at the code, but one thing that looks a bit funny to me is, that you increase iterator_a before writing the entry to the page-directory. In my opinion it should actually be the other way round..

regards,
gaf
User avatar
matthias
Member
Member
Posts: 158
Joined: Fri Oct 22, 2004 11:00 pm
Location: Vlaardingen, Holland
Contact:

Post by matthias »

gaf wrote:
mikae wrote:Well, this is very strange. I see no way for CPU to reboot the system without any notification. (snip) Also, what about paging for handlers?
That exactly the point: If the memory system doesn't work the handlers are most likely not mapped either. The CPU thus triple-faults when it tries to switch to the page-fault handler.

I only had a peek at the code, but one thing that looks a bit funny to me is, that you increase iterator_a before writing the entry to the page-directory. In my opinion it should actually be the other way round..

regards,
gaf
Yes it should, but the first 4mb is mapped in OK. But I'll test it ;)

Doesn't work, so it's somethin' else

I now changed addr_t base to 0x00001000 but doesn't work either, but addresses now align to eachother. What is wrong then? :?
The source of my problems is in the source.
User avatar
matthias
Member
Member
Posts: 158
Joined: Fri Oct 22, 2004 11:00 pm
Location: Vlaardingen, Holland
Contact:

Post by matthias »

Halleluja I changed the code like gaf said, and set base back to zero and now it works!!!! Thanks gaf & Mikea for helping :D

Working code:

Code: Select all

/* we map all physical memory into kernel virutal memory */
   /* 1:1 mapping */

   /* calculate number of pte's and pde's we need */
   num_pte = (mem * 1024) / PAGE_SIZE;
   num_pde = (num_pte / 1024) + 1; // round up

   size_t iterator_a = 0;
   size_t iterator_b = 0;

   addr_t base = 0x00000000;

   // allocate memory for pd
   page_directory = (addr_t*)AllocPage();

   if((addr_t)page_directory % 4096) // error if not aligned (debug function)
   {
      printf("Error: %d 0x%08x", (addr_t)page_directory % 4096, (addr_t)page_directory);
      asm("hlt");
   }

   memset(page_directory, 0, PAGE_SIZE); /* force empty table */

   addr_t* pde; // page table entry

   // fill pd with pde's and pte's
   while(iterator_a < num_pde && num_pte)
   {
      pde = (addr_t*)AllocPage();

      // fill it with pte's
      while(iterator_b < 1024 && num_pte)
      {
         pde[iterator_b] = base | 3; // attribute set to: supervisor level, read/write, present(011 in binary)
         base += PAGE_SIZE; // 4096 = 4kb

         iterator_b++;
         num_pte--;
      }

      iterator_a++;
      iterator_b = 0;

      page_directory[iterator_a] = (addr_t)pde;
      page_directory[iterator_a] = page_directory[iterator_a] | 3;
   }

   num_pte = (mem * 1024) / PAGE_SIZE; // restore

   asm("movl %0,%%eax\n"
   "movl %%eax,%%cr3\n"
   "movl %%cr0,%%eax\n"
   "orl $0x80000000,%%eax\n"
   "movl %%eax,%%cr0\n"
   "jmp 1f\n"
   "1:\n"
   "movl $1f,%%eax\n"
   "jmp *%%eax\n"
   "1:":: "m" (page_directory));

   puts("OK\n"); 
Now I'm going to acces a address which is not mapped in and I'll have a look if it faults ok :)
The source of my problems is in the source.
User avatar
matthias
Member
Member
Posts: 158
Joined: Fri Oct 22, 2004 11:00 pm
Location: Vlaardingen, Holland
Contact:

Post by matthias »

Ok it faults correctly, but what I still didn't get is why it rebooted. The 1st four MB were mapped in correctly. Only 4MB+ wasn't mapped correctly, I do not use these addresses in my kernel so it's strange it rebooted or not? Still I should have gotten to my fault handlers even if 4MB+ wasn't mapped correctly right?
The source of my problems is in the source.
User avatar
gaf
Member
Member
Posts: 349
Joined: Thu Oct 21, 2004 11:00 pm
Location: Munich, Germany

Post by gaf »

matthias wrote:Ok it faults correctly, but what I still didn't get is why it rebooted.
Hard to say as the "working" code you posted is exactly the same as the old code :). If you really only changed what I proposed, the problem probably was that you've written the mapping to the second pde, rather than to the first: page_directory[1] is the second entry of the page-directory and spans the region from 4-8 MiB.

Code: Select all

while(iterator_a < num_pde && num_pte) 
{ 
   pde = (addr_t*)AllocPage(); 

   // fill it with pte's 
   while(iterator_b < 1024 && num_pte) 
   { 
      pde[iterator_b] = base | 3;  
      base += PAGE_SIZE;

      iterator_b++; 
      num_pte--; 
   } 

   page_directory[iterator_a] = (addr_t)pde; 
   page_directory[iterator_a] = page_directory[iterator_a] | 3; 

   iterator_a++; 
   iterator_b = 0; 
}
matthias wrote:The 1st four MB were mapped in correctly. Only 4MB+ wasn't mapped correctly, I do not use these addresses in my kernel so it's strange it rebooted or not?
That's indeed strange.. Is is possible that your debugging code just made the same offset mistake ?

regards,
gaf
User avatar
matthias
Member
Member
Posts: 158
Joined: Fri Oct 22, 2004 11:00 pm
Location: Vlaardingen, Holland
Contact:

Post by matthias »

My debug code was correct, strange :?
The source of my problems is in the source.
Post Reply