Memory Manager

[daisy]

Does this look like a start to a good physical memory manager?

Code: Select all

////////////////////////////////////////////////////
//	PHYSICAL MEMORY MANAGER


char physPagesBitmap[65536]; // 1 Byte represents 8 Pages of 4KB

void _setPhysPageState( unsigned long page, char state ) {
	unsigned long off = page / (8 * 4096);
	unsigned long bit = (page % (8 * 4096)) / 4096;

	physPageBitmap [off] = state << bit;
} 

char _queryPhysPageState( unsigned long page) {
	unsigned long off = page / (8 * 4096);
	unsigned long bit = (page % (8 * 4096)) / 4096;
 
	return physPageBitmap [off] & ( 1 << bit);
}

	

int initPhysMemMgr(unsigned long physSize) {

	unsigned long addrSEnd = (unsigned long)&end; //End of kernel (somewhere above 1MB)

	unsigned long startMem = addrEnd + (addrEnd%4096==0?0:4096-addrSEnd);
	unsigned long endMem = physSize - (physSize%4096==0?0:physSize%4096);

	if(startMem % 4096 != 0) {  // Should never get here, unless something wrong above
		////////////////////
		//Error Handling
		return 0;
	}

	unsigned long pageIndex = 0;
	for(;(pageIndex < 2048 * 1024 * 1024) & pageIndex < ;pageIndex += 4096) {
		if(pageIndex < startMem) {
			_setPhysPageState(pageIndex,1); // 1 Means allocated
		} else if(pageIndex >= startMem && pageIndex < endMem) {
			_setPhysPageState(pageIndex,0); // 0 means unallocated
		} else // pageIndex >= endMem
			_setPhysPageState(pageIndex,1); // 1 means allocated + reserved
	}
		
	return 1; // always here
}

void* allocPhysPage( void ) {
	unsigned long base = 0;

	for(;base < 65536;base++)
		if(physPagesBitmap[base] != 0xFF)
			break;
	if(base >= 65536) 
		return NULL;


	unsigned long page = base * (8*4096) ;
	unsigned long offset = 0;

	for(;offset<8;offset++;page+=4096) {
		if( _queryPhysPageState(page) == 0) {
			_setPhysPageState(page, 1);
			break;
		}
	}
	
	if(offset >=8) 
		return NULL;
	else
		return (void*)page;
}

void freePhysPage( void* page ) {
	unsigned long _page = (unsigned long)page;
	
	if(_page < startMem)
		return;
	else if(_page >= endMem) 
		return;
	else
		_setPhysPageState(_page,0);
} 

[astrocrep]

Seems pretty good on first glance.

May I suggest two things.

First:
I think by default alloc_page should try to return a page after the 16mb mark, if there is any, before it returns the lowest page free, this save the bottom 16mb for dma operations.


Second:
Some sort of account code, to keep track of the number of free pages above and below 16mb along with the total number of pages.

On the whole though, it looks like a fine base to attach a vmm on to.

-Rich

[daisy]

Thanks. Any easy way to implement a VMM?

[astrocrep]

One other thing.

You should also implement a ReservePhysical function that you can specify a starting addr, and a length, and it will go and mark allocated all of those pages for you. Heres my logic below, also note that in my linker script I define a variable called :kernel_end, so I know where my kernel ends.

For instance,

On boot I call something like:

Physical_Memory_Man_INIT(Kernel_End, grub->Total_system_Ram)

What he does is find out the size of the bitmap needed to define all of the physical pages, and start that bitmap at something like kernel_end + 128k. At this point I also reserve the first 2 MB of physical ram, the first 1mb is kept intact for a vm86 to be implemented, and the 1mb - 2mb is my kernel and various data structures.

Next I loop through grubs memory map info and alloc all of the ranges that it says are in use.

Next I start the virtual memory manager. This part was quiet hard.

I have three main function in the vmm.

vmm_install()
vmm_map(virt_page, phys_page)
vmm_free(virt_page)

Before implementing your virtual memory manager you should decide on a memory layout, for me it:

0m - 4m : Reserver for the kernel
4m - 16m : Reserver for DMA (if the system has > 32mb total ram)
16m - 508m : Free Ram.
508m - 512m : My Recursivly mapped Page Directory
512m - 1g : Start of my kernel Heap
1g - ... : User Applications

vmm_install basically setups your initial Page Directory and Page Table.
For me it maps the first 4mb physical to the first 4mb virtual, and them sets a self-reference for the PD to the 508m mark. And lastly turns on paging.

It took my a while to wrap my head around this, but think of it as such:

1 PD = 1024 4-byte Pointers to Page Tables
1 PT = 1024 4-byte Pointer to Physical Pages

So By Mapping the PD to virtual 508m, 508m - 5012m gives me r/w access to all of the physical page address in all of the page tables.

So if I want to map a physical page to 1g mark, I would:
1.) Check to see if there is a page table to address the 1024m-1028m range.
You can do this by basically checking to see the value stored at virtual address:
508m (Base of the PD) + 127 * 4096 (The offset where the PD stored) + the virtual_page / 1024 (In this case, 262144 / 1024 = 256)

Basicall what that is saying is if the virtual_page_dir is at 508m + (127 * 4096)

Then 256 4 byte entries in would be the LOCATION of the PHYSICAL Page Table that would map the 1gb range.

If it blank, the you need to allocate a PHYSICAL Page into the spot,

2.) Now that we know there is a valid PT for mapping address at the 1gb mark, we need to actually load the new physical pages that are actually going to be used to hold the data at the 1 g mark.

So to find the page table for the 1g mark, you would be looking at 508m + (256 * 4096)
That would be the start of the 1g to 1g + 4m page table.

Here is my mapping function...

Code: Select all

unsigned long *vpt_base = (unsigned long *) 0x1FC00000;   //508mb 
unsigned long *vpd_base = (unsigned long *) (0x1FC00000 + 127 * 4096);

int vmm_map(unsigned long virt_page, unsigned long phys_page, unsigned long PTE_flags, unsigned long PDE_flags) 
{ 
	unsigned long temp_phys_addr; 
	
	if (phys_page == 0)
	{
		return -1;
	}

	if( (vpd_base[virt_page / 1024] & 1) == 0 ) 
	{ 
		// No page table mapped 
	  	temp_phys_addr = (mm_alloc() * 4096); 
	  	vpd_base[virt_page / 1024] = temp_phys_addr;
	  	vpd_base[virt_page / 1024] = vpd_base[virt_page / 1024] | PDE_flags | 1; 
	  	_vmm_invalidate_addr(vpd_base + (virt_page / 1024));
	} 
	if( (vpt_base[virt_page] & 1) != 0 ) 
	{
		return 0; 
	}
	
	vpt_base[virt_page] = (phys_page * 4096);
	vpt_base[virt_page] = vpt_base[virt_page] | PTE_flags | 1; 
	_vmm_invalidate_addr(virt_page * 4096);

	return 0; 
}

Free is just the reverse, get the address stored in you PT, and then free the physical page too...

Good luck,
Rich

[daisy]

Right, so does this look OK for memory map?

Code: Select all

	//////////////////////////////////////////////////////////////
	// 	0 - 4K		V86
	//	4K - 640K	DMA 
	//	640 - 1023K	BIOS Services / Video
	//	1MB - ~4MB	Kernel Code
	//	~4MB - ~508MB	Reserved for Kernel [code] + Drivers
	//	508MB - 512MB	Memory Services 
	//	512MB - 2GB	Kernel Memory Heap
	//	2GB - 2GB + 4KB Unallocated
	//	2GB + 4KB - 4GB User Space

Also, when you say 'virt_page' in your code, does it mean the page #, or the virtual address?

[daisy]

Also, here's the beginning to my VM. Looks ok?

Code: Select all

char kernelMemHeapMap[49152]; // Bitmap that manages the shared kernel memory space between 512MB - 2GB

unsigned long* pageTables = 0x1FC00000; // Controls which pages are mapped
unsigned long* pageDirectory = 0x1FC00000 + (127 * 4096);
	

int setPageInfo( unsigned long vaddr, unsigned long paddr, unsigned long pte_flags, unsigned long page_flags)	{

	if( (pageDirectory[ (vaddr / 4096) /1024] & 1) == 0)  {
		pageDirectory[ (vaddr / 4096) / 1024] = (unsigned long)allocPhysPage() | pte_flags | 1;
		// invalidatePage( pageDirectory );
	}
	if( (pageDirectory[ (vaddr / 4096) / 1024] & 1) == 0)
		return 0;
	vpt_base[ vaddr / 4096 ] = paddr | page_flags | 1;
	// invalidatePage( pageTables );
	return 1;
}

unsigned long getPageInfo( unsigned long vaddr ) {
	return pageDirectory[ (vaddr / 4096) / 1024];
}

int getPageFlags( unsigned long vaddr ) { return getPageInfo(vaddr) & 0xFFFF; }
int getPagePhys( unsigned long vaddr) {return getPageInfo(vaddr) & ~0xFFFF;}

[astrocrep]

Yea,

virt_page is just the virtual address / 4096, so it gives you the page number.

Good Luck
Rich

[daisy]

Nevermind, here's the new code

Code: Select all

int initVirtMemManager( void ) {
	//////////////////////////////////////////////////////////////
	// 	0 - 4K		V86
	//	4K - 640K	DMA 
	//	640 - 1023K	BIOS Services / Video
	//	1MB - ~4MB	Kernel Code
	//	~4MB - ~508MB	Reserved for Kernel [code] + Drivers
	//	508MB - 512MB	Memory Services 
	//	512MB - 2GB	Kernel Memory Heap
	//	2GB - 2GB + 4KB Unallocated
	//	2GB + 4KB - 4GB User Space

	//////////////////////////////////////////
	//	Init so we have mapped the kernel + everything below < 1MB, and the Memory Services 

	void* pageDirectory = allocPhysPage();
	memset( pageDirectory , 0 , 4096 );

	pageDirectory[127] = pageDirectory;
	unsigned long index = 0;
	for(index = 0; index < startMem; index += 4096) 
		setPageInfo ( pageDirectory, pageDirectory + 127,index, index, 3, 3,0);


	writeCR3( pageDirectory );
	writeCR0( readCR0() | 0x80000000);
	

}


///////////////////////////////
//	For Processes
unsigned long* creatVMMContext() {
	void* ctxPageDirectory = allocVirtPage();
	memset( ctxPageDirectory, 0 , 4096);

	ctxPageDirectory[127] = ctxPageDirectory;
	unsigned long index = 0;
	for(index = 0; index < startMem; index += 4096) 
		setPageInfo ( pageDirectory, pageDirectory + 127,index, index, 3, 3,0);

	return ctxPageDirectory; // Virt address of a new page directory [for processes]
}