Cost of enabling / disabling pages?

[pcmattman]

Maybe it only maps the first 1GB (excluding the 1gb mark)?

Edit: did you try writing to a lower part of memory, to test it?

[Brendan]

Hi,

Ive copy/pasted what you wrote, and it works (compiles)...

But I still page fault when I try to write to the 1gb mark,,,

Do you know why it page faults at the 1 GB mark?

For e.g. in the page fault handler, what are CR2 and the error code pushed on the stack equal to?

Alternatively, you could stop it just before the "strcpy()" (with a "for( ; ; )" in your code and "control+C" in Bochs) and use Bochs debugger to get the physical address of the page directory ("dump_cpu" and look for CR3), then examine the physical page that CR3 points to ("xp 0x?????????") to see if it's correct, and then get the physical address of the page table from the page directory and examine it too.

In this case you should be able to determine if it's caused by a page not present, a page table not present, bad protection flags, if "strcpy()" fails to read the string or fails to write the string, non-zero data segment bases, etc.

One last thing - it's possibly a good idea to check if the 2 calls to "vmm_map()" return an error during "vmm_install()". For e.g.:

Code: Select all

     if(vmm_map(262144,mm_alloc(), 3, 3) != SUCCESS) panic();
     if(vmm_map(262145,mm_alloc(), 3, 3) != SUCCESS) panic();

I'm also wondering what happens if there is no free memory - how does "mm_alloc()" return an error?


Cheers,

Brendan

[astrocrep]

I do not use bochs its something Ive been meaning to switch to for a while... but right now, I am using vmware / qemu

Its not the mm_alloc... its returning valid pages... and vmm_map is returning with out error...

I had to slightly modify your new vmm_map to the code below:

Code: Select all

int vmm_map(unsigned long virt_page, unsigned long phys_page, unsigned long PTE_flags, unsigned long PDE_flags) 
{ 
	int pd_index; 
	int pt_index; 
	unsigned long *vpt; 
	unsigned long temp_phys_addr; 
	
	kprintf("+ Enter vmm_map: virt addr = %d, phys addr = %d\n", virt_page * 4096,phys_page * 4096);

	if( (vpd_base[virt_page / 1024] & 1) == 0 ) 
	{ // No page table mapped 
	  	temp_phys_addr = (mm_alloc() * 4096); 
	  	kprintf ("   temp_phys_page = %d\n",temp_phys_addr);
	  	vpd_base[virt_page / 1024] = temp_phys_addr;
	  	vpd_base[virt_page / 1024] = vpd_base[virt_page / 1024] | PDE_flags | 1; 
	  	kprintf("   Mapped in new PT virt addr = %d, phys addr = %d\n", vpd_base + (virt_page / 1024) ,temp_phys_addr);
	  	vmm_disable_paging(); 
	  	vmm_enable_paging(); 
	} 
	if( (vpt_base[virt_page] & 1) != 0 ) 
	{
		kprintf("- Leaving vmm_map (Page already mapped!)\n");
		return 0; 
	}
	
	vpt_base[virt_page] = (phys_page * 4096);
	vpt_base[virt_page] = vpt_base[virt_page] | PTE_flags | 1; 
	vmm_disable_paging(); 
	vmm_enable_paging(); 

	kprintf("- Leaving vmm_map(Page mapped!)\n");
	return 0; 
}

Attached is a screen-shot of the out put. The screen-shot shows the debugging output of what happens the two times it enter the vmm_map in the vmm_install function.

The Page-fault occurs either writing (which happens first) or with reading if the writing is commented out.

I will try to get bochs install and do as you suggested.

EDIT: Here is a bochs "dump_cpu" AFTER the page fault...
EDIT 2: Is it wierd that CR2 contains the 1gb mark, well I guess that makes sense, cause that the "offending" address
EDIT 3: Value of CR3 = physical address of PD.
EDIT 4: What you see after os boot is the product of saying: *s1 = 1gb mark, printf("%d || %s",s1,s1); so you see the address of s1 = 1gb mark, but as soon as it hits it for data... boom... (note: that in the screenshot below, the strcpy has been commented out.)

Thanks,
Rich

[Brendan]

Hi,

Attached is a screen-shot of the out put. The screen-shot shows the debugging output of what happens the two times it enter the vmm_map in the vmm_install function.

The Page-fault occurs either writing (which happens first) or with reading if the writing is commented out.

Everything there shows me things that are correct, and I still can't see where something is incorrect...

The "vmm_map()" function doesn't generate a page fault when it puts the page into the page table, so the part that allocates the page table and puts it in the page directory, the self-reference and "vpd_base" must all be correct.

The Bochs log shows that paging is enabled, that CR2 = EDI (segment bases aren't wrong or non-zero) and CR2 = 1 GB (not trying to access an area that isn't meant to be present). The VMware screenshot also shows sane addresses for everything.

My best guess is that there's something wrong with the page table entries, however I can't find anything wrong in the source code.

The only other thing I can think of is memory corruption - something trashing physical pages after they're allocated.

I'm wondering if you'd mind using Bochs to check the paging structures (get a dump of the contents of the page directory and page tables from physical memory (using "xp /32 0x12345678" where 0x12345678 is the address you want to look at - the physical address of the page directory from CR3, or the physical address of page tables from the page directory).

Alternatively, is there a floppy boot image I could download somewhere?

BTW it's currently not causing any problems, but when "vmm_map()" allocates a new page table it should fill it with zeros (unless pages returned by "mm_alloc()" are already filled).


Cheers,

Brendan

[astrocrep]

I know its driving me bonkers...

Sorry I was away on vacation.

Here is the bootable floppy image... zipped.

One thing I noticed, is It doesn't seem to care if I reset paging, or rewrite cr3...

If I comment out:

Code: Select all

 vmm_disable_paging(); 
 vmm_enable_paging(); 

Nothing happens...

Perhaps... the TLB isn't being flushed or something??

Code to enable and disable paging...

Code: Select all

int vmm_enable_paging()
{
	write_cr0(read_cr0() | 0x80000000); // set the paging bit in CR0 to 1 
	return TRUE;
}

int vmm_disable_paging()
{
	write_cr0(read_cr0() ^ 0x80000000); // set the paging bit in CR0 to 0
	return TRUE;
}

Although I am pretty sure that works ok, because in the being of this thread I was working paging by disabling and enabling the paging bit in cr0 to write to my pdes...

Thanks,
Rich

[Brendan]

Hi,

I know its driving me bonkers...

Doh! It's my fault...

In "vmm_install()" change this code:

Code: Select all

    // Do self-reference

   kernel_pd[127] = *kernel_pd | 3;

To something like this:

Code: Select all

    // Do self-reference

   kernel_pd[127] = (unsigned long)kernel_pd | 3;

Basically it was using the first page directory entry instead of the address of the page directory, and therefore mapping the first 4 MB of phyical memory to 508 MB instead of mapping the page directory and page tables to 508 MB. Then when it tried to add a page table for the area at 1 GB it would've put the new page directory entry in physical memory somewhere instead of putting it in the page directory, leaving no page table for the area at 1 GB.

Once this is fixed it doesn't page fault anymore, and there's a 8 KB area full of zeroes at 1 GB. I'm not sure what's meant to happen after that though - it's goes into an endless loop that looks like it's waiting for an IRQ handler to set a global variable to 1 (e.g. "while(someGlobalVariable != 1) { /* keep waiting */ }").


Cheers,

Brendan

[astrocrep]

BEAUTIFUL!!!

It works Perfectly!

Thanks x 100,000,000,000!

Yea, thats my half implements getch() function....

So just to be clean, every time a page is mapped in , I should goto its virtual address and 0 the whole thing out? Wouldn't that be a performance hit? I mean I honestly agree, that we should wipe it out, as it would be a security issue too...

-Rich

[Brendan]

Hi,

So just to be clean, every time a page is mapped in , I should goto its virtual address and 0 the whole thing out?

If the physical page is being used for a page table, then you must fill the physical page with something - otherwise the CPU (and the kernel itself) could think any garbage left in the page is valid page table entries.

If the kernel supports "allocation on demand" (where the kernel pretends that pages were allocated/mapped, but doesn't allocate/map the pages until the application accesses them) then it's a good idea to fill the page/s with zero. This is especially true for a process' ".bss" section (where using allocation on demand can save RAM, but where the process expects the ".bss" section to be full of zeros).

In all other cases it's good practice and/or good for security - it means a page full of passwords can't be freed by one process and allocated (and read) by another process (for e.g.).

Wouldn't that be a performance hit? I mean I honestly agree, that we should wipe it out, as it would be a security issue too..

It would cost a little CPU time, but not really too much.

It's also possible to keep track of "dirty" free pages and "clean" free pages seperately. When a page needs to be allocated you'd try to get a "clean" page that's already filled with zeros, and when the CPU has nothing better to do it can fill pages with zeros (i.e. convert pages from "dirty" to "clean" during idle time to reduce the overhead of allocating pages when the CPU isn't idle).


Cheers,

Brendan

[Candy]

If the kernel supports "allocation on demand" (where the kernel pretends that pages were allocated/mapped, but doesn't allocate/map the pages until the application accesses them) then it's a good idea to fill the page/s with zero. This is especially true for a process' ".bss" section (where using allocation on demand can save RAM, but where the process expects the ".bss" section to be full of zeros).

Forgetting that just after implementing both the kernel page fault handler for the task area and task switching results in a pretty cool instruction trace - I had it run for 160M in random space until it hit a 0-entry, which caused it to run the idle task. Happened to me 3.5 years ago on 0.0.1.