Page Table Cloning Frustration

[kfreezen]

My page table cloning functions just clearly do not work. The first page table will be mapped, but go beyond that and the emulator will triple fault (at least according to my observation). So I guess I'm going to post some code and let you help me figure out where the problem is.

First, my bochs error log:

Code: Select all

...
01131409938i[CPU0 ] CPU is in protected mode (active)
01131409938i[CPU0 ] CS.d_b = 32 bit
01131409938i[CPU0 ] SS.d_b = 32 bit
01131409938i[CPU0 ] EFER   = 0x00000000
01131409938i[CPU0 ] | RAX=0000000000400000  RBX=0000000000400000
01131409938i[CPU0 ] | RCX=0000000002805000  RDX=00000000e0000011
01131409938i[CPU0 ] | RSP=00000000dfffff54  RBP=00000000dfffff88
01131409938i[CPU0 ] | RSI=0000000000401000  RDI=0000000002806000
01131409938i[CPU0 ] |  R8=0000000000000000   R9=0000000000000000
01131409938i[CPU0 ] | R10=0000000000000000  R11=0000000000000000
01131409938i[CPU0 ] | R12=0000000000000000  R13=0000000000000000
01131409938i[CPU0 ] | R14=0000000000000000  R15=0000000000000000
01131409938i[CPU0 ] | IOPL=0 id vip vif ac vm RF nt of df if tf SF zf af PF cf
01131409938i[CPU0 ] | SEG selector     base    limit G D
01131409938i[CPU0 ] | SEG sltr(index|ti|rpl)     base    limit G D
01131409938i[CPU0 ] |  CS:0008( 0001| 0|  0) 00000000 ffffffff 1 1
01131409938i[CPU0 ] |  DS:0010( 0002| 0|  0) 00000000 ffffffff 1 1
01131409938i[CPU0 ] |  SS:0010( 0002| 0|  0) 00000000 ffffffff 1 1
01131409938i[CPU0 ] |  ES:0010( 0002| 0|  0) 00000000 ffffffff 1 1
01131409938i[CPU0 ] |  FS:0010( 0002| 0|  0) 00000000 ffffffff 1 1
01131409938i[CPU0 ] |  GS:0010( 0002| 0|  0) 00000000 ffffffff 1 1
01131409938i[CPU0 ] |  MSR_FS_BASE:0000000000000000
01131409938i[CPU0 ] |  MSR_GS_BASE:0000000000000000
01131409938i[CPU0 ] | RIP=0000000000100331 (0000000000100331)
01131409938i[CPU0 ] | CR0=0xe0000011 CR2=0x00000000dfffff50
01131409938i[CPU0 ] | CR3=0x0011f000 CR4=0x00000000
01131409938i[CPU0 ] 0x0000000000100331>> popfd  : 9D
01131409938e[CPU0 ] exception(): 3rd (14) exception with no resolution, shutdown status is 00h, resetting
01131409938i[SYS  ] bx_pc_system_c::Reset(HARDWARE) called
01131409938i[CPU0 ] cpu hardware reset
01131409938i[APIC0] allocate APIC id=0 (MMIO enabled) to 0xfee00000
01131409938i[CPU0 ] CPUID[0x00000000]: 00000003 756e6547 6c65746e 49656e69
01131409938i[CPU0 ] CPUID[0x00000001]: 00000f20 00000800 00002000 078bfbff
01131409938i[CPU0 ] CPUID[0x00000002]: 00410601 00000000 00000000 00000000
01131409938i[CPU0 ] CPUID[0x00000003]: 00000000 00000000 00000000 00000000
01131409938i[CPU0 ] CPUID[0x00000004]: 00000000 00000000 00000000 00000000
01131409938i[CPU0 ] CPUID[0x80000000]: 80000008 00000000 00000000 00000000
01131409938i[CPU0 ] CPUID[0x80000001]: 00000000 00000000 00000101 2a100800
01131409938i[CPU0 ] CPUID[0x80000002]: 20202020 20202020 20202020 6e492020
01131409938i[CPU0 ] CPUID[0x80000003]: 286c6574 50202952 69746e65 52286d75
01131409938i[CPU0 ] CPUID[0x80000004]: 20342029 20555043 20202020 00202020
01131409938i[CPU0 ] CPUID[0x80000006]: 00000000 42004200 02008140 00000000
01131409938i[CPU0 ] CPUID[0x80000007]: 00000000 00000000 00000000 00000000
01131409938i[CPU0 ] CPUID[0x80000008]: 00003020 00000000 00000000 00000000
01131409938i[PLGIN] reset of 'unmapped' plugin device by virtual method
01131409938i[PLGIN] reset of 'biosdev' plugin device by virtual method
01131409938i[PLGIN] reset of 'speaker' plugin device by virtual method
01131409938i[PLGIN] reset of 'extfpuirq' plugin device by virtual method
01131409938i[PLGIN] reset of 'gameport' plugin device by virtual method
01131409938i[PLGIN] reset of 'pci_ide' plugin device by virtual method
01131409938i[PLGIN] reset of 'acpi' plugin device by virtual method
01131409938i[PLGIN] reset of 'ioapic' plugin device by virtual method
01131409938i[PLGIN] reset of 'keyboard' plugin device by virtual method
01131409938i[PLGIN] reset of 'harddrv' plugin device by virtual method
01131409938i[PLGIN] reset of 'serial' plugin device by virtual method
01131409938i[PLGIN] reset of 'parallel' plugin device by virtual method
01131413244i[BIOS ] $Revision: 1.247 $ $Date: 2010/04/04 19:33:50 $
01131728057i[KBD  ] reset-disable command received
01131852371i[VBIOS] 
VGABios $Id$
...

Then my CloneTable function:

Code: Select all

static PageTable* CloneTable(PageTable* src, UInt32* phys) {
	#ifdef PAGING_TRACE
	kprintf("CloneTable(%x,%x)\n", src, phys);
	#endif
	
	PageTable* table = (PageTable*)kmalloc_ap(sizeof(PageTable), true, phys);
	memset(table, 0, sizeof(PageTable));
	
	int i;
	for(i=0; i<1024; i++) {
		// If its 0, continue.
		if(!(src->t[i]&0xFFFFF000)) {
			continue;
		}
		
		// Allocate a new page.
		int ip = alloc_frame();
		
		// Map the page into the table
		Pointer phys_addr = (Pointer)(ip*0x1000);
		table->t[i] = AssemblePTE(phys_addr, (src->t[i]&(0xFFF)));
		
		// Copy the contents of one frame to another.  physical addresses only, please.
		copy_page_phys((src->t[i]&0xFFFFF000), (table->t[i]&0xFFFFF000));
	}
	
	return table;
}

copy_page_phys code.

Code: Select all

copy_page_phys:
	push ebx
	pushf
	
	cli
	
	mov ebx, [esp+12]
	mov ecx, [esp+16]
	
	mov edx, cr0
	and edx, 0x7fffffff
	mov cr0, edx
	
	mov edx, 1024
	
	mov esi, ebx
	mov edi, ecx
	
	; ecx = ebx
.loop:

	movsd
	dec edx
	jnz .loop
	
	mov edx, cr0
	or edx, 0x80000000
	mov cr0, edx
	
	popf
	pop ebx
	ret

If you could help me, that would be very much appreciated.

[gerryg400]

I think

Code: Select all

      if(!(src->t[i]&0xFFFFF000)) {
         continue;
      }

should be

Code: Select all

      if(!(src->t[i]&0x1)) {
         continue;
      }

IOW, if the page is not present, continue. Your code won't clone a page mapped to physical address 0.

[kfreezen]

That solved one of the problems; however, the code still triple faults.

[gerryg400]

When you enable or disable paging you must be executing code that is identity mapped. (Actually there are ways around this but they tricky).

There are also performance and other issues that you haven't considered yet, so I advise you to find another way to clone your physical pages, one that does not require disabling and re-enabling paging. There are several methods available.

[kfreezen]

Would one be as simple as getting the virtual addresses of both and using
memcpy()?

[gerryg400]

Yes, that's the goal. The difficulty is that when you have lots of processes it takes a lot of kernel virtual memory to keep all those pages mapped so various techniques have been devised to solve this. For example.
1. If the amount of physical memory is small you can map an alias of the entirety of RAM into the kernel - Linux used to do this at 0xc0000000, perhaps it still does. You can make it work even if there is lots of RAM if you make sure to use only the aliased RAM for page tables/dirs.
2. Have a special place in the kernel where you can map in/out pages as you need to access them.
3. Use the self-mapping trick. This still requires some help when copying from one set of mapping to another.

I use 2) and 3) in my 32bit memory manager and 1) in my long mode memory manager.

[gerryg400]

Now I've got a question for you. Why do you need to clone page tables ?

[kfreezen]

I guess just because its the only way I currently know to create a new address space for a thread.

Edit: It appears as if I have gotten the cloning code to work.

Code: Select all

PageDirectory* CreateNewAddressSpace(PageDirectory* krnl) {
	int i;
	
	UInt32 phys;
	PageDirectory* dir = (PageDirectory*) kmalloc_ap(sizeof(PageDirectory), true, &phys);
	
	for(i=0; i<1024; i++) {
		// If table is not in krnl then continue
		if(!(krnl->d[i]&0x1)) {
			continue;
		}
		
		// If table is in krnl and it is not in orig_krnl_dir then clone it.
		if(krnl->d[i] == orig_krnl_dir->d[i]) {
			// Since it is, copy it over to the new directory
			dir->d[i] = krnl->d[i];
			dir->kd[i] = krnl->kd[i];
		} else {
			int j;
			PageTable* pt = (PageTable*)(krnl->kd[i]&0xFFFFF000);
			PageTable* npt = (PageTable*) kmalloc_ap(sizeof(PageTable), true, &phys);
			
			for(j=0; j<1024; j++) {
				if(!(pt->t[i]&1)) {
					continue;
				}
				int afi = alloc_frame();
				npt->t[i] = afi*0x1000;
				
				copy_page_phys((pt->t[i]&0xFFFFF000), (npt->t[i]&0xFFFFF000));
			}
		}
		
	}
	
	return dir;
}

[gerryg400]

But why like this ?

If you are trying to implement an old-fashioned fork() then it is the contents of the pages,( i.e. the code and data that they contain) that must be cloned. Not the page tables. All you need to do is create a new address space the same size as the original with the same protection settings. Usually the protection settings for each app will be the same (e.g. user-exe-ro for code, user, rw for data) so it's only the size that needs to be reproduced.

Once the new address space exists you need to be able to copy data from one address space to the other.

But I can not see the need to copy page tables.