[SOLVED] How to get virtual address of page table

[Aerath]

I used to identity map parts of the memory space as needed with the following function.
My problem is that I can only get the physical address of PTE and I don't know how to get the virtual one. However I need it to change it's flags or page address.
It was not a problem when tables where all identity mapped, but now they can be anywhere.

I tried to find a solution by myself but with no success, and same result by looking on the internet. It seems impossible to me since the only place this information is in the PTE, which is what I'm try to access ... And the only solutions I found online assumed a fixed offset (something like

Code: Select all

#define P_TO_V(p) (p+0xC0000000)

)

Code: Select all

va_t vmap(va_t vstart, pa_t pstart, size_t size)
{
	if(!size) return -1;

	uintptr_t v = (uintptr_t)vstart;
	uintptr_t p = (uintptr_t)pstart;

	const size_t page_num = size/PAGE_SIZE + (size%PAGE_SIZE ? 1 : 0);
	for(size_t i=0; i<page_num; i++)
	{
		pde_t* const pde = &page_dir[v>>22];
		pde->writeable = 1;
		pde->present = 1;
		if(!pde->table_address)
		{ // Allocate and map a table
			va_t table = find_unmapped_vpages(PAGE_SIZE);
			if(!table)
			{
				pde->present = 0;
				return NULL;
			}
			/* With identity mapping
			pde->table_address = (uintptr_t)table>>12;
			vmap(table, table, PAGE_SIZE);
			*/
			// Now, with "random" physical and virtual allocation
			pa_t paddr = palloc();
			pde->table_address = (uintptr_t)paddr>>12;
			vmap(table, paddr, PAGE_SIZE);
			//

			mprotect(table, PAGE_SIZE, PROT_WRITE);
			memset(table, 0, PAGE_SIZE);
		}

		// Physical address of PTE. Not usable to access it.
		pte_t* pte = &((pte_t*)(pde->table_address<<12))[(v>>12)&0x3FF];
		pte->page_address = p>>12;
		pte->present = 1;

		p += PAGE_SIZE;
		v += PAGE_SIZE;
	}
	return vstart;
}

void vunmap(va_t vstart, size_t size)
{
	if(!size) return;

	uintptr_t v = (uintptr_t)vstart;

	size_t page_num = size/PAGE_SIZE +(size%PAGE_SIZE ? 1 : 0);
	for(size_t i=0; i<page_num; i++)
	{
		pde_t* pde = &page_dir[v>>22];
		if(pde->table_address)
			((pte_t*)(pde->table_address<<12))[(v>>12)&0x3FF].present = 0;
		v += PAGE_SIZE;
	}
	// TODO: Clear PDE present bit and free unused PTE if the whole directory has been unmapped
}

[iansjack]

The logical address of the page table is whatever you want it to be. You control the page table mappings, so you control which logical address maps to a particular physical page.

[Aerath]

Yes, I know, currently it's done with

Code: Select all

find_unmapped_vpages(PAGE_SIZE);

but this address is not stored, only the physical one is saved in the PDE/PTE. So I have no way to change an existant mapping : either the page address or any flag (present, writeable...). You can see the virtual address is only available in the context of

Code: Select all

if(!pde->table_address){

[iansjack]

Either you do store the address (it's your OS, so you can do what you like) or else use the recursive page table trick ( http://wiki.osdev.org/Page_Tables ). Alternatively, it doesn't need a huge amount of space to map the whole physical address space to some logical range, although this is probably only useful for 64-bit code where you have an effectively limitless range of logical addresses to play with.

Actually, I'd recommend that your OS uses 64-bit code anyway, for a number of reasons.

[Aerath]

I was hoping there might be a way to do it without additional overhead. Anyway thanks for your answers and hints, I'll think about them.
I will keep the topic unresolved for the next two days, curious to see if someone has figured some creative way of working around it.

[Brendan]

Hi,

I was hoping there might be a way to do it without additional overhead. Anyway thanks for your answers and hints, I'll think about them.
I will keep the topic unresolved for the next two days, curious to see if someone has figured some creative way of working around it.

The recursive mapping trick (already mentioned by iansjack) has almost no overhead. It doesn't cost additional memory (like identity/linear mapping physical memory does) and finding the virtual address of a page table is very fast (e.g. for plain 32-bit paging, maybe "pointer_to_page_directory_entry = (virtual_address >> 20) & 0xFFFFFFFC + 0xFFFFF000" and "pointer_to_page_table_entry = (virtual_address >> 10) & 0xFFFFFFFC + 0xFFC00000").

The only real problems are that it's a little harder to understand, and you can't use it to modify a different virtual address space (but normally you only want to modify the current virtual address space, and there's various ways to handle rare "cross-modifying" cases cheaply).


Cheers,

Brendan

[Aerath]

If I understood recursive mapping, it has a fixed layout. I mean, given a virtual address, it will always have the same physical translation (and that's why one can use formulas to translate physical to virtual). But I want to be free to map any virtual address anywhere in the physical space.

Besides, Brendan, you said

finding the virtual address of a page table is very fast (e.g. for plain 32-bit paging, maybe "pointer_to_page_directory_entry = (virtual_address >> 20) & 0xFFFFFFFC + 0xFFFFF000" and "pointer_to_page_table_entry = (virtual_address >> 10) & 0xFFFFFFFC + 0xFFC00000").

I guess by pointer_to_page_directory_entry you mean the virtual address, since finding it is what I am looking for and what you are talking about in your previous sentence.
However your formulas use virtual_address as an input. It seems strange to me, since it's the result we are looking for. Or maybe I guessed wrong.

[Brendan]

Hi,

If I understood recursive mapping, it has a fixed layout. I mean, given a virtual address, it will always have the same physical translation (and that's why one can use formulas to translate physical to virtual). But I want to be free to map any virtual address anywhere in the physical space.

I'm not sure if this matters (if it's an irrelevant typo or a fundamental misunderstanding of how paging works), but you can't map a virtual page into the physical address space - you can only map physical pages into (one or more) virtual address spaces.

An OS developer is free to do whatever they like with virtual address spaces; however typically they start by splitting it into "fixed layout" areas. For example:

Code: Select all

0x00000000 to 0xBFFFFFFFF = user-space
0xC0000000 to end of kernel's .bss = used by kernel's executable
End of kernel's .bss to 0xFFBFFFFF = used by kernel's dynamic memory management ("heap" or whatever)
0xFFC00000 to 0xFFFFFFFF = used by recursive mapping

Besides, Brendan, you said

finding the virtual address of a page table is very fast (e.g. for plain 32-bit paging, maybe "pointer_to_page_directory_entry = (virtual_address >> 20) & 0xFFFFFFFC + 0xFFFFF000" and "pointer_to_page_table_entry = (virtual_address >> 10) & 0xFFFFFFFC + 0xFFC00000").

I guess by pointer_to_page_directory_entry you mean the virtual address, since finding it is what I am looking for and what you are talking about in your previous sentence.
However your formulas use virtual_address as an input. It seems strange to me, since it's the result we are looking for. Or maybe I guessed wrong.

The typical scenario is that you have a virtual address of something (e.g. of an area where "sbrk()" wanted the kernel to allocate more pages, the virtual address that caused a page fault, the virtual address that an executable loader wanted to memory map part of an executable file, etc), and need to find the virtual address of the page directory/table entries that correspond to that original virtual address.


Cheers,

Brendan

[Aerath]

Ok, looks like I did not understand everything about recursive mapping. I will look at it later (not enough spare time currently).

No worries, I (think I) know how paging works, it's neither a typo nor a misunderstanding but how I am thinking of it : CPU instructions use virtual addresses that are translated to physical ones and then sent to the memory controller. So it seems more logical to me to "map virtual to physical" rather than the opposite. Furthermore that's how we are accessing paging structures : given a virtual address, we get or set the physical equivalent. And it's precisly this one way link that made me write this topic.

[LtG]

Ok, looks like I did not understand everything about recursive mapping. I will look at it later (not enough spare time currently).

No worries, I (think I) know how paging works, it's neither a typo nor a misunderstanding but how I am thinking of it : CPU instructions use virtual addresses that are translated to physical ones and then sent to the memory controller. So it seems more logical to me to "map virtual to physical" rather than the opposite. Furthermore that's how we are accessing paging structures : given a virtual address, we get or set the physical equivalent. And it's precisly this one way link that made me write this topic.

It usually takes people a few moments to wrap their head around the recursive paging trick, often people also recommend using pen and paper and working a few examples out.

Note, you are free to use any physical pages for all mappings and you can allocate any virtual addresses to any processes, except of course the address space used by the page directory itself. So there's really no limitations, except it takes 4MiB of VAS (Virtual Address Space, not memory) from each process, but there's nothing you can really do about that if you want to modify the paging while "in process"..