Questions about Paging (theory)

[Jef]

Hi,

I have some questions about paging.

If i use paging, all processes (including ring-0 kernel) must use paging?
Or when i switch task i can choose if the current process use or not paging?

A process (not kernel) can only "see" the virtual memory that page directory "says" ?
So, i have to add an entry for kernel address (so can call kernel functions) ?

I must create a page directory for all physical memory presents (?), or only for that i use (allocate) ?

I have to put page entries for every 4k memory?
I mean, if i want to allocate 2 MB memory, i have to make 512 entries ?
(and the limit of page entries are 256 per directory) ?

[bewing]

You can have some processes use virtual memory (paging) and some others using physical memory at the same time. My kernel uses physical memory, all my executive modules use physical memory, some drivers use physical mem, some use virtual, and all user apps use virtual. Many other developers here use virtual memory for everything. Your choice.

A process (not kernel) can only "see" the virtual memory that page directory "says" ?

Yes.

So, i have to add an entry for kernel address (so can call kernel functions) ?

That is the way most developers do it. It is very fast if userapps can call kernel functions directly. I don't allow that, myself.

I must create a page directory for all physical memory presents (?), or only for that i use (allocate) ?

It is best if you only do it for pages that are currently allocated. You may have to "protect" each page in the table from the current app, if the app is not supposed to be able to see it. The more pages that are in the table, the more work you have to do to protect them.

I have to put page entries for every 4k memory?

Yes. That really is best. The other choice is for every 4MB -- and that is just unworkable.

if i want to allocate 2 MB memory, i have to make 512 entries ?

Yes.

(and the limit of page entries are 256 per directory) ?

I don't remember.

[JamesM]

(and the limit of page entries are 256 per directory) ?

I don't quite understand what you are asking here. A page directory is a two level structure. It holds (in standard 32-bit mode) 1024 4-byte pointers to "page tables". Each page table holds 1024 4-byte "page table entries", that is, a physical address plus some access / reserved bits. There is a tutorial here that may help your understanding.

So, you do not create page table entries for EVERY 4k of virtual memory. If you have an entire page table that has no allocated page table entries, you can leave that page table 'unallocated' and save memory.

[AJ]

Hello,

Yes - you do have to add entries in the page directory (and associated page tables) for all memory you want processes to see. Just map in a new page for each new page of virtual RAM you want to access.

One mothod of doing this that you may like to consider, is that the last Page Directory Entry (PDE) points to the Page Directory itself. This means that you access the PD via address 0xFFFFF000 (virtual). Your page tables are now consecutive in memory from 0xFFC00000. Whenever you assign a new Page Table (PT), add the appropriate PTE. As each PDE represents 4MB, you would therefore, for mapping in virtual address 0x400000, do the following:

Code: Select all

uint32 *PD = (uint32*)0xFFFFF000;
PD[1] = allocate_physical() | PAGE_PRESENT | PAGE_WRITE;

Because the PD is mapped to itself, the PT you have just allocated can now be accessed at 0xFFC01000; Sticking with the 'mapping in 0x400000 example, you then do:

Code: Select all

uint32 *PT = (uint32*)0xFFC00000;
PT[0] = allocate_physical | PAGE_PRESENT | PAGE_WRITE;

You should now find that you can read/write at memory address 0x400000.

Just a couple of points about the assumptions you have made:
Yes, pages are normally 4KiB in size, BUT
1. If using protected mode, they can also be 4MiB or 2MiB depending on whether you are using Physical Address Extensions and Page Size Extensions.
2. If you are using long mode, they can also be 2MiB or 1GiB if you like.
3. You can actually have 0x400 (1024) entries in a single page directory if you are in protected mode without PAE. This means a single PD can map 4MiB.

But my suggestion would be to follow a paging tutorial of your choice (see http://www.osdever.net for a start) and get paging working in its most basic version first.

Cheers,
Adam

[Jef]

thanks to everyone,

i understand.

the tutorial is also good enough.

[Jef]

after a lot of days, i start again the code of paging.
But, i probably missing something.

How/where you write the virtual address?
Or just you have to find the correct entry (index) that is the wanted address?

for example, if you want to allocate the virtual address 0x400000 (size 8192),
you must fill the first and the second entry of the table which is at the 4096th entry in the page directory (is fact first entry of the 4th page directory) ?

[pini]

The easiest way to think of entries to do the following:

Remove the lowest 12 bits, so address 0x400000 gives you 0x400
This gives you a kind of "absolute" entry in the page directory. Here, the entry is number 0x400=1024, so it's in fact the 1025th entry (counting starts at 0).

Then you perform a division by 1024. Here 1025 = 1 * 1024 + 1.
Here you get the entries to use: first entry of page directory gives the page table and then first entry in that table gives you the entry to fill.

In your code, it's even easier to deal with: just do what the VMM would do to locate the entry. Take the highest 10 bits of the address gives you the directory entry. This entry gives you the table. Then take the bits 12 to 21 gives you the entry in the table, and here you are with the correct entry to fill.

Hope paging is clear to you now

[Jef]

ok. so far so good.
But how you protect a page?
i mean, how you "say" that a process uses only these pages?

and if you want a page to be read-write for one process, and read only for another process, you have to give a different virtual address for the same physical address ?

[AJ]

Hi,

The usual way to do this is to have a separate Page Directory for each process (which just means changing CR3 when you do a task switch). If your kernel is in high address space, say at 0xC0000000, all page tables above 3GB are shared kernel-privilege pages.

All page tables below 3GB can be unique to an individual process with whatever mappings you like. In this way, all processes can be linked to run at 0x100000, for example.

Cheers,
Adam

[Jef]

Hi,

The usual way to do this is to have a separate Page Directory for each process (which just means changing CR3 when you do a task switch). If your kernel is in high address space, say at 0xC0000000, all page tables above 3GB are shared kernel-privilege pages.

All page tables below 3GB can be unique to an individual process with whatever mappings you like. In this way, all processes can be linked to run at 0x100000, for example.

Cheers,
Adam

sounds good but if a single process want to allocate 100 MBytes ?
Any way, for now a have some other bugs and cannot test it, but thanks for the info. I'll keep it for later use

[AJ]

sounds good but if a single process want to allocate 100 MBytes ?

Nothing stopping that - in this system a single process can see up to 3GiB of its own space (+- a bit for control structures, if appropriate).

Cheers,
Adam

[Jef]

oh, you say for every process to have one page directory. I think you say page table. Now i re-read it.
Ok.

[Jef]

The easiest way to think of entries to do the following:

Remove the lowest 12 bits, so address 0x400000 gives you 0x400
This gives you a kind of "absolute" entry in the page directory. Here, the entry is number 0x400=1024, so it's in fact the 1025th entry (counting starts at 0).

Then you perform a division by 1024. Here 1025 = 1 * 1024 + 1.
Here you get the entries to use: first entry of page directory gives the page table and then first entry in that table gives you the entry to fill.

In your code, it's even easier to deal with: just do what the VMM would do to locate the entry. Take the highest 10 bits of the address gives you the directory entry. This entry gives you the table. Then take the bits 12 to 21 gives you the entry in the table, and here you are with the correct entry to fill.

Hope paging is clear to you now

I try this but seems to be wrong
Init_Paging:

Code: Select all

	mov 	edi, PAGE_DIR
	mov	ecx, 4096
	mov	eax, PAGE_TABLE	
.loop1:
	and	eax, 0FFFFF000h
	or	eax, 11b
	mov	dword [edi], eax
	add	edi, 4
	add	eax, 4096
	loop	.loop1		;clear all entries
	
	mov	edi, PAGE_TABLE
	mov	ecx, 10000h
	mov	eax, 10b	;read/write, not present
	rep	stosd	

SetMemPage:

Code: Select all

mov	edi, PAGE_DIR
	mov	eax, dword [ebp + 8]	;physical address
	and	eax, 0FFFFF000h	
	ror	eax, 12		;remove lower 12 bits
	inc 	eax
	xor	edx, edx
	mov	ecx, 1024
	div	ecx
	add	edi, eax
	mov	eax, dword [edi]
	or	eax, 11b
	mov	dword [edi], eax

	and	eax, 0FFFFF000h	
	mov	edi, eax
	mov	eax, dword [ebp + 8]
	and	eax, 3FF000h	;keep 12-21 bits
	ror	eax, 12		;remove lower 12 bits
	add	edi, eax
	
	mov	eax, dword [ebp+8]
	mov	ecx, dword [ebp+4]          ;size in blocks (of 4096 bytes)
.loop1:
	and	eax, 0FFFFF000h
	or	eax, 11b
	mov	dword [edi], eax
	add	edi, 4
	add	eax, 1000h
	loop	.loop1

Any idea ?

[pini]

A few things are wrong (to me) in your code :

Init_Paging:

Code: Select all

	mov 	edi, PAGE_DIR
	mov	ecx, 4096
	mov	eax, PAGE_TABLE	
.loop1:
	and	eax, 0FFFFF000h
	or	eax, 11b
	mov	dword [edi], eax
	add	edi, 4
	add	eax, 4096
	loop	.loop1		;clear all entries
	
	mov	edi, PAGE_TABLE
	mov	ecx, 10000h
	mov	eax, 10b	;read/write, not present
	rep	stosd	

You're clearing too much entries!
The page directory has only 1024 entries, and you're cleaning 4096.
Same for the page table: only 1024 entries, not 0x10000.

SetMemPage:

Code: Select all

mov	edi, PAGE_DIR
	mov	eax, dword [ebp + 8]	;physical address
	and	eax, 0FFFFF000h	
	ror	eax, 12		;remove lower 12 bits
	inc 	eax
	xor	edx, edx
	mov	ecx, 1024
	div	ecx
	add	edi, eax
	mov	eax, dword [edi]
	or	eax, 11b
	mov	dword [edi], eax

	and	eax, 0FFFFF000h	
	mov	edi, eax
	mov	eax, dword [ebp + 8]
	and	eax, 3FF000h	;keep 12-21 bits
	ror	eax, 12		;remove lower 12 bits
	add	edi, eax
	
	mov	eax, dword [ebp+8]
	mov	ecx, dword [ebp+4]          ;size in blocks (of 4096 bytes)
.loop1:
	and	eax, 0FFFFF000h
	or	eax, 11b
	mov	dword [edi], eax
	add	edi, 4
	add	eax, 1000h
	loop	.loop1

Computation for the directory entry is wrong (I think you misunderstood my explanation). Try this:

Code: Select all

mov	edi, PAGE_DIR
	mov	eax, dword [ebp + 8]	;physical address
	and	eax, 0FFC00000h	; Keep directory entry index
	shr	eax, 22		;remove lower 22 bits
	add	edi, eax
	mov	eax, dword [edi]
	or	eax, 11b
	mov	dword [edi], eax

	and	eax, 0FFFFF000h	
	mov	edi, eax
	mov	eax, dword [ebp + 8]
	and	eax, 3FF000h	;keep 12-21 bits
	shr	eax, 12		;remove lower 12 bits
	add	edi, eax
	
	mov	eax, dword [ebp+8]
	mov	ecx, dword [ebp+4]          ;size in blocks (of 4096 bytes)
	shr	ecx,12 ; divide by 4096 to get the number of _pages_
.loop1:
	and	eax, 0FFFFF000h
	or	eax, 11b
	mov	dword [edi], eax
	add	edi, 4
	add	eax, 1000h
	loop	.loop1

But be aware that your code only works if all the requested pages fit in a single page table.

Also note that you're doing 1:1 paging here, i.e. the linear and physical (paged) addresses are the same. Sometimes you want this (in kernel space mostly), sometimes you don't care (in user space mostly).

If you don't need 1:1 paging, you don't need to find specific directory and table entries, instead you just pick the first entry that fits your need and returns the linear address.

[Jef]

You're clearing too much entries!
The page directory has only 1024 entries, and you're cleaning 4096.

True

Same for the page table: only 1024 entries, not 0x10000.

if you have 1024 entries in page directory, you have 1024 page tables.
Every table has 1024 entries. So 1024 * 1024 = 100000h
Page Table(s) size is 10000h.
Where is the fault ?

But be aware that your code only works if all the requested pages fit in a single page table.

Why?

Also note that you're doing 1:1 paging here, i.e. the linear and physical (paged) addresses are the same. Sometimes you want this (in kernel space mostly), sometimes you don't care (in user space mostly).

For now, i don't have userland, so its ok for 1:1.
I have in my back of my mind later to make something different.

If you don't need 1:1 paging, you don't need to find specific directory and table entries, instead you just pick the first entry that fits your need and returns the linear address.

I know

Code: Select all

mov   ecx, dword [ebp+4]          ;size in blocks (of 4096 bytes)
   shr   ecx,12 ; divide by 4096 to get the number of _pages_ 

Why you divide with 4096 ?
Parameter has size / 4096 already (i say in comment : "size in blocks (of 4096 bytes)"

But still code is not working (after your changes).
Maybe something else is wrong, but i appreciate your help.