Virtual Memory Goals

[RezaNoei]

Hi friends.

We use virtual memory to make our physical address space securely accessible for each processes (either if it is a user process or a kernel process).
and it has very noticeable benefits for our OS.

Question :

Since we have a 1024*1024*4 byte of page tables for our kernel, what will happens if we have 1024 running process ? 4gb of page tables?

if we must have a 4mb of page tables (like a big bag) for each process while we have a multitasking OS ?
I think that would not be a good idea as I cant imagine a good one. because each process must have a full access to 2^32 bit of address space while we enabled paging.

is there a better idea to saving the main memory from this wildly consumption ?

[Sourcer]

You can use the swapping concept and save memory in a special file.

[SpyderTL]

I think you can actually swap entire page tables when switching from one process to another, so it's entirely possible that each process could have its own paging table. This is mentioned, briefly, on the Context Switching page.

[Ch4ozz]

Well, you only map the memory you need, and the kernel memory for example will be the exact same so you can just use the same page tables without the need to allocate them more then once.

[iansjack]

each process must have a full access to 2^32 bit of address space while we enabled paging.

That's quite wrong. That would give each process access (on a 32-bit processor) to the whole address space, negating one of the prime advantages of paging (protecting processes from each other).

Each process should only access a small subset of the address space - the virtual addresses will be the same for each process but they will be mapped to different physical pages.

[shmx]

You maped memory addresses at once. Why are you doing it? The memory can be mapped when necessary. Kernel memory should be mapped for all processes (simply copy PTE items).

[neon]

To add to what was already posted, typically you wouldn't even map the whole process into the address space - only what is actually referenced. This is done through demand paging - don't map any part of the process at the start and let it page fault. Map only what you need, when you need it. Your operating system should have a way to reserve regions of user space (like a list of free regions or a tree) which can be used to determine valid addresses when the page fault happens.

[gerryg400]

I think you are all missing the point. The OP is asking how to prevent using the entire kernel address space for page tables.

RezaNoei, it's true. This is a problem. One way of addressing it is by using Recursive mapping as described in the wiki here. Under this scheme you generally only have the page tables of the current process in memory at any time.

[shmx]

This is a problem.

I don't see problem. Can you explain more detail?

[Brendan]

Hi,

Since we have a 1024*1024*4 byte of page tables for our kernel, what will happens if we have 1024 running process ? 4gb of page tables?

I never map it all into kernel space. Normally I map the highest level paging structure (e.g. page directory for "plain 32-bit paging") for all processes into kernel space, and all paging structures for the current process only into kernel space. This means (for "32-bit plain paging") that with 1024 processes I need 1024 * 4096 + 4 MiB = 8 MiB of kernel space; and with 256 MiB of kernel space used for mapping page directories (plus 4 MiB for the current process) I can have up to 65536 processes.

Of course (on 32-bit systems) you're much more likely to run out of RAM before you ever hit the "too many processes" limit, and on 64-bit systems there's plenty of kernel space (128 TiB) so you can have far more processes (and still run out of RAM before you ever hit the "too many processes" limit).

I think that would not be a good idea as I cant imagine a good one. because each process must have a full access to 2^32 bit of address space while we enabled paging.

No.

Each process has access to "a maximum of up to 4 GiB of virtual address space", but typically a quarter is "kernel space" that's the same in all virtual address spaces so it's more like "a maximum of up to 3 GiB of virtual address space per process", and for most processes almost all of it is "not present" and costs nothing.

For example, if you have a 1234 KiB executable that has a ".bss" section of 1234 KiB and then dynamically allocates another 1234 KiB of memory; then it's only using a total of 3702 KiB of its virtual address space (and not the entire 3 GiB) and (for "plain 32-bit paging") will probably end up only needing one page table (and not 768 page tables).

Also note that there are many tricks used to reduce RAM usage; like "copy on write" (where the same physical page is mapped into multiple different virtual address spaces, and you create a copy of it if anyone writes to it), and "allocate on demand" (where you have a single page full of zeros mapped everywhere and allocate a real physical page if someone writes to it), swap space, memory mapped files, etc. Most of these tricks can be applied to page tables, etc too - there's no reason you can't have a "copy on write page table", or have a "page table filled with the same page full of zeros", or send a process' page table to swap space, etc.

The real limit is that (for worst case) the CPU needs to be able to access up to 6 pages at once; which means (for worst case, for "32-bit plain paging") one page directory, 6 page tables and 6 pages of data (13 pages). Basically; as long as you have 52 KiB of RAM left for processes to use everything else can be elsewhere (in swap space, in memory mapped files, etc) and more RAM just means better performance (less disk IO).


Cheers,

Brendan

[gerryg400]

This is a problem.

I don't see problem. Can you explain more detail?

Naah, it's quite clear in the original post.

[Boris]

Hi,
Don't allocate every sub-tables immediately.
Allocate them on the fly .
Recursive mapping allows you to alter pagination tables without physical memory cost.
If you want to map arbitrary physical memory to a virtual location, you have to check and allocate intermediary tables if they are not present.
That way instead of eating 4.4Mb per address space tables, you just have 4kb x ( 1 + amount of tables needed immediately)

[shmx]

Naah, it's quite clear in the original post.

I just allocate physical memory (and PTEs, if necessary) when first accessing the page in #PF handler without any problems.

[LtG]

The real limit is that (for worst case) the CPU needs to be able to access up to 6 pages at once; which means (for worst case, for "32-bit plain paging") one page directory, 6 page tables and 6 pages of data (13 pages). Basically; as long as you have 52 KiB of RAM left for processes to use everything else can be elsewhere (in swap space, in memory mapped files, etc) and more RAM just means better performance (less disk IO).

Out of curiosity, where'd you get the number 6?

13 seems reasonable based on the 6, but the 6 itself?

[alexfru]

Out of curiosity, where'd you get the number 6?

Something like a movsd instruction crossing 3 page boundaries (1 code and 2 data).