Page 1 of 1
x86_64 support for 1GB pages
Posted: Mon Sep 22, 2014 11:22 am
by AndrewAPrice
How common is support for 1GB pages on modern x86_64 desktop CPUs?
x86_64 CPUs have 4 levels of page tables/directories. This complicates things - if I don't have any free entries left if my page table - how am I supposed to allocate page tables/directories to store the page table/directory I want to work on if there's no temporary place to put them to write them?
I thought that I could probably preserve 4 kernel pages tables entries that I could map page tables/directories into when I'm working with them - then I don't need to keep my paging structures in virtual memory. But I wonder if this would be slow?
Alternatively, if support for 1GB is common, I could identity map the top 512GB of virtual memory to the first 512GB of physical memory, then my kernel has the freedom to write to any physical memory address - simplifying things considerably.
What are your thoughts?
Re: x86_64 support for 1GB pages
Posted: Mon Sep 22, 2014 2:45 pm
by Nable
AFAIR, I've saved this link from some thread of osdev forum:
http://www.pvk.ca/Blog/2014/02/18/how-b ... -pages-be/
Re: x86_64 support for 1GB pages
Posted: Mon Sep 22, 2014 4:14 pm
by AndrewAPrice
Thank you for replying, Nable. Unfortunately, I didn't find anything relevant to my question of identity mapping the top 512GB of virtual memory to 512GB of physical memory, or how common support for 1GB pages are.
Re: x86_64 support for 1GB pages
Posted: Tue Sep 23, 2014 1:38 am
by Nable
MessiahAndrw wrote:I didn't find anything relevant to my question of identity mapping the top 512GB of virtual memory to 512GB of physical memory, or how common support for 1GB pages are.
Oh, sorry, my answering bot matched keywords but didn't get the sense. Ok, I'll try to partly fix the situation.
1. 1G pages are common for server CPUs since
Intel Westmere, wiki
says: "newer x86-64 processors, such as AMD's newer AMD64 processors and Intel's Westmere[15] and later processors can use 1 GB pages in long mode". I couldn't find such information about desktop CPUs but it looks like such support isn't common for them.
2. 512G to one CPU? It doesn't seem useful. There are no such large TLBs, common config looks like 64*4k + 32*2M + 4*1G entries. Identity mapping also doesn't seem to be a good idea, as one shouldn't assume anything about physical address space layout. You can find some discussion of idea "mapping the whole physical memory" here:
http://forum.osdev.org/viewtopic.php?p=231539#p231539
Re: x86_64 support for 1GB pages
Posted: Tue Sep 23, 2014 3:08 am
by alexfru
MessiahAndrw wrote:... if I don't have any free entries left if my page table - how am I supposed to allocate page tables/directories to store the page table/directory I want to work on if there's no temporary place to put them to write them?
You could always reserve a few for this very purpose, couldn't you? You probably don't need many. And you could also use a counting semaphore to limit the number of simultaneously used entries. When you attempt to go over the limit, you block.
Also, have you considered recursive page tables/directories? Just how bad would it be to deal with non-uniform PTEs/PDEs/etc?
Re: x86_64 support for 1GB pages
Posted: Tue Sep 23, 2014 6:19 am
by Brendan
Hi,
MessiahAndrw wrote:What are your thoughts?
Fortunately, most CPUs (even those that don't support fancy stuff like 1 GiB pages) do support conditional branches. This means it'd be trivial to do something like:
Code: Select all
if( CPU_supports_1GiB_pages() ) {
// Setup the hideously wasteful "physical memory mapping" using 1 GiB pages
} else if( CPU_supports_2MiB_pages() ) {
// Setup the hideously wasteful "physical memory mapping" using 2 MiB pages
} else {
// Setup the hideously wasteful "physical memory mapping" using 4 KiB pages
}
Cheers,
Brendan
Re: x86_64 support for 1GB pages
Posted: Tue Sep 23, 2014 6:29 am
by bluemoon
MessiahAndrw wrote:Alternatively, if support for 1GB is common, I could identity map the top 512GB of virtual memory to the first 512GB of physical memory, then my kernel has the freedom to write to any physical memory address - simplifying things considerably.
Why 512G? IMO, having magic number like this is not elegant.
While today's i7 CPU can address like 32GB of physical memory, having a limit of 512GB seems reasonable enough, yeah Bill Gates thought the same (64 kb is enough for anyone).
If, for some unexpected breakthrough on memory technology & manufacturing, that consumer PC has easily get into the TB scale (as currently already existed on supercomputers, a quick search give 710TB of memory). You'll then have to redo the kernel design.
You may think this is over-consideration, but once you completed your kernel one or more decade later, the average digital clock might be more powerful then today's supercomputer.
On the other hand, the traditional design would have less impact on the available memory.
Re: x86_64 support for 1GB pages
Posted: Tue Sep 23, 2014 6:57 am
by Brendan
Hi,
bluemoon wrote:If, for some unexpected breakthrough on memory technology & manufacturing, that consumer PC has easily get into the TB scale (as currently already existed on supercomputers, a quick search give 710TB of memory). You'll then have to redo the kernel design.
It's not just unnecessary/artificial limitations.
In general, we use paging because it's
powerful and flexible, and lets us do various tricks to improve performance (e.g. NUMA optimisations) and reduce RAM consumption (e.g. copy on write), and lets us do them efficiently. People that do the "map all of physical memory into kernel space" thing typically take something powerful/flexible and then throw all of that power/flexibility away.
Cheers,
Brendan
Re: x86_64 support for 1GB pages
Posted: Thu Sep 25, 2014 10:21 am
by AndrewAPrice
bluemoon wrote:Why 512G? IMO, having magic number like this is not elegant.
I agree with you. I'm abandoning this idea, and there are limitations - physical memory can be extremely fragmented - after enough time running, it might be hard to find space to allocate large data structures in the kernel.
alexfru wrote:You could always reserve a few for this very purpose, couldn't you? You probably don't need many. And you could also use a counting semaphore to limit the number of simultaneously used entries. When you attempt to go over the limit, you block.
That's the approach I'm going with now. I'm going to reserve a page table (that I keep in virtual memory) - that gives me 512 entries to I can write any arbitrary page into to access any part of physical memory. Now I can use these to set up my page tables/directories/pml* without needing to keep them in memory.
Re: x86_64 support for 1GB pages
Posted: Mon Sep 29, 2014 1:01 pm
by eryjus
I have been looking at my own paging question: 4K vs. 2M vs. 1G and I came across the following post from
Brendan. I thought it might be relevant to your question (not really "how common" but maybe "why").