2MiB vs 4KiB pages in 64-bit mode

[IanSeyler]

Currently we are using 2MiB pages in BareMetal OS. Are there any advantages/disadvantages to using 4KiB pages instead? Which offers better performance?

[Owen]

"It depends"

There are more TLB slots for 4kb pages; so, if you're accessing data spread out throughout the address space, then 4kb pages are better. However, the small number of large page TLB entries cover more RAM.

Also note that, with the Phenom, AMD introduced dedicated TLB entries for 1GB pages; I wonder if Intel have done the same? This of course indicates that one should consider a three tier strategy.

For conventional operating systems, a possible layout would be

• Where possible, map the first 2GB of RAM (or so) at the top of the address space using 1GB pages. This minimizes the space taken up by static data and code. On older CPUs, use 2MB pages.
• Use 4kb pages for application RAM and the kernel heap
• Use 2MB pages for mapping large contiguous ranges (E.G. graphics card frame buffer)

Caching should also be set as appropriate; for example, the aforementioned frame buffer should be write combining. The idea is to maximize use of the TLB; as you cannot easily allocate the larger pages, then use them opportunistically in order to minimize the impact on the most heavily used portion.