OSDev.org

Hi,
I know this may not be the right place but I have found no answer elsewhere...

I am deploying a virtual lab based on vSphere ESXi 5.5 running on HP server with dual Intel Xeon E5620 processor supporting EPT (Extended Page Table) technology.
From vmware kb documentation, in "normal" condition ESXi 5.5 running on processor supporting EPT (or AMD RVI) make use of host large page (2MB) to back VM's guest physical pages (guest OS is unaware of that and can use 4KB or even 2MB as page size width)

Now, AFAIK, just when ESXi host is under memory pressure large pages are broken into 4k pages to allow Transparent Page Sharing (TPS) to work.

Now my doubt is: when we say "large pages are broken into small page to allow TPS to work" actually means the (EPT) address translation scheme is changed from now on and ESXi VMkernel has to build up on the fly the page tables needed to support the last level of page walk required by the new (EPT) translation scheme ?

thanks

ESXi host is under memory pressure large pages are broken into 4k pages

If a system has been running for a while and getting low on memory, there will be less contiguous chunks of memory because intermittent frees leave relatively small holes. At some point, fragmentation prevents whole 2MB pieces to be no longer available and the system has to hand out the next smaller size at 4K.

In addition, when running several VMs, the chance that you find the same 4K page in more than one VM is significantly larger than finding an identical 2M page. For that reason, if you want to reclaim memory from VMs by sharing, you might want to check on a 4K basis and arrange the backing page tables on a 4K basis, even if you do have 2M pages available. This includes removing 2M page entries and replacing them with a page table and a series of 4K pages of which you can share a few between VMs.

None of this should actually have to alter guest states.

Thanks.

Have you an idea how to check if hypervisor is making use of large page to back guest OS pages ?

A version of "Cache missing for fun and profit" would be the most VM-agnostic way to do it. There might be MSRs or VM specific methods of querying though that can prove to be more helpful, but because all of these are basically side-channel attacks on the host you might find them blocked or otherwise impaired.

The more important question is however, do you even need to know how the VM maps guest to host memory? A VM is meant to work transparently after all.

A related question: can a VMM/Hypervisor force guest OS running into a VM to NOT make use of Large Page ? Previously in this thread I assumed is not possible because guest OS can independently choose the page size to use...

Does it make sense ?

The VM normally shouldn't care what page size the client uses.

That said, you can prevent large pages to some extent, by faking that PSE/PSE36/PAE doesn't exist:

AMD SVM Manual wrote:Various instructions and events (such as exceptions) in the
guest can be intercepted by means of control bits in the VMCB.
The two primary classes of intercepts supported by SVM are
instruction and exception intercepts.
(...)
CPUID
(...)
MOV to/from CRn

This is completely ineffective for 64-bit guests as long mode guarantees large page support.