OSDev.org

Posted: **Mon Mar 14, 2016 4:28 pm**

I strongly recommend on enabling EPT. Set them to identity map (GPA = HPA) with RWX rights and caching according to the MTRRs for now, but at least have it. It will save you from a lot of problems.

When the guest will try to enable and activate long mode (by writing to the EFER MSR) you will get an exit and you must update the guest area. Don't corrupt the host EFER MSR

Also, you may want to look at bit 9 inside the VM-Entry Controls (IA 32e mode guest) and at bit 15 (table 24-12 in the documentation).
Somehow related (and the reason I reply when there is already a good answer), special care should be taken for 32 bit guests that use PAE.

Posted: **Tue Mar 15, 2016 4:02 am**

Hi,

zdz, Thank for your reply but you are talking Chinese for me

You need someone who already understand to understand what you are explaining

You are trying to help so consider that I am trying to understand not to get more confused

Try please to explain to me in a simpler thorough way. It is good especially when someone like me in the future come to osdev with the same questions, he/she can make use of this thread.

But thanks anyway for spending the time.

For alexg,

Okay I got that in the manual and will try it out.

But surprisingly in section 26.4 page "vol 3c 26-17" in the foot notes it says the following:
1. Because attempts to modify the value of IA32_EFER.LMA by WRMSR are ignored, attempts to modify it using the VM-entry MSR-
load area are also ignored.
2. If CR0.PG = 1, WRMSR to the IA32_EFER MSR causes a general-protection exception if it would modify the LME bit. If VM entry has
established CR0.PG = 1, the IA32_EFER MSR should not be included in the VM-entry MSR-load area for the purpose of modifying the
LME bit.

So does this apply to the entry of my VM. My question is if the LME bit is ignored why would I use the IA32_EFER MSR in the VM entry to enable it in my guest.

This sounds to be contradicting with 24.4.

Please let me know if you have explanation or if I am misreading it.

Just a verification question, I should be using the offsets in the table in B.2.3 to set the guest EFER with vmwrite right?

Thanks a lot,
Karim.

Posted: **Tue Mar 15, 2016 6:15 am**

Do things as easy as possible and do programming incrementally. In first hypervisor enable as little of vmx features as possible. You will be happy when you first time hit you vm exit handler (no errors/mistakes/invalid settings in VMCS fields, so everything passes and vmentry completes). Then you will work on improving vm exit handlers. Then you return back and enable more features in VMCS fields on hypervisor startup (like EPT, unrestricted guest possiblity and so on).
In first attempt, do not modify MSR EFER at all. Just copy guest EFER.LMA (bit 10) into IA-32e mode guest (bit 9) of vmentry controls VMCS field.
And do not yet enable MSR load feature (you may do it later in development when you have simplest possible hypervisor running well).
When guest enables/disables long mode your hypervisor has 2 choices:
[0] do not care (guest will do it without notifying hypervisor)
[1] do care, that's enabling vmexits on WRMSR into MSR EFER and enabling vm exits on CR0.PG modification (you must enable CR0 guest/host mask bit 31, set CR0 read shadow to 0 when guest is in long mode and to 1 when guest leaves long mode) and the same also on CR0.PE. Your vmexit handler have to set VM entry controls bit 9 to 1 when guest enters long mode and set that bit to 0 when it leaves long mode (the only way is when guest disables CR0.PG and then it may modify MSR EFER via WRMSR instruction). Remember you will handle situation when guest CR0.PG=1 and guest attempts to modify MSR EFER by disabling EFER.LME by injecting exception into guest. That's again and again complication. You will do these steps when you have first simplest hypervisor running well.

And remember, EFER.LMA bit is set by hardware, not by you. When CR0.PE=CR0.PG=EFER.LME=1, then CPU sets EFER.LMA to 1 also (long mode active).
When e.g. CR0.PE=EFER.LME=1, but CR0.PG=0, then value of EFER.LMA is still 0 (long mode not active, long mode is only enabled in EFER.LME, but not yet active in EFER.LMA) and when you then change CR0.PG from 0 to 1, on this instruction completion, CPU activates long mode and changes EFER.LMA from 0 to 1. You can modify CR0.PE, CR0.PG, EFER.LME, but you can't EFER.LMA (that bit is modified by hardware according settings of previous 3 bits).

Posted: **Tue Mar 15, 2016 7:38 am**

Thank you so much for your thorough reply.

When you say:

in first attempt, do not modify MSR EFER at all. Just copy guest EFER.LMA (bit 10) into IA-32e mode guest (bit 9) of vmentry controls VMCS field.

Which "MSR EFER" I should not modify?
What is the difference between guest EFER.LMA and IA-32e mode guest?
As per your explanation the guest EFER.LMA (bit 10) is not initially set as initially the guest is not in long mode.
Why do I have to copy it, I thought I have to set bit 9 right away?

[0] do not care (guest will do it without notifying hypervisor)

What do you mean by the "guest will do it without notifying hypervisor" ? do what? switch to long mode?

I am not sure, do you mean that the guest code should execute the instructions of switching to long mode as if it is a normal machine?

I was under the impression that if I set the appropriate VMCS environment of the guest and do vmlaunch the guest starts executing in long mode right away and that I do not need to execute any setup instructions from within the guest??

Are you referring to starting the guest in real mode?

I understand that if I am running a normal commercial OS from within my guest I need to start the guest in real mode and I have to emulate the reaction to VM exists by the VMM based on the sequence of the instructions executed by the OS from within the hypervisor as you have explained under [1]. e.g. so when the guest execute instructions for changing LME for example this will cause VM exit and hence this will trap to the VMM and consequently the VMM should modify the VMCS to have the LME bit enabled and vmlaunch again, right?

But what I am aiming at initially is that I would like to launch the VM in long mode and make it execute long mode code without having the guest code go through the long mode initialization at all. Is that possible? If that possible I need a page table attached to the VM anyway to be able to run the guest in long mode, and that is why I asked to use the page table I have for my host OS just to get the VM running and see it running and then I will modify my code to build a page table for each VM I create (in the future).

I am sorry for the unorganized questions by it seems that I am missing something.

Thanks,
Karim.

Posted: **Tue Mar 15, 2016 12:44 pm**

Hi,

zdz, Thank for your reply but you are talking Chinese for me You need someone who already understand to understand what you are explaining You are trying to help so consider that I am trying to understand not to get more confused

Sorry, I was in a bit of a hurry and my post was more a list of places to start looking for if you get into trouble.

I think @feryno answered your previous questions.

I was under the impression that if I set the appropriate VMCS environment of the guest and do vmlaunch the guest starts executing in long mode right away and that I do not need to execute any setup instructions from within the guest??

The guest starts executing from the environment you set up. And you need to set up almost everything. But let's take it step by step: I'm guessing your current problem is that vmlaunch fails? Can you please tell us the steps you made for initializing the guest and what exactly isn't working. I'm guessing that you are dealing with an invalid guest state, but let's not jump ahead.

Posted: **Tue Mar 15, 2016 1:08 pm**

Hi

Thanks for your reply,

I don't have a problem yet

I did as instructed earlier in this thread to read the manuals and look at the code of the tutorial code project; that took a quite long time from me especially when interleaved with work

I decided to do a lot of reading before I start getting my hands dirty.

As I read and look at the code questions started to pop up, and this is how this thread got building up

I will probably slow down on reading and start doing some coding tomorrow.

What I plan to do, or what I aim at is the following (to run on bochs):

1. I already have the code booting and getting to long mode.
2. A page table of 2 MB like the osdev long mode directly tutorial.
3. I will check and enable VMX.
4. Create 2 VMCS regions each of 4KB; one physical for the VMM and the other virtual for the guest.
5. Initialize both VMCS using the 64-bit offsets as in the table of section B.2.3 of the intel manual, right ? I will try to follow the tutorial in codeproject to do that.
6. Will make the guest VMCS point to the same page table ( 2MB) of the host OS.
7. set LME bit in the guest EFER.
8. vmlaunch. (The most important thing for me is to start the vm in long mode right away)
9. All what I intend to do in my quest is to write a byte in the memory ( the mapped 2 MB)
10. Upon vmexit I will print the byte to see if it is affected as expected.

Is that doable.

Just let me know if I am missing anything.

When I try the above tomorrow, I can tell where it fails:) or more about my problems

Thanks guys for your help, I really do appreciate it.

Karim.

Posted: **Thu Mar 17, 2016 4:16 pm**

Hi,

I have started coding and I have a problem in the vmxon when run on bochs. Here is my code after switching to long mode and mapping all memory:

Code: Select all


    uint32_t cpuid_eax = 0;
    uint32_t cpuid_ebx = 0;
    uint32_t cpuid_ecx = 0;
    uint32_t cpuid_edx = 0;
    uint32_t msr_eax_value,msr_edx_value;

asm volatile
      ("cpuid" : "=a" (cpuid_eax), "=b" (cpuid_ebx), "=c" (cpuid_ecx), "=d" (cpuid_edx)
       : "a" (1), "c" (0));  
// after this call cpuid_ecx= 0x2028

    asm volatile ( "rdmsr" : "=a" (msr_eax_value),"=d" (msr_edx_value) : "c" (0x480) );
// after this call msr_eax_value = 0x2b , msr_edx_value = 0x581000

    asm volatile("         \
      in      $0x92, %al; \
      or      $0x02, %al; \
      out     %al, $0x92; \
      mov %cr4,%rax;       \
      bts $13, %rax;        \
      mov %rax,%cr4;    \
      mov %cr0,%rax;       \
      bts $5, %rax;        \
      mov %rax,%cr0;");

    uint64_t vmx_region = kmalloc_ptr->kmalloc_a(0x1000);
// after this call vmx_region = 0x2d5f000
    uint32_t * vmx_region_ptr = (uint32_t *) vmx_region;
    vmx_region_ptr[0] = msr_eax_value;
    
    asm volatile ("vmxon %0"::"m"(vmx_region));

    asm volatile ("vmxoff");

When running this in bochs it generates:

Code: Select all

05609069988e[CPU0  ] #GP: VMXON is not allowed !

Any ideas what could be wrong?

Thanks,
Karim.

Posted: **Fri Mar 18, 2016 1:29 am**

Related to the VMXON failure see 31.5 in Intel manual (VMM SETUP & TEAR DOWN), in short you need to set the VMXE bit in your CR4 register before executing VMXON, also you need to specify a physical address as the operand to VMXON not as a virtual one.

Relating to the EFER MSR, you can read 26.3.2.1. The section you cited 26.4 refers to the MSR-load area, that is a structure which specifies what MSRs should be loaded with what values on guest entry. However, for some MSRs (EFER, SYSENTER_CS, SYSTENTER_EIP, ...) there are explicit VMCS fields which specify what values those MSRs have on guest load (so you don't have to use MSR-load area).
Going back to 26.3.2.1 it states that if “load IA32_EFER” VM-entry control is 1, the IA32_EFER MSR is loaded from the (VMCS) IA32_EFER field, else, “load IA32_EFER” VM-entry control is 0 then the values of LMA and LME are taken from the “IA-32e mode guest” VM-entry control.

Posted: **Fri Mar 18, 2016 2:11 am**

Hi,

Thanks alegx,

While I was waiting for a reply, I decided to have a look within bochs source code to see the conditions that causes the error message and I found it exist once in the cpu.vmx.cc. Here is bochs code:

Code: Select all

 if (CPL != 0 || ! BX_CPU_THIS_PTR cr0.get_NE() ||
        ! (BX_CPU_THIS_PTR cr0.get_PE()) || BX_GET_ENABLE_A20() == 0 ||
        ! (BX_CPU_THIS_PTR msr.ia32_feature_ctrl & BX_IA32_FEATURE_CONTROL_LOCK_BIT) ||
        ! (BX_CPU_THIS_PTR msr.ia32_feature_ctrl & BX_IA32_FEATURE_CONTROL_VMX_ENABLE_BIT))
    {
      BX_ERROR(("#GP: VMXON is not allowed !"));
      exception(BX_GP_EXCEPTION, 0);
    }

A you can see a lot of conditions are the cause of the error, so I have added the following code prior to that code to print exactly which condition fires the GPF.

Code: Select all

if ( CPL != 0 )  BX_ERROR(("kemosparc: CPL != 0"));
        if ( ! BX_CPU_THIS_PTR cr0.get_NE() )  BX_ERROR(("kemosparc: ! BX_CPU_THIS_PTR cr0.get_NE()"));
        if (  ! (BX_CPU_THIS_PTR cr0.get_PE()))  BX_ERROR(("kemosparc:  ! (BX_CPU_THIS_PTR cr0.get_PE())"));
        if ( BX_GET_ENABLE_A20() == 0 )  BX_ERROR(("kemosparc: BX_GET_ENABLE_A20() == 0 "));
        if (  ! (BX_CPU_THIS_PTR msr.ia32_feature_ctrl & BX_IA32_FEATURE_CONTROL_LOCK_BIT) )  
                 BX_ERROR(("kemosparc:  ! (BX_CPU_THIS_PTR msr.ia32_feature_ctrl & BX_IA32_FEATURE_CONTROL_LOCK_BIT)"));
        if (  ! (BX_CPU_THIS_PTR msr.ia32_feature_ctrl & BX_IA32_FEATURE_CONTROL_VMX_ENABLE_BIT) )  
                 BX_ERROR(("kemosparc:  ! (BX_CPU_THIS_PTR msr.ia32_feature_ctrl & BX_IA32_FEATURE_CONTROL_VMX_ENABLE_BIT)"));

It turned out to be

Code: Select all

 ! (BX_CPU_THIS_PTR msr.ia32_feature_ctrl & BX_IA32_FEATURE_CONTROL_LOCK_BIT)

Replying to your answer:
1. CR4: VMXE is bit 13 and I set this in the code I provided in my previous post
2. I set also bit 5 which is CR0:NE
3. I enabled A20

But I don't understand what I should set to pass the condition I mentioned above. What is msr.ia32_feature_ctrl ? I will check the intel documentation for that.

Thanks,
Karim.

Posted: **Fri Mar 18, 2016 2:28 am**

Okay great,

I got it working after setting the lock bit of the IA32_FEATURE_CONTROL MSR.

I will start now working on building the VMCS.

Will keep you posted guys.

Thanks a lot,
Karim.

Posted: **Fri Mar 18, 2016 10:45 am**

Hi,

I was able to make vmxon work on bochs. It works perfectly and continue after the vmxon and vmxoff and execute the rest of the code after them.

The problem is that when I run the same image on KVM it get stuck in vmxon.

The processor just get stuck in this instuction and never leave it forever.

I have started qemu/kvm with "-monitor stdio" and issued "info registers"

The RIP has the address of the vmxon instruction. I picked up that from objdump.

It never continues to execute post this point.

I understand that bochs is pure emulator and that Qemu/KVM is hypervisor, but what could be the cause.

The return of the rdmsr 0x480 in case of the KVM is
EDX:EAX = 0x181000:0x11e57ed0

Which is not like that when run from bochs, and the revision identifier (eax) in case of bochs is 0x2d?

Any ideas what it could be ?

Thanks,
Karim.

Posted: **Fri Mar 18, 2016 11:15 am**

Hi,

Another difference between bochs and kvm is that bits 29 and 30 in CR0 are set in boch while not in KVM.

The CR0 in bochs is 0xe0000031 and in kvm 0x80000031

I thought this might be of any indication ?

Thanks,
Karim.

Posted: **Sat Mar 26, 2016 8:20 am**

I was able to make vmxon work on bochs. It works perfectly and continue after the vmxon and vmxoff and execute the rest of the code after them.

Great

Congrats, that is already a great step you have managed to achieve!

I got it working after setting the lock bit of the IA32_FEATURE_CONTROL MSR.

Normally, the BIOS locks the associated MSR. In this way, you cannot "disable" VMX support at runtime. The BIOS used by Bochs (apparently) does not lock it, which however is required for the VMXON instruction to work. Bu you have already figured that our by yourself

The problem is that when I run the same image on KVM it get stuck in vmxon.

At this point you need to distinguish between an an emulator and a hypervisor. An emulator (in your case Bochs) attempts to provide the same CPU/system behavior as a real CPU/system. Therefore, every instruction (including the VMXON) is emulated (you can say implemented) in software. In this way, you can use most of the emulated systems resources including the VMX functionality. This is not the case with hypervisors (virtual machine monitors or VMMs), such as KVM. For performance reasons, KVM tries to execute most of the guest instructions on bare metal. Therefore, it utilizes your system's VMX capabilities and hence blocks further VMX initialization attempts. This is exactly the behavior you are experiencing right know

One solution to this problem is to use "nested virtualization". I know that KVM supports nested virtualization but never tested it (There is also a quite interesting comment inside the KVM's code stating that you should use nested virtualization on your own risk

). Another solution is to continue using Bochs or even switch to bare metal with VMX support.

Posted: **Sat Apr 02, 2016 5:27 am**

Hi,

Thanks for the reply.

I processed with bochs.

But now I have a problem launching my VM.

All what I need is that I want to launch a VM in 64-bit that execute a VMCALL and return to the VM exit.

Here is my code:

Code: Select all

/**************************************************************************
                               VMX test Code
**************************************************************************/
static void vm_exit ()
{
    
    kernel.getVideo()->setPosition(10,10);
    kernel.getVideo()->putString("This is VMEXIT",COLOR_WHITE,COLOR_RED);
    for (;;);

}

static void vm_entry ()
{
    
    asm volatile("vmcall");

}
extern "C" void bsp_kernel_main(uint64_t p_start_stack,uint64_t p_end_stack)
{    
    kernel.setupVideo();
    kernel.initializePhysicalMemory();
    kernel.setupGlobalDescriptorTable();
    if ( !kernel.setupACPI ())
    {
        kernel.getVideo()-> putString("Halting: ACPI Setup Failed \n",COLOR_RED,COLOR_WHITE);
        return;
    }
    kernel.startupMemoryManager(p_start_stack,p_end_stack);
    kmalloc_ptr = kernel.getKMallocPTR();
    kernel.scanPCIBus();
    uint32_t cpuid_eax = 0;
    uint32_t cpuid_ebx = 0;
    uint32_t cpuid_ecx = 0;
    uint32_t cpuid_edx = 0;
    


    
    asm volatile
      ("cpuid" : "=a" (cpuid_eax), "=b" (cpuid_ebx), "=c" (cpuid_ecx), "=d" (cpuid_edx)
       : "a" (1), "c" (0));    
    if ( isBitSet(cpuid_ecx,0x5)) // Check if0x480 VMX is supported
    {
        kernel.getVideo()->putString("\nCPUID ECX: ",COLOR_BLUE,COLOR_WHITE);
        kernel.getVideo()->putHexa(cpuid_ecx,COLOR_BLUE,COLOR_WHITE);
        
        // Read MSR and store revision number
        uint32_t msr_eax_value,msr_edx_value;
        asm volatile ( "rdmsr" : "=a" (msr_eax_value),"=d" (msr_edx_value) : "c" (0x480) );
       
        
        // Disable A20, set bit 0 of MSR 0x3a, set bit 13 of CR4 and set bit 5 of CR0
        asm volatile("         \
          in      $0x92, %al; \
          or      $0x02, %al; \
          out     %al, $0x92; \
          mov $0x3a,%ecx;    \
          rdmsr; \
          bts $0, %rax;        \
          wrmsr; \
          mov %cr4,%rax;       \
          bts $13, %rax;        \
          mov %rax,%cr4;    \
          mov %cr0,%rax;       \
          bts $5, %rax;        \
          mov %rax,%cr0;");

        uint64_t cr0,cr4;
        
        asm volatile("mov %%cr0, %0": "=r"(cr0)); // Get CR0 register value
        asm volatile("mov %%cr4, %0": "=r"(cr4)); // Get CR4 register value
        
        // Allocate host and guest VMCS and set VMX revision number
        uint64_t vmx_phy_region = 0;
        uint64_t vmx_region = kmalloc_ptr->kmalloc_ap(0x1000,&vmx_phy_region);
        uint32_t * vmx_region_ptr = (uint32_t *) vmx_region;
        vmx_region_ptr[0] = msr_eax_value;

        uint64_t vmcs_phy_region = 0;
        uint64_t vmcs_region = kmalloc_ptr->kmalloc_ap(0x1000,&vmcs_phy_region);
        uint32_t * vmcs_region_ptr = (uint32_t *) vmcs_region;
        vmcs_region_ptr[0] = msr_eax_value;
        
        
        // VMXON and loading the guest VMCS region
        asm volatile ("vmxon %0"::"m"(vmx_region));
        asm volatile ("vmclear %0"::"m"(vmcs_region));
        asm volatile ("vmptrld %0"::"m"(vmcs_region));
        
        
        //******************************************************************
        //              Host VMCS Setup
        //******************************************************************
        uint64_t val = 0x13ff;
        uint64_t encoding = 0x4012;
        asm volatile ("vmwrite %0, %1"::"r" (val), "r" (encoding):"flags");
        
        val = 0x1f; encoding = 0x4000;
        asm volatile ("vmwrite %0, %1"::"r" (val), "r" (encoding):"flags");
        
        val = 0x8401e9f2; encoding = 0x4002;
        asm volatile ("vmwrite %0, %1"::"r" (val), "r" (encoding):"flags");

        val = 0x2; encoding = 0x401E;
        asm volatile ("vmwrite %0, %1"::"r" (val), "r" (encoding):"flags");
        
        val = 0x36fff; encoding = 0x400C;
        asm volatile ("vmwrite %0, %1"::"r" (val), "r" (encoding):"flags");

        encoding = 0x6C00;
        asm volatile ("mov %0,%%rbx;\
                       mov %%cr0,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");

        encoding = 0x6C02;
        asm volatile ("mov %0,%%rbx;\
                       mov %%cr3,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");


        encoding = 0x6C04;
        asm volatile ("mov %0,%%rbx;\
                       mov %%cr4,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");

        encoding = 0xC02;
        asm volatile ("mov %0,%%rbx;\
                       mov %%cs,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");

        encoding = 0x6C16;
        asm volatile ("mov %0,%%rax;\
                       vmwrite %%rax,%1;" :: "r" (vm_exit), "r" (encoding):"flags");

        encoding = 0xC04;
        asm volatile ("mov %0,%%rbx;\
                       mov %%ss,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");
                
        encoding = 0x6C14;
        asm volatile ("mov %0,%%rbx;\
                       mov %%rsp,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");

        encoding = 0xC06;
        asm volatile ("mov %0,%%rbx;\
                       xor %%rax,%%rax;\
                       mov %%ds,%%ax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");



        encoding = 0xC00;
        asm volatile ("mov %0,%%rbx;\
                       xor %%rax,%%rax;\
                       mov %%es,%%ax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");

        encoding = 0xC08;
        asm volatile ("mov %0,%%rbx;\
                       xor %%rax,%%rax;\
                       mov %%fs,%%ax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");
        
        encoding = 0xC0A;
        asm volatile ("mov %0,%%rbx;\
                       xor %%rax,%%rax;\
                       mov %%gs,%%ax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");
        

        uint64_t tss_address = (uint64_t) kernel.getGlobalDescriptorTable()->getTSSEntry();
        encoding = 0xC0C;
        asm volatile ("vmwrite %0, %1"::"r" (tss_address), "r" (encoding):"flags");
        

        encoding = 0x6C0C;
        asm volatile ("vmwrite %0, %1"::"r" (m_globalDescriptorTablePointer.getBase()), "r" (encoding):"flags");

        encoding = 0x6C0E;
        asm volatile ("vmwrite %0, %1"::"r" (m_interruptDescriptorTablePointer.getBase()), "r" (encoding):"flags");
        
        //******************************************************************
        //              Guest VMCS Setup
        //******************************************************************
        
        encoding = 0x6800;
        asm volatile ("mov %0,%%rbx;\
                       mov %%cr0,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");
        
        

        encoding = 0x6802;
        asm volatile ("mov %0,%%rbx;\
                       mov %%cr3,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");
        

        encoding = 0x6804;
        asm volatile ("mov %0,%%rbx;\
                       mov %%cr4,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");

        encoding = 0x6820;
        asm volatile ("mov %0,%%rbx;\
                       mov $0x2,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");
        

        encoding = 0x802;
        val = 0x8;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");


        encoding = 0x4802;
        asm volatile ("mov %0,%%rbx;\
                       mov $0xfffff,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");
                

        encoding = 0x4816;
        val = AR_PRESENT | AR_CODE | DPL_KERNEL | AR_READABLE;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");

        encoding = 0x6808;
        asm volatile ("mov %0,%%rbx;\
                       mov $0x0,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");

        encoding = 0x681E;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (vm_entry):"flags");
        
        
        encoding = 0x6816;
        asm volatile ("vmwrite %0, %1"::"r" (m_globalDescriptorTablePointer.getBase()), "r" (encoding):"flags");
       
        encoding = 0x4810;
        asm volatile ("vmwrite %0, %1"::"r" ((uint64_t)m_globalDescriptorTablePointer.getLimit()), "r" (encoding):"flags");

        // Here BOCHS hangs forever
        encoding = 0x6818;
        asm volatile ("vmwrite %0, %1"::"r" (m_interruptDescriptorTablePointer.getBase()), "r" (encoding):"flags");
        encoding = 0x4812;
        asm volatile ("vmwrite %0, %1"::"r" ((uint64_t)m_interruptDescriptorTablePointer.getLimit()), "r" (encoding):"flags");

        
        // Setting ES, DS, SS, FS, GS selectors
        val = 0x10;
        encoding = 0x800;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x804;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x806;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x808;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x80A;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        

        // Setting ES, DS, SS, FS, GS limits
        val = 0xfffff;
        encoding = 0x4800;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x4804;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x4806;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x4808;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x480A;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");


        // Setting ES, DS, SS, FS, GS Access

        val = AR_PRESENT | AR_DATA | AR_WRITABLE | DPL_KERNEL;
        encoding = 0x4814;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x4818;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x481A;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x481C;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x481E;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");

        // Setting ES, DS, SS, FS, GS bases

        val = 0x0;
        encoding = 0x6806;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x680A;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x680C;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x680E;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");
        
        encoding = 0x6810;
        asm volatile ("mov %0,%%rbx;\
                       mov %1,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding),"r" (val):"flags");

        
        // Setting DR7
        encoding = 0x681A;
        asm volatile ("mov %0,%%rbx;\
                       mov $0x400,%%rax;\
                       vmwrite %%rax,%%rbx;" :: "r" (encoding):"flags");
        
        asm volatile ("VMLAUNCH");

        
        asm volatile ("vmxoff");
        
    }
    else
    {
        kernel.getVideo()->putString("\nVMX is not supported: ",COLOR_BLUE,COLOR_WHITE);
    }   
}

This VMX code is inserted in my kernel main c function after switch to long mode and mapping the whole memory.

Interrupts are not enabled and I start the VM with only one core (BSP).

I run this at bochs and it get stuck running for ever when I try to set the IDT base of the guest as per my code comments above.

Can you please have a look at the code and let me know what is wrong and if there are other problems that will stop the VM from launch.

Thanks,
Karim.

Posted: **Sun Apr 03, 2016 5:45 am**

Hi,

I enabled debugging on bochs, and the instruction of setting the IDT base address using vmwrite causes this message to occur:

Code: Select all

05639221626d[CPU0  ] BxError: Encountered an unknown instruction (signalling #UD)
05639221626d[CPU0  ] exception(0x06): error_code=0000
05639221626d[CPU0  ] interrupt(): vector = 06, TYPE = 3, EXT = 1
05639221626d[CPU0  ] interrupt(long mode): INTERRUPT TO SAME PRIVILEGE
05639221711d[CPU0  ] LONG MODE IRET
05639221712d[CPU0  ] BxError: Encountered an unknown instruction (signalling #UD)
05639221712d[CPU0  ] exception(0x06): error_code=0000
05639221712d[CPU0  ] interrupt(): vector = 06, TYPE = 3, EXT = 1
05639221712d[CPU0  ] interrupt(long mode): INTERRUPT TO SAME PRIVILEGE
05639221797d[CPU0  ] LONG MODE IRET
.
.
.
.
.

And bochs kept on printing it forever.

I then commented the instruction that causes the problem so the code continued to execute and the VMLAUNCH was fired but the same message appeared again at the VMCALL inside my vmentry routine.

I was able to identify the instructions that caused the problem using objdump and printing the stack from bochs

I tried to look up this message on google but with no luck, or meaningful search hits.

Anyone knows what might be the problem ?

Thanks,
Karim.

OSDev.org

A Beginner VMX Question

Re: A Beginner VMX Question

Re: A Beginner VMX Question

Re: A Beginner VMX Question

Re: A Beginner VMX Question

Re: A Beginner VMX Question

Re: A Beginner VMX Question

Re: A Beginner VMX Question (VMXON throws GP fault in bochs)

Re: A Beginner VMX Question

Re: A Beginner VMX Question

Re: A Beginner VMX Question

Re: A Beginner VMX Question (Problem when run on KVM)

Re: A Beginner VMX Question

Re: A Beginner VMX Question

Re: A Beginner VMX Question

Re: A Beginner VMX Question