OSDev.org

Hi,

I encounter an issue with "VMLaunch" that I am trying to track down. If I keep a breakpoint at the first instruction in guest before calling "VMLaunch" (opcode 0f 01 c2), I hit that breakpoint and then successfully execute guest code. If I don't keep that breakpoint, guest hangs the moment I run "VMLaunch". Obviously, the int 3 trap handler in Windows is doing something that helps. Can anyone think of a reason why it might help? Thanks.

Kamala

Did you try to run it with Bochs ?

Stanislav

Windows is doing something

Are you trying to make your own VM application for Windows?

No, I haven't tried on Bochs.

I am writing a light weight OS agnostic layer which is like a VMM but not quite. It it virtualization enabled.

I seem to vmexit with a double fault - either a bad eip or esp after some guest instructions are executed. Something that doesn't happen when I attach to a debugger and the first guest instruction executed after a vmlaunch is an int 3.

Kamala

Still kind of short on details here. What kind of code are you running on this VM? What mode is the processor operating in behind the VM? Are you using VMX features specifically to trap int3 while other interrupts are redirected back to the VM itself?

Since it appears something is faulting, I'd suggest switching ints/faults to trigger back to the VMM and logging where the first error occurs. From there it should be simple enough to figure out what's wrong with the VMX State, or the code you're running.

> Still kind of short on details here.
Apologies.

> What kind of code are you running on this VM?
It's preinitialized Windows OS that was frozen at a certain point and resurrected in guest mode. So, inital instruction to run as guest would be Windows kernel code.

> What mode is the processor operating in behind the VM?
Not real mode. Ring 0, operating system agnostic code running in VMX root mode controls the guest. You could call it a VMM for the sake of this discussion.

> Are you using VMX features specifically to trap int3 while other interrupts are redirected back to the VM itself?
Not really. I do take control of certain software interrupts but int 3 is not in that list.

> Since it appears something is faulting, I'd suggest switching ints/faults to trigger back to the VMM and logging where the first error occurs.
I did and I always end up in a guest double fault with either invalid eip or esp. This doesn't happen when I am attached to a debugger and the first instruction to run in the guest is int 3.

> From there it should be simple enough to figure out what's wrong with the VMX State, or the code you're running.
Do you have any insight into what it is that I might have configured incorrectly at VMCS level for the guest to fail with a double fault if the initial instruction is not an int 3 that traps into a debugger?

Thanks for your help.

Kamala

It could be a multitude of things, once again it's very hard to say without looking at the initial exception that is occuring. The exception bitmap register should allow you trap the first exception that occurs and examine it (You could simply intercept the exception, log it and then reinject it back into the VM until you double fault).

A short list of possibilities:

• Anything dealing with segmentation (GDT/LDT)
  The TSS and TR register
  Guest paging state, which can be complicated if the processor doesn't support EPT.

> It could be a multitude of things, once again it's very hard to say without looking at the initial exception that is occuring.
I did look at the initial exception. More on that below.

> The exception bitmap register should allow you trap the first exception that occurs and examine it (You could simply intercept the exception, log it and then reinject > it back into the VM until you double fault).
I do set the exception bitmap to capture exceptions of interest and the first one to hit in that case is a double fault. Examining the double fault shows incorrect eip or stack when the first instruction to execute in guest space is not a breakpoint.

> A short list of possibilities:
> Anything dealing with segmentation (GDT/LDT)
> The TSS and TR register
> Guest paging state, which can be complicated if the processor doesn't support EPT.
Except I don't understand why those won't matter when the first instruction to execute in guest space is a breakpoint instruction.

Thanks.

Kamala

One thought just occured while perusing through something relevant -

Triggering that initial breakpoint does create a trap frame and may be that makes all the difference given where I fail otherwise - Double fault happens around the area where sysexit happens or int 2* is called which almost implies the trap frame created during that time get corrupted when we fail but keeping a breakpoint fixes that issue.

Does that make sense?

Kamala

It could indicate a stack problem, I don't really have enough experience with kernel mode debugging on windows to comment though.

Yes, it appears to be so.

I tried to focus on the instruction executed within the guest before the double fault. One point where we failed was when int 2b call is executed from guest userspace. It resulted in a double fault. So I am assuming the fault happens when the processor tries to switch to kernel stack, the value of which it gets from the task state structure. When I look at the task state structure, the kernel stack value looks valid. Does this give any clue as to what else might be going wrong around this time? Thanks.

Kamala

Is the ept mapped correctly?
Is the guest loaded correctly ?
Could you dump the vmcs fields?

Hi,

> Is the ept mapped correctly?
This happens even if EPT is disabled.

> Is the guest loaded correctly ?
Yes.

> Could you dump the vmcs fields?
I can reproduce the problem with minimal/default vmcs control fields set as well.

If I am clear on the exact set of steps taken by the processor when int 2b is executed, that might help solve the problem. My understanding is -

#1 - Processor switches to kernel stack (as int 2b is called from user mode code)
#2 - Push eflags, cs, eip etc.
#3 - Push error code but not for this interrupt

I suspect we choke in step #1. For the processor to switch stack, it takes the kernel mode stack address from task state structure. When I look at the address in the tss, it looks right. So I am not sure what I could be missing. Thanks.

Kamala

I would suggest you again to use Bochs and see he internals of your operation in details.
VMX is fully supported in Bochs 2.5.1 release and you should be able to reproduce your problem exactly.

Stanislav

> I would suggest you again to use Bochs and see he internals of your operation in details.
> VMX is fully supported in Bochs 2.5.1 release and you should be able to reproduce your problem exactly.

I would very much like to try that but the last time I tried with Bochs (which was sometime back) I didn't have much success with Windows though it was very useful with other OSes. Do you have a quick set of steps I could use to setup Bochs for Windows? I have a virtual disk I created with VirtualBox. Will I be able to use that with Bochs as well? Please let me know. Thanks.

Kamala