Issues with VBE in VMWare

[max]

Hey guys,

I've got my VBE driver working well in Bochs, QEMU and VirtualBox, but I still can't figure out whats wrong with it in VMWare. It already fails at loading the VBE info block, giving me an invalid instruction code. Kernel log:

Code: Select all

        [log] <1036> starting window manager
        [log] <1036> setting video mode to 800x600x24
        [log] <1036> VBE setting mode
        [log] <1036> VBE sending request
  [scheduler] waiting for all threads of process 1000 to exit: all finished
        [log] <1036> VBE sent request
        [log] <1036> VBE receiving with topic
        [log] <1021> got lower memory block 32268
  [exception] invalid operation code in task 1060 (process 1060)
                 eip: 0x00009130   eflags: 0x00030246
                 eax: 0x00008900      ebx: 0x00000000
                 ecx: 0x00000000      edx: 0x000076BE
                 esp: 0x00000FF6   state@: 0xFD005FA8
                intr: 0x00000006    error: 0x00000000
  [exception] process 1060 killed due to invalid operation code 0x000000FF
        [log] <1021> could not load VBE info block
        [log] <1021> vbe: unable to switch to video mode 800x600x24
        [log] <1036> VBE received with topic
        [log] <1036> VBE received false
        [log] <1036> VBE driver reported that mode-set was not successful, quitting window manager

My function for loading this isn't very complex. the given VbeInfoBlock* points to a struct in lower memory that has the appropriate size. It looks like this:

Code: Select all

// FP-Macros
#define FP_SEG(fp)        			(((FarPointer) fp) >> 16)
#define FP_OFF(fp)        			(((FarPointer) fp) & 0xffff)
#define LINEAR_TO_FP(linear)		((linear > 0x100000) ? 0 : ((((linear >> 4) & 0xFFFF) << 16) + (linear & 0xFL)))

...

bool loadVbeInfo(VbeInfoBlock* target) {

	ghost::Virtual8086Registers out;
	ghost::Virtual8086Registers in;

	FarPointer vbeInfoBlockFp = LINEAR_TO_FP((uint32_t ) target);

	in.ax = 0x4F00;
	in.bx = 0;
	in.cx = 0;
	in.dx = 0;
	in.es = FP_SEG(vbeInfoBlockFp);
	in.di = FP_OFF(vbeInfoBlockFp);
	in.ds = 0;
	in.si = 0;

	ghost::SystemCalls::callVm86(0x10, in, out);

	return (out.ax == 0x4F);
}

Did any of you have problems with VBE in VMWare? Is there anything special about VMWare that I have to consider?

Thank you in advance.
Greets, Max

[Brendan]

Hi,

It's unlikely that the VBE "Get controller information" function is broken on VMWare; as any software that uses VBE would need use this BIOS function. With that in mind, the first thing I'd do is test if the VBE function works on VMWare in plain real mode, to determine if it's an unlikely problem in VMWare (or a more likely problem in your virtual8086 mode code or something else you've done).


Cheers,

Brendan

[max]

Okay, I just wanted to assure first that the call parameters are whats expected & see if someone had similar issues.
I'll look into my virtual 8086 monitor

[alexfru]

I'd really make sure that the receiving structure is located below 1MB. If it's not, the macro will force the address to 0 and, guess what, there can be some important things in the system near address 0 (the real-mode interrupt vector table and the BIOS data area).

Also, if, for whatever reason, the int 10h handler enables interrupts, the v86 monitor should be able to handle the situation.

[max]

I'd really make sure that the receiving structure is located below 1MB. If it's not, the macro will force the address to 0 and, guess what, there can be some important things in the system near address 0 (the real-mode interrupt vector table and the BIOS data area).

It is, as it says in the log "got lower memory block 32268" which is at 0x7E0C (i should have used %x )

Also, if, for whatever reason, the int 10h handler enables interrupts, the v86 monitor should be able to handle the situation.

I could handle that interrupt... but i think theres actually a bug in my 8086 monitor, maybe regarding 16/32bit instruction switching...

[xenos]

I guess you already checked the memory location where the invalid opcode occurs and figured out which instruction is causing this problem?

[max]

I guess you already checked the memory location where the invalid opcode occurs and figured out which instruction is causing this problem?

Yes I did. But I found out the problem was a bug in my V8086 monitor. Calculated the wrong addresses from the IVT when handling interrupts within the monitor

Thanks though!
... and sorry for wasting your time