VESA bios functions, QEMU does it, VMware and Bochs crash

[max]

Heyho,

I've got a problem when trying to do Vm86 calls to enable a VESA graphics mode. The thing is, it works in QEMU, switches to the graphics mode and everything is fine - but when running the same in VMware or Bochs, there are some weird characters on the screen and then it resets. Bochs log tells me the following, maybe one of you knows what it means:

Code: Select all

00026305774e[CPU0 ] interrupt(): gate.type(3) != {5,6,7,14,15}

Full logfile: http://pastebin.com/Mqt3eC5L

Does one of you have an idea what could be wrong?

Thanks in advance

[alexfru]

Code: Select all

00026305774e[CPU0 ] interrupt(): gate.type(3) != {5,6,7,14,15}

Bad IDTR/IDT?
See http://wiki.osdev.org/Interrupt_Descrip ... #Structure, etc.

Looks like you don't have a proper exception handler in your kernel (there's none or something's broken).

[max]

Bad IDTR/IDT?
See http://wiki.osdev.org/Interrupt_Descrip ... #Structure, etc.

Looks like you don't have a proper exception handler in your kernel (there's none or something's broken).

Hmmm I don't think so... you can see the values I'm inserting to my IDT here (the bases always leed to asm stubs): http://pastebin.com/t7xt2qD8

And my entry struct and filling looks like this:

Code: Select all

struct IdtEntry {
	uint16_t baseLow;
	uint16_t kernelSegment;
	uint8_t zero;
	uint8_t flags;
	uint16_t baseHigh;
}__attribute__((packed));
...

IdtEntry idt[256];
...

// For each entry you see in the log file:
idt[index].baseLow = base & 0xFFFF;
idt[index].baseHigh = (base >> 16) & 0xFFFF;
idt[index].kernelSegment = kernelSegment;
idt[index].zero = 0;
idt[index].flags = flags;

Afaik this should be proper... any more suggestions?

[Brendan]

Hi,

Code: Select all

00026305774e[CPU0 ] interrupt(): gate.type(3) != {5,6,7,14,15}

Translated: The type field in the IDT entry was not "task gate", "16-bit interrupt gate", "16-bit trap gate", "32-bit interrupt gate" or "32-bit trap gate"; and is therefore invalid.

Looks like you don't have a proper exception handler in your kernel (there's none or something's broken).

Hmmm I don't think so... you can see the values I'm inserting to my IDT here (the bases always leed to asm stubs):

Either:

• the IDT entry was created wrong to start with
• the IDT entries were right but the "LIDT" instruction wasn't
• both the IDT entry and the "LIDT" were correct, but something trashed one or both afterwards

If you've thoroughly ruled out the first 2 possibilities, then that leaves the "was right but then got trashed" option.


Cheers,

Brendan

[max]

Thanks for the answer Brendan!

Either:

• the IDT entry was created wrong to start with
• the IDT entries were right but the "LIDT" instruction wasn't
• both the IDT entry and the "LIDT" were correct, but something trashed one or both afterwards

If you've thoroughly ruled out the first 2 possibilities, then that leaves the "was right but then got trashed" option.

Okay i did a little more exploring to isolate the bug, and I'm nearly sure that it's caused either by a messed up TSS or the VM86 setup does not do what it should. Now I get the infamous following error message which is also listed in the wiki, but I'm nearly sure that my GDT is correct. And it occurs when the scheduler tries to switch to VM86 task.

Code: Select all

00759402983e[CPU0 ] interrupt(): gate descriptor is not valid sys seg (vector=0x0d)
00759402983e[CPU0 ] interrupt(): gate descriptor is not valid sys seg (vector=0x08)

Just for taking a little look over it - this is what I initially set the registers of my VM86 to:

Code: Select all

VM86-Process 1002 created
  user stack foot:  0x00008018 // the stack i allocated in lower memory
  kernel stack foot:  0x0037F144 // the stack that is set to the TSS when switching to this task
  interrupt code entry (FP):  0xC0004C2B
 registers set to:
   ss: 0x00000900
   sp: 0x00000FF0
   eflags: 0x00020202
   cs: 0x0000C000
   ip: 0x00004C2B
   ax: 0x00004F00
   bx: 0x00000368
   cx: 0x00000010
   dx: 0x00000020
   si: 0x00000000
   di: 0x0000000C
   gs: 0x00000000
   fs: 0x000007E0
   es: 0x00000000
   ds: 0x00000037
Switching to task 1000 - ESP: 0x0036F038, vm86: false // some other task
Switching to task 1002 - ESP: 0x0038F0FC, vm86: true // trying to switch to vm86 -> crash

The weird thing is, that it works in QEMU... oO maybe its just much too non-restrictive...

I'm quite sure that you don't feel like reviewing my code, but if you want to take a small look at my GDT code it's here: http://pastebin.com/XxH7NT0q (feel free to copy the flags-macros if you like them ) (EDIT: wrong link, updated)

[alexfru]

A couple of things about TSS and task switching...

Do you use one TSS for more than one task? Are those real tasks from the CPU's point of view or do you try to reuse and update your single TSS?

If you reuse a TSS, remember about the "busy" bit. You can't switch to a task that has this bit already set to 1.

Do you know/remember that switching to a task simply makes the task continue whatever it was doing when it was last switched from and that the task must essentially be a loop as it has no way to be called at a predefined entry point or returned from unlike a regular subroutine or ISR?

[Brendan]

Hi,

Either:

• the IDT entry was created wrong to start with
• the IDT entries were right but the "LIDT" instruction wasn't
• both the IDT entry and the "LIDT" were correct, but something trashed one or both afterwards

If you've thoroughly ruled out the first 2 possibilities, then that leaves the "was right but then got trashed" option.

Okay i did a little more exploring to isolate the bug, and I'm nearly sure that it's caused either by a messed up TSS or the VM86 setup does not do what it should. Now I get the infamous following error message which is also listed in the wiki, but I'm nearly sure that my GDT is correct. And it occurs when the scheduler tries to switch to VM86 task.

Code: Select all

00759402983e[CPU0 ] interrupt(): gate descriptor is not valid sys seg (vector=0x0d)
00759402983e[CPU0 ] interrupt(): gate descriptor is not valid sys seg (vector=0x08)

That says the IDT entry for the general protection fault exception handler is broken, and the IDT entry for the double fault exception handler is broken. It doesn't say there's anything wrong with the GDT or any of the GDT entries.

Just for taking a little look over it - this is what I initially set the registers of my VM86 to:

This looked quite messed up to me. Any task may switch between protected mode an virtual8086 mode, and you do not need a specific task for virtual8086 mode.

Mostly, to switch to virtual8086 mode you set the "VM" flag in EFLAGS. This is done with an "IRET" that includes a CPL=0 to CPL=3 privilege level switch and causes the "real mode compatible" segment registers to be loaded. The reverse (switching from virtual8086 mode to protected mode) is typically done by interrupts; which clear the "VM" flag in EFLAGS while doing a privilege level switch to CPL=0.

Also note that schedulers typically only ever switch from one task running at CPL=0 to another task running at CPL=0; and therefore the scheduler never has anything to do with virtual8086 at all. To understand this, consider this state diagram:

• {CPL=3} ---IRQ-or-syscall--> {CPL=0 -> scheduler -> CPL=0} ---IRET-or-sysret--> {CPL=3}

Cheers,

Brendan

[max]

Do you use one TSS for more than one task? Are those real tasks from the CPU's point of view or do you try to reuse and update your single TSS?

I use the TSS for only 1 task at a time, i describe that a little later here in my answer

If you reuse a TSS, remember about the "busy" bit. You can't switch to a task that has this bit already set to 1.

But, this busy bit is only relevant if there are multiple tasks using the same TSS, right?

Do you know/remember that switching to a task simply makes the task continue whatever it was doing when it was last switched from and that the task must essentially be a loop as it has no way to be called at a predefined entry point or returned from unlike a regular subroutine or ISR?

Yes..

This looked quite messed up to me. Any task may switch between protected mode an virtual8086 mode, and you do not need a specific task for virtual8086 mode.

The thing is, in my kernel all processes run in ring 0. Therefore my scheduler only switches between ring 0 and vm86 tasks. I know that I don't specifically need a process for vm86 mode, but I thought it would be the cleanest solution..

This is done with an "IRET" that includes a CPL=0 to CPL=3 privilege level switch and causes the "real mode compatible" segment registers to be loaded.

That's what my scheduler does when switching to a Vm86 task. It first sets that tasks kernel stack into the TSS as ESP0 and then returns this address to my asm stub that pops off the registers and does an iret.
The IRET then does a switch to the virtual 8086 mode and the BIOS function is executing its code. Then, when this task causes a GPF, my GPF handler checks if the last task was a v86-task, if so it calls a routine to handle the GPF. This routine handler handles the faulty opcode, and everytime an INT is done I count up a variable. Everytime an IRET is called, i count this variable down, and if the variable is 0 when IRET occurs, I know that the BIOS function is done and I can quit the process.

As I currently only have 1 task using the TSS at all, this shouldn't be the problem..
Another thing - the interrupt descriptors that I have in my IDT apply to ring 0, but do they also work for ring 3 tasks? The vm86 task is a ring 3 task, and if the interrupt handlers (should) only work for ring 0 (and QEMU still lets them work for some reason...) that could be bad^^

thanks a lot so far ^-^

EDIT: if i set the idt descriptors to DPL 3, bochs gives me:

Code: Select all

stackPrefetch(4): access [0xfffffffc] > SS.limit [0x0fffffff]
interrupt(): gate descriptor is not valid sys seg (vector=0x08)

[max]

Finally got the solution. The problem was, that my GDT entries were not properly set up - I simply forgot to set the higher bits of the limit. QEMU ignores this, but Bochs/VMware crash.