VGA heisenbug

[TheQuux]

I'm just getting started with osdeving, but, about a week into my OS, I tried to set up hardware scrolling on the text console. I have a status bar at the bottom of the display (just a standard 80x25 text console, but with a custom font) that I set up to be below the "split".

However, when I started it up in qemu, the status bar (or the area dedicated to it) showed up black


So, I thought I'd try it in vmware, which has supposedly perfect emulation. The result? The same exact thing:


So, there must be some probelm with my code... I'll just runit in bochs and debug it there. The result?


So... Is this unusual at all, or have I made a mistake?

(My full source is avaliable at http://thequux.com/xenon-0.01.tar.bz2 )

[Mike]

I can't think of anything at the moment... but I'll take a look at your source. I guess I should re-download the latest version...

I never had this problem with my OS, and I have tested it in VmWare, Bochs, and on a real pc. So my guess is it's a subtle bug.

What character do you write for the blank space in the statusbar?

[TheQuux]

I was using an ASCII space (0x20), but I've also tested it with a hash mark and a '-'; none of these worked.

[Mike]

I never knew about the split technique. But it makes perfect sense... I think I'll use it on my os once I get the kernel together again.

What happens if you write one more line beyond the 'bottom line', as if the split wasn't there? Does that output show up in place of the status line? It's possible qemu and vmware don't implement this. But I highly doubt that! But it's about all I can think of at the moment.

[TheQuux]

According to FreeVGA, the bottom part of the split appears first in memory (in fact, it must start on the first byte of the VGA plane), and the top part, which is able to be scrolled in hardware, appears after it. So, I suppose that if you tried to do full ywrap-based scrolling (which the VGA scrolling code can do, according to all documentation that I've seen), the bottom pane will turn up in there somewhere...

Granted, my easily-accessable documentation is somehat old (one of the books is titled "Assembly Language for the Intel 8086", and the other book covers the preliminary VGA spec... I need new manuals :- ), so I could be wrong there.

The biggest problem, though is that the bottom panel can't scroll... it's fixed to the forst byte of VGA memory.

Also, one bit of eye-candy that you could do with it is to, when switching terminals, copy the new screen so that it appears immediately afterthe end of video memory, set the line counter register to 0, shift the start address to the new position, and then increase the line counter at some rate... the way that the splitting works will give an "uncovering" kind of effect...

actually, nah. Too much like PowderPoint

But it still would be something that you don't see very often...

[Brendan]

Hi,

According to FreeVGA, the bottom part of the split appears first in memory (in fact, it must start on the first byte of the VGA plane), and the top part, which is able to be scrolled in hardware, appears after it. So, I suppose that if you tried to do full ywrap-based scrolling (which the VGA scrolling code can do, according to all documentation that I've seen), the bottom pane will turn up in there somewhere...

Just wondering - are Qemu and VMware 100% VGA compatible at the hardware level, or are they "almost VGA compatible", or not VGA compatible at the hardware level at all (e.g. supporting BIOS functions for VGA compatability only)? If they aren't 100% VGA compatible at the hardware level, then the hardware scrolling and split screen stuff probably isn't supported.

To be honest, I wouldn't be surprised if most SVGA video cards aren't 100% VGA compatible at the hardware level either.

Have you considered emulating it in software using 640 * 480 * 16 colour graphics mode? That way it'd work on a much wider variety of hardware, and you could scroll anything you like....


Cheers,

Brendan

[TheQuux]

I'm not sure whether VMWare is VGA compatible.. but qemu should be at least as compatible as bochs is, as they use the same implementation, AFAICT.

Next time I reboot, I'll give it a test... I don't do anything with hard disks yet, so I'm not too worried.


On the other hand, if somebody would be willing to test it on actual hardware, that would be ideal; the kernel is multiboot compliant, and the two modules that it loads are optional fonts.

The quick test is to boot the kernel, and, when you get the ascii/CP437 chart, just hit enter until it scrollf off the screen. The thing to test for is that the bright blue status bar remains visible. Ignore what happens to the scrollbar.

*edit... I should really look at who I'm responding to... *

[Mike]

If I had an external floppy drive for my laptop, I'd test it tonight. I guess GRUB should be able to load off a usb flash drive, though, so all I'd need to do is figure out how.

[TheQuux]

I can put grub on a flash drive for you tomorrow, if you want... I just don't have a flash drive here. Ever since the Linux "Low Performance USB Block Driver" ate mine...

[Mike]

Here's what happens in VMWare when you scroll to the very end of the buffer. (The next enter key returns to the top).

Each enter keypress takes about one second to register. It was way faster on bochs, surprisingly. So something seems to be wrong with VMWare's VGA driver.

[TheQuux]

According to the qemu developers,

> The VMware driver idea is just an extension of it to add some
> acceleration primitives, and of course it is much more complicated.

The Vmware driver more simples of Bocsh VBE really.
The Bochs VBE have banking video memory and used standart VGA registers
for control.
The Vmware chip have framebufer only, don`t control by VGA-registers
and its "acceleration primitives" may be switch off by capatibiles mask
(to 0).
But Vmware chip is PCI only.

> Adding real hardware support (such as a Cirrus Logic VGA card) will also
> be more complicated than the Bochs VESA VBE layer.

http://lists.gnu.org/archive/html/qemu- ... 00033.html

So... that would explain why vmware is slower... bochs is running code natively that vmware is emulating.

My feeling is that, it's hard to say which is the most compliant.

On one hand, because of the further complexity in Qemu's model, there could be extra bugs. However, Qemu is supposedly emulating a real chip, and it does so well enough to pass off as that chip according to both linux and windows.

Bochs doesn't need to worry about emulating a specific chip, so theirs may have fewer opportunities for bugs; but then again, there'sno way to check.

As for VMware, considereig the number of people who pay them huge amounts of cash for virtualization products, you woul dhope that they'd get it rght.

But then again, considering the amount of mony people give a certain other company, for their OS, you would expect that they, too would put out a decent showing, yet experience indicates otherwise...

[Brendan]

Hi,

I did a some research....

VMware doesn't actually emulate a true video card at all. It does have support for the standard video BIOS functions (including VESA/VBE functions), but for OS's like Linux and Windows you need to obtain a "special" video driver for it. It isn't compatible with CGA, VGA or any flavour of SVGA at the hardware level.

Qemu doesn't emulate a VGA card either. Instead it emulates a Cirrus Logic GD5446 video card.

Bochs emulates either a VGA card, a Cirrus Logic GD5430 video card (ISA) or a Cirrus Logic GD5446 video card (PCI), depending on how it's configured.

I guess the next question is whether the Cirrus Logic video cards (if perfectly emulated) are VGA compatible at the hardware level, or not.

First I tried searching for: "CL-GD5446" datasheet

After finding and downloading the datasheet, I found out that it does have a "100% VGA compatible BIOS" (which is meaningless). There's also one sentence (section 2.4 "Compatability") which states: The CL-GD5446 includes all registers and data paths required for VGA controllers, and is updward-compatible with the CL-GD542X family. I also scrolled down to see the registers, and found "CR18 - CRTC Line Compare" listed, and everything else I (vaguely) remember about VGA registers. IMHO this probably means that it does at least attempt to be 100% VGA compatible at the hardware level.

The next step is to check some source code. For Bochs, I found things like the line compare register in the source code (vga.c) fairly quickly. The source code for the Cirrus Logic video card calls the VGA video code for anything it doesn't understand. This means, at least for Bochs, the emulated video card is 100% VGA compatible at the hardware level (or as close a practically possible), regardless of whether it's emulating a "pure VGA" or Cirrus Logic video card. Checking this source code I found that the line compare (and split screen) is supported for all VGA video modes, but it isn't supported for any SVGA video modes.

Now the last thing is to check the source code for Qemu. I compared "vga.cc" (Bochs) to "vga.c" (Qemu) - it is completely different source code. The copyrights at the top are different (Qemu's "vga.c" is copyrighted by Fabrice Bellard while for Bochs it's copyrighted by MandrakeSoft S.A.). For bochs the entire file is 3481 lines long, while for Qemu it's only 1946 lines. My general impression is that Boch's video code is much more complicated, and handles things like double scan modes correctly, while Qemu's code is much simpler.

For Qemu, there's this function:

Code: Select all

/*
 * Text mode update
 * Missing:
 * - double scan
 * - double width
 * - underline
 * - flashing
 */
static void vga_draw_text(VGAState *s, int full_update)
{

I searched this function for anything that would indicate support for the line compare register, but found nothing. There's also another function:

Code: Select all

/*
 * graphic modes
 */
static void vga_draw_graphic(VGAState *s, int full_update)
{

This function does support the line compare register, even for SVGA modes.

The Cirrus Logic GD5446 video card supports 1600 * 1200 * 256 colour video mode. The line compare register is 10 bits wide, and the only way to disable "split screen" is by setting the line compare register to a value higher than the video mode's resolution. For a resolution like 1600 * 1200, Qemu doesn't disable the line compare, and (AFAIK) would fail to display the 1600 * 1200 * 256 colour video mode correctly because of this - it would display the first 1024 lines correctly, and then wrap back to the start of video memory when this isn't desired (you'd never be able to see the last 476 lines of display memory). Real hardware would have the same problem, so I'd assume that real hardware ignores the line compare for at least some SVGA video modes.


Summary

For VMWare you're screwed.

For Bochs, split screen should work in text modes and standard VGA modes, but not SVGA modes.

For Qemu, split screen should work for all graphics video modes, but not text modes.


Cheers,

Brendan

[TheQuux]

Thanks...

I'm considering sticking this split-screen code in a subversion branch and just using my older text-blitting code for now for now.

However, the fact that Bochs emulates the vga stuff in the emulator, rather than the vga bios, would explain why the using the bochs vga bios in qemu didn't work.

Ah, well.

If bochs can be considered a "reference" VGA implementation, I'll throw together a VGA test suite so that we can see just how compatible various chips are... would people be willing to run something like that, and record the results?

--q

[Combuster]

Ah, well.

If bochs can be considered a "reference" VGA implementation, I'll throw together a VGA test suite so that we can see just how compatible various chips are... would people be willing to run something like that, and record the results?

Good idea.
Actually, when i was thinking about gfx drivers i quickly browsed the web to see if i could come up with some sort of compatibility list, but so far no success, so i kindof had the same idea , you only beat me to the point of launching it.

However, i dont think considering Bochs being reference material is a good choice. There are a few things that bochs can only wish to emulate: CRTC registers. And especially arcane usage of those. (and i have noticed their effects vary greatly between cards) So i'd suggest to add tests for things that arent necessarily workable in bochs.
And since i kindof intend to do some CRTC tweaking myself, having a few tests for those would be helpful.

Still, if you got a test os i'll be happy to help compile a compatibility list.