[SOLVED] Windows VGA driver question

[theesemtheesem]

Hi,

Does anyone know how the Windows 7 Standard VGA driver works? What i'm especially interrested in is how does it change the
video mode. Does it return to real mode and invoke the BIOS int 10h (possibly problematic for 64 bit versions)? Does it use Vesa extensions? Obviously using virtual 8086 is out of the question afaik, especially in 64 bit versions (as far as I understand one can't use virtual 86 mode from long mode). From what research I've been able to conduct that driver is actually quite capable, supports a lot of resolutions including very high ones, works even on strange and obscure / old cards, supports true color, and is actually quite fast (especially compared to the vista / winxp generic vga driver, which only supported standard vga modes and with 640x480x4bpp as max). Did anyone ever look into this? I did a lot of googling and found nothing about it.

Cheers,
Andrew

[Gigasoft]

The BIOS ROM is executed using an emulator which resides in hal.dll.

[theesemtheesem]

Hi,

Thanks, that clears things up. So what it does exactly is copy the video bios somewhere and then execute it in an emulator, right? Any idea which features of "real" hardware should be emulated in order for it to work? Or does it just let the code run and only intercepts port IO and memory access? I'm asking because it would be fun and useful to create something similar. Sorry if it seems as if I'm pulling Your tongue, but somehow the prospect of disassembling hal.dll is a bit frightening, and in addition I don't want to see too many implementation details, I want to develop it on my own.

I guess I should start by disassembling some video bioses to see what makes them tick - that will give me some details of what sort of emulation has to be provided. Obviously the code must work in real mode, but once int 10h is invoked, no one really knows what's happening. For anything else than old isa cards (and I don't care about these) access to high memory would be needed in order to do mmaped io. That suggests that the video bios enters either protect or unreal mode, does what has to be done, and then returns to real mode. What sort of instructions are used? 8086? 386? 586? sse? mmx?

One interesting approach would be to use a sort of instruction rewriting to "convert" any real mode code (mostly memory access in seg:ofs form) to run in protected mode (in ring 3), put the video bios in it's own address space and intercept any page faults / GPFs caused by memory or port access, have the emulator do these on real hardware, then pass them back to the 'trapped' bios. Is that even feasible? Anyone tried something like this?

Cheers,
Andrew

[Brendan]

Hi,

I guess I should start by disassembling some video bioses to see what makes them tick - that will give me some details of what sort of emulation has to be provided. Obviously the code must work in real mode, but once int 10h is invoked, no one really knows what's happening. For anything else than old isa cards (and I don't care about these) access to high memory would be needed in order to do mmaped io. That suggests that the video bios enters either protect or unreal mode, does what has to be done, and then returns to real mode. What sort of instructions are used? 8086? 386? 586? sse? mmx?

I'd expect most video card ROMs work in virtual 8086 mode, which means they can't/don't use protected mode. This is why the video card needs to be in a legacy mode, with its registers mapped to obsolete VGA IO ports and with the legacy "display memory windows" at 0x000A0000 (so that the ROM doesn't need protected mode).

If you're writing an emulator, then you'd write an emulator that supports real mode (e.g. including writing real mode test code to ensure it does things right). Then you'd setup the pretend IVT, map display memory at 0x000A00000 and map the video card's ROM at 0x000C00000 and see what happens. If the ROM does something you don't handle your emulator is going to know, and you can add support for whatever is missing then, until it supports everything it needs to.

The main problem here is that you won't have any video, so it'll be hard for the emulator to tell you (the user) when an unsupported instruction, memory read/write, IO port access, etc has occurred (and therefore hard for you to add support for whatever was missing). For that reason I'd implement some way for a remote computer to display the error messages (e.g. simple dumb terminal using serial ports).

One interesting approach would be to use a sort of instruction rewriting to "convert" any real mode code (mostly memory access in seg:ofs form) to run in protected mode (in ring 3), put the video bios in it's own address space and intercept any page faults / GPFs caused by memory or port access, have the emulator do these on real hardware, then pass them back to the 'trapped' bios. Is that even feasible? Anyone tried something like this?

There's lots of ways it could be implemented. Most people just use interpreting (because that's the easiest way if you're in long mode), but if you want to spend years writing a JIT compiler or doing instruction level recompiling then that can work in theory too.

However; it's a very good idea to figure out how much effort is justified (e.g. do you really want to spend 4 years implementing an instruction level recompiler?). The fact is that regardless of what you do "int 0x10" sucks badly and doesn't avoid the need for native video drivers (e.g. so that you can use the full range of video modes rather than just the old ones VBE provides, and support 2D/3D hardware acceleration that's required to get acceptable performance out of modern graphics). The other thing to consider is that end users almost never switch video modes after boot.

Basically what I'm saying is that it's a lot of work for very little practical benefit; and the time/effort involved for instruction level recompiling isn't justified, given that (99% of the time) you're only going to use it once during boot. For the same reason, it's also unlikely that supporting high performance JIT is justified. An interpreter is less work and therefore it's easier to justify that work.

To be honest, about 20 years ago I added support for virtual8086 mode to a version of my OS (for video mode switching). This is the easiest way (or at least it was back then when you didn't need to worry about long mode and/or UEFI) - much easier than any other way (including an interpreter). Ever since then I've been convinced that the effort needed to support virtual8086 wasn't justified and that it's better to setup a default video mode during boot (when you're in real mode anyway) and forget about allowing "driverless video mode switching that's almost never needed or used by end users" after boot. I haven't bothered with virtual8086 mode (or interpreting/JIT/recompiling) in any version of my OS since.


Cheers,

Brendan

[theesemtheesem]

Hi,

Many thanks to Brendan for his insightful remarks. I know that once a system boots people very rarely change video
modes, and I understand that having it done properly would require enormous effort (i was thinking about something
like a JIT compiler, but as You pointed out that would take YEARS to write). Besides, I'm actually happy with my OS being
in 80x25 textmode for now anyway, but it is sometimes fun to reach into the realm of 'what if'

Still the way M$ has done it is really nifty, and despite not being a M$ fan i have to give them respect for that.

Anyway, i'll have a look at some video bioses just for kicks, especially that I didn't find any information anywhere about
how these work internally. I mean there is bochs' video bios to start with, but what I'll do is look into some
real ones from real cards. I'll definitely post whatever I find, maybe someone will have some use for this in the future.

Cheers,
Andrew

[Gigasoft]

Writing an emulator is probably overkill. In my OS, I just go to real mode whenever I need to call int 10h, but using virtual 8086 mode would also have worked fine. With some care, you can switch between PM and LM as needed, and use the same page tables.

[Brendan]

Hi,

Writing an emulator is probably overkill. In my OS, I just go to real mode whenever I need to call int 10h, but using virtual 8086 mode would also have worked fine. With some care, you can switch between PM and LM as needed, and use the same page tables.

The main problem with switching back to real mode is IRQs.

In real mode, all interrupts (including software interrupts) behave like interrupt gates - the CPU automatically disables IRQs. To avoid severe IRQ latency problems in real mode, BIOS functions that take more than about 50 cycles (e.g. the relatively expensive "set video mode" BIOS functions) can/should explicitly re-enable IRQs (e.g. "STI"). Also some BIOS functions must enable IRQs (e.g. "get keypress" and anything involving time-outs and/or disk IO).

Because you can't assume that IRQs won't interrupt BIOS code; you have to be able to handle IRQs interrupting BIOS functions correctly. For the simple case (no native drivers or anything) you can let the BIOS do all IRQ handling. However...

Once you've got a real OS (e.g. with PIC chips reprogrammed and/or IO APIC and/or MSI; with IRQs for native device drivers, IPIs, etc. occurring all the time) it becomes more complicated. To avoid losing IRQs (and breaking native drivers and everything else) you must have a dummy IVT in real mode with a set of real mode IRQs handlers; where your real mode IRQ handlers switch back to protected/long mode, pass control to the OS's native interrupt handlers, then switch back to real mode and return to the interrupted code. However...

This is still not quite enough. Nothing prevents an NMI or machine check exception from occurring while you're in the middle of switching CPU modes at any point; such that the NMI or machine check exception handler starts while the CPU is in one of several "half real mode and half protected mode" states. It actually is theoretically possible to handle this correctly, but it's insanely complex and extremely difficult to test and/or guarantee reliability.

Mostly; in my experience; projects that switch to real mode to use BIOS functions after boot are done by people that either aren't aware of the complications or ignore complications. They take the easy way, and have relatively fast initial development while everything "seems to work" (because they aren't dealing with complications for this or anything else); and then either crumble under the project's lack of stability (once they start trying to implement native drivers) or never grow beyond a certain point (e.g. become "BIOS extensions" with no native drivers at all).


Cheers,

Brendan

[Bender]

Writing an emulator is probably overkill. In my OS, I just go to real mode whenever I need to call int 10h, but using virtual 8086 mode would also have worked fine. With some care, you can switch between PM and LM as needed, and use the same page tables.

Why bother to do all the hard work and possibly put the whole OS in a sort of dead state or use V8086 mode?
Just use an existing emulator like libx86emu, I guess someone on this board has made one too.
IMO it depends, monotasking OSes should be OK with this, but multitasking and feature-heavy OSes will have a great problem when switching to Real Mode. Maybe one core can switch to Real Mode and do the interrupt while other cores can do the OS stuff.

[theesemtheesem]

Hi,

Mostly; in my experience; projects that switch to real mode to use BIOS functions after boot are done by people that either aren't aware of the complications or ignore complications. They take the easy way, and have relatively fast initial development while everything "seems to work" (because they aren't dealing with complications for this or anything else); and then either crumble under the project's lack of stability (once they start trying to implement native drivers) or never grow beyond a certain point (e.g. become "BIOS extensions" with no native drivers at all).

Why bother to do all the hard work and possibly put the whole OS in a sort of dead state or use V8086 mode?
Just use an existing emulator like libx86emu, I guess someone on this board has made one too.

Yeah, that is exactly my point. From 'day zero' i have promised myself to at all cost avoid switching to real mode, exactly because it is such a pain in the backside to do properly, and even then there is really no such thing as doing it properly. And that becomes especially important for a 64 bit OS running in long mode, where even virtual mode is out of the question, because it does not exist.
My OS is strictly 32 bit at this point, being my first project, but I do have grand plans for the future to move to long mode land once my understanding of OS programming becomes good enough, and for starters, 32 bit is easier and there is more information available.

I have done some headway and managed to have a look at a few video bioses, and it turns out that using a ready made emulator might be the only reasonable way to achieve results. In theory a suitable emulator could be written by myself from scratch, but it would take too much time and in this case redoing all the work is pointless. Some video bioses do a lot of crazy stuff, and really do need full x86 support from the emulation layer to work.

Cheers,
Theesem

[bluemoon]

Many thanks to Brendan for his insightful remarks. I know that once a system boots people very rarely change video
modes

I want to point out that, it is way more common for notebook to switch video mode for external display.
If the notebook is with mirror mode and user want to optimize for external display, it may also affect the primary display.

However, if you are on a driverless situation, or fallback driver, you should not worry too much on this.

[theesemtheesem]

Hi,

However, if you are on a driverless situation, or fallback driver, you should not worry too much on this.

Yes, I am talking about a 'default' driver, or as You nicely put it, fallback driver. As I said, I am impressed by the Windows 7 standard vga driver, which seems really capable for a 'standard vga' driver. It would be nice to have something like that in one's OS. I've managed to install Win 7 (32 bit) on a Pentium 3 machine to see just how capable that driver is. It turns out that as long a graphics card supports vesa 2.0, the driver will work, and reasonably fast. It only breaks down on ancient isa cards, but that's reasonable, since nobody would expect something that old in a win7 machine.

Cheers,
Andrew

[Gigasoft]

The main problem with switching back to real mode is IRQs.

In real mode, all interrupts (including software interrupts) behave like interrupt gates - the CPU automatically disables IRQs. To avoid severe IRQ latency problems in real mode, BIOS functions that take more than about 50 cycles (e.g. the relatively expensive "set video mode" BIOS functions) can/should explicitly re-enable IRQs (e.g. "STI"). Also some BIOS functions must enable IRQs (e.g. "get keypress" and anything involving time-outs and/or disk IO).

No interrupts are passed to the BIOS. Any IRQ that happens while executing the real mode function will merely set a pending flag which will be processed after the real mode function returns. The VBE ROM is assumed to not rely on disk IO or keyboard input. At worst, it will need a working timer tick counter.

This is still not quite enough. Nothing prevents an NMI or machine check exception from occurring while you're in the middle of switching CPU modes at any point; such that the NMI or machine check exception handler starts while the CPU is in one of several "half real mode and half protected mode" states.

The same applies when switching between LM and PM, too. If it is impractical or undesired to disable NMIs and MCEs, having a temporary identity-mapped IDT with entries for RM, PM and LM handlers at appropriate offsets, and an identity-mapped GDT, takes care of the problem. Once again, when using RM we have to assume that the VBE ROM does not call int 12h to retrieve the conventional memory size, if we decide to leave MCEs enabled.

However, getting real mode calls to work flawlessly in all cases is not a priority, as it is eventually going to be replaced with Virtual 8086 mode.

[Brendan]

Hi,

The main problem with switching back to real mode is IRQs.

In real mode, all interrupts (including software interrupts) behave like interrupt gates - the CPU automatically disables IRQs. To avoid severe IRQ latency problems in real mode, BIOS functions that take more than about 50 cycles (e.g. the relatively expensive "set video mode" BIOS functions) can/should explicitly re-enable IRQs (e.g. "STI"). Also some BIOS functions must enable IRQs (e.g. "get keypress" and anything involving time-outs and/or disk IO).

No interrupts are passed to the BIOS. Any IRQ that happens while executing the real mode function will merely set a pending flag which will be processed after the real mode function returns. The VBE ROM is assumed to not rely on disk IO or keyboard input. At worst, it will need a working timer tick counter.

So, you'd have IRQ handlers for real mode that set pending flags (and check those pending flags and start the real IRQ handler/s when you get back to protected mode)? That would work, if you like poor IRQ latency.

This is still not quite enough. Nothing prevents an NMI or machine check exception from occurring while you're in the middle of switching CPU modes at any point; such that the NMI or machine check exception handler starts while the CPU is in one of several "half real mode and half protected mode" states.

The same applies when switching between LM and PM, too. If it is impractical or undesired to disable NMIs and MCEs, having a temporary identity-mapped IDT with entries for RM, PM and LM handlers at appropriate offsets, and an identity-mapped GDT, takes care of the problem.

It's excessively difficult to solve all the problems with a hybrid IDT (and still impossible to test that you can correctly handle an NMI or MCE after every single instruction involved in mode switches).

The best possible end result is having reliable code (excluding modern/UEFI systems) to do "video mode switching without a driver after boot" that sucks just as badly as VBE does. Instead of wasting your time making reliable code that sucks, it'd be far better to not bother (set a mode during boot) and spend that time on something that doesn't suck (like a native video driver for Intel or ATI, that's capable of doing video mode switches efficiently with a much wider range of video modes that includes the monitor's native resolution, that does work for UEFI and BIOS and systems with multiple video cards, and also gives you something you can build on later when you want 2D/3D acceleration).


Cheers,

Brendan