Hi,
AFAIK both AMD and VIA released (not necessarily well documented) register information and an open source driver for Linux. Intel did the same, but Intel's register information includes lots of documentation. There's also some information for a few older cards around (Cirrus Logic, S3, 3dfx/voodoo).
Unfortunately, even with the best possible documentation modern video cards are extremely complex; and it'd take an extremely long time to write a "full featured" device driver just for one video card. Basically, IMHO writing a full featured video driver isn't practical unless you've got lots of video driver experience beforehand.
The real mode interface for VBE (all versions) work fine (but not in protected mode), and it's relatively easy to write code that uses any information it can get (e.g. supports all versions of VBE at the same time). The protected mode interfaces for VBE often aren't supported.
VBE/AF wasn't great when it was released, and VESA tried charging a huge amount of $$$ for a copy of it, so nobody ever implemented anything for it (no support in almost all video cards; and almost no software that would've used VBE/AF if video card manufacturers bothered).
The last option is to program the card as if it's a "standard VGA". This works on lots of cards, but all the video modes have low resolution and/or poor colour depth, and there's no hardware acceleration.
My advice (in order of implementation):
- Setup a video mode using VBE during boot (while you're still in real mode) and try to get DDC/EDID information during boot too. Then implement a simple framebuffer device driver (where everything is done in software, and you need to reboot to change video modes). Note: Some people spend time implementing virtual8086 mode support (and/or 8086 emulators, e.g. for long mode) so that they can change video modes with VBE without rebooting. To be perfectly honest (for my purposes at least) I think this is a waste of time - the goal should be to have a device driver that supports all features of the video card, and the simple framebuffer driver should be considered a temporary thing that's only used when there's nothing better. Basically, in the long run, your time as an OS developer could be better spent elsewhere.
- After that, write a "standard VGA" driver. Let the user install this driver instead of the framebuffer driver if they want, and keep track of which video cards it works on (and which video cards it doesn't). This still means everything is done in software (no hardware acceleration).
- After that, attempt to write a basic video driver for Intel's video controllers. What you learned from writing a standard VGA driver will help, and it'd be best to start with the controller with the most complete documentation. This still means everything is done in software (no hardware acceleration).
- Then, either write basic drivers for other video controllers, or turn the existing basic video drivers into full featured video drivers (starting with Intel's due to better documentation). Note: Once you've got a basic video driver or a full featured video driver for one video card, it should be relatively easy to modify that video driver to support similar video cards (e.g. video cards from the same manufacturer, where "version X" is only slightly different from "version X+1").
The most important thing is the software interface between the video driver/s and other software (e.g. the GUI). This needs to be designed well, with all sorts of things in mind (e.g. hardware acceleration, hardware cursors, device independent shader language and tools/compiler, stereoscopic 3D glasses, 3D layered LCD screens, multiple monitors, multiple video cards, etc). The last thing you want to do is spend years implement device drivers, GUIs, etc; and then realize that the video driver interface you're using isn't adequate and that you need to go back and redesign everything.
Also, sooner or later you'll need to start maintaining a hardware compatibility database. This could be as simple as a text file that says if each piece of hardware works or not, and what level of support there is (e.g. if the driver supports everything, or only some things); but it could be a more complex thing, like a web site with a MySQL back-end with the ability to find contributing factors (e.g. "video card X works unless it's used with chipset Y") and (optional) automated feedback sent from the OS.
Cheers,
Brendan