How does video work?

[Tomaka17]

Hello,

Currently my o/s can read files on the floppy disk, execute ELF files, and even connect to google.fr and download the index
But now I'd like to create a graphical shell and I'm a bit confused with all these things

I have already searched through the forum and the wiki, but I only find incomplete or confusing informations :-/
The purpose of this topic is to sum up how the video works and you correct me if I'm wrong


So :

- almost all video cards conform to the VGA standarts which define a lot of things including ports and memory addresses you can write ; VGA is based on how the first video chipsets were working

- almost all video cards conform to the VBE standarts which define a set of functions accessible using the video BIOS ; the video BIOS can thus be used by operating systems, but in practice it's not really the case because it's very slow

- the VBE standarts only define screen resolutions up to 1280x1024. If you want something higher you have to use a card-specific driver

- SVGA, XGA, etc. are not really standarts ; they are just names for screen resolutions (not part of the VBE standarts)

- OpenGL is just a set of functions that may be directly handled by the video card. In fact each video card has a different way to provide these functions (which is why you have a video card driver). If a function can't be provided by the video card, the o/s has to provide it (in which case its execution will be slower)

- When you call an OpenGL function in a user-mode program, this call is handled by a shared library. This library then makes a syscall, and each syscall is redirected either to the video card (if the video card provides this function) or to the o/s implementation of it

- Any video card may also provide non-opengl and non-vbe functions which may be used by the operating system (like mouse cursor handling, bitblt, etc.) but there are no standarts for this

- functions like "CreateWindow" are syscalls on Microsoft Windows (as windowing is handled by the kernel) or inter-process communication on Linux (as windowing is handled by the software X11)

- In fact OpenGL functions calls first go through the windows manager, which can then adapt the parameters with the position of the window (in case of windowed rendering). Functions like changing rendering viewport are in fact handled by the windows manager, so it can save these values

- Qt, SDL, etc. are just libraries that are hiding OpenGL or o/s-dependent calls

Many of these things are assumptions, that's why I hope you'll correct me

[Tomaka17]

I'm going to give an example of how I think it is working on Windows
Again correct me if I'm wrong


Example 1
There is no card-specific driver installed
The system thus uses the VBE standart functions and implements all the OpenGL functions manually

A program calls "CreateWindow" -> syscall -> the kernel saves the coordinates of the window
Then it calls "SetCurrentViewport(newWindow, 0, 0, sizex, sizey)" -> syscall -> the kernel saves this

Then it calls "glRectangle" -> syscall -> the windows manager tranforms glRectangle(0, 0, 10, 10) to glRectangle(200, 250, 210, 260) if the coordinates of the window are (200, 250)

The function glRectangle is not provided by the video card (as there is no driver installed), thus the O/S uses the video BIOS call defined by the VBE standarts in order to draw this rectangle

If the function was not glRectangle but something a bit more complicated (like 3D rendering), the O/S would have to use its own implementation of the function which uses simplier calls (like the "plot" function)


Example 2
The user installs a card-specific driver

A program calls "CreateWindow" -> syscall -> the kernel saves the coordinates of the window
Then it calls "SetCurrentViewport(newWindow, 0, 0, sizex, sizey)" -> syscall -> the kernel saves this

Then it calls "glRectangle" -> syscall -> the windows manager tranforms glRectangle(0, 0, 10, 10) to glRectangle(200, 250, 210, 260) if the coordinates of the window are (200, 250)

The function glRectangle is provided by the video card
The call is redirected to the video driver, which then does something secret (like writing to a specific memory address) to draw the rectangle

If the function was not glRectangle but something a bit more complicated (like 3D rendering), the O/S would have done it exactly the same way : not using its own implementation but telling the video driver to call the function

[gzaloprgm]

VBE standard functions are ONLY used to change video mode.
In case there's no graphic card driver, Windows would use "default vesa compatible video card", which uses VM86 to execute the bios 16 bit code to switch the graphic mode.

To draw things like lines, curves, rectangles, circles, etc, Windows has GDI, the component of the OS that is responsible for representing graphical objects and transmitting them to output devices such as monitors and printers.

video BIOS call defined by the VBE standarts in order to draw this rectangle

This "BIOS call defined by standards" doesn't exist, drawing a rectangle is just drawing 4 lines. Drawing 4 lines is just setting some values in a linear frame buffer provided by the video card.

Then you have user32.dll, which allows programs to implement a graphical user interface, contains basic functions such as window management, user input, text, etc. This library AFAIK uses GDI to draw the interface.

But, in case the OS has the video card driver, somethings will be faster:
The mouse can be drawed via Hardware (e.g. you provide coordinates, the card does the rest)
The memory from RAM to the card can be copied faster: Bitblt: You provide from and to boxes and the card does the rest.

In case you wanna use the card to do 3d acceleration, I suppose you send to the card the list of vertexes and location in memory of the textures, camera viewpoint and the card does the rest.

Cheers,
Gonzalo

[Brendan]

Hi,

- almost all video cards conform to the VGA standarts which define a lot of things including ports and memory addresses you can write ; VGA is based on how the first video chipsets were working

Yes, but even when the video card is "VGA compatible" at the hardware level different cards are more compatible than others. It also doesn't help when there's 2 or more video cards because only one video card can use the "standard VGA" I/O ports and memory ranges.

- almost all video cards conform to the VBE standarts which define a set of functions accessible using the video BIOS ; the video BIOS can thus be used by operating systems, but in practice it's not really the case because it's very slow

Almost all video cards conform to a version of the VBE standards (e.g. newer cards might support VBE 3, older cards might support VBE 2 or VBE 1). VBE is only really useful for setting video modes (and only for the first video card when there's 2 or more video cards).

- the VBE standarts only define screen resolutions up to 1280x1024. If you want something higher you have to use a card-specific driver

The VBE standards will handle resolutions up to 65535 * 65535. However, because VBE is considered a legacy thing some video cards don't mention some video modes (e.g. a video card might support higher resolution video modes and VBE won't tell you).

Also, VESA introduced a method of handling "arbitrary resolution" video modes, where there's a calculation to determine timings (where you only need to make sure that the timings are supported by both the video card and the monitor); but AFAIK this came after the VBE standards and isn't included in the VBE standards. In this case VBE won't tell you about the thousands of possible combinations of refresh rate, resolution and colour depth, and will probably only tell you about standard video modes.

- When you call an OpenGL function in a user-mode program, this call is handled by a shared library. This library then makes a syscall, and each syscall is redirected either to the video card (if the video card provides this function) or to the o/s implementation of it

How OpenGL is implemented depends on how OpenGL is implemented (different OSs may implement it in entirely different ways, as long as the user-level OpenGL interface is the same)....

For example, I'd assume that on most Unix clones OpenGL functions cause packets to be sent via. a socket (to the X server).

- Any video card may also provide non-opengl and non-vbe functions which may be used by the operating system (like mouse cursor handling, bitblt, etc.) but there are no standarts for this

Video cards mostly provide hardware and code in their ROM, where the code in their ROM normally only includes VBE and standard BIOS functions. OpenGL isn't included in any video card - it's implemented in software (e.g. a device driver).

Things like mouse cursor handling, bitblt, etc are features the hardware might support, that may be used by software (e.g. the video driver).

There was a "VBE/AF" standard that attempted to provide an interface to some limited hardware acceleration features. Nobody ever used it (it's still not an open stadard - VESA have always tried to charging far too much for it, and it was far too limited, so software developers ignored it and hardware manufacturers didn't have a reason to bother with it because no software used it).

- Qt, SDL, etc. are just libraries that are hiding OpenGL or o/s-dependent calls

Yes.


Cheers,

Brendan

[neon]

In case there's no graphic card driver, Windows would use "default vesa compatible video card", which uses VM86 to execute the bios 16 bit code to switch the graphic mode.

From my understanding, doesn't Windows just use its default VGA-compatible driver? I personally never heard of Windows using VBE so I am somewhat interested.

[JohnnyTheDon]

I'm pretty sure it uses VGA. Whenever I install Windows it always is stuck in low res and low color modes (640x800 and the like). This is definitely true in x64 because there is no v86 mode.

[JackScott]

800x600 is not a VGA mode. The "VGA compatible" driver uses VBE. That's how I can use 1440x900@32b on my laptop screen for months before realising that I don't have a driver installed.

[JohnnyTheDon]

x86 or x64?

[JackScott]

Both. I'm currently using Vista x64, but the same is true of WinXP and Vista x86.

[Tomaka17]

Thank you for answering so fast =)

The VBE standards will handle resolutions up to 65535 * 65535. However, because VBE is considered a legacy thing some video cards don't mention some video modes (e.g. a video card might support higher resolution video modes and VBE won't tell you).

I meant that the VBE standards define video mode numbers up to 1280 * 1024 but there may be other video modes
These video modes can be retreived by calling a VBE-standard function in the video BIOS

To draw things like lines, curves, rectangles, circles, etc, Windows has GDI, the component of the OS that is responsible for representing graphical objects and transmitting them to output devices such as monitors and printers.

GDI is something that is still very confusing for me
Isn't that what I call o/s implementation of OpenGL functions calls?

video BIOS call defined by the VBE standarts in order to draw this rectangle

This "BIOS call defined by standards" doesn't exist, drawing a rectangle is just drawing 4 lines. Drawing 4 lines is just setting some values in a linear frame buffer provided by the video card.

I just checked the VBE 3.0 documentation and realised I was wrong
I thought there were more functions but in fact the only available functions are for video modes, palette data and video memory start
If I'm right, when you say drawing a rectangle is drawing 4 lines, etc. the function that draws these lines is part of what is called GDI ?

In case you wanna use the card to do 3d acceleration, I suppose you send to the card the list of vertexes and location in memory of the textures, camera viewpoint and the card does the rest.

Yes but the location to write these data is kept secret and will only be done by a non-opensource video driver



To sum up, when you want let's say draw a rectangle :
- the OpenGL shared library may directly handle the call (certainly not the case for rectangle drawing, but perhaps for functions whose goal is just to set a state)
- if not, it makes a syscall
- the O/S checks if the video driver provides a "drawrectangle" function
- if not, the O/S manually draws it (that's called GDI for Windows) using the video memory (it knows the address thanks to VBE)
- if the driver provides the "drawrectangle" function, the O/S simply calls it ; then the driver may either directly write into video memory or send data to i/o ports, ie. the fastest way to do its job

Some functions like "drawrectangle" may have an O/S implementation, but some others (like for 3D rendering) may not because the way video cards are working is kept secret

On Unix/Linux it may even be more complicated because of XWindow's client-server architecture


You didn't talk about my assumption that "drawrectangle(0, 0, 10, 10)" would be turned into "drawrectangle(200, 250, 210, 260)" by the kernel because we are drawing in a window located at 200,250
Is that right or is there an hardware implementation for windowing?

[Combuster]

I meant that the VBE standards define video mode numbers up to 1280 * 1024 but there may be other video modes
These video modes can be retreived by calling a VBE-standard function in the video BIOS

Only VBE1 defined specific video modes. On all non-prehistoric video cards (which are VBE2/VBE3) you should use a bios call to get the list of modes and their corresponding numbers.

GDI is something that is still very confusing for me
Isn't that what I call o/s implementation of OpenGL functions calls?

It's not the same. Windows has three (semi-direct) ways of communicating with graphics hardware: DirectX, GDI, OpenGL. Each of them translates a set of function calls (glBegin / BitBlt / ISurface::Flip) into something the OS understands. Each works differently, you get the choice which one you see most appropriate.

Yes but the location to write these data is kept secret and will only be done by a non-opensource video driver

Only NVidia works that way. Get a different card if that concerns you

You didn't talk about my assumption that "drawrectangle(0, 0, 10, 10)" would be turned into "drawrectangle(200, 250, 210, 260)" by the kernel because we are drawing in a window located at 200,250
Is that right or is there an hardware implementation for windowing?

You can't assume any of that. The graphics card may support hardware overlays. Blitfills may be done in some random way specified by the graphics card. The only valid assumption that you can make is that drawrect() will draw a rectangle. How it works under the hood is undefined.
Basically, we can't see if you made the drawrect behave like you said.

[Tomaka17]

You can't assume any of that. The graphics card may support hardware overlays. Blitfills may be done in some random way specified by the graphics card. The only valid assumption that you can make is that drawrect() will draw a rectangle. How it works under the hood is undefined.
Basically, we can't see if you made the drawrect behave like you said.

My goal is to build my own windowing system, that's why I was asking this question :p
I still don't clearly know the link between 3D rendering and windowing


If I'm right, DirectX, OpenGL and GDI are just shared libraries that may use the same syscalls?
And from the kernel's point of view there is not difference between these?

I need a glue betwenn a video driver and my O/S
At the beginning I was thinking of simply using OpenGL functions, but I think I'm just going to make a custom list of functions video drivers may handle or not

[Brendan]

Hi,

If I'm right, DirectX, OpenGL and GDI are just shared libraries that may use the same syscalls?
And from the kernel's point of view there is not difference between these?

There's about 4 different interfaces:

• The interface between the video driver and the video card's hardware (defined by the video card manufacturer).
• The interface between the video driver and the GUI or virtual terminal layer or whatever (defined by the OS developer). For example, Windows Display Driver Model.
• The interface between the GUI (or virtual terminal layer or whatever) and the application's binaries (defined by whoever writes the GUI or virtual terminal layer or whatever). For example, the X protocol, GDI.
• The interface between the programmer's source code and the library they use (defined by anyone that feels like inventing a graphics library). For example, SDL, OpenGL.

IMHO it's a good idea to make the interface between the video driver and the GUI (or virtual terminal layer or whatever) similar to the interface between the GUI (or virtual terminal layer or whatever) and the application's binaries, so that it's easier for the GUI to translate one interface to another. However, the interface between the GUI (or virtual terminal layer or whatever) can handle more than just video - it can also include sound, music, microphone, keyboard, mouse, joystick, touchscreen, etc. Also, the GUI might need to talk to several video drivers rather than just one. For example, imagine 2 monitors next to each other being used as a single large desktop, with an application's window covering the right half of the left monitor and the left half of the right monitor.

Code: Select all

.__________   __________
|          | |          |
|       ###| |####      |
|       #HE| |LLO#      |
|       ###| |####      |
|__________| |__________|

To complicate things, the same protocol can be used recursively. For example, (in theory) you could use OpenGL for your video driver interface, and use OpenGL for (part of) your GUI interface, and then you'd be able to run a GUI inside a GUI inside a GUI inside a GUI...


Cheers,

Brendan

[Love4Boobies]

SVGA, XGA, etc. are not really standarts ; they are just names for screen resolutions (not part of the VBE standarts)

Most of the other things are covered in other posts, I didn't read all of them, but I saw no one commeting on this. Actually, VBE is the BIOS interface that most SVGA cards adopt. SVGA stands for "Super VGA", and unlinke VGA, isn't standard. VGA is standard (but not a standard) in the sense that all VGA cards are fully compatbile with each other from an implementation point of view. XGA and XGA-2 were meant to replace VGA, but they never actually cought on.

Another problem with VBE is that it doesn't work under 64-bit mode, unless you have some way of translating 32-bit BIOS code intro 64-bit code safely.

[Tomaka17]

Thank you Brendan

In fact it's very similar to the network (ws2_32.dll -> kernel's network manager -> NIC-specific driver to send/recv data) or the sound (fmod -> kernel's sound manager -> device-specific driver)
The only difference is that you have to add a GUI interface between the library and the video managing system