Accessing GPU directly without kernel, driver, etc.

[LieutenantHacker]

First off, don't start with any assumptions or criticisms ... yet. I am well ready for naysayers, banter, and ridicule, but that will not stop me from knowing this can be done.

What I am trying to do is getter a better understanding of the GPU pipeline, and how that's implemented on the architectural level. For example, regardless if using a polished device driver or not there's only a specific way, or specific set of ways that the GPU will be accessed, programmed for, etc. I realize there's direct memory access, etc.

But what I want to know is the process of which the GPU is programmed for, and I want to compartmentalize that understanding into a device driver for an actual operating system later on.

As of now I'd like to now the method of which the GPU is programmed for, i.e. is there an "assembly" that modern GPUs use, and when writing code straight for the CPU (x86 Assembly, IA 64-32, Intel GMA 4500, Intel G41 Express Chipset), how does the CPU map instructions to the GPU via buses? Or is the entire process of actually using a GPU dependent solely on a sort-of GPU assembly/DMA combination? If so, and assuming device drivers directly interface the hardware, in what general way is it done on the lower-level that I can use as a guideline to replicate/disassemble to use as an idea to access the GPU itself as I want?

I realize 100% that every GPU is different, and I have already picked a few specific GPUs/boards in mind, and I just want to do this to:

1.Understand the GPU pipeline directly, chipset involvement, circuit, and addressing methods via the motherboard accurately.

2.Be able to directly test the GPU's capabilities without the abstraction of APIs, kernels, etc.

3.Gain more knowledge of the whole process on the circuit-level(for computer engineering interest).

PS: What I wanted to do is specify a square, and rotate it via the screen. You can laugh at me all you want, call me a noob, idiot, come up with dozens of reasons to tell me what I'm doing is "impossible" when drivers do the same thing, but this desire is not going to the trash yet.

EXTRA: I also know somebody is going to say "reverse engineering", "trade secrets", "use this open-source blah, blah, only way", but I'm not falling victim to the naysayers, and since I want to be a computer engineer as well as general programmer, I won't let anyone tell me accessing a GPU is "impossible" without vendor specific files when vendor specific files that access it are not "magical".

[metallevel]

Have you read the wiki?
Accelerated_Graphic_Cards

This may sound stupid, but what exactly are you considering a "GPU"? Some people around here (myself included) have successfully used hardware acceleration to draw graphics. But I don't think anyone here has managed it on a graphics card made in the last ten years. If you don't consider a Voodoo, an S3, or a Rage series card a GPU, then you have a lot of work to do. Correct it's technically not impossible, but the open source movement has spent YEARS trying to do this with limited success (especially with Nvidia GPUs).

Now things are better now than they used to be a few years ago. AMD has released several documents describing (at least in part) how to use their GPUs, including their recent 7000 series. But the register references alone can be hundreds of pages! And yes there is an instruction set that modern GPUs use; it is complex and specific to every generation (or even every model) of GPU. DMA is also handled by the GPU in a custom manner, in fact DMA hasn't had a standard interface since the ISA DMA (or the short lived MCA and EISA DMA) back in the 1980's.

If you're just curious about the detailed architecture of modern GPUs, I'd start by reading some of the documentation that AMD has released:
http://www.x.org/docs/AMD/

[LieutenantHacker]

I am considering a GPU exactly as described by Wikpedia:

A graphics processing unit (GPU), also occasionally called visual processing unit (VPU), is a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display.

And you are right ... I passed through a few PDFs of AMD's GPU documentation, and there's thousands of pages!

At least there's hope, though, on getting something done.

[Combuster]

The bottom line is, the more modern a video card is, the more it looks like a full-blown computer of its own - with processor cores that need to be bootstrapped, memory controllers you need to configure, firmware you need to upload, CRT controllers you need to tell not to blow up the monitor, and even with their own sound and networking, and there's relatively little new things involved compared to programming out a regular kernel. Regardless of the video card, you're not going to get just the GPU to deal with - even though "the rest" can be dealt with in 20 lines of code with magic numbers on the oldest of devices.

The only thing you will need to realize that claiming you want to do it without either a kernel or driver is nonsense - because you'll be writing exactly that. You're just seemingly going to do that the rebellious way without any plan or structure which in turn probably comes to haunt you again later.



You should realize just one thing from the "naysayers": you will be on your own. Several brave people have gone this path before and most of them were never heard of again.

[Rew]

I don't see what is wrong with trying to understand the depth of the GPU "problem". I'll leave LT hacker to defend (which he shouldn't have to) his interest in learning about how things work (isn't that why we are all here?).

I did have a question to append to this discussion about modern graphics acceleration. Does anyone know anything about utilizing current APU's? Do they have a similar control concept as standard GPU's? Are they "easier" due to the merging with CPU? Are they going to change the way the open source community looks at graphics acceleration?

[metallevel]

Does anyone know anything about utilizing current APU's? Do they have a similar control concept as standard GPU's? Are they "easier" due to the merging with CPU? Are they going to change the way the open source community looks at graphics acceleration?

I don't know much about the APUs, but I think AMD's APUs are just an AMD cpu and gpu bolted together, and I don't see why they wouldn't be. There might be a couple minor differences in the registers dealing with parts that are shared between the two (ex. caches), but if AMD already has working cpus and gpus, why would they make radical changes to the architectures (and their own drivers) to make a cpu-gpu hybrid? I wouldn't be surprised if the integrated gpu even showed up as being on a pcie bus.

The bottom line is, the more modern a video card is, the more it looks like a full-blown computer of its own - with processor cores that need to be bootstrapped, memory controllers you need to configure, firmware you need to upload, CRT controllers you need to tell not to blow up the monitor, and even with their own sound and networking, and there's relatively little new things involved compared to programming out a regular kernel. Regardless of the video card, you're not going to get just the GPU to deal with - even though "the rest" can be dealt with in 20 lines of code with magic numbers on the oldest of devices.

And that's just the setup. To get it to actually do something you have to send it command packets (of which there are numerous types) and worse yet, shader code (which must be compiled specifically for that gpu). Plus you need a system to manage and control the gpu, servicing requests, handling exceptions, etc.

Before all of this is done, chances are you will have no indication of whether or not you're doing it right. If you get a blank screen when your finished (and you will), debugging will be a nightmare.

If you want to do hardware accelerated graphics, at least start with an old card with good documentation. Voodoo is often recommended around here. Personally I've worked on the S3 Virge, which I'd recommend because they're common in old computers (and if you don't have one they're cheap), there is a full programming manual for it, they're easy to get working, and it has basic 3d capability (though I'm not sure I'd call the 3d 'accelerated' ).

[Combuster]

And I'd recommend a Rendition video card, because they are quite simple, and they have an onboard processor that runs firmware and in that regard look much more like a video card from a decade later, save from the additional complexity.

They're pretty much undocumented though - or used to be before I ran through it...

[LieutenantHacker]

I fully understand the gritty complications of the GPU, but some of you here might be getting the idea wrong between "writing a device driver from scratch for an OS", and "just initializing the GPU, and using it for one purpose, one time, from a bare machine".

This would be analogous to the difference between writing a full-fledged keyboard driver via Windows Driver Model for a Windows OS, assuming someone has already programmed Windows (and they have), and writing this below:

Code: Select all

ClearB:      
mov al, 0ff
mov     ah, 1           
                int     16h
                jz      BuffC     
                mov     ah, 0           
                int     16h             
                jmp     ClearB

BuffC:    mov     cx, 0           
RandKey:      inc     cx
                mov     ah, 1           
                int     16h
                jz      RandKey
                xor     cl, ch          
                mov     ah, 0           
                int     16h

Basically, reading a key, and using the keyboard from a bare machine is a lot simpler than implementing an entire device driver which, in some respects, may be coined as "middleware". Doing the same, just via a GPU, should not and I would not expect a single function of the GPU to be considered a "device driver", especially when there's no API.

[metallevel]

Except perhaps for setting the video mode, BIOS functions won't help you when programming a GPU. In fact, the BIOS functions you're using in that keyboard routine are essentially a device driver. You're not talking to the "bare machine" there, you're just using the API the BIOS provides. It's the equivalent of using "cin" to get data from the keyboard. To talk to the "bare machine", you need to setup the ps/2 controller and keyboard controller. See [wiki]PS/2 Keyboard[/wiki] and [wiki]"8042" PS/2 Controller[/wiki] to see what I mean. THAT is what Windows does on the lowest level, and that's also what you'll have to do to talk with the bare hardware. You may not call a program not running inside an OS a "device driver", but if you're accessing bare hardware you must do the same setup and ultimately perform at least some of the runtime functions a full featured device driver would. You are making a device driver, it just isn't integrated into an OS and is very minimalistic.

I fully understand the gritty complications of the GPU, but some of you here might be getting the idea wrong between "writing a device driver from scratch for an OS", and "just initializing the GPU, and using it for one purpose, one time, from a bare machine".

I don't mean to sound rude, but I don't think you're quite getting it. Everything we've mentioned so far IS required to "just initialize a GPU, and use it for one purpose, one time, on a bare machine". On a modern GPU, if you don't configure all the controllers, upload the firmware, setup the context, send a number of special command packets, and upload shader code, NOTHING will happen. In the past you could get 2D acceleration on GPUs a bit easier, but nowdays I think even the 2D functions are handled by the same 3D hardware.

Also I hate to ask this, but have you done anything in Protected Mode yet?

[iansjack]

I'm afraid that the example that you give, reading a keyboard, indicates - as metallevel points out - your lack of understanding of the complexity of the problem that you are trying to undertake. Calling a BIOS function to read a keystroke is trivial and is just making use of someone else's library routines; in fact, talking to the bare metal to read the keyboard is not a particularly difficult task and is something that is documented comprehensively. Proprietary video cards are not documented in the same way and, even if they were, controlling them is orders of magnitude more complicated than reading a key from a keyboard.

I think you would learn more by curbing your ambitions a little. Start by learning how to read a keystroke only by accessing the bare metal - no BIOS interrupts allowed. Then do the same for a disk controller. You might then try something a little more ambitious, such as talking to a USB device (difficult, but not impossible). Once you have mastered those then you will be ready to think about the much, much more difficult task of getting a video card to do something; who knows, you might eventually succeed where everyone is telling you it's nearly impossible. But forget about the BIOS altogether - we're not in Kansas any more.

[LieutenantHacker]

I do understand the complexity, trust me. Maybe not to the extent of those who have actually managed to, and succeeded with this as a project, but I am certain it's possible, regardless of difficulty or convenience.

My BIOS interrupts code was just an example! I am currently working on exactly as you said, except not a "driver" in that sense (I do not consider software to be a driver unless it is actually exposing a protocol to a routine, and acting as middleware, i.e. a library in an OS, like Winmm.lib).

I realize that a driver would be doing the same thing, but the semantic difference, I believe, is how it's employed, and how it's interfaced.