Drawing triangles on Raspberry PI.

[AlfaOmega08]

Oh, it's been a long time since my last post. First of all I want to thank you all on the forum for your precious help in the past. I was 14 when I joined the forum and I was a true pest. Here I finally understood how the pointer dereferencing operator works. All the way up to having a self-compiling OS with gcc, smp, acpi and a network stack (all my AWWWW YEAH moments)

I have received some RTFM and STFW, I deserved them and I learned to think and search a thousand times before posting. And this time I did it, I really did.

So, what am I looking for today?

I recently bought a Raspberry PI (shipped, still not received). Fascinated by the infinite possibilities, the little OSDever buried inside me rise again.
Just before clicking the "Complete Purchase" button I read an article by Broadcom, declaring that the R-PI is the only 100% open source computer.

My first thought was to the GPU: maybe I can finally draw a rotating 3D cube without having to draw every single pixel using REP STOSB?
I've looked all over the web. I downloaded and quickly read the available documentation out there:

1) the official BCM2835 chip documentation (just a list of memory mapped IO registers and some other fuzz), no mention of the GPU.
2) the official VideoCore IV documentation. Another bunch of registers and the Control Lists, the way to tell the GPU to do things. (still no mention - at first sight - on how to get to the right resolution, how to enable the HDMI port instead of the composite, etc.)
3) https://github.com/hermanhermitage/videocoreiv a GitHub documentation project about the R-PI GPU details (could be useful if complete)

I downloaded the official VideoCore IV driver for Android, but the lack of comments and poor naming made the code pretty useless (still, I didn't read it thoroughly).

I downloaded through the Wayback Machine the source code of DexOs who claimed of having the HDMI working, but still I could only find the old boring way of doing the "Painter Algorithm" drawing each pixel "by hand".

All other hobby OS out there just claim of having a pointer to the FB and again, they just put raw colors there. All of them use a mysterious "mailbox" (to obtain the FB address and resolution details) I could find no mention about on both the official specs (except for an IRQ number). No specification, no protocol, no registers.

Still, no apparent mention anywhere on "how to tell the GPU to draw a f******g triangle". (Control Lists maybe? Still how do I communicate them to the GPU?)

Now, I don't consider myself an expert in English reading, but man, I made it through the ACPI specification. These are just ~120 pages long (and tables most of the time).
I'm pretty sure I have not missed them. I've used some synonyms during the search. I've jumped back and forth for a week all over the web for these piece of informations.

Sure I could find my way reading source codes with more attention, but wasn't this meant to be a 100% open source platform?

So here are my questions:
1) Are those all the source of informations about the inner details of the R-PI?
2) Really it is not documented anywhere how to work with the GPU in a hardware accelerated fashion?
3) Did you try this way and if you succeeded, can you point me to your source of documentation?
4) Given that the Linux driver for the gpu is the only complete and reliable source of documentation (cryptic, but working), can you tell me where to find the code that actually does the rendering?

5) Is all this worth it? Or am I better to just put colors on the frame buffer?

As always, thanks in advance.

[Brendan]

Hi,

My first thought was to the GPU: maybe I can finally draw a rotating 3D cube without having to draw every single pixel using REP STOSB?
I've looked all over the web. I downloaded and quickly read the available documentation out there:

1) the official BCM2835 chip documentation (just a list of memory mapped IO registers and some other fuzz), no mention of the GPU.
2) the official VideoCore IV documentation. Another bunch of registers and the Control Lists, the way to tell the GPU to do things. (still no mention - at first sight - on how to get to the right resolution, how to enable the HDMI port instead of the composite, etc.)
3) https://github.com/hermanhermitage/videocoreiv a GitHub documentation project about the R-PI GPU details (could be useful if complete)

For video there's 2 "mostly separate" pieces - one piece to feed pixel data into a monitor (likely detailed in the BCM2835 chip documentation) that has nothing to do with rendering the data; and another piece (the GPU) that does the rendering (that's detailed in the official VideoCore IV documentation). The BCM2835 chip documentation won't include information for the GPU (as the GPU is a different part), and the VideoCore IV documentation won't include information for video mode setting (as that is a different part).

I only had a brief look at the official VideoCore IV documentation (mostly because I was surprised it existed - I think it might be the first time Broadcom provided any documentation for anything); but it does seem to include everything relevant for the GPU (although it is considerably "terse").

Also note that you will not find a "draw triangle" feature. Instead, you'll find information on using the GPU to execute code. It's your job to write code ("shaders") to run on the GPU; where your code (e.g.) transforms vertices, generates "fragments", applies textures to fragments, etc. Of course you could just use it for GPGPU instead - that'd be fun too.

5) Is all this worth it? Or am I better to just put colors on the frame buffer?

It depends too much on your goals; but if you're going to implement a software renderer for 3D/OpenGL it'd make more sense implementing it using "GPU assembly" than expecting the ARM core to handle the processing.


Cheers,

Brendan

[Owen]

The other bit you're missing is that the VideoCore IV is a complete processor in its' own right (actually, the ye olde ARM11 is a coprocessor to it). That processor runs some firmware it loads off the SD card; the shipped firmware listens for commands in its' mailbox, a combination of which you will want to use to draw graphics (unless you're writing your own firmware).

This may be the undocumented gap you're finding; I don't know if the shipped firmware is completely documented anywhere (or if you would need to reverse engineer everything from the Linux drivers)

[embryo]

2) Really it is not documented anywhere how to work with the GPU in a hardware accelerated fashion?

No. But the problem is there is a problem

Hardware acceleration is a complex area. First - it is REQUIRED (it means it's a must have knowledge) to know some mathematics. Second, it is required to have a deep understanding of relationships between the math and the hardware. And third, it is required to have deep understanding of the particular hardware.

As an example of the set of knowledge any 3D low level programmer must have I can show my googling results:

• Here is some info about required math (use arrows on the top to navigate)
• Here is an example connection between the math and the hardware
• Here is an example of the hardware description

3) Did you try this way and if you succeeded, can you point me to your source of documentation?

I haven't tried the 3D programming in it's full power, but I have completed a similar way for other programming technologies.

4) Given that the Linux driver for the gpu is the only complete and reliable source of documentation (cryptic, but working), can you tell me where to find the code that actually does the rendering?

It's a really interesting question. There should be some classification of the Linux drivers, but as far as I know it is missed. It would be great if low level developer community will ever unite and produce such classification.

5) Is all this worth it? Or am I better to just put colors on the frame buffer?

Is it worth to study assembler (or any other language)? Or it is better to just use MS Office?

[SpyderTL]

The VideoCore IV Wikipedia page (http://en.wikipedia.org/wiki/VideoCore) actually has a link to the "Architecture Reference Guide" PDF (http://www.broadcom.com/docs/support/vi ... G100-R.pdf), which contains the VC4 instruction set and encodings that you will need to write code (functions/shaders/etc.) for the VC4.

The VC4 has it's own instruction set, so you will either need to write your own compiler, or find one that already exists. Then you will need to compile it, and include it in your OS as a resource that can be loaded, at runtime, and executed.

The Raspberry Pi is a platform, like the XBOX 360 or PlayStation 3. These platforms have "platform specific" functionality that only works in that configuration, unlike most OS development platforms, that are expandable, and components may or may not exist, and must be detected, dynamically, at run time. The advantage is that you can simplify your applications and drivers, because you can safely make assumptions about what is available on the system. The disadvantage is that the individual component interfaces are not well defined and not well documented, because they really have no context on their own. In other words, you won't find any stand-alone PDF files explaining how the "Mailbox" works on the Raspberry Pi, because it is so tightly coupled with the rest of the system. The OSDev wiki page on the ARM RaspberryPi has a link to a GitHub wiki page about the mailbox. (https://github.com/raspberrypi/firmware ... -mailboxes)

The Rasperry Pi Wikipedia page (http://en.wikipedia.org/wiki/Raspberry_Pi) talks about the open/closed source VideoCore driver issue:

Raspberry Pi can use a VideoCore IV GPU via a binary blob, which is loaded into the GPU at boot time from the SD-card, and additional software, that initially was closed source. This part of the driver code was later released, however much of the actual driver work is done using the closed source GPU code. Application software uses calls to closed source run-time libraries (OpenMax, OpenGL ES or OpenVG) which in turn calls an open source driver inside the Linux kernel, which then calls the closed source Videocore IV GPU driver code. The API of the kernel driver is specific for these closed libraries. Video applications use OpenMAX, 3D applications use OpenGL ES and 2D applications use OpenVG which both in turn use EGL. OpenMAX and EGL use the open source kernel driver in turn.

Trying to write your own driver is probably going to be quite difficult, given the nature of the system, the lack of documentation and sample code, and the lack of a true virtual machine that can be used during development. But we will help wherever possible -- I've got 2 RPi Model B's at home, collecting dust. I'd love to be able to render 3D graphics on them, coding from scratch.

[SpyderTL]

If it helps, I have the Raspberry Pi video core tag numbers defined in XML here:
https://ozone.codeplex.com/SourceContro ... /VideoCore

Now that I know where they are, I'll probably have the VC4 instructions defined in XML in a day or two.

[SpyderTL]

I found this post on reddit that gives even more information about the Raspberry Pi processors/co-processors, and has various links to (potential) tutorials and overview documentation.

http://www.reddit.com/r/ReverseEngineer ... pberry_pi/

The SoC in the Raspberry Pi has 3 processors / instruction sets - (1) An ARM v6 @700MHz for userland, (2) a VideoCore IV VPU @250MHz with dual core, dual issue and 16-way SIMD integer vector processor for running the blob, codecs, 2d acceleration etc, and (3) a 24 GFLOPS shader processor (QPUs) for OpenGL ES and OpenVG.

On a side note, I tried reading through the VC4 instruction set and programming manual, and now I have a massive headache... I'm sure it's just a coincidence...

Core Pipeline Operation

The front end of each QPU pipeline receives instructions from a shared instruction cache (icache). As one
icache unit serves four QPUs in four successive clock cycles the front end pipelines of each of these four QPUs
will be at different phases relative to each other. After instruction fetch there is a ‘re-synchronisation’ pipeline
stage which brings all of the QPUs into phase with each other. The re-synchronised parts of the QPU pipeline
are shown in Figure 2 along with the names assigned to each pipeline stage.

Processor Registers

The QPU contains two sets of physical registers consisting of a set of four general-purpose accumulators, two
special-purpose accumulators, and two large register-file memories. The register space associated with each
of the A and B regfiles can address 64 locations, 32 of these are backed by the physical registers while the other
32 are used to access register-space I/O.

The QPU pipeline is constructed such that register files have an entire pipeline cycle to perform a read
operation. As the QPU pipeline length from register file read to write-back is greater than four cycles, one
cannot write data to a physical QPU register-file in one instruction and then read that same data for use in the
next instruction (no forwarding paths are provided). QPU code is expected to make heavy use of the six
accumulator registers, which do not suffer the restriction that data written by an instruction cannot be read in
the next instruction.

I miss the good ol' days when CPU instruction references said something along the lines of:

ADD - Adds two operands together, and stores the result in the specified register.

Life was so simple back in the 60's...

[SpyderTL]

Also found this overview here: http://codedivine.org/2014/03/03/broadc ... -overview/

In the Raspberry Pi and Nokia flavors, the VideoCore IV consists of two distinct processors. The first processor is the actual programmable graphics core, which I will refer to as PGC. The second processor is a coprocessor. This embedded processor, not to be confused with the main CPU, runs its own operating system and handles almost all the actual work of the OpenGL driver. For example, shader compilation is done on this embedded processor and not on the main CPU in the Raspberry Pi and Nokia flavors. The OpenGL driver on these devices just is a shim that passes calls to the embedded coprocessor via RPC-like mechanism.

There's a lot more high-level information on that page, but I thought that this would clear things up a bit. Apparently, the GPU has its own OpenGL API endpoint, which can be called by the main CPU. Somehow...

I believe that I read that it also supports another API for 2D graphics functionality... I'll have to find it again.

This gives me hope that rendering triangles on the RPi could be as simple as sending a few OpenGL messages to the GPU.

Please let it be that easy!

[SpyderTL]

Found more information about registers and instructions here:
https://github.com/hermanhermitage/vide ... ers-Manual

And that page has a link to an instruction set defined in YAML:
https://github.com/mgottschlag/vctools/ ... tions.yaml

Edit: Just noticed you already had the first one...

[AlfaOmega08]

Sorry for the late reply bit I have been away from home, and even though I have read and appreciated alla of your answers, it is not very comfortable to read source codes or heavy documentations on a smartphone. Ad soon ad possible I hope to continue this discussion on a decent PC.

Thanks.

[AlfaOmega08]

Also note that you will not find a "draw triangle" feature. Instead, you'll find information on using the GPU to execute code. It's your job to write code ("shaders") to run on the GPU; where your code (e.g.) transforms vertices, generates "fragments", applies textures to fragments, etc. Of course you could just use it for GPGPU instead - that'd be fun too.

Having done some DirectX programming in the past I expected something like this. When I said "draw a triangle" I actually meant a function, or list of opcodes that translated roughly to:
1) take this vertex list
2) take this transformation matrix
3) use this texture
4) draw

So all of this is made through compiled shader, but still, the VC's QPU instruction set does not make very clear how to do this . Also, somewhere I read that there's more than QPUs in the VC.
Something called VPU (Vector Processing Unit) and TMU (Texture and Memory Lookup Unit), more misteries to be solved. How to interface with these???

Hardware acceleration is a complex area. First - it is REQUIRED (it means it's a must have knowledge) to know some mathematics. Second, it is required to have a deep understanding of relationships between the math and the hardware. And third, it is required to have deep understanding of the particular hardware.

As an example of the set of knowledge any 3D low level programmer must have I can show my googling results:
Here is some info about required math (use arrows on the top to navigate)
Here is an example connection between the math and the hardware
Here is an example of the hardware description

Luckily, having done DX in the past I already studied the math required and the various shaders involved in the rendering. Even though I'm absolutely not an expert, I think I can put at least two ✓ on that list.

The VC4 has it's own instruction set, so you will either need to write your own compiler, or find one that already exists.

That may be a problem, given the half-unknown instruction set (and probably, the multiple instruction sets - one for the QPUs, one for the VPU and one for the TMU), I highly doubt there's any in the wild. But maybe an assembler may be an easier task.

Also found this overview here: http://codedivine.org/2014/03/03/broadc ... -overview/

In the Raspberry Pi and Nokia flavors, the VideoCore IV consists of two distinct processors. The first processor is the actual programmable graphics core, which I will refer to as PGC. The second processor is a coprocessor. This embedded processor, not to be confused with the main CPU, runs its own operating system and handles almost all the actual work of the OpenGL driver. For example, shader compilation is done on this embedded processor and not on the main CPU in the Raspberry Pi and Nokia flavors. The OpenGL driver on these devices just is a shim that passes calls to the embedded coprocessor via RPC-like mechanism.

There's a lot more high-level information on that page, but I thought that this would clear things up a bit. Apparently, the GPU has its own OpenGL API endpoint, which can be called by the main CPU. Somehow...

I believe that I read that it also supports another API for 2D graphics functionality... I'll have to find it again.

This gives me hope that rendering triangles on the RPi could be as simple as sending a few OpenGL messages to the GPU.

Please let it be that easy!

That would just be great. Even though I always talked about triangles and 3d transformations, what I'm really interested in is 2D graphics (windows, cursors and shadows), which could be simulated using 3D and a constant Z value, but I hope there's something easier.

After all, what I really missed is the fact that the mailbox is a protocol of the blob firmware (bootcode.bin I suppose?) and not something integrated in the ARM or GPU (I tought it was something like a DMA controller). It's just software. So in the end it looks like the strongest references available are the linux source and the firmware (which I'm not going to decompile using the provided VideoCore instruction set )