How to draw graphics to a high resolution and high refresh rate monitor with vsync?

[bellezzasolo]

It's likely to land up running into the 10's if not 100's of pages - it needs a book not a wiki!
but a reasonable shout non the less, I'll start putting something together.

I was also considering if, anyone is interested in this area and wants to setup a Slack Channel or Discord group etc, to collaborate on and discuss - this is the sort of topic that is going to need a lot of informal chat/debug/rinse and repeat work to get solid answers on.

I started having a go trying to distill the spec into a page, really for purposes of understanding it myself. It would need to be at least a category in its final form.
[wiki]User:Bellezzasolo/Intel Graphics Family[/wiki]

Existing pages:
Intel HD Graphics
Native Intel graphics

It's on my "might eventually have a go" list, incidentally, I'm getting an A770 arriving today, whereas I haven't seen Battlemage specs.

[CasualKyle]

It's likely to land up running into the 10's if not 100's of pages - it needs a book not a wiki!
but a reasonable shout non the less, I'll start putting something together.

I would love this as I am struggling to accomplish the most basic thing: submitting a logical context to the blitter command streamer.

As mentioned, this stuff is extremely complicated but I think I understand the idea behind command streamers. You define something called a logical context, the logical context contains a whole bunch of stuff that I have yet to understand but importantly, the logical context has a section in it called the logical ring context. The logical ring context allows you set up the ring buffer registers which are registers that configure a list of commands which the corresponding engine will execute.

To get the command streamer to actually start executing commands in the ring buffer, you have to submit the logical context. This is done by creating a context descriptor which contains the memory location of your logical context. Then you write the context descriptor to the Execution List Submission Port (ELSP). The context descriptor is 64 bits but the port only accepts 32 bits, so you have to do two writes. I found the "Command Stream Programming" document (for my Tiger Lake iGPU) actually gives a nice overview of all of this.

So with that all said, has anyone actually been able to do this? When I submit my context via writing to the ELSP, then read from the Execlist Submission Queue Contents registers to actually verify the submission, I get no results back. The entire submission queue is empty.

I wonder if the whole blitter command streamer is not enabled but I've yet to find anything that mentions somehow enabling or turning it on.

The document does mention this in the section explaining context submission:

Note that this mechanism cannot be used when the Execlist Enable bit in the corresponding engines
MODE register is not set, i.e GFX_MODE register for Render Engine, BLT_MODE register for Blitter Engine, VCS_MODE register for Video Engine, or VECS_MODE register for Video Enhancement Engine.

But the problem is, and unfortunately this is actually a pretty common problem I've seen after reading a lot of the documents, the BLT_MODE register does not exist in the register documents. The GFX_MODE register does exist and there is one for the blitter (GFX_MODE_BCSUNIT) however, there is no bit documented for enabling the execlist.

So I'm not sure where to go from here, has anyone accomplished this?

[johnsa]

I've just finished up all my other bits and am joining you now in copy-engine and blitter hell

I think this is just a misnomer in the PRM .. I believe (could be totally wrong), as I've not found any mention of those registers in the PRM or i915 driver, that what they actually mean is :

0x2209c MI_MODE_BCSUNIT (bit 13) Disable MI_SET_CONTEXT for execution list. That seems to be the closest thing I can find to establishing whether a context can be loaded via ELSP.

It also states that once you've written to the SQ via ELSP, you have to trigger the actual load by writing bit 0=1 in the EXECLIST_CONTROL_BCSUNIT (0x22550).

Do you have the format/layout of the descriptor?

[johnsa]

Ok .. I think I figured it out

what they mean by execlist enable, IS actually in the GFX_MODE register ..

0x2229c (GFX_MODE_BCSUNIT)
bit 3 - disable legacy mode (this effectively means enable execlist mode) - NB it's a masked register so you need to set the high word bit and the low word bit.

Before using execlists and BCS I think you need to ensure you've done a full BCS reset, then the setup looks like (based on Linux i915):

; 1. Enable ExecList mode (disable legacy mode)
; GFX_MODE uses masked writes: upper 16 bits = write mask
mov eax, (1 SHL (3 + 16)) OR (1 SHL 3) ; Enable bit 3
mov [rdi + GFX_MODE_BCSUNIT], eax

; 2. Disable STOP_RING in MI_MODE
mov eax, (1 SHL (8 + 16)) OR (0 SHL 8 ) ; Clear bit 8
mov [rdi + MI_MODE_BCSUNIT], eax

; 3. Set Hardware Status Page (need to find register)
mov eax, hwsp_ggtt_addr
mov [rdi + HWS_PGA_BCSUNIT], eax

So you need a 4kb page in the GGTT for the hardware status page.

HWS_PGA_BCSUNIT EQU 0x22080
MI_MODE_BCSUNIT EQU 0x2209c
GFX_MODE_BCSUNIT EQU 0x2229c

[CasualKyle]

It also states that once you've written to the SQ via ELSP, you have to trigger the actual load by writing bit 0=1 in the EXECLIST_CONTROL_BCSUNIT (0x22550).

Yes, I just haven't even gotten to the point where the ELSP is accepting my context descriptor. Writing the load bit appears to be the next step and what actually triggers the engine to start executing the commands.

Do you have the format/layout of the descriptor?

Yes, I've found that mentioned in two places. "Context Descriptor Format" section in the command stream programming document. And the "Context Descriptor Format" of the command reference structures document.

what they mean by execlist enable, IS actually in the GFX_MODE register ..

0x2229c (GFX_MODE_BCSUNIT)
bit 3 - disable legacy mode (this effectively means enable execlist mode) - NB it's a masked register so you need to set the high word bit and the low word bit.

Interesting! I wonder how you worked that out. It does seem to make sense because the GFX_MODE register description (at least for Tiger Lake) does mention the execlist as being "new".

This register contains a control bit for the new execlist and 2-level PPGTT functions.

So maybe by disabling legacy mode, we get the execlist enabled.

Before using execlists and BCS I think you need to ensure you've done a full BCS reset

Yeah the documentation for that bit in GFX_MODE does say that. Thanks for sharing the Linux example, I guess this will be my next step.

[johnsa]

Let us know how you get on I'm just finishing off a generic set of GTT allocators to setup the required buffers this will need,
1 page for the HW Status, 2 pages for the HW context I think and pages for the ring buffers, then i'll try write the context descriptor and see how it goes my end.

[Gigasoft]

I have a driver for Brookdale, a very old Intel GPU with no programming manual. It can do the following:
- Read EEPROM
- Read BIOS Data Blocks
- Set up resolution and display format
- Upload and display cursor
- Copy between surfaces of same type or from monochrome to color
- Fill with solid color
- Fill with pattern
- Draw with solid color and bit mask

I also have all the 3D stuff figured out, but not implemented yet.

[johnsa]

There is another completely undocumented register you're going to need..
It's value needs to be set in the Logical Ring Context packet you build up and point to with the LRCA field in the descriptor.

It's the Context Control register, and it's address is 0x22244.
The LRC registers are encoded as offsets from the unit base, so 0x22000 is the base for BCS, so you encode it as 0x244.

The bit definition for the register is:
PXP_ENABLE bit 10
OAC_CONTEXT_ENABLE bit 8
RUN_ALONE bit 7
INDIRECT_RING_STATE_ENABLE bit 4
INHIBIT_SYN_CTX_SWITCH bit 3
CTX_RESTORE_INHIBIT bit 0 <<<<<< This is the one you really want as the PRM refers to this all over the place.

So my understanding is that the LRC is made up of 2x4kb pages, the first page is the per-process hardware status page - you don't fill anything in to this, the GPU will. Page 2 is where you construct the hardware context as described in the command stream programming volume, this is 5 cachelines - so 320 bytes and described by the table in the PRM on page 49. After this comes the engine state, which we also don't touch, and is why we want the above bit, so the GPU doesn't load it when submitting the execlist, and would instead fill it in for us if/when we swap the context out.

What I'm not sure about yet, is if we can ignore the PPGTT entries - leave them all 0 as I'm trying really hard to avoid setting up the PPGTT - it's complete overkill at this stage and am hoping I can rely on just the GTT.

[johnsa]

It seems like the whole BCS/Blitter is somehow disabled, all the registers read back as 0 for me - is there some sort of master enable we're missing?

[eeeee]

Hello everyone!

The original topic was about VSYNC, and I think writing a custom driver isn't actually necessary. The tearing issue exists because our rendering program has no feedback from the monitor about when it draws a frame. Here's a (maybe silly) idea: use our eyes and hands to give feedback to the program manually. Basically:
1) Set up a timer to trigger an interrupt every 1/144 second.
2) When the interrupt fires, write the frame to the framebuffer as fast as possible (memcpy) if the frame is ready; otherwise, do nothing.

If tearing appears, it means our memcpy is somehow colliding with the monitor's refresh cycle. In that case, all we need to do is slightly adjust (shift/offset) the timer phase and try again until the tearing disappears.

[johnsa]

We have a working driver..

and when you say vsync, which pipe, which monitor, do we not want multiple screens, 60hz, 120hz, VRR, PSR?
refresh rate can vary depending on power management.

If you just want tear-free drawing on UEFI/GOP and that is the end of it, you could enable a vsync only driver on Pipe A and be done.
This is probably 200 lines of code at most - there is no way you can manually synchronise by sight to the vsync, if you're off by 0.01ms it will drift and you'll be constantly having to manually re-adjust.

I can share exactly how to enable the vsync IRQ (msi or msix) - which will work on top of the UEFI/GOP display.

[CasualKyle]

So my understanding is that the LRC is made up of 2x4kb pages, the first page is the per-process hardware status page - you don't fill anything in to this, the GPU will. Page 2 is where you construct the hardware context as described in the command stream programming volume, this is 5 cachelines - so 320 bytes and described by the table in the PRM on page 49. After this comes the engine state, which we also don't touch, and is why we want the above bit, so the GPU doesn't load it when submitting the execlist, and would instead fill it in for us if/when we swap the context out.

Yes, the sizes you found match what I found as well.

What I'm not sure about yet, is if we can ignore the PPGTT entries - leave them all 0 as I'm trying really hard to avoid setting up the PPGTT - it's complete overkill at this stage and am hoping I can rely on just the GTT.

I've been wondering this as well. Setting up the PPGTT seems like a pain but I haven't really read up on it in the docs yet. In the Context Descriptor Format documentation for the "Privilege Access" bit 8, it says if that field is set, then PPGTT is enabled. And below that it says it must be set so I guess we need to consider the PPGTT as enabled when we submit a context to ELSP. Whether or not "enabled" means we must use it instead of GTT? That I am not sure.

It seems like the whole BCS/Blitter is somehow disabled, all the registers read back as 0 for me - is there some sort of master enable we're missing?

Yeah this is what I was seeing as well, did you try the idea you had of enabling that bit which disabled legacy mode in GFX_MODE register?

The original topic was about VSYNC, and I think writing a custom driver isn't actually necessary.

I originally set out with this idea in mind. Then I read the command stream programming documentation and it explains the idea of submitting commands to the device well enough that I'm inspired to try and get a basic driver for 2D acceleration working for now.

[johnsa]

I think the entire GT register space is gated .. all registers < 0x40000 drop writes and read back 0 for me, mostly - they're just nonsense.

I've just tried re-creating all the domain forcewake, full BCS reset protocol .. forcewake times-out, due to the registers in the GT range not
doing anything useful.. I just spent the last 4 hours digging through all the Workarounds, specifically ones for Gen12 that are MCS and GT related and applying all those before trying to forcewake the render/gt domains ... nothing..

I'm no wondering if the GT simply doesn't work without the full HuC/GuC firmware blob load and microcontroller init.

These are the GEN12.x workaround/bug-fixes for GT:

Code: Select all

    ; Set MCR steering to slice 0, subslice 0
    ; MCR_SELECTOR format: bits [3:2] = slice, bits [1:0] = subslice
    xor eax, eax  ; 0 = slice 0, subslice 0
    mov [rdi + 0xfdc], eax  ; GEN8_MCR_SELECTOR 

    ;/* Wa_14011059788:tgl,rkl,adl-s,dg1,adl-p */
    mov eax, [rdi + 0x9550] ; GEN10_DFR_RATIO_EN_AND_CHICKEN
    or eax, (1 SHL 9)
    mov [rdi + 0x9550], eax
    
    ; 3. Wa_14015795083: MISCCPCTL - disable DOP_CLOCK_GATE_RENDER
    mov eax, [rdi + 0x9424]  ; GEN7_MISCCPCTL
    or eax, (1 SHL 1)  ; GEN12_DOP_CLOCK_GATE_RENDER_ENABLE
    mov [rdi + 0x9424], eax

Then there is the forcewake code.. which variers per generation - of course...

[johnsa]

Ok .. some progress, I've gotten the first ExecList to submit, GT registers now read back without returning 0's and I can see the values specified in the execlist in the registers!

The issue WAS the forcewake + BCS reset. The forcewake uses masked registers and posted reads, otherwise it doesn't work.
Also there are different ACK registers depending on the generation.

The problem now is - I don't know if the contents of the execlist are correct, and I don't seem to be able to get a 2nd execlist submit to do anything.
I setup the Context ID in both packets with SW Counter as 1, 2 and it stays stuck on 1. Not sure if a broken context in 1 causes 2 to never work.

It seems like you have to submit a new context every time you update the ring buffer, then in the context modify the ring.tail position.

The BCS reset and forcewake needs these registers (undocumented mostly):

Code: Select all

FORCEWAKE_GT        EQU 0xa188
FORCEWAKE_RENDER EQU 0xa278

FORCEWAKE_ACK_GT EQU 0x130044
FORCEWAKE_ACK_GT_MTL EQU 0xdfc       ; Use this one if your hw is GEN 12.7+
FORCEWAKE_ACK_RENDER EQU 0xd84

RESET_CTRL_BCSUNIT      EQU 0x220d0
GDRST                             EQU 0x0941c ; bit 2 = blitter, 1 to initiate domain reset, cleared by CP once reset complete.

For gen 12.5 there is a register you can read to get the GT version like 1250 1270 etc, if not, you have to fall back to a table of device ids.

The reset process involves writing RESET_CTRL_BCSUNIT first and waiting for the hw is ready to reset flag.
Then you write GDRST to triger the actual soft reset.

This can ONLY be done after forcewake.

Forcewake requires writing 0x00010001 (ie masked high 16 bits specify which low 16 bit values can change) to the FORCEWAKE registers
then you poll on the ACK register bit 0=1. This brings the GT domain online as it's offline after reset - what you're left with from UEFI/GOP etc.
You MUST read the FORCEWAKE register back immediately after writing it so it posts.

[CasualKyle]

Awesome, you've really figured a lot out! Thanks for continuing to share. I wish I wasn't so busy with work right now so I can get back into this stuff. Once I get some free time, I will be reading your posts very closely and try it on my end .