[Solved] xHCI Interrupt Troubles

[iloveosdev]

here it is:

Note, the “last processed Event TRB” includes the case where software
detects a Cycle bit mismatch when evaluating an Event TRB and the ring is empty.

[foliagecanine]

@iloveosdev Thank you for your help.
About the picture:
Do you mean Figure 4-11 or 4-7? 4-7 is referring to non-segmented rings.
Fig. 4-11 however does make it look like the DQ pointer points to the next executable TRB. It even shows the internal Enqueue pointer.

I saw a state machine for the Enqueue pointer in the spec, but I can't find one for the Dequeue pointer. Did I just miss it?

About the quote:
Ah. So its just a terminology misunderstanding. So the "last processed TRB" includes the one where the cycle bit doesn't match.

[iloveosdev]

Fig. 4-11 however does make it look like the DQ pointer points to the next executable TRB.

figure 4.11, yes.

Yes, when the eq pointer is where it is, then the dq pointer should be moved all the way over there in one turn (that is dq_pointer = dq_pointer + 16*x where x is bigger than 1 in this case, or dq_pointer = eq_pointer) after having processed all the data in between (of course that depends on how your driver does that). From what i see when looking at figure 4.11, , the eq pointer is at a trb with a cycle bit mismatch and that trb can have no data in it except the cycle bit (which can either be 1 or 0. i guess it's 0 in this case).

So, regarding figure 4.11, which is what I meant, and from how I see it (figure 4.11), this is a state where the producer (the Xhc) has written event rings and has pushed forward the eq pointer. The dq pointer is at a state just before software will advance it to the same spot as the eq pointer.

[BenLunt]

By the way Ben, I don't think your GDevDesc xHCI program works on QEMU:

Code: Select all

qemu-system-i386 USB/freedos_hd.img -device qemu-xhci,id=xhci -device usb-mouse,bus=xhci.0

Code: Select all

GD_xHCI -- xHCI: Get Device Descriptor.   v1.10.20
Forever Young Software        (C)opyright 1984-2016

PCI: Found a USB controller entry: Bus = 0, device = 4, function = 0
  Found xHCI controller at 0xFEBB0000
C:\>

I will have to have a look and see. I am sure it did at the time I tested it, but things change.

Thanks,
Ben

[BenLunt]

By the way Ben, I don't think your GDevDesc xHCI program works on QEMU:

Code: Select all

qemu-system-i386 USB/freedos_hd.img -device qemu-xhci,id=xhci -device usb-mouse,bus=xhci.0

Code: Select all

GD_xHCI -- xHCI: Get Device Descriptor.   v1.10.20
Forever Young Software        (C)opyright 1984-2016

PCI: Found a USB controller entry: Bus = 0, device = 4, function = 0
  Found xHCI controller at 0xFEBB0000
C:\>

I will have to have a look and see. I am sure it did at the time I tested it, but things change.

Thanks,
Ben

Hi,

I remember now why I didn't mess with xHCI and QEMU. QEMU doesn't allow non-dword aligned mem-mapped reads (or writes) to the capabilities/operational registers or to the extended capabilities area.

For example, reading the controller Version Register in the Capabilities Registers, requires a dword read from offset 0x00000000 and then a shift by 16 and then a (redundant) AND 0xFFFF. Real hardware allows you to read a WORD from offset 2 to get the version.

Real Hardware:

Code: Select all

#define xHC_CAPS_CapLength      0x00
#define xHC_CAPS_IVersion       0x02

size_t bar = base_address_of_capabilities_registers;
version = * (bit16u *) (bar + xHC_CAPS_IVersion);

QEMU:

Code: Select all

#define xHC_CAPS_CapLength      0x00
#define xHC_CAPS_IVersion       0x02

size_t bar = base_address_of_capabilities_registers;
version = * (bit32u *) (bar + xHC_CAPS_CapLength);
version = (version >> 16) & 0xFFFF;

Second, when reading from the Extended Capabilities area (which is in the mem-mapped area of the xHCI, not the PCI config space), you need to read bytes and words to parse the port protocols. QEMU returns zero's for all non-dword aligned reads. The demo code you speak of does not pair up the ports correctly because of this.

My OS code has a work-around for this by reading in all of the Extended Capabilities registers as dwords into a memory buffer, then parsing the buffer using byte and word accesses. The demo code you speak of does not.

This is really something every newbie here needs to know about QEMU.

For example, do the following:

Code: Select all

#pragma optimize( "", off )  // see note below
  struct FOO {
    bit8u offset;
    bit8u resv;
    bit16u version;
  };

  struct FOO *bar = (struct FOO *) BASE_ADDRESS_RETURNED_BY_XHCI_BAR;
  printf("  %02X  %04X\n", bar->offset, bar->version);

Note that it prints the correct offset value (0x40 in QEMU), but the version is 0000.

However, now do the following:

Code: Select all

  printf("  %04X\n", (* (bit32u *) &bar->offset) >> 16);

The version will now be printed correctly.

PLEASE NOTE: For this example to work, you have to have optimizations off. If not, the above code will be optimized to:

Code: Select all

  // this line of code:
  printf("  %04X\n", (* (bit32u *) &bar->offset) >> 16);
  // will be optimized to:
  printf("  %04X\n", (* (bit16u *) &bar->offset + 2);
  // and since it is reading a non-dword aligned value, it will still print zero.

Therefore, to get around this, you must always read a dword-aligned dword from the XHCI registers when using QEMU. ALWAYS.

Ben

P.S. Don't get me wrong, I think QEMU is an excellent emulator and use it often. However, if you have been using emulators and have been programming hardware as long as I have, you tend to find errors in these things. I have commented this error to the QEMU maintainers, but since Linux works as is, they didn't have any desire to make my corrections. Oh well, nothing is perfect, especially me.

[foliagecanine]

Yeah. I saw this problem, but since I can't get USB to work in bochs to test, I just assumed that it was the norm.
I use this code for any access to controller registers:

Code: Select all

// Force a 64 bit read
inline volatile uint64_t _rd64(volatile void *mem) { 
	return *(volatile uint64_t *)mem; }

// Force a 32 bit read
inline volatile uint32_t _rd32(volatile void *mem) { 
	return *(volatile uint32_t *)mem; }

// Force a 16 bit read
inline volatile uint16_t _rd16(volatile void *mem) { 
	return (uint16_t)((*(volatile uint32_t *)(void *)((uint32_t)mem&~3))>>(((uint32_t)mem&3)*8)); }

// Force an 8 bit read
inline volatile uint8_t _rd8(volatile void *mem) { 
	return (uint8_t)((*(volatile uint32_t *)(void *)((uint32_t)mem&~3))>>(((uint32_t)mem&3)*8)); }

// Force a 32 bit write
inline void _wr32(void *mem, uint32_t b) { 
	__asm__ __volatile__("movl %%eax, %0":"=m"(*mem):"a"(b):"memory"); }

// Emulate a 64 bit write
inline void _wr64(void *mem, uint64_t b, xhci_controller *xc) {
	_wr32(mem,(uint32_t)b);
	if (xc->params&1)
		_wr32(mem+4,(uint32_t)(b>>32)); }

It seems to work in all the emulators I've tried so far (QEMU, VMWare, and I think Virtualbox).
The only ones you might need to modify are the _wr32 to work with DJGPP assembly and the _wr64 function to use the hccparams1 variable.
Unfortunately I haven't gotten around to getting DJGPP set up in DOS to test my fixes to GDevDesc.

[Octocontrabass]

I remember now why I didn't mess with xHCI and QEMU. QEMU doesn't allow non-dword aligned mem-mapped reads (or writes) to the capabilities/operational registers or to the extended capabilities area.

The xHCI specification says this about the operational registers:

Unless otherwise stated, all registers should be accessed as a 32-bit width on reads with an appropriate software mask, if needed. A software read/modify/write mechanism should be invoked for partial writes.

So, QEMU is correct about those at least.

[BenLunt]

I remember now why I didn't mess with xHCI and QEMU. QEMU doesn't allow non-dword aligned mem-mapped reads (or writes) to the capabilities/operational registers or to the extended capabilities area.

The xHCI specification says this about the operational registers:

Unless otherwise stated, all registers should be accessed as a 32-bit width on reads with an appropriate software mask, if needed. A software read/modify/write mechanism should be invoked for partial writes.

That is only for the Operational Registers. xHCI specification, version 1.0, section 5.4. "Host Controller Operational Registers"

It states nothing like this for the Capability Registers or the Extended Capability space, which is the two sets of registers I reference in this thread. All of the Operational Registers (which are a part of that quote you mention) are 32-bit (or 64-bit) on a DWORD boundary anyway, so I have no issue here. It is the other two register sets that have registers that are not DWORD aligned. This is where the problem with QEMU arises.

So, QEMU is correct about those at least.

Therefore, I have to disagree with you on that statement.

Edit: Octocontrabass: Sorry. I was reading through some older posts, re-read your comment, and realized you meant that QEMU gets it correct for the Operational Registers at least. I took your comment wrong. Sorry about that. You are actually correct in that comment. Ben

Ben

[BenLunt]

By the way Ben, I don't think your GDevDesc xHCI program works on QEMU:

I will have to have a look and see. I am sure it did at the time I tested it, but things change.
Ben

Okay, I did a few modifications to get the code to work for QEMU.

The code was actually correct, no fixing was necessary. The code worked on real hardware as well as Bochs and VirtualBox. However, (IMHO) QEMU has a few oddities/bugs that I believe need to be fixed.

First, and as explained earlier in this thread, you should be able to read bytes and words from the Capability Registers. QEMU does not allow this.

Second, another issue is actually how QEMU incorrectly assumes there will always be a STATUS TRB after a SETUP/IN TRB transfer. I have reported this issue, but unfortunately, it will take quite a bit of re-writting of the QEMU xHCI emulation to get it fixed.
See the comments and code starting at Line 1033 in my new code for an explanation.

Anyway, the updated code is now posted. I did a few other (non-function changing) modifications, as well as updated a little bit of the code.

Thank you for letting me know,
Ben
- http://www.fysnet.net/the_universal_serial_bus.htm