[Solved] Getting a USB Hub to work on xHCI

[foliagecanine]

Hello again, OS Developers!
I know xHCI is a much more complicated process of getting a hub to work, but that's my next step.
I've read the parts in Ben Lunt's book that talk about it, but it still isn't working.
It's failing at the Set Address command (see the bottom of the post).
Also, if I enable the SetDepth request, it causes the hub to hang and stop responding.

Here's what I'm doing:

• Activate hub like any other device
• Set the configuration: type=0,request=SetConfig,value=config_value,index=0,length=0
• Set the HUB bit in the slot context and send a Evaluate Context command where A=1
• Set the hub depth (on both USB2 and 3?). type=0x20,request=0xC,value=[routing value for device, same as slot ctx],index=0,length=0
  At this point it stops functioning, starts timing out or sending bad status code. If I don't do the set depth command, it continues.
  .
• Request hub descriptor: type=0xA0,request=GetDesc,value=0x2900,index=0,length=7 (This is only to get number of ports)
  For each port:
• Set feature PORTPOWER: type=0x23,request=SetFeature,value=8,index=port+1,length=0
• Clear feature PORTCNCTN: type=0x23,request=ClrFeature,value=0x10,index=port+1,length=0
• Check Status: type=0xA3,request=GetStatus,value=0,index=port+1,length=4
  Check if HUB_STATUS_CONNECTION bit is on. Break if not.
  .
• Set Feature RESET: type=0x23,request=SetFeature,value=4,index=port+1,length=0
  (Wait 50ms)
• Check Status: type=0xA3,request=GetStatus,value=0,index=port+1,length=4
  Check if HUB_STATUS_CONNECTION bit is on. Break if not.
  .
• Clear Feature CRESET: type=0x23,request=ClrFeature,value=0x14,index=port+1,length=0
  If this succeeds, a valid device is found. My code gets here as long as the SetDepth request is disabled.
  .
• Get Hub status to see if connected device is lowspeed or not: 0xA3,GetStatus,0,port+1,4
• Generate new routing value for the device
• Create a new slot (and context) for the device:
  .......place the routing value in the slot context
  .......set the RH port num to the parent's RH port num
• Send SetAddress. This always fails. It returns 0x5000000, or TRB error?

Code: Select all

For reference, here is the TRB. It doesn't look like there's anything wrong:
Param: 8CE000
Status: 0
Command: 5002C00
And here's the slot context I'm trying to enable:
08100000 00040000 00000000 00000000 00000000 00000000 00000000 00000000

Sorry this is such a huge list, but xHCI is complicated and I need to make sure I'm doing all the steps in the correct order.
Any help is appreciated

[BenLunt]

You are right, external hubs on the xHCI are quite a bit more complicated.

For example, with a regular device, such as a mouse, the hardware will use the USB 2 port register set for that socket, and leave the USB 3 port register set "empty". Just the opposite if you plug a super-speed device into the socket. Now the USB 3 port register set is used, and the USB 2 port register set is "empty".

However, here is where it gets interesting with hubs and the xHCI.

If you plug a USB 2 external hub into an xHCI socket, only the USB 2 port register set will be used. If you plug a USB 3 external hub into an xHCI socket, *both* register sets are now used. All Super-speed traffic is on the USB 3 port register set and all USB 2 traffic is on the USB 2 port register set. You now have two hubs attached, a Super-speed hub and a High-speed hub.

Read the note on page 21-2 of Edition 3 of my book for more on this subject.

Anyway, please give more details. Are you using an emulator or real hardware? If it is real hardware, is the hub a Super-speed hub, or a USB 2 hub?

Did you get the BOS descriptor. (BOS = Binary Device Object)

On page 21-12 of my book, there is a note about a Catch-22 when getting the HUB descriptor before setting the Configuration. Depending on the hub's speed, you must set the configuration before or after getting the HUB descriptor.

You need to set the hub depth *after* setting the configuration and *before* you get the hub descriptor.

Hope this helps,
Ben

[foliagecanine]

Hi.
In my emulator, I have 3 mice connected to USB2 ports 0-2 and a hub (with a mouse on its port 0) connected to port 3
I don't know whether its a USB 2 or 3 hub though.

You say that there's a Catch-22 with the config, but I don't see where you need the number of ports to configure it.

And do I need to get the BOS for even 2.0 devices connected to a 3.0 hub?

Thank you.

[BenLunt]

Hi.
In my emulator, I have 3 mice connected to USB2 ports 0-2 and a hub (with a mouse on its port 0) connected to port 3
I don't know whether its a USB 2 or 3 hub though.

Which emulator are you using? QEMU?

It has been a while, but last time I checked, QEMU only emulated the hub as a USB 2 hub, and did not allow any devices attached to it. Maybe they have improved the emulation since then.

What is you command line?

Ben

[foliagecanine]

Yes, I am using QEMU, but there is a (weird) way to get devices onto hubs.

There are 4 USB2 ports and 4 USB3 ports on the virtual xHCI root hub. When you add a device, it fills up a port on the root hub.
This would make it impossible to attach more than four devices, but QEMU fixes this.
At 4 devices, it actually creates a (I believe 4-port) hub which it connects the fourth device to. Then it starts adding devices to that hub.

Here is the command line for a successful enumeration of a hub (which only has 2 ports on UHCI) on UHCI

Code: Select all

qemu-system-i386 -m 512M -cdrom myos.iso -s -serial stdio -usb -device usb-mouse -device usb-mouse

The second usb-mouse will be added to a hub. This works entirely and I can address it and send it requests with no problems.

Here's my command line for xHCI

Code: Select all

qemu-system-i386 -m 512M -cdrom myos.iso -s -serial stdio -device qemu-xhci,id=xhci -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0

My code manages to reset exactly one port on the hub (indicating one device attatched), so I know it's working.

I have seen the -device usb-hub, but I messed around with it and could not get it to attach devices to it, so I went back to using the method explained above.

[linuxyne]

Code: Select all

qemu-system-x86_64 -m 512M -cdrom debian-10.6.0-amd64-netinst.iso -s -serial mon:stdio -device qemu-xhci,id=xhci -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0
QEMU 5.1.50 monitor - type 'help' for more information
(qemu) stop
(qemu) info usb
  Device 0.1, Port 1, Speed 480 Mb/s, Product QEMU USB Mouse
  Device 0.4, Port 2, Speed 480 Mb/s, Product QEMU USB Mouse
  Device 0.3, Port 3, Speed 480 Mb/s, Product QEMU USB Mouse
  Device 0.2, Port 4, Speed 12 Mb/s, Product QEMU USB Hub
  Device 0.5, Port 4.1, Speed 12 Mb/s, Product QEMU USB Mouse

After increasing the # of ports:

Code: Select all

qemu-system-x86_64 -m 512M -cdrom debian-10.6.0-amd64-netinst.iso -s -serial mon:stdio -device qemu-xhci,p2=8,p3=8,id=xhci -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0 -enable-kvm
QEMU 5.1.50 monitor - type 'help' for more information
(qemu) stop
(qemu) info usb
  Device 0.1, Port 1, Speed 480 Mb/s, Product QEMU USB Mouse
  Device 0.4, Port 2, Speed 480 Mb/s, Product QEMU USB Mouse
  Device 0.3, Port 3, Speed 480 Mb/s, Product QEMU USB Mouse
  Device 0.2, Port 4, Speed 480 Mb/s, Product QEMU USB Mouse
(qemu) 

With usb-hub, but unfortunately v1.1 only

Code: Select all

qemu-system-x86_64 -m 512M -cdrom debian-10.6.0-amd64-netinst.iso -s -serial mon:stdio -device qemu-xhci,p2=8,p3=8,id=xhci -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0 -device usb-mouse,bus=xhci.0 -device usb-hub,bus=xhci.0,port=4 -device usb-mouse,bus=xhci.0,port=4.4 -enable-kvm
QEMU 5.1.50 monitor - type 'help' for more information
(qemu) info usb
  Device 0.1, Port 1, Speed 480 Mb/s, Product QEMU USB Mouse
  Device 0.4, Port 2, Speed 480 Mb/s, Product QEMU USB Mouse
  Device 0.3, Port 3, Speed 480 Mb/s, Product QEMU USB Mouse
  Device 0.2, Port 4, Speed 12 Mb/s, Product QEMU USB Hub
  Device 0.5, Port 4.4, Speed 12 Mb/s, Product QEMU USB Mouse

The location, on the commandline, of usb-hub relative to other consumers of the bus affects the upstream port=## parameter for the hub device.

[1] https://www.kraxel.org/blog/2018/08/qemu-usb-tips/
[2] https://github.com/qemu/qemu/blob/master/docs/usb2.txt

[foliagecanine]

Wow, linuxyne, I had looked into it quite a bit but you seem to have found out how to use the usb-hub. Thank you!
New test command line:

Code: Select all

qemu-system-i386 -m 512M -cdrom myos.iso -s -serial stdio -device qemu-xhci,p2=1,p3=1,id=xhci -device usb-mouse

Still returns Error 0x5 in the SetAddress command though (although that was sort of expected).

I still do not understand the Catch-22. There does not appear to be any space in the setup packet for the number of ports. Am I missing something?

And I asked this earlier, but it sort of got buried, but do you need to get the BOS descriptor even if you are plugging in only full speed devices to the hub?

(I've just committed to the repository, so if you need to look at the code, it is located here. Specifically xhci.c and usbhub.c)

[linuxyne]

Still returns Error 0x5 in the SetAddress command though (although that was sort of expected).

The error seems to be because of missing route info in the route string field of the slot context. It seems the set address is sent for port 1 and not for port 1.1. The device resides at port 1.1.

Manually fixing qemu allows the guest to print messages:

Code: Select all

Found endpoint!
Configuring HID Mouse
Installed HID Mouse

I still do not understand the Catch-22. There does not appear to be any space in the setup packet for the number of ports. Am I missing something?

And I asked this earlier, but it sort of got buried, but do you need to get the BOS descriptor even if you are plugging in only full speed devices to the hub?

I can't answer either of them. Perhaps Ben and/or anybody else familiar with the details can help.

[foliagecanine]

I'm sorry, linuxyne, I don't understand what you are saying.
Is there a bug in QEMU or my code?

[linuxyne]

Apologies. "fixing" wasn't the most appropriate term.

qemu was modified to compensate for the missing routing info in your slot context. The root cause is most likely in the way your driver sets up the slot context.

Edit: You may want to check the loop which prepares the path inside xhci_lookup_uport, inside qemu.

Edit2:

In the first 32-bits, 0x8100000, the bits [3-0] (i.e. the least significant nibble) being 0 is what seems to be causing the failure.

Code: Select all

08100000 00040000 00000000 00000000 00000000 00000000 00000000 00000000

[BenLunt]

I still do not understand the Catch-22. There does not appear to be any space in the setup packet for the number of ports. Am I missing something?

The Catch-22 is mostly for USB 2 hubs, but you may not know if it is a USB 2 hub or a USB 3 hub at this point.

Part of configuring the hub is to retrieve the Hub Descriptor. However, the length of the Hub Descriptor is unknown until you get the count of ports on the hub. The count of ports on the Hub is only defined in the Hub Descriptor. The Catch-22 is that you need to know how many ports are on the Hub to know the length of the Hub descriptor.

On most devices it isn't really a big deal because you can handle it in a few different ways.
1a) request the first 8 bytes of the Hub descriptor, getting byte 2, indicating how many ports the Hub has, now calculating the actual length of the Hub Descriptor and requesting it again.
1b) request the first 8 bytes of the Hub descriptor, getting byte 0 which contains the actual length of the Hub Descriptor and requesting it again.
or
2) simply request 256 bytes, (possibly) receive a Short Packet, and then using Byte 2 of the returned data, determine how many more bytes are valid.

However, at least one of the USB 3 hubs that I tested with, Stalled if I requested more than the actual length of the Hub Descriptor, making option 2 above not work. All hubs I tested with, USB 2 and USB 3, all work with option 1 above.

do you need to get the BOS descriptor even if you are plugging in only full speed devices to the hub?

When I asked this, I didn't know if you were on real hardware or not, using USB 3 devices or not. The BOS Descriptor is a USB 3 descriptor and has sub-descriptors within.

Ben

[foliagecanine]

Thank you so much linuxyne!
It appears I forgot to actually pass the routing information to the function:

Code: Select all

void *xhci_init_device(usb_device *usbdev, xhci_controller *xc, uint32_t routing) {
...
}

bool xhci_init_port_dev(xhci_controller *xc, uint8_t port, uint32_t routing, uint8_t speed) {
...
xhci_init_device(usbdev,xc,0);
// Should have routing variable as last parameter
...
}

Well, now it works and the mouse installs like it should. I'll add USB3 support soon though.

The Catch-22 is that you need to know how many ports are on the Hub to know the length of the Hub descriptor.

Is there a reason it should be configured twice then?

Thank you everyone!