[SOLVED] xHCI: Command Ring does not work on my laptop

[tokusan]

Hello. I created a USB keyboard driver and checked that it worked on QEMU. However, this driver does not work on my laptop.

I confirmed that the Event Ring works. It receives the Port Status Change TRBs. However, the Command Ring does not work. I sent a Noop TRB, but no Command Completion TRBs were issued to the Event Ring. Also, I sent an Enable Slot TRB, but could not receive the Command Completion TRB.

Command Ring is page-aligned. It does not cross the page boundary.

Code: https://github.com/toku-sa-n/ramen/tree ... l_hardware The name of branch is "fix_not_working_command_ring_on_real_hardware". xHCI codes are in kernel/src/device/pci/xhci. This code is incomplete because it does not handle the Port Status Change TRBs.

Am I missing something?

[nullplan]

Laptops often use PS/2 keyboards. Have you ascertained the nature of your keyboard?

[tokusan]

Laptops often use PS/2 keyboards. Have you ascertained the nature of your keyboard?

No, but I don't think this is related because the current problem is that the xHC does not respond to the Noop TRB.

[rdos]

Hello. I created a USB keyboard driver and checked that it worked on QEMU. However, this driver does not work on my laptop.

I confirmed that the Event Ring works. It receives the Port Status Change TRBs. However, the Command Ring does not work. I sent a Noop TRB, but no Command Completion TRBs were issued to the Event Ring. Also, I sent an Enable Slot TRB, but could not receive the Command Completion TRB.

Command Ring is page-aligned. It does not cross the page boundary.

Code: https://github.com/toku-sa-n/ramen/tree ... l_hardware The name of branch is "fix_not_working_command_ring_on_real_hardware". xHCI codes are in kernel/src/device/pci/xhci. This code is incomplete because it does not handle the Port Status Change TRBs.

Am I missing something?

I fail to understand how you could send Enable slot TRBs without having port connection logic. I think that the command ring needs to be enabled too, but the manual does go into it.

Also, I've yet not encountered a single laptop that has an internal USB keyboard. You need to connect an external USB keyboard to an XHCI port, and then you must know which port it is to be able to Enable a slot.

[tokusan]

I fail to understand how you could send Enable slot TRBs without having port connection logic. I think that the command ring needs to be enabled too, but the manual does go into it.

Also, I've yet not encountered a single laptop that has an internal USB keyboard. You need to connect an external USB keyboard to an XHCI port, and then you must know which port it is to be able to Enable a slot.

Command ring is enabled here: https://github.com/toku-sa-n/ramen/blob ... mod.rs#L76 and here: https://github.com/toku-sa-n/ramen/blob ... mod.rs#L24 These are done before running the xHC.

I haven't checked whether the keyboard of my laptop uses USB or PS/2. However, the problem is that xHC does not respond to the Noop TRB. My problem is rather the xHC problem than the keyboard driver one.

[rdos]

I fail to understand how you could send Enable slot TRBs without having port connection logic. I think that the command ring needs to be enabled too, but the manual does go into it.

Also, I've yet not encountered a single laptop that has an internal USB keyboard. You need to connect an external USB keyboard to an XHCI port, and then you must know which port it is to be able to Enable a slot.

Command ring is enabled here: https://github.com/toku-sa-n/ramen/blob ... mod.rs#L76 and here: https://github.com/toku-sa-n/ramen/blob ... mod.rs#L24 These are done before running the xHC.

I haven't checked whether the keyboard of my laptop uses USB or PS/2. However, the problem is that xHC does not respond to the Noop TRB. My problem is rather the xHC problem than the keyboard driver one.

Have you checked in the USB status register so the controller is running and if you have an event interrupt or not?

The CRCR contains information about the state of the command ring that you should dump.
'
You also need to ring the doorbell to execute things on the command ring.

[tokusan]

Have you checked in the USB status register so the controller is running and if you have an event interrupt or not?

Yes, after setting the "Run/Stop" bit in USB Command Register, I looped while the "HCHalted" bit in the register is 1. I don't use interrupts. I use polling.

The CRCR contains information about the state of the command ring that you should dump.

I confirmed that CRCR = 0x08 after issuing a Noop TRB.

You also need to ring the doorbell to execute things on the command ring.

Yes, I write 0 to doorbell[0] after issuing a Command TRB.

[tokusan]

I solved this problem partially. This was because of the "read-modify-write" cycle. When reading the CRCR register, some fields are always 0 regardless of the actual value. This caused the inconsistency of values.

Still, the problem exists on my desktop. I'll investigate this.

[rdos]

I solved this problem partially. This was because of the "read-modify-write" cycle. When reading the CRCR register, some fields are always 0 regardless of the actual value. This caused the inconsistency of values.

Still, the problem exists on my desktop. I'll investigate this.

Reading the XHCI document again since I'm updating my driver, and No Op is not allowed on the command ring, only on transfer rings.

[tokusan]

Reading the XHCI document again since I'm updating my driver, and No-Op is not allowed on the command ring, only on transfer rings.

Really? No-Op TRB for the Command Ring certainly exists. 4.6.2 and 6.4.3.1 of the xHCI manual revision 1.2 describes it.

[rdos]

Reading the XHCI document again since I'm updating my driver, and No-Op is not allowed on the command ring, only on transfer rings.

Really? No-Op TRB for the Command Ring certainly exists. 4.6.2 and 6.4.3.1 of the xHCI manual revision 1.2 describes it.

That's a bit strange, but table 6-86 says that it's not supported on the command ring, only on the transfer ring.

Regardless of that, every command should result in a response on the event ring, and so if the controller doesn't support No op it should still send a response back on the event ring that it is unsupported. If that doesn't happen, your command ring probably is misconfigured given that you get other events.

Another thing you might want to check is if the controller wants a scratch-pad area. Apparently, some do while others don't. If you ignore this it could mean the controller doesn't work. You also need to program a few other settings, like the config reg that your emulator might not care about.

Real-world controllers also don't keep their registers page-aligned, and both I'm testing with now have all their registers in a single page.

Some controllers also need you to perform "BIOS handoff", which is relatively easy to do.

[iman]

Do you acknowledge the interrupts properly?
Maybe there is still a pending interrupt.

Look at my working xHCI driver https://github.com/ImAbdollahzadeh/LiBO ... HCI.c#L862

[rdos]

Do you acknowledge the interrupts properly?
Maybe there is still a pending interrupt.

Look at my working xHCI driver https://github.com/ImAbdollahzadeh/LiBO ... HCI.c#L862

I don't think the interrupt handler needs to acknowledge anything. At least not if it is setup to use MSI or MSI-X, which both of my PCs support.

[rdos]

Another possible problem with XHCI is the size of the context entries that can be both 32 and 64 bytes. This is a bit problematic since these structure must be placed in a sequence and so it's not possible to always pad to 64 bytes.

XHCI doesn't need to support 64-bit addresses and if such a chip is placed in a system with more than 4G of memory, there is a need to copy to below 4G buffers. Still, both of my XHCI controllers support 64-bits, and so maybe this is not an issue.

An interesting feature is that XHCI supports multiple event rings & interrupters. Both my XHCI chips support 8 IRQs, but given that IRQs are a scarce resource, I decided to only use max 4. Then one can be used for the command ring and the others can be used for transfer rings for USB devices.

[tokusan]

Thanks for the replies and sorry for the late reply. I noticed that my Event Ring receives no TRBs on my desktop.

Reading the XHCI document again since I'm updating my driver, and No-Op is not allowed on the command ring, only on transfer rings.

Really? No-Op TRB for the Command Ring certainly exists. 4.6.2 and 6.4.3.1 of the xHCI manual revision 1.2 describes it.

That's a bit strange, but table 6-86 says that it's not supported on the command ring, only on the transfer ring.

Regardless of that, every command should result in a response on the event ring, and so if the controller doesn't support No op it should still send a response back on the event ring that it is unsupported. If that doesn't happen, your command ring probably is misconfigured given that you get other events.

Another thing you might want to check is if the controller wants a scratch-pad area. Apparently, some do while others don't. If you ignore this it could mean the controller doesn't work. You also need to program a few other settings, like the config reg that your emulator might not care about.

Real-world controllers also don't keep their registers page-aligned, and both I'm testing with now have all their registers in a single page.

Some controllers also need you to perform "BIOS handoff", which is relatively easy to do.

Table 6-86 is about the Link TRB. Could you quote the sentence which says the No-op TRB is supported only on the Transfer Ring?

Also, I did "BIOS handoff", but I'm not sure I did it correctly. xHCI dev manual says there is the USB Legacy Support Capability at xECP + 0x00, but this capability does not exist on QEMU.

Do you acknowledge the interrupts properly?
Maybe there is still a pending interrupt.

Look at my working xHCI driver https://github.com/ImAbdollahzadeh/LiBO ... HCI.c#L862

I don't use interruption for xHCI. I just poll the Event Ring.