Different transfer type using uhci

[Bonfra]

I've implemented the basic USB functionalities with a uhci controller. Meaning I can enumerate the connected devices and query the various descriptors to get info about that device and pass it to the correct driver. This is the general function I use to initiate an in transfer:

Code: Select all

usb_transfer_status_t usb_transfer_in(const usb_bus_t* bus, uint64_t addr, uint64_t endpoint, void* setup, void* payload, size_t size)
{
    size_t num_packets = 2 + size / 8 + (size % 8 != 0);
    usb_packet_t packets[num_packets];
    memset(packets, 0, sizeof(usb_packet_t) * num_packets);

    packets[0].type = USB_PACKET_TYPE_SETUP;
    packets[0].maxlen = 8;
    packets[0].buffer = setup;

    int toggle = 1;
    int pid = 1;
    for(int i = 0; i < size; i += 8)
    {
        packets[pid].type = USB_PACKET_TYPE_IN;
        packets[pid].maxlen = 8;
        packets[pid].buffer = payload + i;
        packets[pid].toggle = toggle;
        toggle = !toggle;
        pid++;
    }

    packets[pid].type = USB_PACKET_TYPE_OUT;
    packets[pid].maxlen = 0x800;
    packets[pid].buffer = NULL;
    packets[pid].toggle = 1;

    return bus->hci.driver->transfer_packets(bus->hci.data, addr, endpoint, packets, num_packets);
}

From what I understand (so very little), these are sent as control packets. But how do I use a different protocol? say I need to send/receive some bulk stuff, how do I tell the controller "hey please take these packets and send them as bulk, not as control". I read in the uhci specs that a bit for isochronous transfer exists in the transfer descriptor, doesn't something like that but also for bulk (or other protocols) exist?

[nullplan]

Yes, what you've written there is a function that does control transfers. That's why it sends a setup packet, then IN data packets for the length of the usable data, and then an OUT packet for the status. You may want to generalize the function slightly to be able to do both input and output control transfers (in case of output control transfers, IN and OUT packets are reversed), but for the most part, this is it. The output direction may be needed for some class driver later. In standard USB, it is only needed for SET_DESCRIPTOR, which is used infrequently.

Interrupt and bulk transfers are different by the fact that the communication is unidirectional. So you only put in IN or OUT packets depending on endpoint direction. However, in case of interrupt IN endpoints, you typically want to have one function put in an IN transfer for the maximum packet size the endpoint supports, and another function to query if the transfer is done. That is because the transfer will only succeed when the device is sending you an interrupt. However, you can treat an interrupt OUT endpoint like a bulk endpoint. Whether you need to call the query function from a timer or if you have a dedicated interrupt provided by the HCI depends on the HCI.

Isochronous endpoints are the same as bulk on the USB (as in, you only send IN or OUT packets depending on endpoint direction). Here the difference is how they are handled in the USB controller. For an isoch OUT endpoint, the controller will send your packets out at a predetermined rate, for isoch IN it will receive those packets at a predetermined rate. These are typically used for A/V applications, so I won't beat around the bush: You need to set these up so that the device has little to buffer, but your OS can manage to fill or read the transferred data quickly enough, and there is little latency. The smaller you make the transfer buffer, the less latency you have, but the more often you must fill or read the buffer. If that happens too often, the application itself may not have the time to provide your kernel with the data it needs, and then you get issues with sound cutting out.

At your stage of development, I would leave isochronous transfers by the wayside until I have figured out how I want to handle audio/video applications. That topic is far from simple.

[Bonfra]

I surely agree with you about leaving aside isochronous transfers, it is a too advanced topic for my current knowledge.

Instead, about bulk and interrupt, I could recycle most of the code I use for the control transfers but I send the packets in one endpoint and read from another; correct? also, I don't need to send a setup packet for this type of transfer? can I just send an in/out packet on the out endpoint and expect something to appear in the in one?

I'm a bit confused about how I would implement this system... I mean I can't treat an endpoint as a pipe and read raw data from it. Or can I? I tried to stick to a non-interrupt approach for my driver at the moment so maybe I'm missing the knowledge that I would receive an interrupt as soon as some data is present on the endpoint(?).
But still, all I know how to do at the moment is to ask the device to fill a buffer with some data (like a descriptor). How would that work with this double endpoint approach?

Sorry for the large number of stupid questions but I'm having a really hard time finding the answers in the various specs :(

[BenLunt]

As you have probably found out, a device will identify how many endpoints (EPs) it has. For example, a thumb-drive, when using BBB protocol, will have a Control EP (one EP, both directions), a Bulk In EP and a Bulk Out EP.

Once you have enumerated the device (using the Control EP), you can now use the Bulk EPs to transfer "sectors". You send a 31-byte command packet to the Out EP, (optionally) receive or send one or more packets on the Bulk In or Bulk Out EPs (respectively), and finally receive a 13-byte packet on the Bulk In EP. This is done per the BBB protocol (or Bulk/Bulk/Bulk, aka Bulk-only transport protocol).

UASP is a bit more involved, but much faster, and in my opinion, once you have UASP figured out, it is a bit easier anyway. However, UASP isn't available on the UHCI and the device should not show this protocol capability when enumerated. On an EHCI and especially an xHCI, UASP is preferred. You tell the device (Control EP) which protocol to use.

Isn't USB fun? :-)

Ben
- https://www.fysnet.net/the_universal_serial_bus.htm

[Bonfra]

Isn't USB fun? :-)

Well yea coming from a really troubled journey trying to implement ahci in a decent way this is a piece of a cake :P

Anyway so it should be pretty easy: I send a cbw to the out endp and I receive data + csw on the emdp in. pretty straightforward as a pure concept. What I'm missing is still the implementation detail, to send the cbw I can just send it split into multiple IN packets I guess, but then how do I physically read from the other endpoint? I mean do I send also an address with the cbw or something? some pseudo code would be more clear than what the specs are trying to explain with just words :(

[BenLunt]

You do it the same way you do Control transfers. Create a Queue and enough TD's to transfer the data. However, this time there are no SETUP/STATUS packets, just DATA packets. Set the endpoint value in the TD to the IN or OUT EP found when enumerating the device. For example, a thumb-drive may have an EP of 0 for the Control EP (required), a value of 1 (0x81) for the IN EP, and a value of 2 (0x02) for OUT EP. Instead of setting the TD's EP to zero (Control EP), set it to 1 for the IN EP or 2 for the OUT EP.

Control Transfers are:
SETUP, zero or more DATA, STATUS

BULK Transfers are:
one or more DATA packets.

When sending the 31-byte Command Packet, imagine sending it as a Control transfer, but just without the SETUP and STATUS packets. Just a 31-byte DATA packet, now with a different TD.Endpoint value (as well as a few minor other differences). Also, take note of the 'toggle' bit. It must toggle each and every packet, with each EP having a separate toggle. It must be cleared after a Clear_Stall on that endpoint.

Ben

[Bonfra]

So i could use this pair of functions to send/receive the 31-byte data packet?

Code: Select all

usb_transfer_status_t usb_transfer_bulk_out(const usb_bus_t* bus, uint64_t addr, uint64_t endpoint, void* payload, size_t size)
{
    size_t num_packets = size / 31 + (size % 31 != 0);
    usb_packet_t packets[num_packets];
    memset(packets, 0, sizeof(usb_packet_t) * num_packets);

    int toggle = 1;
    int pid = 0;
    for(int i = 0; i < size; i += 31)
    {
        packets[pid].type = USB_PACKET_TYPE_OUT;
        packets[pid].maxlen = 31;
        packets[pid].buffer = payload + i;
        packets[pid].toggle = toggle;
        toggle = !toggle;
        pid++;
    }

    return bus->hci.driver->transfer_packets(bus->hci.data, addr, endpoint, packets, num_packets);
}

usb_transfer_status_t usb_transfer_bulk_in(const usb_bus_t* bus, uint64_t addr, uint64_t endpoint, void* payload, size_t size)
{
    size_t num_packets = size / 31 + (size % 31 != 0);
    usb_packet_t packets[num_packets];
    memset(packets, 0, sizeof(usb_packet_t) * num_packets);

    int toggle = 1;
    int pid = 0;
    for(int i = 0; i < size; i += 31)
    {
        packets[pid].type = USB_PACKET_TYPE_IN;
        packets[pid].maxlen = 31;
        packets[pid].buffer = payload + i;
        packets[pid].toggle = toggle;
        toggle = !toggle;
        pid++;
    }

    return bus->hci.driver->transfer_packets(bus->hci.data, addr, endpoint, packets, num_packets);
}

I tried playing around with these functions but I always get no significant result... this the function that uses these two functions for bulk (USB msd stuff):

Code: Select all

bool send_scsi_command(void* device, scsi_command_t* command, void* data, size_t transfer_length)
{
    usb_msd_device_t* dev = (usb_msd_device_t*)device;

    usb_msd_cbw_t cbw;
    cbw.tag = dev->tag++;
    cbw.signature = 0x43425355;

    cbw.transfer_length = transfer_length;
    cbw.flags = command->write ? CBW_DIRECTION_OUT : CBW_DIRECTION_IN;
    cbw.lun = 0;
    cbw.command_length = command->packet_len;
    memset(cbw.scsi_command, 0, 16);
    memcpy(cbw.scsi_command, command->packet, command->packet_len);

    usb_transfer_bulk_out(dev->device->bus, dev->device->addr, dev->endpoint_out->descriptor.endpoint_number, &cbw, sizeof(usb_msd_cbw_t));

    usb_msd_csw_t csw;
    usb_endpoint_descriptor_t* ep = command->write ? &dev->endpoint_out->descriptor : &dev->endpoint_in->descriptor;
    if(command->write)
        usb_transfer_bulk_out(dev->device->bus, dev->device->addr, ep->endpoint_number, data, transfer_length);
    else
        usb_transfer_bulk_in(dev->device->bus, dev->device->addr, ep->endpoint_number, data, transfer_length);
 
    if(csw.signature != 0x53425355)
        return false;

    if(csw.status != 0)
        return false;

    return true;
}

But maybe obviously the csw is not populated with anything meaningful (tried this with an inquiry command)

[BenLunt]

What is the difference between your usb_transfer_bulk_in and usb_transfer_bulk_out functions? A single declaration? If this is the case, try using only one function. Building a Transfer Descriptor, whether it be in or out, is (almost) identical, with only a few minor differences, easily passed in a flags parameter. Also, you should send the length (in this case, 31) as a parameter, though I get the idea you are simply 'experimenting' at the moment anyway.

Also, does your compiler return an error on:

Code: Select all

packets[pid].buffer = payload + i;

It should.

On an IN transfer, your transfer_packets() function is waiting for a response, correct? You have to wait for the 'bulk in' to actually take place. It can take a little while (depending on your stack frame, milliseconds in fact) before it happens. If the Queue for the CSW is in the same frame as the Queue for the CBW, the devices hasn't had time to respond before the controller executes the CSW. You need to wait for the CBW to be executed by the device before you send the CSW.

Since you are not using an interrupt interface, if your transfer_packets() function doesn't wait for the device to execute the Command phase, then the Response phase, your Status phase will be zeros. Just a thought, since there is no reference to your transfer_packets() function except for the call itself, it is waiting for the device to fill the result phase, correct?

In experimentation, wait 10mS or so before you retrieve the CSW. See if there is something there.

A few more notes. Your toggle needs to be aware of short packets. If you create, for example, 6 IN packets, each toggling with an ending toggle bit of 1 (so that the next new packet will have a toggle of 0), let's say that you short packet on the fifth packet. The device will expect the next packet to have a toggle of 1, but your code will give it a toggle of 0. The error in the toggle will invalidate any remaining packets until you clear_endpoint() or fix the toggle bit.

Also, you are assuming that the LUN is zero. This is the case for most devices, however some devices will have multiple LUN values starting with 1. Have you sent the Get_Max_LUN request yet?

Ben

[BenLunt]

After reading over my reply, it sounds a lot like I am 'directing' instead of 'helping'. Sorry it is not my intent to be so direct.

With a little sarcasm, I do actually mean "USB is fun". I spent years trying to get it to work, but every time I 'figured' something out, it was an enjoyable accomplishment. So please don't get frustrated. You will figure it out, and when you do, you too will think that "USB is fun", though you will be thinking it with a bit of sarcasm as well.

I definitely enjoyed it, all 735 pages worth. :-)

Ben

[Bonfra]

Yes all of your considerations about by horrible code are correct XD indeed it should be parameterized in only one function and yes gcc gives me a warning for void pointer arithmetic but it behaves like a uint8 pointer so 'till I'm in this prototyping state I'm writing this code like a stream of consciousness (a really messy one )

On an IN transfer, your transfer_packets() function is waiting for a response, correct?
[...]
If the Queue for the CSW is in the same frame as the Queue for the CBW, the devices hasn't had time to respond before the controller executes the CSW

Yes correct, the function returns only when the interrupt bit on the status register is set in the uhci controller and only the last td of the packets I send has the IOC bit set, plus the uhci schedule is executed always from the start so all packets should be processed in the correct order.
So given that every call of that function is an independent (and not thread-safe) unit of execution of the TDs in the framelist, when I send the CWS it should be granted that the CBW has already been processed.

Since you are not using an interrupt interface, if your transfer_packets() function doesn't wait for the device to execute the Command phase, then the Response phase, your Status phase will be zeros. Just a thought, since there is no reference to your transfer_packets() function except for the call itself, it is waiting for the device to fill the result phase, correct?

I'm not sure about how to answer to this, it surely waits that all the TDs are processed but I don't know how to also achieve this other thing.

In experimentation, wait 10mS or so before you retrieve the CSW. See if there is something there.

Nope, no luck

A few more notes. Your toggle needs to be aware of short packets. If you create, for example, 6 IN packets, each toggling with an ending toggle bit of 1 (so that the next new packet will have a toggle of 0), let's say that you short packet on the fifth packet. The device will expect the next packet to have a toggle of 1, but your code will give it a toggle of 0. The error in the toggle will invalidate any remaining packets until you clear_endpoint() or fix the toggle bit.

I'm kinda leaving error handling aside for the moment, being in an emulated environment i'm pretty much assuming no errors are happening hardware side, also executing the thing more than one should eliminate any risk of a bad result caused by casual error, theoretically

Also, you are assuming that the LUN is zero. This is the case for most devices, however some devices will have multiple LUN values starting with 1. Have you sent the Get_Max_LUN request yet?

Yup, I get a max LUN of 0, so only 1 LUNis present (index 0)

[Bonfra]

Answering your latest post, don't even worry about the way you are helping me! I'm extremely grateful to you and to your patience with my dumb questions and silly mistakes.
The kind of frustration I feel about this project is a positive one, the kind that leads me to go on figuring out more things and understanding better this now complicated system that (I hope) one day will have few secrets for me.

So thanks a lot and please feel free to answer in any manner you feel more appropriate based on the level of my "not understanding basic stuff"

[BenLunt]

Thank you for the kind words.

I think you said you have used Bochs to test your code. If so, can you set the debug messages to log all packets in and out? If so, can you post that log, or better yet, at least those packets?

Bochs should show you what packets are being received. If I see those packets, I can try to tell you if they are correctly built or not.

If you are not using Bochs, or cannot get it to print the packet data, can you add a 'dump' feature to your code to dump the bytes of the packet.

For example, in usb_transfer_bulk_out(), call usb_dump_packet(payload, size). Or better yet:

Code: Select all

for(int i = 0; i < size; i += 31)
    {
        packets[pid].type = USB_PACKET_TYPE_OUT;
        packets[pid].maxlen = 31;
        packets[pid].buffer = payload + i;
        packets[pid].toggle = toggle;
        toggle = !toggle;
        usb_dump_packet(packets[pid].buffer, packets[pid].maxlen);
        pid++;
    }

Have it dump it in the old DOS style DUMP app style:

Code: Select all

00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00-00 00 00 00 00 00 00

You can then look to see if the packet is correctly built. I can have a look and help determine so.

Ben

[Bonfra]

I'm using QEMU so no fancy Bochs output sorry, anyway here is the dump in the DOS style format:

Code: Select all

55 53 42 43 00 00 00 00-24 00 00 00 80 00 06 12
00 00 00 24 00 00 00 00-00 00 00 00 00 00 00

And with the help of some gdb magic here it is in a more human readable format:

Code: Select all

{signature = 0x43425355, tag = 0x0, transfer_length = 0x24, flags = 0x80, lun = 0x0, command_length = 0x6, scsi_command = {0x12, 0x0, 0x0, 0x0, 0x24, 0x0 <repeats 11 times>}}

[BenLunt]

I can confirm, that is a valid CBW packet for the Inquiry command.

Just a few questions, you may have answered already:
1) you can confirm that the Status byte in the TD has been modified by the controller, indicating that it has executed that TD?
2) You start the toggle bit at zero for each Bulk EP?
3) You have successfully addressed the device and are now using this device number in your TD(s).
4) You are setting the PID to OUT in the CBW TD?
5) The new Endpoint number is being placed in the TD? (not zero anymore)
6) The MaxLen field is now 64-1 instead of 8-1. The Endpoint Descriptor will indicate the MaxPacket Size, most likely 64.
7) The device is a full speed device, so you are setting the LowSpeedDevice bit accordingly?
8) can you confirm that the Status byte in the DATA IN TD has been modified? The TD that should receive the Inquiry data? It too should now use a MaxLen of 64-1, new DevAddr, PID=IN, etc.
9) The Toggle bit should be 0 for the CBW, 0 for the DATA IN, and 1 for the CSW.
10) You have sent the SetConfiguration command before attempting to read from the device?
11) The Interface subclass field is 6, BBB or Bulk-only protocol?
12) You have sent the SetInterface command?

Those might not be in order, but they are questions I have off the top of my head, to make sure you have addressed them as well.

Ben

[Bonfra]

so, brief list of answers:
1) yes
2) yes
3) yes
4) Yes
5) Yes
6) it was 32-1 but now that you made me notice, I've switched to 64-1 (all 31s in the above code are now 63s), sadly no changes
7) Yes
8) Yes
9) I need to double check that
10) Yes
11) Yes
12) No

rn here is a bit late in the night so I'll update (or make a new post) tomorrow to answer the missing question (9) and check what happens if i set the correct interface (idk why but i didn't thought that just setting the configuration was enough.. silly me)

also, may I push some of this non-working code on my GitHub repo to simplify this process so that you can take a better look at what you need without waiting for me to answer in a completely different timezone of yours XD

EDIT: In the end i went for a new post ;)