Playing audio through USB ADC?

[Ethin]

As a part of Google Summer of Code I'm working on an audio output protocol for UEFI. This is in an effort to standardize UEFI accessibility for people with disabilities like myself.
Originally, I was going to use the VirtIO sound device for audio, as a starting point, since its simple and appears easy to work with. However, I found out that QEMU does not yet actually implement the device; there is an RFC for it, but it is just a set of patches that have yet to be upstreamed. As such, I had a few options:

1. I could implement the device anyway and bank on the (very unlikely) possibility that a QEMU update is released just in time to demonstrate my implementation;
2. I could implement the Intel HDA specification; or
3. I could implement either the USB audio device class specification or the basic audio device definition. (One of my mentors says that there's a "generic USB audio device", but I can't seem to find any kind of reference to it other than a few generic descriptors.)

The first option wasn't very tenable. I'm required to demonstrate that my protocol works, since it'll be a driver and not an application, but I (obviously) can't test it if I can't even use the device in QEMU to begin with. The second option might've worked if it weren't for the fact that I'd need to implement separate drivers for each HDA codec I wanted to support, so my mentors and I ruled that out as more of a long-term project than anything else. We'll get there but I don't think I could do that in three months.
The final option was USB audio. I chose this for a few reasons:

1. Its already in QEMU and it works
2. The specification is well written (though quite large), and I don't need to implement all of the specification anyway -- you don't need a graphical equalizer, for example.
3. The specification is relatively easy to implement and UEFI makes USB pretty simple.

The problem is that I'm getting lost within the specification. The main ADC specification is massive and contains tons of descriptors and units, but the basic audio device definition is almost too small for me to actually figure out anything. I'm able to find the device on the USB bus and read its interface, device, and endpoint descriptors, and the USB I/O protocol provides functions like EFI_USB_IO_CONTROL_TRANSFER, EFI_USB_IO_BULK_TRANSFER, etc., so I (think) I can retrieve other descriptors for audio, but the difficulty I'm struggling with is knowing what descriptors I need to look for and then how to actually play something. I'm not looking for anything fancy; I don't need mixing or audio spacialization or anything like that.
Does anyone here know the process for doing this?

[rdos]

I don't think you need per-codec HDA support. I've tested my generic HDA on many different platforms, and it works without any kind of special code. What is a bit complex is to handle the deivce connections that differs between hardware configurations, but it is doable, and the components are the same (it's just the connections that differ).

I think USB audio is probably a lot more complex than HDA (I haven't done USB audio drivers). You both need to write the USB driver (and there are four different standards + Hubs), and the USB audio driver. I haven't done this yet, and I'm not sure that it is as simple as writing a single driver. USB networks at least are not. The USB CDC only specifies the basics. Same with USB printers.

[Ethin]

UEFI already has USB support, so I don't need to worry about that.

[rdos]

UEFI already has USB support, so I don't need to worry about that.

I suspect USB audio will use isochronous transfers, and are you sure that UEFI supports that? USB keyboards & mice use interrupt transfers, and USB discs use bulk transfers. Thus, UEFI really has no use for isochronous transfers with typical devices.

[Ethin]

Yes. The EFI_USB_IO_PROTOCOL supports control, interrupt, bulk, and isochronous transfers, with interrupt and isochronous transfers supporting both a synchronous mode and an asynchronous version via EFI_USB_IO_PROTOCOL.UsbControlTransfer(), EFI_USB_IO_PROTOCOL.UsbBulkTransfer(), EFI_USB_IO_PROTOCOL.UsbAsyncInterruptTransfer(), EFI_USB_IO_PROTOCOL.UsbSyncInterruptTransfer(), EFI_USB_IO_PROTOCOL.UsbIsochronousTransfer(), and EFI_USB_IO_PROTOCOL.UsbAsyncIsochronousTransfer().

[Ethin]

Bumping this topic to ask if anyone has any tips. I've gotten a hold of a packet capture of USB exchanges from someone who plugged in a USB audio device, and from what it tells me I have to initially do the following to figure out what it is:

1. Send a get device descriptor request to check device information and read the response
2. Send a get configuration descriptor request and read the response and, in particular, read wLength
3. Send another get configuration descriptor request containing the actual length of the descriptor and read the full descriptor
4. Send a set configuration request with bConfigurationValue set to 1 and wIndex and wLength to 0

After these packets I don't know what to do next. The system appears to send at least 3 requests with a URB transfer type of 0xFF, which Wireshark doesn't understand and claims is malformed. There then are a bunch of get string descriptor requests. Then a bunch of input control transfer noise. So I'm not really sure how to proceed. Naturally, the USB specifications are completely useless.

[nullplan]

Naturally, the USB specifications are completely useless.

I suspect they hold the answers. But anyway, the process you described is the normal part of enumerating a USB device. You turn on power to the port, then address the device, then configure the device. The config descriptor will contain interface descriptors and endpoint descriptors. I have no experience with USB audio devices myself, but I suspect there are going to be isochronous endpoints involved.

What you are supposed to do is find the right driver for the device, based on device class, interface class, or device and vendor IDs. If there are actual class values available, now is the time to look up the specifications for the class. Or find the data sheet of the device if it has no good class values. Or else reverse engineer how the device is supposed to work from a published driver, e.g. from Linux. If I were to design such hardware myself, I would probably have the host send commands on the default control pipe and exchange audio data on the isochronous endpoints. But that doesn't help you, since (a) this is speculation on my part, and (b) it doesn't tell you what those commands look like. So happy googling!

[Ethin]

Naturally, the USB specifications are completely useless.

I suspect they hold the answers. But anyway, the process you described is the normal part of enumerating a USB device. You turn on power to the port, then address the device, then configure the device. The config descriptor will contain interface descriptors and endpoint descriptors. I have no experience with USB audio devices myself, but I suspect there are going to be isochronous endpoints involved.

There are. The difficulty is getting to that point. For context, I'm writing a UEFI driver for USB audio, so this is why I'm asking. UEFI gives me the USB protocols to send control/bulk/interrupt/isochronous transfers, but I don't know if I can just read the descriptors and then throw audio at the device. And no, the USB specs don't hold the answers, so far as I can tell.

What you are supposed to do is find the right driver for the device, based on device class, interface class, or device and vendor IDs. If there are actual class values available, now is the time to look up the specifications for the class. Or find the data sheet of the device if it has no good class values. Or else reverse engineer how the device is supposed to work from a published driver, e.g. from Linux. If I were to design such hardware myself, I would probably have the host send commands on the default control pipe and exchange audio data on the isochronous endpoints. But that doesn't help you, since (a) this is speculation on my part, and (b) it doesn't tell you what those commands look like. So happy googling!

Again, I'm *writing* the driver. There is an ADC specification, but its quite brief on how the device is supposed to work.

[nullplan]

Again, I'm *writing* the driver. There is an ADC specification, but its quite brief on how the device is supposed to work.

I know you are writing a driver. I am suggesting you look up a working driver to figure out how this device is supposed to work. Unfortunately, not having a programmer's reference seems to be the norm. If there was a USB class for these kinds of device, there may have been a USB class specification available from the USB-IF. However, I have also seen a lot of devices where the class is just "vendor specific" and the vendor doesn't feel like writing programmer's references.

For example, the other day I was interested in how popular USB-serial converters work. I found out, however, the the Universal Serial Bus doesn't have a standard for USB-RS232 converters. So basically it comes down to the two main chip vendors: FTDI and Prolific. In both cases, the cable says the class is "vendor-specific", and, as far as I could find at least, Prolific actually doesn't publish programmer's references. They publish data sheets, but those only tell you what you need to develop the hardware around the chip, like the physical dimensions, electrical characteristics, and the functions of each pin, but when it comes to actually driving these chips, there's only deafening silence. So the only viable choice was to download the Linux source code (which admittedly I already had on my disk) and read the driver code. For FTDI the situation was a little better, because for some of their chips there are programmer's guides, but not for all of them, and for the FT4222 there is only an advert for their library.

If there is no programming guide, but at least an open-source driver, that may be bad, but is at least an order of magnitude better than not even having that. If there is no open-source driver, your only choices are to reverse-engineer the closed-source one, or to reverse-engineer the hardware itself by sniffing the USB traffic. But for that you have to have a lot more knowledge about USB. I don't have an explanation for the invalid URB types you are seeing. And anyone else here could only guess, unless they happen to have the exact same hardware and have already reverse-engineered it.

I just noticed, that you want to play audio through an ADC. Don't you need a DAC for that? I admit I am not an electrical engineer, so maybe I misunderstood something here.

[Ethin]

Yeah, I've been thinking about examining an audio driver, just not sure which one (the USB audio driver in Linux is a bit too much since it implements all 3 standards and I just want audio 1.0). And no, I don't mean ADC as in the converter, I mean ADC as in the audio device class of the USB specifications library.

[nullplan]

And no, I don't mean ADC as in the converter, I mean ADC as in the audio device class of the USB specifications library.

So, there is a standard device class, and you even know the name, and you say you didn't find the spec? Are you aware of this page? That would be the v1.0 spec of the USB AV class. It's a bit of a longer read, but it actually defines how USB AV devices are supposed to work. In case that wasn't what you meant, here is the Basic Audio Device Class specification. More is available with just a bit of googling.

In case of the Basic Audio Device, pretty much everything is clear. The audio device consists of a USB input, a mixer, a stereo or mono speaker, and optionally one or two aux inputs with a mixing device. The document defines the descriptor formats for getting all of that. The audio data is interleaved PCM, in a fixed format and sample rate. Biggest pitfall is probably that you need to activate AltSetting 1 on the streaming interface in order to turn it on.

[Ethin]

No, I have the specifications locally. I was just really confused on how to actually initialize the device to get things going. I've made some progress analysing the packet dump, so I (think) I know what to do. I'll take what I know about the packet dump and try following that. Perhaps I'll end up describing my process on the OSDev wiki in case anyone else goes for USB audio and gets confused on what exactly to do.

[nullplan]

No, I have the specifications locally. I was just really confused on how to actually initialize the device to get things going. I've made some progress analysing the packet dump, so I (think) I know what to do. I'll take what I know about the packet dump and try following that. Perhaps I'll end up describing my process on the OSDev wiki in case anyone else goes for USB audio and gets confused on what exactly to do.

If I understood the Basic Audio spec correctly, there is not a whole lot to initialize. Set the correct alternative interface, and then you just send your audio data to the isochronous endpoint. The audio data in that case is interleaved PCM, with parameters specified in the output speaker report, and no way to switch anything. The only other thing you could be doing is change mixer levels, but the Basic Audio spec makes it clear that the device should initialize those to "comfortable" levels at power-on.

[rdos]

The USB AV specification seems equal in complexity as the HID. Just like with HID, there is a legacy short-cut that boot devices might use. I'd say that the legacy variant is most suitable for this project. To be able to handle non-legacy USB AV device, there will be a need to parse XML, setup the hardware configuration and select the best configuration. This is quite similar to how HID descriptors must be parsed, and how the HDA Codecs must be configured to achive the paths you want. Thus, the complexity of the full-blown AV driver is similar to a writing a full-blown HID driver.

I think it would be fun & useful to write a driver for a USB AV camera, since this hardware is part of most laptops. For USB AV audio, things are different. Basically all laptops have their audio function implemented as HDA, not USB AV. Thus, I don't find it useful to write only a USB audio AV driver as then you only support external USB-video cams, which typical users don't have.

[Ethin]

The USB AV specification seems equal in complexity as the HID. Just like with HID, there is a legacy short-cut that boot devices might use. I'd say that the legacy variant is most suitable for this project. To be able to handle non-legacy USB AV device, there will be a need to parse XML, setup the hardware configuration and select the best configuration. This is quite similar to how HID descriptors must be parsed, and how the HDA Codecs must be configured to achive the paths you want. Thus, the complexity of the full-blown AV driver is similar to a writing a full-blown HID driver.

I think it would be fun & useful to write a driver for a USB AV camera, since this hardware is part of most laptops. For USB AV audio, things are different. Basically all laptops have their audio function implemented as HDA, not USB AV. Thus, I don't find it useful to write only a USB audio AV driver as then you only support external USB-video cams, which typical users don't have.

UEFI doesn't need camera support. This is for UEFI, not for an operating system. We're therefore not using the USB AV specification but the USB audio/basic audio specifications. AV support would be ridiculous. Nobody needs to access their video camera in a pre-boot environment.