Re: [developer] USB driver design question

[Robert Mustacchi <rm@fingolfin.org>]

Hi Carsten,

On 3/1/24 13:10, Carsten Grzemba via illumos-developer wrote:
> Some time ago I made the urtwn driver available for review. 
> https://code.illumos.org/c/illumos-gate/+/2383. 
> Originally the driver is developed by Masayuki Murayama published on http://freenicdrivers.la.coocan.jp. 
> With this I was able to use the Edimax wifi Stick N150 under Illumos. Now I wanted to update the whole thing again. 
> So I thought to myself, buy another Edimax N150, test everything and update everything again for the Gerrit review. I ordered an Edimax N150 and got an Edimax N150 V2. I didn't think anything bad with V2 but the test showed that the Edimax N150 that worked with the urtwn had an RTL8192C chip, the Edimax N150 V2 has an RTL8188E chip. I compared the urtwn drive with its BSD  ancestors and found that the RTL8188E requires major adjustments. 
> I have integrated the adjustments into the urtwn driver followed the example of the BSD driver, the driver is linked to the hardware and everything looks good so far. 
> However, when I call 'dladm scan-wifi', the WIFI interface doesn't find any WIFI networks, but there are no errors either. When I compare the same thing with the older WIFI USB Edimax N150, I see that the uwgem_bulkin_cb function is called.
> The uwgem_bulkin_cb function is provided with usb_pipe_bulk_xfer(). The question for me now is why uwgem_bulkin_cb is not executed from the RTL8188E hardware. 
> I see 3 possible reasons: 
> 1. The initialization of the USB framework is not correct, so uwgem_bulkin_cb is not called. 
> 2. The interrupt that triggers the call is not initialized correctly and will not call the call back function
> 3. The RTL8188E hardware is not initialized correctly, so no interrupt is triggered. 
> The question for me now is, how do I analyze the USB framework, whether the call back function is configured correctly using the usb_pipe_blk_xfer function and which interrupt should trigger the callback function and whether it is configured correctly?
> Can anyone give me some hints to analyze the USB framework settings?

So let me try to give some suggestions here of where to look. Without
datasheets, it's going to be hard to be definitive, but before that let
me go into some background of how USB works and how this all fits
together, because interrupts don't quite work the same way as they do on
other devices. If this is stuff that you're already familiar with, then
please just skip ahead and consider it something that'll hopefully help
different folks (there is more here in the theory statement for the xhci
controller -- uts/common/io/usb/hcd/xhci/xhci.c).

A USB device is made up of different kinds of endpoints. A device's USB
descriptor describes the kinds of endpoints that exist. Endpoints are
broken into four broad classes:

1) Control endpoints
2) Bulk endpoints
3) Interrupt endpoints
4) Isochronous endpoints

These types of endpoints describer the nature of the data that flows in
and out of them. In addition, each of these (aside from control) has an
associated direction that data flows in. That is a pipe is specifically
a combination of a type and direction.

Control endpoints are for misc. control of the device and every USB
device has a default control endpoint that always exists. Bulk endpoints
are used for transferring a chunk of data in a given direction. For
example, USB storage devices use these for reads and writes. Several
USB->Ethernet devices use this for transmitting and receiving packets.

Interrupt endpoints are used for small amounts of data. The Interrupt-IN
endpoint can be used for a polled mode. However, there are one-shot
transfers as well. Note while these are called interrupts and they are
used for the reasons that a device might generate an interrupt for
things like status events (a USB hub generates an event on this pipe
whenever a port has a device physically plugged and unplugged from it),
they aren't related to interrupts on say PCIe devices and I/O completes
on other pipes regardless of anything on these.

Isochronous are the final type, and are endpoints that require some kind
of periodic transfer to happen. A good example here are audio devices.
We can ignore these for the most part for this kind of driver.

A USB's device descriptor and class descriptors describe the kinds of
endpoints that exist in it and which of the different pipe slots
correspond to what kind of endpoint. Despite the fact that the pipe has
directions, which say which way data flows, the host must always
initiate the request. That is, the USB device cannot initiate a request
for bulk-in data, the host must and for a NIC that'll probably be
outstanding until there is data available for it.

Behind the scenes, the USB host controller driver (e.g. xhci or ehci)
will receive an interrupt when the request is transferred by the device
based on the ring configuration (using xhci here). So that interrupt
that xhci uses is really not something that is controlled by the end
device, it's about when I/O occurs so it's not really something that I
think of as directly impacted there absent some deeper bug in the xhci
driver (which while always possible, is not where I'd start). Put
differently, if the device generated data on that pipe, your callback
would be called. If it doesn't, it may not be doing so.

I haven't looked at this code and maybe can try to (though it'll be a
bit especially if there aren't docs available), but I would expect that
this bulk-in endpoint is where data is expected to be received and the
reason that nothing calls the callback is that the device isn't in the
condition where such data would be generated.

This to me suggests that this is most likely in option (3) of the above.
So here are questions for you to answer that I think I would expect to help:

1) What does this bulk-in data pipe represent? A device can have more
than one set and it's often the case that control interfaces are also
done over bulk endpoints as well, so does this represent packets coming
in, responses to commands, something else?

2) What are the conditions under which the device would send data to
that pipe?

3) What is the internal state of the chip? What registers does it expose
and what are its values? Is the antenna on?

4) How does the scan-wifi command get translated into a command for the
device? What pipes does it use? How would you expect this to flow if it
was all working?

I hope this helps a little bit. I realize it can be challenging to
answer without the programmer's manuals, but this is what I'd start
with. While there are deeper things one can do like trying to get a USB
protocol analyzer, I don't think that'll be required per se and I've not
used one to write USB drivers or the xhci driver (though it would have
certainly helped there).

Robert