Re: usbd - revision (Was: [9fans] USB developments)

[Lucio De Re <lucio@proxima.alt.za>]

On Thu, Jan 15, 2004 at 06:01:41PM +0000, C H Forsyth wrote:
>
> PCI enumeration would probably become more complex
> for similar reasons when hot-swapping is possible.
>
Naturally.  But I agree with you that the kernel view of the USB
need only be flat, not hierarchical.

> i'll say a bit more about the original intention, and which parts i now dislike.
>
Thank you.  Every bit of information will help me understand the
subject better.  There is no substitute for experience.

> i observed that the linux and freebsd (and one other i can't mention)
> usb implementations were rather complex.  much of that was related
> to insisting on enumerating and configuring the devices in the kernel, and there was
> quite a bit of non-trivial state.
>
Device configuration in the kernel strikes me as absurd unless
you're sold, as Torvald is, on monolithics.  I reserve judgement
on enumeration.

> in particular, to enumerate, you typically build a tree of devices.
> if you're only enumerating in the kernel
> (ie, assigning IDs) then a user-level program
> will do configuration.  that needs to maintain a similar tree.

I'm assuming, possibly without justification, that the in-kernel
data structure is far simpler, but I'm listening...

> if a hub is removed, you need to delete chunks of substructure
> but if both kernel and usbd are keeping a tree, they'll update
> at different times (eg, kernel then usbd).  alternatively, they
> can exchange data to try to keep things in sync, but then there's the potential for races.
>
Let's see if our paths will cross.  Kernel enumerates based on
notification (an interrupt, perhaps?) that the structure has changed.
Am I reading the spec right that enumeration is the only way to
determine the structure and that it should occur frequently?  That's
certainly what it sounds like, I'm assuming that instead of polling
one can wait for interrupts, but whichever applies, the code is no
doubt in the existing driver code already as there is nothing optional
about it.

Now, the kernel driver reports, by writing to one of its files the
new address space, optionally adding the devices that have been
dropped, it seems to me one could get away with leaving them out
and letting userland figure out for itself what has changed, but
perhaps that's suboptimal.

Userland has the only copy of the tree structure that models the
previous state of the USB and now rewalks the USB creating an up
to date copy.  Userland and the kernel have only the flat list of
devices present in common, with the kernel copy specifically kept
to the bare essentials: index, ID, type, status.s currently
applicable.

> furthermore, for the purposes of IO through the kernel, the tree structure is
> irrelevant, since the ID space is actually flat and only endpoints within
> devices matter.
>
Hence my assumption that the kernel can have a very narrow view of
it.  Userland keeps a mapping between the kernel entries and the
tree.  It rebuilds it whenever it starts up or the kernel recommends
a rescan.  It may be useful to allow user requests too.

> it therefore seemed more straightforward to have the user process build the only
> copy of the tree (which really is needed for enumeration and hot-plugging/unplugging)
> and maintain it.  it then tells the kernel to enable devices.

I imagined that the actual userland enumeration operation would be
more of bus scan in which each device is examined and if it requires
enabling, configuring or initialising, it can be done based on the
reported status change.  As long, as you suggest, that race conditions
don't cause the kernel and userland daemon to get out of sync.
Using the kernel functionality would prevent such loss of
synchronisation, in my opinion.

> the kernel provides files it can read to see the status of the root hub.
> it monitors the state of individual hubs itself, using a process.
> (ie, usbd is the class/device-specific manager for hubs.)
> it's easy for usbd to use libthread, and do quite a bit of potentially
> messy error handling that again i thought would be painful in the kernel.
>
Yes.  It doesn't take much effort for usbd to notify the kernel
when something needs kernel intervention.

> a device-specific device managing process can do device-specific
> configuration including parsing complex structures like the HID,
> and it enables endpoints, within the given device directory.
>
Here my pet obsession makes me interject that it makes little
difference whether the device managing process is kernel or userland
resident.  At the risk of showing how little I know, I suggest that
as long as usbd and the device driver exchange information via
control files, the choice remains open.  But, yes, I see that things
are not quite this straightforward.

> the kernel needs to manage only structures for devices and endpoints
> (which believe me are complex enough), and it does that in response to
> instructions from user-mode programs.  usbd is in control.
>
Sounds OK to me.  The devil will no doubt be in the details.

> i was broadly happy with the kernel/user split, and
> the overall structure.  the kernel device is essentially
> an IO multiplexor and i think that's about right.  the kernel doesn't
> need to know about fancy hub trees or HID descriptors,
> and that also seems good to me.  (for mouse and keyboard there
> are or were special boot-time modes one could use before usbd could run.)
> (in fact, usbd could probably devise/check the USB IO schedule too.)
>
Enter Nemo.  I'll leave him to comment, I'm just a poorly informed
spectator.

> at the time i wrote it five or six years ago (!), i had trouble finding
> USB devices that kept to the spec--or even correct PIIXn implementations--
> and decided to do more urgent things.  that's one reason that i
> never supported iso mode (someone else did that and did a lot
> of work on the driver and usbd).
>
Complaints about spec conformance still abound, but it is purely
a matter of dealing with each quirk.  It's O(n) unless somebody
wants to produce a single driver for all devices.  Quirk descriptions
can bring it down to O(log n) if tracking all the different VID/DID
or serial numbers etc. becomes too painful.  Keep in mind that XP
deals with it, why shouldn't Plan 9?

> had i realised that i wouldn't really get back to it, i'd have made several
> more passes over it then.  for instance, when i first wrote it, qio.c didn't exist in Plan 9,
> and i used Stream Blocks instead.  i was never very happy with
> the amount of code in the driver overall, and it was ugly.
> i did have a quick go at putting Queues in properly but didn't complete it.
>
Noted, even though I understand very little of it :-)

> i was much less happy with the name space.   i think it was right
> that it's flat, an array of devices (rather than attempting to mimic the connection hierarchy
> which seemed an irrelevant physical detail).   the naming within
> the directories is fussy, though, and the use of physical

I wasn't going to say so, at least not until I was a lot more
comfortable, but I agree.  Could I ask you to put some thought into
that, I can do the implementation.  Like passing a string to the
kernel to do the prefixing or suffixing?  ep3data is a terrible
name for a mass-storage interrupt pipe and mv ep3data intrpipe
isn't likely to work, either.

> names (device/endpoint) to access logical things such as printer
> or ether is troublesome.  i thought of allowing a logical name to be created
> below usb, to refer to an endpoint or endpoints,
> but not having tried it, i don't know whether that would be better.

It sounds a lot more liveable with, if nothing else.

> sometimes a device presents one endpoint for reading and
> one for writing, but interfaces such as ethermedium expect
> a single name.   there are other peculiar cases.

They strike me as needing more of a policy than much hard implementation
effort, but you're the one with the T-shirt :-)

Based on the NetBSD philosophy, I can safely claim that #U is
optional and therefore it can be bloated, at least while still
under development.  I would prefer not to have to list all the
possible devices at kernel compile time (can you say distribution?)
which weighs seriously in favour of userland drivers, but I still
believe that brocee's work should be reconsidered.  Specially if
it then involves a merger of technologies rather than a switch of
philosophy.

Thanks, Charles.  I wish you had more time to spend explaining #U
to me (usbd I can grasp, I think), you and whoever added that
seeming extremely complex paradigm of isochronous transfer.  This
last message surely covered a lot of ground, but also exposed a
glimpse of the real complexity lurking.

++L