discussion: general fix for 9p status files which don't accept multiple readers

discussion: general fix for 9p status files which don't accept multiple readers

[Silas McCroskey]

In my time with 9front I've found a few fileservers which expose a
"status" file to which a reader can listen for updates from the
fileserver. Some concrete examples are /dev/window, /dev/mouse,
/dev/kbd, /dev/tablet, etc. The reader maybe gets an initial message
about the current state, and then blocks until the state changes, at
which point it gets another message. Really simple, easy-to-use
interface, which is great.

I've noticed a common theme among a lot of these is that they either
poorly handle or outright disallow the case of multiple concurrent
readers, as handling this rare case quickly becomes fraught with the
usual perils of concurrency and locking. I've hit this issue myself in
at least one fileserver I've written on my own. As in practice it
seems very few have hit these problems, it seems like the authors did
make the right call re: simplicity vs. completeness.

That said, as this is a commonly-recurring problem, I do wonder about
the ability to solve this generally without heavy complication of the
fileservers in question. A similar problem with /dev/audio was solved
in a pretty slick way by mixfs -- mixfs opens /dev/audio for writing,
binds over it, then accepts multiple writers and combines their
streams into one. (Side note: this is a fun example of why plan9 is an
awesome ecosystem: I recognized this problem, came up with this exact
same design, then asked if it already existed. Someone else (cinap,
iirc) had gone through the exact same steps (plus implementation), so
it did!).

I feel like a similar approach could be readily applied to these
status files (in the opposite direction), and wrote a little toy fs
which could be used for doing that sort of thing in simple cases (note
this isn't heavily tested, likely to have memory leaks or locking
problems): http://sam-d.org/src/thumbs/9pmultipipe.c.

I see a few approaches to solving this "for real":

1. Client fileserver calls a 9p library: one could conceivably write
functions which take in an arbitrary Req pointer and handles all the
reqqueue/lock/etc. bookkeeping. The problem I remember hitting with
this approach was that I couldn't figure out how to do it without at
least partially taking over the r->fid->aux struct, which either leads
to the client not being able to use it, or to the library needing to
jump through hoops to figure out how to access its needed fields. I
suspect it also wouldn't work with fileservers that don't use lib9p,
like rio, but I could be wrong. Maybe someone knows how to better
solve these problems, and this is the right approach.

2. A separate fileserver 'multireadfs', which binds itself over (and
is the sole reader of) the status file: this seems like it would work
well, but is limited. First of all, it puts the onus on the client to
always run this immediately after any FS with this problem. Secondly,
that first problem would be horribly exacerbated by cases like
/dev/wsys/N/window, where new status files are created during normal
operation.

3. Client fileserver starts multireadfs itself and delegates reads to
it in the background: Rio could potentially start up a new multireadfs
in its namespace for each /dev/wsys/N/window, and route read requests
and send status messages through that appropriately. This handles the
status-files-on-demand problem discussed above since it puts the
client back in control, but also leaves a lot of the complication on
the client (e.g. it still needs to handle its own reqqueues since its
read requests to multireadfs will block). It'd also likely cause
problems for fileservers not using lib9p, again. I had trouble getting
this approach off the ground to test viability, but that's likely
because I'm not as good at C and lib9p as I'd like to be.

4. Extending multireadfs to allow the creation of multiple nodes and
accept signalling from the client fileserver that a new node needs to
be created: the biggest problem with (2) imo is the fact that it can't
cover cases like /dev/wsys/N/window, but this seems easily remedied.
If rio can be configured to notify multireadfs of a new status file
path when it's created, multireadfs can create that node, and union
mount rules will have it bound over rio's file to intercept any
opens/reads headed there.

(4) seems like the best path to me, but it only occurred to me
recently, so I haven't fleshed it out or tried anything yet.
Theoretically the only change required to the client fs (e.g. rio) is
the ability to signal multireadfs when a new node needs to be created.
It already occurs to me, for example, that the ordering would have to
be somewhat complicated to avoid races... it'd have to be something
like (create multireadfs node (reading nothing) -> create status node
-> signal to multireadfs that the status file now exists and can be
read).

Curious what others think about this problem or any of these
suggestions. Hopefully I didn't just write up this text dump for the
benefit of no one other than myself!

- sam-d

Re: discussion: general fix for 9p status files which don't accept multiple readers

[Silas McCroskey]

> http://sam-d.org/src/thumbs/9pmultipipe.c

linked to the wrong file, http://sam-d.org/src/thumbs/multipipe.c is
correct. 9pmultipipe.c is my attempt at the library approach I
discussed in (1).

-Silas