Re: [9fans] ideas for helpful system io functions

[Nathaniel W Filardo <nwf@cs.jhu.edu>]

[-- Attachment #1: Type: text/plain, Size: 2478 bytes --]

On Mon, Dec 07, 2009 at 02:36:36PM +0000, roger peppe wrote:
> 2009/12/7 Sam Watkins <sam@nipl.net>:
> > I meant for example if a process is reading from its stdin a open file 'A' and
> > writing to stdout the input of a pipe 'B', rather than looping and forwarding
> > data it may simply "join" these two fds, and exit.  The OS will then do what is
> > necessary to make sure the data can travel from A to B (and/or vice versa) with
> > the minimum effort needed.
> 
> i'm not sure how you think this would work.

The pipe would have to be a bit smarter than Plan 9's pipes currently are,
or the attempts to join to a pipe would have to skip over the pipe and join
with the other descriptor.  It's certainly _possible_ to do, and AFAIK the
Linux guys do so with abandon. ;)
 
> a file descriptor is essentially a passive object - it responds
> to read, write, etc requests on it, but it doesn't do anything
> of its own accord.
> 
> if i do:
> 
> fd1 := open("/foo1", ORDWR);
> fd2 := open("/foo2", ORDWR);
> fd3 := fdjoin(fd1, fd2);
> 
> what is going to happen?
> something has got to initiate the requests to actually
> shift the data, and it's not clear which direction the
> data will flow.

"file to file" joins like that are not the typical case and might even be an
error to attempt.  Linux's equivalent APIs (yes, plural, sigh) always hook
an "active" component somewhere...  sendfile() for example is typically
employed as a crude hook on the TCP stack's "I could accept some bytes from
a write() from userland now" "event" and turn that into a read() of the
sendfile()d thing (which must be a pagecacheable thing... wtf.  splice()
fixes at least some or perhaps all of that).  splice()d file descriptors
just forward read()s and write()s across the splice.

> this is an optimisation, right? what parts of the current system
> could be speeded up by the use of this primitive?

A typical *nix use case is sending a prefix and static file to a socket
(e.g. nonencrypting, nonchunked httpds, ftpds, etc.).

Of note, more generally, splice() and friends are approximating something
possible and (relatively) easy in the capability kernel world: some process
has capabilities to two objects and wishes to introduce those objects to
each other (and further wishes that those objects would stop bothering it.
:) ).  i.e. "Please resend all outstanding and forward all future requests
to this other capability."

--nwf;

[-- Attachment #2: Type: application/pgp-signature, Size: 204 bytes --]