Re: [9fans] ideas for helpful system io functions

[Sam Watkins <sam@nipl.net>]

On Sat, Dec 05, 2009 at 12:59:34PM -0800, Bakul Shah wrote:
> You cut out the bit about buffering where I explained what I meant.

Your idea seems good, so long as the OS buffers data and keeps it around until
all readers have consumed it there would be no problem.  This would be another
possible solution to my problem, you could fork the fd before reading the http
headers, read the headers on one fd, find how long they are, and seek forward
over the exact length of the headers in the other fd before execing the script.

The only problem would be that the OS might be required to keep an arbitrarily
large buffer for the fd, if one forked fd reads a long way ahead, but the other
stays still.

I do think it is a good idea to have shared pipes / input streams though, there
are many cases where two or more processes need to read the same input; and
it's inefficient to have multiple pipes for this purpose, when they could
easily share a single buffered "multi-pipe".  Perhaps a limitation could be
that if a process tries to read too far ahead from the other processes, it may
block.  This limit might be configurable as the "pipe size".  An httpd
application might (should) reject requests with over-large headers, so this
limitation would be is okay.

I still like my "join" function.  It can be used for other cases, such as when
you have to prepend a header before connecting the input stream to your CGI
script (or whatever it is).  It would make it easy to implement zero-copy, as
all plain in-to-out copying can be delegated to the OS and in many cases will
not require the OS to do any work, just to collapse two pipes together or
something simple like that.

Sam