On Sat, Dec 05, 2009 at 08:24:45AM -1000, Tim Newsham wrote: > ps. if you wanted to hide this ugliness of passing a buffer and > fd to a child process instead of just passing an fd, you could > still solve it in userland without a syscall. Write a library > that does buffered IO. Include unget() if you like. Write the > library in a way that you can initialize it after a fork/exec > to pick up state from the parent (ie. by taking two fds, > reading the buffer from the first, and continuing on with the > 2nd when it is exhausted). > > Is there much benefit in doing this in the kernel instead? it's all library code, and it loses the "everything is a file (descriptor)" advantage. you cannot pass that library state to another program. you could if the state was a file descriptor. for inferno i wrote an http client library that turns a request into an fd to read the data from. that fd has http chunking,gzip,ssl peeled off. now i can pass the fd with the http response to other programs, do buffered i/o on it, etc. this is implemented in user-space btw, with inferno's sys->file2chan (as opposed to pipes, you can do error message propagation over file2chan's). since file descriptors are so essential, it may help to have "tools" to use them. yesterday evening i hacked up devbuf.c and devjoin.c after reading this thread. both offer a file "new". for devbuf.c you can write data to it, then later consume it (yes, you could just use a pipe instead). for devjoin.c, you can write fd numbers (of open files) to register an fd, then later reads will get data from the first registered file, when that returns 0 it continues on the next, and so on. so fd's can be chained for reading (not writing). i know this "join" functionality is different from what sam originally described. i've attached devbuf.c and devjoin.c, as example (for inferno). they have bugs (don't assign qid.path, probably *walk is broken too). testbufjoin.b is an example of how the dev's can be used. it creates a new fd that has a buffer at the front (e.g. leftovers from http header reading), then continues on stdin (where the leftover may have come from). then it reads the new fd and writes its data to stdout. these devices are not for performance. perhaps they make working with one of the most basic OS concepts (fd's) a bit easier. but perhaps this problem is not common enough, or can be handled (with fd's preferrably) in a better way. mjl