On Fri, 30 Apr 2010 12:13:59 EDT erik quanstrom <quanstro@labs.coraid.com>  wrote:
> > > i don't see any reason why 9p couldn't use some of the same
> > > congestion control ideas.  the trick would be to feed back packet loss
> > > detection and retransmission info to the point where file io gets
> > > turned into rpcs→the mount driver
> >
> > Agreed -- sort of what I meant by "I hope 9p evolves".
> > Though I'd probably stick this in a layer below 9p.
>
> i suggested putting it above 9p.  i don't understand your
> suggestion.  how would a layer below know that there is
> more data?

I was just musing aloud -- just some vague ideas.  9p already
allows 2^16 outstanding messages so in theory one can turn a
read() or write() into a sequence of 9p messsages and only
wait for acks at the end (but process any acks as they
arrive).  Perhaps ack receipt time can be used to measure
latency & bandwidth to transparently control how many 9p
messages can be blasted. The other vague thought was that one
can do this:

	char buf[SOME_LARGE_CONSTANT];
	int count = read(src, buf, sizeof buf);
	if (count > 0) write(dst, buf, count);

But read will wait until buf is filled up or file ends &
write will wait until the entire buf is sent or there is an
error. Now may be this is good enough in that a reading thread
can read in big gulps and pass on filled buffers to a writing
thread. And the writing thread writes out and sends empty
buffers back to the reader.  But one can also think of a
lower level facility that in effect "offload" actual transfer
to a coprocessor.  This allows read()/write() to return much
quicker (but obviously some errors will be delayed). One can
implement something like this using mmap(). Now I am well
aware of issues with mmap (we don't have to go through that
discussion again) but it can be an option for improving
throughput.

Neither of these ideas requires changes to 9p per se.  To
take this any further, I would want to prototype something.
I have coded some bits in Scheme but no time really to do
much.

In turn some examples to explain your suggestion would be
helpful.

> > It can
> > backpressure a 9p client if it tries to send too much (either
> > by blocking or returning with equiv of EWOULDBLOCK).
>
> isn't that what you just complained about—fcp having to
> do the kernel's work?

Blocking would backpressure in that fcp wouldn't be able to
send more data! fcp doesn't have to do anything more.