Re: [9fans] A simple experiment

[erik quanstrom <quanstro@quanstro.net>]

> > 9p, like aoe, is a ping-pong protocol.  each message requires an ack.
> > therefore, the transport layer doesn't need flow control.
>
> Therefore, it is also not able to utilise bandwidth
> effectively over longhaul links. As an example, US coast
> to coast round trip time latency is about 100ms. Now consider
> fcp. Each worker thread of fcp does 8K read/writes. Due to
> pingponging, the *most* a thread can xfer coast to coast is
> 80KBps (for 16 threads, 1.28MBps).  It is actually much worse
> since each thread doesn't even overlap reads with writes.

i think you are conflating a ping-pong protocol with the
limitation of a single outstanding message.  there is no
reason that one needs a 1:1 correspondence between threads
and outstanding messages.  nor does the protocol inherently
prohibit 1 write from becoming multiple messages.

in the case of aoe, you can have up to 2^32 - 1 outstanding
packets at the same time to each target.  thus the theoretical
maximum would be
	80 * (2^31 - 1) kbps per target.
the aoe driver (in contrib) has a limit of 128 8k jumbo frames
outstanding per target.  the aoe driver has a fixed number of
thread per target.

> Short of a sliding window that is as large as the capacity of
> the pipe, you can't expect to keep it full.  As usual one has
> to trade off simplicity vs performance.

i don't think so.  the only thing a sliding window gives you
is < 1 ack per message sent.

-erik