Re: [9fans] Help with two small shared file servers

[Bakul Shah <bakul@bitblocks.com>]

On Thu, 18 Aug 2011 01:29:29 EDT erik quanstrom <quanstro@labs.coraid.com>  wrote:
> > > Is 9p suitable for this? How will the 40ms latency affect 9p
> > > operation? (I have 100Mbit).
> >
> > With a strict request/response protocol you will get no more
> > than 64KB once every 80ms so your throughput at best will be
> > 6.55Mbps or about 15 times slower than using HTTP/FTP on
> > 100Mbps link for large files.  [John, what was the link speed
> > for the tests in your thesis?]
>
> i calculate 1/0.080s/rt * 64*1024 = 819 kbytes/s.  (i think your
> conversion from mbits/s to bytes/s is faulty.)

*8 for bits. 2^16*8/0.080/10^6 Mbps == 6.5536 Mbps.

MBps == 10^6 bytes/sec.
Mbps == 10^6 bits/sec.

But it is not accurate in another way. 40ms for the server to
receive the request. 40 ms for the client to receive the first
byte of data + time for transmission of 64KB == 2^16*8/10^8 ==
about 5ms. Actually a bit more due to ethernet, ip & tcp
overhead. So the total time will be over 85ms for a  BW of
under 6.17Mbps.

> also you do get 1 oustanding per read or write, not per machine.
> so the real limit is wire speed, but for most programs most of the
> time the limit for a single file transfer would be as you specify.

It is three things:

RTT= round trip time = 2*latency
BW = bandwidth
DS = datasize that can be sent without waiting.

Time to deliver DS bytes will be at least
	RTT+DS/BW

Consider three cases:
1) local (small RTT)
2) remote slow (large RTT, small BW)
3) remote fast (large RTT, large BW)

When RTT effects are small (1 & 2) 9p does fine. For case 3,
to minimize RTT effects you must make DS large but that can't
be done in 9p beyond 64k.

In John's test it was 10Mbps @ 25ms latency. To transfer 64KB
is 50ms rtt + 52ms or about 102ms. He should see 5.1Mbps or
about half the throughput compared to HTTP streaming case but
he says it was 1/4th so there must be other inefficiencies.