Re: [9fans] netpipe results

[Dean Prichard <dpx@acl.lanl.gov>]

Andrey and I spend some time trying out different values
of QMAX in /sys/src/9/ip/tcp.c. Increasing the size of QMAX
seems to improve netpipe large packet behavior quite a bit.
All tests were done using 2 800Mhz PIII machines with onboard
i82557 ethernet. The graphs are available from
http://www.acl.lanl.gov/plan9/netpipe/i82557.html
The curves still flatten out above 32k data size, but it is much
better than before once QMAX >= 256k.

-dp

On Sat, 9 Mar 2002 presotto@plan9.bell-labs.com wrote:

> Date: Sat, 9 Mar 2002 08:52:00 -0500
> From: presotto@plan9.bell-labs.com
> Reply-To: 9fans@cse.psu.edu
> To: 9fans@cse.psu.edu
> Subject: Re: [9fans] netpipe results
>
> Looks good.  Is the 'data size' the size of a single write i.e. what the
> NetPIPE paper calls 'block size'?  If the latter, I can guess at some
> explanations to start working at improving things.
>
> Looks like things start to fall off around 32K byte writes.  Just looking
> at numerology, that's what Maxatomic is set to in port/qio.c.  It's
> the point we start breaking writes into multiple blocks.  I assume that
> will have some affect though not a huge one.  The drop we see at that
> point is more than I would expect.  The real nose dive happens
> twixt 64k and 100k.  64k is the size of the per channel output queue.
> After that, the system has to wait for the queue to half empty before
> queuing any more.  Just for a wild guess, it looks like we're getting
> screwed by interaction twixt that histeresis and the process getting
> rescheduled.  There's similar problems with the size of the input
> per channel queue (64k).  Perhaps we should have spent more time
> reading Kleinrock.
>
> These are just wild guesses.  I'll start playing to see if I can explain
> it, and preferably make it go away.  I welcome anyone else doing the
> same.
>
> Thanks much.  It'll make a fun distraction.
>