Re: [9fans] network throughput measurements

[presotto@plan9.bell-labs.com]

[-- Attachment #1: Type: text/plain, Size: 360 bytes --]

You might want to send a byte back at the end to make sure the
receiver is done.  What you're measuring is the amount of time
to get the data into the pipeon the write side, not the amount
of time to complete the transmission.  I don't see that you are
measuring latency anywhere.

As is, I'ld say you're measuring our inability to do Nagel's
algorithm.

[-- Attachment #2: Type: message/rfc822, Size: 10691 bytes --]

[-- Attachment #2.1.1: Type: text/plain, Size: 1167 bytes --]

Hello, 

Here are a few numbers with regards to network throughput of p9 and
other operating systems.  No substantial discussion is presented, but
we would like to draw your attention towards the small data size
transmission results.


The setup is as follows: EtherExpress100 network cards on pentuim 800s
for plan9, Netgear GA620 for the GigE tests.

The test involves:


Sender:
for(i = 1; i < somesize; i= i<<1) {
	buf = mallocz(i, 1);
	tbegin = nsec();
	for(j = 0; j < 10000; j++)
		write(fd, buf, i);
	tend = nsec();

	/* calculate and print result */
}


Receiver:
buf[8192];
dnull = open("/dev/null", OWRITE);
while((n = read(fd, buf, 8192)) > 0) 
	write(dnull, buf, n);


The same code was used for linux/fbsd but all the headers and crappy
TCP initialization crud was added in front of it.

Also, the Linux/FBSD code uses ftime(2) for calculating the elapsed
time, which gives resolutions down to a microsecond.  We found no
better way to measure elapsed time.

Note: we're currently looking at ways to improve the latency penalty
for small data size packets and suggestions are most welcome.


cheers, andrey


[-- Attachment #2.1.2: Type: message/rfc822, Size: 7937 bytes --]


100BaseT:

10000 iterations over data size 1:     0.18 mbits/s...     0.43 seconds
10000 iterations over data size 2:     0.37 mbits/s...     0.43 seconds
10000 iterations over data size 4:     0.87 mbits/s...     0.37 seconds
10000 iterations over data size 8:     1.72 mbits/s...     0.37 seconds
10000 iterations over data size 16:     3.53 mbits/s...     0.36 seconds
10000 iterations over data size 32:     7.18 mbits/s...     0.36 seconds
10000 iterations over data size 64:    14.12 mbits/s...     0.36 seconds
10000 iterations over data size 128:    30.32 mbits/s...     0.34 seconds
10000 iterations over data size 256:    52.98 mbits/s...     0.39 seconds
10000 iterations over data size 512:    92.56 mbits/s...     0.44 seconds
10000 iterations over data size 1024:    94.13 mbits/s...     0.87 seconds
10000 iterations over data size 2048:    94.44 mbits/s...     1.73 seconds
10000 iterations over data size 4096:    94.58 mbits/s...     3.46 seconds
10000 iterations over data size 8192:    94.45 mbits/s...     6.94 seconds
10000 iterations over data size 16384:    93.68 mbits/s...    13.99 seconds
10000 iterations over data size 32768:    92.27 mbits/s...    28.41 seconds
10000 iterations over data size 65536:    92.15 mbits/s...    56.89 seconds


GigE:

10000 iterations over data size 1:     0.23 mbits/s...     0.35 seconds
10000 iterations over data size 2:     0.49 mbits/s...     0.33 seconds
10000 iterations over data size 4:     1.09 mbits/s...     0.29 seconds
10000 iterations over data size 8:     2.32 mbits/s...     0.28 seconds
10000 iterations over data size 16:     4.79 mbits/s...     0.27 seconds
10000 iterations over data size 32:    10.01 mbits/s...     0.26 seconds
10000 iterations over data size 64:    19.99 mbits/s...     0.26 seconds
10000 iterations over data size 128:    40.50 mbits/s...     0.25 seconds
10000 iterations over data size 256:    68.98 mbits/s...     0.30 seconds
10000 iterations over data size 512:   133.30 mbits/s...     0.31 seconds
10000 iterations over data size 1024:   160.94 mbits/s...     0.51 seconds
10000 iterations over data size 2048:   170.62 mbits/s...     0.96 seconds
10000 iterations over data size 4096:   171.85 mbits/s...     1.91 seconds
10000 iterations over data size 8192:   168.40 mbits/s...     3.89 seconds
10000 iterations over data size 16384:   158.23 mbits/s...     8.28 seconds
10000 iterations over data size 32768:   153.45 mbits/s...    17.08 seconds
10000 iterations over data size 65536:   152.23 mbits/s...    34.44 seconds




IL over 100BaseT:

10000 iterations over data size 1:     0.25 mbits/s...     0.32 seconds
10000 iterations over data size 2:     0.50 mbits/s...     0.32 seconds
10000 iterations over data size 4:     0.98 mbits/s...     0.33 seconds
10000 iterations over data size 8:     1.98 mbits/s...     0.32 seconds
10000 iterations over data size 16:     3.94 mbits/s...     0.32 seconds
10000 iterations over data size 32:     7.84 mbits/s...     0.33 seconds
10000 iterations over data size 64:    15.48 mbits/s...     0.33 seconds
10000 iterations over data size 128:    30.50 mbits/s...     0.34 seconds
10000 iterations over data size 256:    59.05 mbits/s...     0.35 seconds
10000 iterations over data size 512:    90.30 mbits/s...     0.45 seconds
10000 iterations over data size 1024:    94.99 mbits/s...     0.86 seconds
10000 iterations over data size 2048:    94.31 mbits/s...     1.74 seconds
10000 iterations over data size 4096:    95.91 mbits/s...     3.42 seconds
10000 iterations over data size 8192:    95.90 mbits/s...     6.83 seconds
10000 iterations over data size 16384:    95.81 mbits/s...    13.68 seconds


--- linux ---

GigE on Linux (Alpha):

10000 iterations over data size 1:     6.15 mbits/s...     0.01 seconds
10000 iterations over data size 2:    11.43 mbits/s...     0.01 seconds
10000 iterations over data size 4:    22.86 mbits/s...     0.01 seconds
10000 iterations over data size 8:    49.23 mbits/s...     0.01 seconds
10000 iterations over data size 16:    85.33 mbits/s...     0.01 seconds
10000 iterations over data size 32:   150.59 mbits/s...     0.02 seconds
10000 iterations over data size 64:   256.00 mbits/s...     0.02 seconds
10000 iterations over data size 128:   393.85 mbits/s...     0.03 seconds
10000 iterations over data size 256:   553.51 mbits/s...     0.04 seconds
10000 iterations over data size 512:   546.13 mbits/s...     0.07 seconds
10000 iterations over data size 1024:   549.80 mbits/s...     0.15 seconds
10000 iterations over data size 2048:   547.96 mbits/s...     0.30 seconds
10000 iterations over data size 4096:   549.80 mbits/s...     0.60 seconds
10000 iterations over data size 8192:   549.80 mbits/s...     1.19 seconds
10000 iterations over data size 16384:   549.80 mbits/s...     2.38 seconds
10000 iterations over data size 32768:   553.98 mbits/s...     4.73 seconds
10000 iterations over data size 65536:   554.39 mbits/s...     9.46 seconds
10000 iterations over data size 131072:   554.71 mbits/s...    18.90 seconds
10000 iterations over data size 262144:   556.79 mbits/s...    37.66 seconds




100BaseT on Linux (RH 7.2):

10000 iterations over data size 1:     2.96 mbits/s...     0.03 seconds
10000 iterations over data size 2:     5.93 mbits/s...     0.03 seconds
10000 iterations over data size 4:    12.80 mbits/s...     0.03 seconds
10000 iterations over data size 8:    27.83 mbits/s...     0.02 seconds
10000 iterations over data size 16:    51.20 mbits/s...     0.03 seconds
10000 iterations over data size 32:    91.43 mbits/s...     0.03 seconds
10000 iterations over data size 64:   111.30 mbits/s...     0.05 seconds
10000 iterations over data size 128:    93.94 mbits/s...     0.11 seconds
10000 iterations over data size 256:    93.94 mbits/s...     0.22 seconds
10000 iterations over data size 512:    93.94 mbits/s...     0.44 seconds
10000 iterations over data size 1024:    94.16 mbits/s...     0.87 seconds
10000 iterations over data size 2048:    94.27 mbits/s...     1.74 seconds
10000 iterations over data size 4096:    94.13 mbits/s...     3.48 seconds
10000 iterations over data size 8192:    94.12 mbits/s...     6.96 seconds
10000 iterations over data size 16384:    94.15 mbits/s...    13.92 seconds
10000 iterations over data size 32768:    94.15 mbits/s...    27.84 seconds
10000 iterations over data size 65536:    94.14 mbits/s...    55.69 seconds
10000 iterations over data size 131072:    94.12 mbits/s...   111.41 seconds


--- freebsd -- 

100BaseT on FBSD (4.4):


10000 iterations over data size 1:     3.64 mbits/s...     0.02 seconds
10000 iterations over data size 2:     7.62 mbits/s...     0.02 seconds
10000 iterations over data size 4:    14.55 mbits/s...     0.02 seconds
10000 iterations over data size 8:    29.09 mbits/s...     0.02 seconds
10000 iterations over data size 16:    49.23 mbits/s...     0.03 seconds
10000 iterations over data size 32:    85.33 mbits/s...     0.03 seconds
10000 iterations over data size 64:    94.81 mbits/s...     0.05 seconds
10000 iterations over data size 128:    93.94 mbits/s...     0.11 seconds
10000 iterations over data size 256:    94.38 mbits/s...     0.22 seconds
10000 iterations over data size 512:    94.38 mbits/s...     0.43 seconds
10000 iterations over data size 1024:    94.16 mbits/s...     0.87 seconds
10000 iterations over data size 2048:    94.11 mbits/s...     1.74 seconds
10000 iterations over data size 4096:    94.08 mbits/s...     3.48 seconds
10000 iterations over data size 8192:    94.05 mbits/s...     6.97 seconds
10000 iterations over data size 16384:    94.05 mbits/s...    13.94 seconds
10000 iterations over data size 32768:    94.08 mbits/s...    27.86 seconds
10000 iterations over data size 65536:    94.08 mbits/s...    55.73 seconds