From mboxrd@z Thu Jan  1 00:00:00 1970
Message-ID: <915c375ec7b54c67fbcb9cd127ccbeae@coraid.com>
From: erik quanstrom <quanstro@coraid.com>
Date: Thu, 22 Mar 2007 22:22:21 -0400
To: 9fans@cse.psu.edu
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Subject: [9fans] 10gbe ethernet for plan 9
Content-Transfer-Encoding: quoted-printable
Topicbox-Message-UUID: 2e76826c-ead2-11e9-9d60-3106f5b1d025

coraid was given the blessing on monday to give the
the myricom 10 gigabit ethernet driver with firmware
for the cpu and fs kernels to the plan 9 community.

the m10g driver drives a pretty nice piece of hardware,
the lanai z8e.  there are no errata to worry about,  the
phy is invisable to the driver.  all packets are transferred
via dma.

the good folks at myricom provided really great help.

if you use this driver, there are a couple of things to
keep in mind.  the dma stats printed on startup
are very important.  you should see 1200 MB/s
r and w and over 1800MB/s r/w dma performance.  if you
don't, it's likely there is a problem with pcie
configuration.  (motherboard manufacturers are still
learning about pcie.)  at 10gb/s, it's important to enable
jumbo frames like so:

	ip/ipconfig -m 9000 gbe /net/ether$n add $ip $mask
	echo -n mtu 9000 >> /net/ipifc/1/ctl

you may also want to increase the default interrupt coalesing
delay of 25=C2=B5s to 80=C2=B5s.  (or decrease it, if you're an sc guy.)
this can be done by

	echo coal 80 > /net/ether$n/0/ctl

if you have any questions about the hardware or the driver,
feel free to email.

- erik