From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: <41514EEA.7060607@9fs.org> Date: Wed, 22 Sep 2004 11:07:38 +0100 From: Nigel Roles User-Agent: Mozilla Thunderbird 0.8 (X11/20040913) MIME-Version: 1.0 To: Fans of the OS Plan 9 from Bell Labs <9fans@cse.psu.edu> Subject: Re: [9fans] Hardware for venti/fossil/cpu server References: <3c38e0973b81ee280f074f4647d3d05a@collyer.net> In-Reply-To: <3c38e0973b81ee280f074f4647d3d05a@collyer.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Topicbox-Message-UUID: e6bd167e-eacd-11e9-9e20-41e7f4b1d025 geoff@collyer.net wrote: >I've never heard of the built-in Gbe. You're probably better off >just buying an Intel PRO/1000 MT and putting it in. > The Marvell device is also in the nForce 3 chipset. There is a reverse engineered driver in Linux which seems fine so far. I thought the driver name quite amusing - forcedeth. It's only 2000 lines. I know this sounds a lot, but it looks straightforward enough, and a lot of the 2000 lines is #defines. >I've always distrusted hardware built into the motherboard >and the current round of built-in Gbe controllers is doing >nothing to change that distrust. They're either broken >(the not-quite igbe), Broadcom or Brand X (Marvell). >I'm not sure how breaking the igbe controllers reduces >anybody's costs. > > > Well, if you want 1Gb out of your 1Gb ethernet chip, then plugging into the PCI bus is a non starter. PCI is just not very fast. We have tried a Tyan tomcat with opteron 248 + 2 broadcom on PCI, and it maxes out at 800mbps and 100% CPU! Some of that is the Broadcom chip's fault, but the rest is the fact that the bus is full. PCI is only just over a Gbit when bursting. You need the Gbe connected either to a better bus (e.g. PCI-X) or the CPU side of the PCI bridge. This can be achieved either by embedding in the chipset (e.g. nForce3, KT800PRO), or by having an alternate ethernet specific bus (Intel CSA). PCI-X is expensive. The sockets are long, and there's no space on the motherboards. You only get it on two uniprocessor boards that I know of (SuperMicro and a very new Gigabyte one), otherwise it has to be dual processor or more. This doubles the cost of the motherboard from 100 to 200 quid. That's how it reduces costs.