From mboxrd@z Thu Jan 1 00:00:00 1970 From: bakul@bitblocks.com (Bakul Shah) Date: Tue, 26 Sep 2017 11:26:39 -0700 Subject: [TUHS] UNIX of choice these days? In-Reply-To: Your message of "Tue, 26 Sep 2017 11:39:56 -0600." References: <20170924140617.GG28606@mcvoy.com> <20170924203621.GA80203@wopr> <49B7FCB8-A086-4FFB-AF3B-4B3BD167EC54@bitblocks.com> <1506297783.59c847b740373@www.paradise.net.nz> <1506386563.59c9a2834015a@www.paradise.net.nz> <20170926144155.GW28606@mcvoy.com> <46A45CD5-45A0-44C8-ABC4-0644D7A23493@bitblocks.com> Message-ID: <20170926182655.2F059156E523@mail.bitblocks.com> On Tue, 26 Sep 2017 11:39:56 -0600 Warner Losh wrote: Warner Losh writes: > > On Tue, Sep 26, 2017 at 11:34 AM, Bakul Shah wrote: > > > You probably know Brantley Coile did ata over Ethernet. AOE is a simple > > protocol so the client side driver is simple. He ran plan9 on his > > controller. One per rack. Basically you have a storage area network. Adding > > disks becomes very easy. His company is back in business if you want to buy > > some! > > > > AOE is probably a better idea than FCoE, fiber channel over Ethernet, with > > its requirement for a reliable transport. On the other hand, may be there > > is need for encrypted channels all the way to disks. > > > AoE: All the quality you'd expect from low end ATA drives couple with all > the reliability of low-end consumer grade ethernet. What could go wrong? > > AoE really punted on all the reliability stuff, so if you have any kind of > congested network it's a total disaster. Plus, who doesn't want to access > storage via an arcane protocol designed so you could latch registers > effective, but not so you can encode requests so. > > Sorry for the rant, but I've had to deal with too much low-end gear > recently and it makes me grumpy and AoE back in the early 2000's was a > total S***show of low-end, poor quality implementations.... I have no practical experience with either so no idea which can be done better. Just an instinct that lower level protocols should be kept simple. Ethernet is not a reliable protocol and I'd rather let an higher level protocol deal with end to end reliability. And while AOE uses ATA commands, attached disks can be SAS or ATA or whatever the controller can handle. The congestion should probably be dealt with at the ethernet switch level. Since each protocol message has a tag, the driver should keep track of what it has not received and retry after timeout. In a previous job I got pretty familiar with ieee 802.1ad and 802.1ah (QinQ and MACinMAC) and they get complex when they add all the other stuff some of which probably belongs at layer 3. FC itself is a transport protocol for SCSI etc. so FCoE adds yet another layer. (And I suppose you can then send ethernet frames over fiber!)