Re: [TUHS] Cole's Slaw

[George Michaelson <ggm@algebras.org>]

I deployed cheap switches, to try and reduce contention on the WiFi
segment in a 3bed flat. I suspect, switching is *slower* than good
quality 4/5G. I deployed cheap unmanaged switches. I think I may need
to re-consider. (it may be slower than good quality 5gz WIFi but I
felt not, lots of concrete, so I went cables in walls. Professionally
installed cables too)

old cheap switches have high port speed, but low cross-backplane
bandwidth. They basically don't do what they say on the label although
I have no proof, my observation is that 100mbit is rarely achieved on
a cheap switching backplane. I think most people won't notice. If you
need to do the full-disk restore, or stream 4K at the same time as
play some call-of-duty *and* do a full-disk streaming restore, I think
you can destroy your cheap home switch quickly. I have been prepping a
ZFS filestore and I calculate I'm getting 30mbit-sec end-to-end from
300mbit capable backplane hosts. The port speeds are gig. The actual
capability once you get over the Raspberry Pi USB3 is a LOT less. (the
receiver is a Pi4 with 8GB of memory, it should *not* be disk I/O
bound)

my various OSX Macs can do roaring speeds if you back-to-back connect
them. Put them into the switching stuff, it drops off really quickly
to boringly slow speeds.

If you go out the fibre router to the public network, I think their
contention model remains interesting: Stuff is being done in layer 2
to manage your bandwidth and the fibre which is capable of gig, is
probably doing bizarre things to pretend to be slower. Forced
bit-drop, random drop, long stalls. The framing at layer2 is really
bizarre. they can do a LOT in layer 2, they don't talk about. we're
all some kind of PPPo<something> inside this, which feels pretty meh.
Why did it have to be like this?

Economies which (unlike Australia) don't do bandwidth limit for
pricing disfunction, probably are saying :what do you mean: at this
point. (The UK, you can get true gig to the home. Parts of the USA
likewise. Lots of europe likewise. Here in Oz, although the bearer is
capable, you can't afford the price they want to charge because they'd
doing ROI sums to pay back $50b of spend, as quickly as possible)

If your router is running the Broadcom chipset most of them are built
around, Its switch is not really very big, in buffer space.  My R7000
is a  Broadcom BCM4709A0 and I don't think its doing very well.

So, I'm dying inside because this is highly contestable FUD. If
somebody with shares in the vendors chipset can tell me "you're just
wrong' I'd believe them but my experience from the field is, white box
domestic switches, and home routers, simply can't "do" full-bore gig
on multiple paths across the switch at once. Ubiquity and other good
vendors? Probably fine. Hell, for all I know, even Mikrotik might be
ok, I've only ever had their toss-away freebies from conference Schwag
and it was cool to have a box the size of a packet of cigarettes which
run BGP, but they felt pretty slow (they even do portspan. amazing
stuff)

-G


On Thu, Dec 10, 2020 at 10:53 AM Erik E. Fair <fair-tuhs@netbsd.org> wrote:
>
> I bet your backups across the gigabit switch to your NAS are (small) incrementals.
>
> You'll care about having 10gigE when you try a restore from backup.
>
> We have a very persistent problem with storage capacity versus I/O bandwidth, which I metaphor as "swimming pools of data, which we fill & drain with garden hoses." How many terabytes of stable storage do you have at home?
>
> Further, LANs have always lagged local I/O bandwidth (across PCI, PCIe, etc), independent of the nastiness introduced by latency and less-than-ideal data transfer protocols. Unix "diskless" workstations were only tolerable because the disks were still expensive and there were good reasons to share.
>
>         Erik