[discuss] Mellanox LACP aggr fails without snoop if using vlan tags

[Josh Coombs <jcoombs@staff.gwi.net>]

[-- Attachment #1: Type: text/plain, Size: 1766 bytes --]

I've got a brand new r151050e install using a Mellanox CNX5 card, dual 25Gb
ports paired up in an aggr to a Juniper EX4650 cluster. It will only work
if I start snoop on the aggr. Without doing so, it won't pass traffic. I
ran into this back in 2019 with bnx devices after upgrading to r151030 and
was never able to find a fix, that box I ended up changing NICs to intel to
get around the problem.

It also works if I do a snoop -P -d aggr0 so it may not be promiscuous mode
directly that's 'fixing' things?

If I rebuild the setup and don't use VLAN tagging, that appears to work as
anticipated which is different from my prior bnx troubles. Those I never
got around to testing with vlans on OmniOS.

When not working, I can see my aggr's mac address upstream on the switches.
I haven't tried manually firing packets out an interface but based on last
time I expect that will work. The issue seems to be on the receive side in
OmniOS, something about starting a snoop suddenly allows OmniOS to accept
the packets fully for some reason.

I don't have a production and lab setup, sadly my test box is in a
datacenter that is an hour away, unmanned so I'm limited in what I can do
at the moment. I've got a little bit of time before I need to lock the conf
for production use, but can only be onsite once or twice a week at best. I
can live without vlan tagging, but if this is an unexpected fail I'd like
to gather what data I can to help before shutting the door to testing?

Joshua Coombs

------------------------------------------
illumos: illumos-discuss
Permalink: https://illumos.topicbox.com/groups/discuss/T608dab80e5db30f6-Mf17e7261eeb23be4da6bab83
Delivery options: https://illumos.topicbox.com/groups/discuss/subscription

[-- Attachment #2: Type: text/html, Size: 2705 bytes --]