From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: from tb-ob1.topicbox.com (tb-ob1.topicbox.com [64.147.108.173]) by inbox.vuxu.org (Postfix) with ESMTP id D2AC521707 for ; Thu, 31 Oct 2024 22:25:13 +0100 (CET) Received: from tb-mx0.topicbox.com (tb-mx0.nyi.icgroup.com [10.90.30.73]) by tb-ob1.topicbox.com (Postfix) with ESMTP id D45573309B for ; Thu, 31 Oct 2024 17:25:12 -0400 (EDT) (envelope-from bounce.mM7a4bf3d343ab2df1faf1eed8.r81958daa-7202-11ef-bbf2-18142b2d11b0@illumos.bounce.topicbox.com) Received: by tb-mx0.topicbox.com (Postfix, from userid 1132) id D1B08166B19; Thu, 31 Oct 2024 17:25:12 -0400 (EDT) ARC-Authentication-Results: i=2; topicbox.com; arc=pass; dkim=pass (1024-bit rsa key sha256) header.d=cooperi.net header.i=@cooperi.net header.b=gHxlEvXA header.a=rsa-sha256 header.s=mx x-bits=1024; dmarc=pass policy.published-domain-policy=quarantine policy.applied-disposition=none policy.evaluated-disposition=none (p=quarantine,d=none,d.eval=none) policy.policy-from=p header.from=cooperi.net; spf=pass smtp.mailfrom="bounce+855386.d86e42f-discuss=lists.illumos.org@cooperi.net" smtp.helo=m43-13.mailgun.net; x-internal-arc=fail (as.1.topicbox.com=pass, ams.1.topicbox.com=fail (message has been altered)) (Message modified while forwarding at Topicbox) ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=sender:message-id:date:mime-version:subject:to :references:cc:in-reply-to:content-type :content-transfer-encoding:list-help:list-id:list-post :list-subscribe:reply-to:from:list-unsubscribe; s=sysmsg-1; t= 1730409912; bh=RoXkvO+IyG3OJHSHKQjCyaEFL/bQlNB5f7GdmaIiSPI=; b=t X4mXF7mAic8lNAi9HmgPv4pV0qOFZqwnK7DlGu2/3reguHq6aRLipCbQHaZU21yg OAa9+vLj7CkPscKztzdTAF519PvHtK7MPJBQ0LO/gYYq/lf6YmAM8qNbsaPie6Nw BLpCnUIAE/CO2Pvr8gikyOcKRYtNvchHOUtbbxFDjo= ARC-Seal: i=2; a=rsa-sha256; cv=pass; d=topicbox.com; s=sysmsg-1; t= 1730409912; b=iqGrT+IueD9bTKcEWV8Tn24k/WNnck7llJVcqCD7ShacjcTWo8 Y0RLfiqHyeVBdbV9HzYQXmVOVmIhQ3k0RI0SwMMyX2C+uxc5TpmUghlSCVtM7qMb qQ922zBTHGywizuqhWLjA9w7hs3aUOMgUZm1CWXdgZysU5CTx3ZcjA5WE= Authentication-Results: topicbox.com; arc=pass; dkim=pass (1024-bit rsa key sha256) header.d=cooperi.net header.i=@cooperi.net header.b=gHxlEvXA header.a=rsa-sha256 header.s=mx x-bits=1024; dmarc=pass policy.published-domain-policy=quarantine policy.applied-disposition=none policy.evaluated-disposition=none (p=quarantine,d=none,d.eval=none) policy.policy-from=p header.from=cooperi.net; spf=pass smtp.mailfrom="bounce+855386.d86e42f-discuss=lists.illumos.org@cooperi.net" smtp.helo=m43-13.mailgun.net; x-internal-arc=fail (as.1.topicbox.com=pass, ams.1.topicbox.com=fail (message has been altered)) (Message modified while forwarding at Topicbox) X-Received-Authentication-Results: ORIGINATING; auth=pass smtp.mailfrom=alex@cooperi.net X-Received-Authentication-Results: tb-mx1.topicbox.com; arc=none (no signatures found); bimi=none (No BIMI records found); dkim=pass (1024-bit rsa key sha256) header.d=cooperi.net header.i=@cooperi.net header.b=gHxlEvXA header.a=rsa-sha256 header.s=mx x-bits=1024; dmarc=pass policy.published-domain-policy=quarantine policy.applied-disposition=none policy.evaluated-disposition=none (p=quarantine,d=none,d.eval=none) policy.policy-from=p header.from=cooperi.net; iprev=pass smtp.remote-ip=69.72.43.13 (m43-13.mailgun.net); spf=pass smtp.mailfrom= "bounce+855386.d86e42f-discuss=lists.illumos.org@cooperi.net" smtp.helo=m43-13.mailgun.net; x-aligned-from=domain_pass (Domain match); x-me-sender=none; x-ptr=pass smtp.helo=m43-13.mailgun.net policy.ptr=m43-13.mailgun.net; x-return-mx=pass header.domain=cooperi.net policy.is_org=yes (MX Records found: lamina.cooperi.net,sorus.cooperi.net); x-return-mx=pass smtp.domain=cooperi.net policy.is_org=yes (MX Records found: lamina.cooperi.net,sorus.cooperi.net); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=0 state=0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lists.illumos.org; h= sender:message-id:date:mime-version:subject:to:references:cc :in-reply-to:content-type:content-transfer-encoding:list-help :list-id:list-post:list-subscribe:reply-to:from :list-unsubscribe; s=dkim-1; t=1730409912; x=1730496312; bh=RoXk vO+IyG3OJHSHKQjCyaEFL/bQlNB5f7GdmaIiSPI=; b=jULjtq5JtkviTPxTe3TX y5JJFIAQgqmjKu+pP4AFWdcIkGHDhsY7cGdmLUciFI6nUO1Zki1BVEvN7sweAwvw 0Q6geT0otrBTMrSpZMfPJrZvWBcXuvmdiMy49yajSZB1qgAQI0Ymx5+DXelNLwKu 4R1DPquW7Rt79FH5DJzFLBQ= Received: from tb-mx1.topicbox.com (localhost.local [127.0.0.1]) by tb-mx1.topicbox.com (Postfix) with ESMTP id 4F463182243 for ; Thu, 31 Oct 2024 17:23:23 -0400 (EDT) (envelope-from bounce+855386.d86e42f-discuss=lists.illumos.org@cooperi.net) Received: from tb-mx1.topicbox.com (localhost [127.0.0.1]) by tb-mx1.topicbox.com (Authentication Milter) with ESMTP id 0F91A11DDF3; Thu, 31 Oct 2024 17:23:23 -0400 ARC-Seal: i=1; a=rsa-sha256; cv=none; d=topicbox.com; s=arcseal; t= 1730409803; b=vSSiTO8EEAYM8pfSrTDzJXp7Jfrg2B8b43TdWON88dtPT9fpXM 3oJ2rE9kVcXliWewk35MvgVcBCV1fg8JcXD0TlaBcmMkVmuqDksZmE3u3xD3BL9+ oodca62o/DDtbQYuU8Ro9CsQtqH2haf8E0p9M1OzWoGbpub9mwqHzm3plhJ6koIF xUwBHrl/pNLZ9wLx+S7U8/ZX2tQfG3RFbg2zdHa8SzzWFhc/M9k+cudI2h0xQ03d /tycOoooUSC/5SLSoJY5ib3UxTnksRfvJ5QwzV/kCnKFUqhZMTbWPRKZn/UykPPY BPuTef1x6ISUsHliPV8RB23fKxoKj4xz71bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=sender:message-id:date:mime-version:subject:to :references:from:cc:in-reply-to:content-type :content-transfer-encoding; s=arcseal; t=1730409803; bh=ojbRkQCg IL4n6bQaz55aLQohYMlncWhqwUfkIigUe4o=; b=C6O8lL8EeobR+PKsT3p5uvWB 5m6kMktfVv7GhICdwAmxJwIS9zH9j+ugd9zOraDsUVdFwU9uktBdMnkIMGEGaNl1 EM9ANLvW1YJ5Lz4+M0cRzr0ZlJsgmi2B2/OQLLbqWQB5JL0hHtVUGAwvEQpoM3+q 2itjMF+JjziA1VneH1corXg/RU6Gd4v6b0nYsanWaaYnKzvsK1S8015Cnp7aAgaw aukpxFypL5/hT4j77pbjnxmYWVOHeGeT+0xUE3hnxA9ojayaiJfl/i6bnskxxiX9 T6fhYrKYQpzyAB/wfJ+KaQJBsALXEbEEDpTU9omW5ZLFO/RFT/tWNdyv+y/ADA== ARC-Authentication-Results: i=1; tb-mx1.topicbox.com; arc=none (no signatures found); bimi=none (No BIMI records found); dkim=pass (1024-bit rsa key sha256) header.d=cooperi.net header.i=@cooperi.net header.b=gHxlEvXA header.a=rsa-sha256 header.s=mx x-bits=1024; dmarc=pass policy.published-domain-policy=quarantine policy.applied-disposition=none policy.evaluated-disposition=none (p=quarantine,d=none,d.eval=none) policy.policy-from=p header.from=cooperi.net; iprev=pass smtp.remote-ip=69.72.43.13 (m43-13.mailgun.net); spf=pass smtp.mailfrom= "bounce+855386.d86e42f-discuss=lists.illumos.org@cooperi.net" smtp.helo=m43-13.mailgun.net; x-aligned-from=domain_pass (Domain match); x-me-sender=none; x-ptr=pass smtp.helo=m43-13.mailgun.net policy.ptr=m43-13.mailgun.net; x-return-mx=pass header.domain=cooperi.net policy.is_org=yes (MX Records found: lamina.cooperi.net,sorus.cooperi.net); x-return-mx=pass smtp.domain=cooperi.net policy.is_org=yes (MX Records found: lamina.cooperi.net,sorus.cooperi.net); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=0 state=0 X-ME-VSCause: gggruggvucftvghtrhhoucdtuddrgeeftddrvdekjedgheefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefkffggfg fuvfhfhfevjggtgfesthekredttddvjeenucfhrhhomheptehlvgigucghihhlshhonhcu oegrlhgvgiestghoohhpvghrihdrnhgvtheqnecuggftrfgrthhtvghrnheptdetvdefke efheehfeejleefteelvddtudekgeelgfefjeegtdduhfehtdehffdunecukfhppeeiledr jedvrdegfedrudefpddvtdefrdduvddrvddruddvvdenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepihhnvghtpeeiledrjedvrdegfedrudefpdhhvghlohepmhegfedq udefrdhmrghilhhguhhnrdhnvghtpdhmrghilhhfrhhomhepoegsohhunhgtvgdokeehhe efkeeirdgukeeivgegvdhfqdguihhstghushhspehlihhsthhsrdhilhhluhhmohhsrdho rhhgsegtohhophgvrhhirdhnvghtqedpnhgspghrtghpthhtohepuddprhgtphhtthhope eoughishgtuhhssheslhhishhtshdrihhllhhumhhoshdrohhrgheq X-ME-VSScore: 0 X-ME-VSCategory: clean Received-SPF: pass (cooperi.net: Sender is authorized to use 'bounce+855386.d86e42f-discuss=lists.illumos.org@cooperi.net' in 'mfrom' identity (mechanism 'include:mailgun.org' matched)) receiver=tb-mx1.topicbox.com; identity=mailfrom; envelope-from="bounce+855386.d86e42f-discuss=lists.illumos.org@cooperi.net"; helo=m43-13.mailgun.net; client-ip=69.72.43.13 Received: from m43-13.mailgun.net (m43-13.mailgun.net [69.72.43.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tb-mx1.topicbox.com (Postfix) with ESMTPS for ; Thu, 31 Oct 2024 17:23:20 -0400 (EDT) (envelope-from bounce+855386.d86e42f-discuss=lists.illumos.org@cooperi.net) X-Mailgun-Sending-Ip: 69.72.43.13 X-Mailgun-Sending-Ip-Pool-Name: X-Mailgun-Sending-Ip-Pool: X-Mailgun-Sid: WyI3MzJmMiIsImRpc2N1c3NAbGlzdHMuaWxsdW1vcy5vcmciLCJkODZlNDJmIl0= Received: from sorus.cooperi.net (sorus.cooperi.net [139.99.250.52]) by 5f049c6be385 with SMTP id 6723f547a3a4e126b2f4ef0a (version=TLS1.3, cipher=TLS_AES_128_GCM_SHA256); Thu, 31 Oct 2024 21:23:19 GMT Sender: alex@cooperi.net Received: from [192.168.0.150] (home.cooperi.net [203.12.2.122]) by sorus.cooperi.net (Postfix) with ESMTPSA id 15217589B7; Thu, 31 Oct 2024 21:23:16 +0000 (UTC) Message-ID: <9682f6e3-b34b-495e-bdea-53e98a4cf9d5@cooperi.net> Date: Fri, 1 Nov 2024 07:23:16 +1000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [discuss] Mellanox LACP aggr fails without snoop if using vlan tags To: discuss@lists.illumos.org References: Content-Language: en-US Cc: rm@fingolfin.org In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: Rspamd (sorus.cooperi.net) Topicbox-Policy-Reasoning: allow: sender is a member Topicbox-Message-UUID: 5edc76ce-97ce-11ef-a431-f2ce13443f88 Archived-At: =?UTF-8?B?PGh0dHBzOi8vaWxsdW1vcy50b3BpY2JveC5jb20vZ3JvdXBz?= =?UTF-8?B?L2Rpc2N1c3MvVDYwOGRhYjgwZTVkYjMwZjYtTTdhNGJmM2QzNDNhYjJkZjFm?= =?UTF-8?B?YWYxZWVkOD4=?= List-Help: List-Id: "illumos-discuss" List-Post: List-Software: Topicbox v0 List-Subscribe: Precedence: list Reply-To: illumos-discuss From: "Alex Wilson via illumos-discuss" List-Unsubscribe: , Topicbox-Delivery-ID: 2:illumos:a9ae1d0a-2aea-11e7-965f-d98f9f16e227:81958daa-7202-11ef-bbf2-18142b2d11b0:M7a4bf3d343ab2df1faf1eed8:1:neeeZwvnXp1s0EDUatdjewO6_5Vp37RZo69XxxbwJ88 Hi Josh, On 1/11/24 06:47, Josh Coombs wrote: > I've got a brand new r151050e install using a Mellanox CNX5 card, dual=20 > 25Gb ports paired up in an aggr to a Juniper EX4650 cluster. It will=20 > only work if I start snoop=C2=A0on the=C2=A0aggr. Without doing so, it wo= n't pass=20 > traffic. I ran into this back in 2019 with bnx devices after upgrading=20 > to r151030 and was never able to find a fix, that box I ended up=20 > changing NICs to intel to get around the problem. >=20 > It also works if I do a snoop -P -d aggr0 so it may not be=20 > promiscuous=C2=A0mode directly that's 'fixing' things? >=20 I've seen this bug as well. When I dtrace the calls into mlxcx what I=20 see is that aggr never gives the driver any VLAN tag filters for the=20 default group (but it does give MAC filters), so no traffic other than=20 on the default tag ends up received. If you perturb the MAC state of the aggr enough it will switch to=20 explicit VLAN tag filters and work fine (e.g. if you add a VNIC as well=20 as the VLAN interface, the existence of the VNIC will fix it since that=20 causes MAC to add an explicit VLAN tag filter for the vlan DL) I think this is a semantic bug here -- I suspect MAC is assuming that if=20 it adds just MAC filters and no VLAN filters to a NIC, that means all=20 tagged traffic for that MAC should be matched, not just un-tagged.=20 Unfortunately the documentation (mac_capab_rings.9e and mac.9e) is not=20 very clear on this point and some drivers (definitely mlxcx I can speak=20 for, since I wrote most of it) have interpreted it differently. For now I've just been always using VNICs, since those always generate=20 explicit filters and work fine. But we should get this fixed up, and=20 probably the documentation adjusted to spell it out more clearly so no=20 other new drivers make the same mistake going forwards. ------------------------------------------ illumos: illumos-discuss Permalink: https://illumos.topicbox.com/groups/discuss/T608dab80e5db30f6-M7= a4bf3d343ab2df1faf1eed8 Delivery options: https://illumos.topicbox.com/groups/discuss/subscription