From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: from tb-ob0.topicbox.com (tb-ob0.topicbox.com [64.147.108.117]) by inbox.vuxu.org (Postfix) with ESMTP id 6FBB830D8D for ; Thu, 31 Oct 2024 21:49:55 +0100 (CET) Received: from tb-mx0.topicbox.com (tb-mx0.nyi.icgroup.com [10.90.30.73]) by tb-ob0.topicbox.com (Postfix) with ESMTP id 4AE9922F4A for ; Thu, 31 Oct 2024 16:49:55 -0400 (EDT) (envelope-from bounce.mMf17e7261eeb23be4da6bab83.r81958daa-7202-11ef-bbf2-18142b2d11b0@illumos.bounce.topicbox.com) Received: by tb-mx0.topicbox.com (Postfix, from userid 1132) id 471E1164A5E; Thu, 31 Oct 2024 16:49:55 -0400 (EDT) ARC-Authentication-Results: i=2; topicbox.com; arc=pass; dkim=pass (2048-bit rsa key sha256) header.d=staff.gwi.net header.i=@staff.gwi.net header.b=AHSPe1Yz header.a=rsa-sha256 header.s=google-1 x-bits=2048; dmarc=pass policy.published-domain-policy=none policy.published-subdomain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,sp=none,d=none,d.eval=none) policy.policy-from=sp header.from=staff.gwi.net; spf=pass smtp.mailfrom=jcoombs@staff.gwi.net smtp.helo=mail-pg1-f169.google.com; x-internal-arc=fail (as.1.topicbox.com=pass, ams.1.topicbox.com=fail (message has been altered)) (Message modified while forwarding at Topicbox) ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=mime-version:from:date:message-id:to :content-type:list-help:list-id:list-post:list-subscribe :reply-to:subject:content-transfer-encoding:list-unsubscribe; s= sysmsg-1; t=1730407795; bh=EVYCvDYZDvfiq9qGL4WG4bOu8neKAfi8kRkfm TTLa4k=; b=eVq+j27QK0W+cv9YMsbFXUIDNrfAnq6q/JaGpkvF2IY9wLg5ShLsS /eBAXwfme26Chd9YhTLVAEPs3m8KrRk8MPw+gFy3V6XblB8i9bXyKstyPoH4gEqw +yoR+K3LB9s5b6qykaEr7f7Ik5rNvw6uSWO78Dlsn9/6u4K/oOHujM= ARC-Seal: i=2; a=rsa-sha256; cv=pass; d=topicbox.com; s=sysmsg-1; t= 1730407795; b=D5Zpl57kq+UYBcZMu/1/ZvRXWRyxFR532Ng8EtE8A9vVz982Br jWOYuIwJ3xk8QRsQwcaBtPn+SRrbuNBMUGyir6V4SzXmBmeqa7je7TKAW92OCXNH ntPfryAekYAUpyvdAsv0qTKqBvnKh7G1Z3X4Y7jlTs7A8t+en9bxbk0Ug= Authentication-Results: topicbox.com; arc=pass; dkim=pass (2048-bit rsa key sha256) header.d=staff.gwi.net header.i=@staff.gwi.net header.b=AHSPe1Yz header.a=rsa-sha256 header.s=google-1 x-bits=2048; dmarc=pass policy.published-domain-policy=none policy.published-subdomain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,sp=none,d=none,d.eval=none) policy.policy-from=sp header.from=staff.gwi.net; spf=pass smtp.mailfrom=jcoombs@staff.gwi.net smtp.helo=mail-pg1-f169.google.com; x-internal-arc=fail (as.1.topicbox.com=pass, ams.1.topicbox.com=fail (message has been altered)) (Message modified while forwarding at Topicbox) X-Received-Authentication-Results: tb-mx1.topicbox.com; arc=none (no signatures found); bimi=skipped (DMARC Policy is not at enforcement); dkim=pass (2048-bit rsa key sha256) header.d=staff.gwi.net header.i=@staff.gwi.net header.b=AHSPe1Yz header.a=rsa-sha256 header.s=google-1 x-bits=2048; dmarc=pass policy.published-domain-policy=none policy.published-subdomain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,sp=none,d=none,d.eval=none) policy.policy-from=sp header.from=staff.gwi.net; iprev=pass smtp.remote-ip=209.85.215.169 (mail-pg1-f169.google.com); spf=pass smtp.mailfrom=jcoombs@staff.gwi.net smtp.helo=mail-pg1-f169.google.com; x-aligned-from=pass (Address match); x-google-dkim=pass (2048-bit rsa key) header.d=1e100.net header.i=@1e100.net header.b=vp0xNniC; x-me-sender=none; x-ptr=pass smtp.helo=mail-pg1-f169.google.com policy.ptr=mail-pg1-f169.google.com; x-return-mx=pass header.domain=staff.gwi.net policy.org_domain=gwi.net policy.is_org=no (MX Records found: aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,alt1.aspmx.l.google.com,aspmx3.googlemail.com); x-return-mx=pass smtp.domain=staff.gwi.net policy.org_domain=gwi.net policy.is_org=no (MX Records found: aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,alt1.aspmx.l.google.com,aspmx3.googlemail.com); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=-100 state=0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lists.illumos.org; h= mime-version:from:date:message-id:to:content-type:list-help :list-id:list-post:list-subscribe:reply-to:subject :content-transfer-encoding:list-unsubscribe; s=dkim-1; t= 1730407795; x=1730494195; bh=rmtCI/C61Pv/uZfeHr4pAttdZphMX+mSWq4 siS5L1Hk=; b=YuywQn1nQbSCkfM7RHoS7Ghw6+reCYhvIpX9HFo0qXEV/oYwViL 0MhNERkBgTzIuEPHBL43cMmWdj8JpA+PyafXjKpeJ8ZoRr2BSfzojq3xNyGOKqSs 2lYeK53jcIKAHTOwAPRjLEDll0dx394v62MtSWtJl681iroGerR8VYDk= Received: from tb-mx1.topicbox.com (localhost.local [127.0.0.1]) by tb-mx1.topicbox.com (Postfix) with ESMTP id 0BDD31808F9 for ; Thu, 31 Oct 2024 16:48:01 -0400 (EDT) (envelope-from jcoombs@staff.gwi.net) Received: from tb-mx1.topicbox.com (localhost [127.0.0.1]) by tb-mx1.topicbox.com (Authentication Milter) with ESMTP id 956D16F2946; Thu, 31 Oct 2024 16:48:01 -0400 ARC-Seal: i=1; a=rsa-sha256; cv=none; d=topicbox.com; s=arcseal; t= 1730407681; b=QfZaMkhYkEgqQwcmX9XuLeg1u3oLQO3RLMA/I9e/QUfXssbhor 928S5MwCAttGrM9M1stZTrJ7W/fFs8e9NtHPpBJAMJi8XshHOODLGtc8XQ3fTiR5 Ni1C0jjFpSStKfbD4Ono9YidP/glRI4fgfcl9u55Suv0XNNBLj7BJZ7bw+4ttv+m HkUQcXTHV3D9wbUW6ns4nM3/vFP9mPf8UG6AXIbYau1AtNytckPRefLvYvaBF9/M Uqg9m2sNIGtNlB+7Q4wGZk07EyT611NUVbar0LB6fOQuPMEyNsdnXGUUVgvBlB1G KD0c+XnOUXcyI6UDBEceci2TchdpFpTqiVSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=mime-version:from:date:message-id:subject:to :content-type; s=arcseal; t=1730407681; bh=XO1DF/9lQ/VtzCdOlSubC EoS4GH2LiSRMBgBe0Akoxs=; b=KC2mmq+fInEPMuKRzWCRd//lffRTirod3+9wj 8RAW4BebiKcg8VUyIvT625/L2qQjBz16boM6j8d52QAqDwV2tzF2WRdgvcZK4bJG rHZ2wWS0Lv9O/0u20RrnYtXMGUG/egdcjj4qD6X34hSXSyIy/7su36jmSR+9mXn0 SUPLrHk05t9ddRp0zwCWPxhxHaOd6ReBd6of/pgeAudfqLVIJHyhf5Hz4Wdmeazh rgmlOIJm1+iLZpx7lsa9rKZJ+iYuKMhPFVNjILX4sO+aUe1Zf8cU37gG/RJL7L45 VGn+gutQ+6mnlJwkLtGgU61PcDou2W2CQfj6CeHelD78b/JWQ== ARC-Authentication-Results: i=1; tb-mx1.topicbox.com; arc=none (no signatures found); bimi=skipped (DMARC Policy is not at enforcement); dkim=pass (2048-bit rsa key sha256) header.d=staff.gwi.net header.i=@staff.gwi.net header.b=AHSPe1Yz header.a=rsa-sha256 header.s=google-1 x-bits=2048; dmarc=pass policy.published-domain-policy=none policy.published-subdomain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,sp=none,d=none,d.eval=none) policy.policy-from=sp header.from=staff.gwi.net; iprev=pass smtp.remote-ip=209.85.215.169 (mail-pg1-f169.google.com); spf=pass smtp.mailfrom=jcoombs@staff.gwi.net smtp.helo=mail-pg1-f169.google.com; x-aligned-from=pass (Address match); x-google-dkim=pass (2048-bit rsa key) header.d=1e100.net header.i=@1e100.net header.b=vp0xNniC; x-me-sender=none; x-ptr=pass smtp.helo=mail-pg1-f169.google.com policy.ptr=mail-pg1-f169.google.com; x-return-mx=pass header.domain=staff.gwi.net policy.org_domain=gwi.net policy.is_org=no (MX Records found: aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,alt1.aspmx.l.google.com,aspmx3.googlemail.com); x-return-mx=pass smtp.domain=staff.gwi.net policy.org_domain=gwi.net policy.is_org=no (MX Records found: aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,alt1.aspmx.l.google.com,aspmx3.googlemail.com); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=-100 state=0 X-ME-VSCause: gggruggvucftvghtrhhoucdtuddrgeeftddrvdekjedggeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepggfhfffkuffvtgesrgdtreertddtjeenucfhrhho mheplfhoshhhucevohhomhgsshcuoehjtghoohhmsghssehsthgrfhhfrdhgfihirdhnvg htqeenucggtffrrghtthgvrhhnpeekjeefheejffeghfeitdetjeelvdegleeiieevfedv jedvvdetieeifeevffegjeenucfkphepvddtledrkeehrddvudehrdduieelnecuvehluh hsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepvddtledrkeehrddvudehrddu ieelpdhhvghlohepmhgrihhlqdhpghduqdhfudeiledrghhoohhglhgvrdgtohhmpdhmrg hilhhfrhhomhepoehjtghoohhmsghssehsthgrfhhfrdhgfihirdhnvghtqedpnhgspghr tghpthhtohepuddprhgtphhtthhopeeoughishgtuhhssheslhhishhtshdrihhllhhumh hoshdrohhrgheq X-ME-VSScore: -100 X-ME-VSCategory: clean Received-SPF: pass (staff.gwi.net: Sender is authorized to use 'jcoombs@staff.gwi.net' in 'mfrom' identity (mechanism 'include:_spf.google.com' matched)) receiver=tb-mx1.topicbox.com; identity=mailfrom; envelope-from="jcoombs@staff.gwi.net"; helo=mail-pg1-f169.google.com; client-ip=209.85.215.169 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tb-mx1.topicbox.com (Postfix) with ESMTPS for ; Thu, 31 Oct 2024 16:48:00 -0400 (EDT) (envelope-from jcoombs@staff.gwi.net) Received: by mail-pg1-f169.google.com with SMTP id 41be03b00d2f7-7ea9739647bso1067315a12.0 for ; Thu, 31 Oct 2024 13:48:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730407678; x=1731012478; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=XO1DF/9lQ/VtzCdOlSubCEoS4GH2LiSRMBgBe0Akoxs=; b=vp0xNniCzQM5gPIdUytucg1qNm760DzwYvKJlNd2dRZe0m8Tv1m0BWYOjYiOo+V7ng nWM7FwDz2hR+s1kk1LoBG8a3zLim1UwG9RIBMgj7JDQiQbJbo0VKwdBrhLc6cmWFG4FA O5uDxxAu6sgxDhCVvKYqgiLiahnzEVsZcZCjIwZksPt3h8ZONOzCx9XO4r+mr/WR8flo K8Acv5oMFGTM2rBZR+l+tzGMaJI0lsuJp9jrnGyz95JUr0eyoUy6P/qF8fOubACdbJ0u gCYE8pPSXdjzsA3086kvIwuvYJeARIDluoscvtQhW5vo8AeWrCuyAIGmrxbo8Inqvuzt IneQ== X-Gm-Message-State: AOJu0Yx0hZgBcMIZWwZavFXdw8VweB66Wb/dH0vzFcqh/mQvpWNMosJK kNwwEs2jPpvinfLGJRFHXItJkSc3/lsZcHTcXS3sGYUuxUnKOJLNvdmOk5msSU4s98OFQCZKPt4 220P/ZL/5QZZ1QdanzXy6XrmKq1Ze/JYBDfTkutjfttAq/JyY X-Google-Smtp-Source: AGHT+IHE6Du/7AvTIRUGhKXYe8eABAc/qq3dF/trGgep7+wQhecN6+qfkS+ShaT4lv3p1rcjDdXPiQ6SDpoNNX7Wmrk= X-Received: by 2002:a17:90a:db87:b0:2e2:cd11:c9b with SMTP id 98e67ed59e1d1-2e8f1058107mr22853767a91.3.1730407678049; Thu, 31 Oct 2024 13:47:58 -0700 (PDT) MIME-Version: 1.0 From: Josh Coombs Date: Thu, 31 Oct 2024 16:47:21 -0400 Message-ID: To: illumos-discuss Content-Type: multipart/alternative; boundary=000000000000f7a3610625cbee52 Topicbox-Policy-Reasoning: allow: sender is a member Topicbox-Message-UUID: 6f756b80-97c9-11ef-8d88-facc234214ba Archived-At: =?UTF-8?B?PGh0dHBzOi8vaWxsdW1vcy50b3BpY2JveC5jb20vZ3JvdXBz?= =?UTF-8?B?L2Rpc2N1c3MvVDYwOGRhYjgwZTVkYjMwZjYtTWYxN2U3MjYxZWViMjNiZTRk?= =?UTF-8?B?YTZiYWI4Mz4=?= List-Help: List-Id: "illumos-discuss" List-Post: List-Software: Topicbox v0 List-Subscribe: Precedence: list Reply-To: illumos-discuss Subject: [discuss] Mellanox LACP aggr fails without snoop if using vlan tags Content-Transfer-Encoding: 7bit List-Unsubscribe: , Topicbox-Delivery-ID: 2:illumos:a9ae1d0a-2aea-11e7-965f-d98f9f16e227:81958daa-7202-11ef-bbf2-18142b2d11b0:Mf17e7261eeb23be4da6bab83:1:4SKxdURhRVA1V8KjOobxA1A26E9xCoIBN6fO4FazD-c --000000000000f7a3610625cbee52 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I've got a brand new r151050e install using a Mellanox CNX5 card, dual 25Gb ports paired up in an aggr to a Juniper EX4650 cluster. It will only work if I start snoop on the aggr. Without doing so, it won't pass traffic. I ran into this back in 2019 with bnx devices after upgrading to r151030 and was never able to find a fix, that box I ended up changing NICs to intel to get around the problem. It also works if I do a snoop -P -d aggr0 so it may not be promiscuous mode directly that's 'fixing' things? If I rebuild the setup and don't use VLAN tagging, that appears to work as anticipated which is different from my prior bnx troubles. Those I never got around to testing with vlans on OmniOS. When not working, I can see my aggr's mac address upstream on the switches. I haven't tried manually firing packets out an interface but based on last time I expect that will work. The issue seems to be on the receive side in OmniOS, something about starting a snoop suddenly allows OmniOS to accept the packets fully for some reason. I don't have a production and lab setup, sadly my test box is in a datacenter that is an hour away, unmanned so I'm limited in what I can do at the moment. I've got a little bit of time before I need to lock the conf for production use, but can only be onsite once or twice a week at best. I can live without vlan tagging, but if this is an unexpected fail I'd like to gather what data I can to help before shutting the door to testing? Joshua Coombs ------------------------------------------ illumos: illumos-discuss Permalink: https://illumos.topicbox.com/groups/discuss/T608dab80e5db30f6-Mf= 17e7261eeb23be4da6bab83 Delivery options: https://illumos.topicbox.com/groups/discuss/subscription --000000000000f7a3610625cbee52 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I've got a brand new r151050e install usin= g a Mellanox CNX5 card, dual 25Gb ports paired up in an aggr to a Juniper E= X4650 cluster. It will only work if I start snoop on the aggr. Wi= thout doing so, it won't pass traffic. I ran into this back in 2019 wit= h bnx devices after upgrading to r151030 and was never able to find a fix, = that box I ended up changing NICs to intel to get around the problem.
<= br />
It also works if I do a snoop -P -d aggr0 so it may not be = promiscuous mode directly that's 'fixing' things?

If I rebuild the setup and don't use VLAN tagging, t= hat appears to work as anticipated which is different from my prior bnx tro= ubles. Those I never got around to testing with vlans on OmniOS.
=
When not working, I can see my aggr's mac address upst= ream on the switches. I haven't tried manually firing packets out an in= terface but based on last time I expect that will work. The issue seems to = be on the receive side in OmniOS, something about starting a snoop sud= denly allows OmniOS to accept the packets fully for some reason.
=
I don't have a production and lab setup, sadly my test= box is in a datacenter that is an hour away, unmanned so I'm limited i= n what I can do at the moment. I've got a little bit of time before I n= eed to lock the conf for production use, but can only be onsite once or twi= ce a week at best. I can live without vlan tagging, but if this is an unexp= ected fail I'd like to gather what data I can to help before shutting t= he door to testing?

Joshua Coombs
<= /div>
= --000000000000f7a3610625cbee52--