From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.zx2c4.com (lists.zx2c4.com [165.227.139.114]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8B431C433FE for ; Fri, 22 Apr 2022 00:45:04 +0000 (UTC) Received: by lists.zx2c4.com (OpenSMTPD) with ESMTP id 2e96d275; Fri, 22 Apr 2022 00:02:30 +0000 (UTC) Received: from dfw.source.kernel.org (dfw.source.kernel.org [2604:1380:4641:c500::1]) by lists.zx2c4.com (OpenSMTPD) with ESMTPS id 398175b4 (TLSv1.2:ECDHE-ECDSA-AES256-GCM-SHA384:256:NO) for ; Fri, 22 Apr 2022 00:02:29 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5DCC261F09; Fri, 22 Apr 2022 00:02:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1FEECC385A7; Fri, 22 Apr 2022 00:02:26 +0000 (UTC) Authentication-Results: smtp.kernel.org; dkim=pass (1024-bit key) header.d=zx2c4.com header.i=@zx2c4.com header.b="U1g8W4w5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zx2c4.com; s=20210105; t=1650585744; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s9YJ+AxOvCGo+608yNkBuz6fniGjDnN27/vTN4qozqQ=; b=U1g8W4w5mXPOSzp02yDVyUOJq3OHK6F9LOhkjdvSk/1MX063a58lwwKfrPDj2Y/UZx+loo Xl7bK2lV1BkP7iJYsjq9sNzhJ57z5tVGLY5eAZpvDF5S/QviE4shKPxlmkmyp5JR9LxTyZ c5LFRt48T/pkfculMStrb+EsyJSr7VI= Received: by mail.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id eaac2843 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); Fri, 22 Apr 2022 00:02:24 +0000 (UTC) Date: Fri, 22 Apr 2022 02:02:21 +0200 From: "Jason A. Donenfeld" To: Charles-Francois Natali Cc: wireguard@lists.zx2c4.com, netdev@vger.kernel.org, linux-crypto@vger.kernel.org, Daniel Jordan , Steffen Klassert Subject: Re: [PATCH] WireGuard: restrict packet handling to non-isolated CPUs. Message-ID: References: <20220405212129.2270-1-cf.natali@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20220405212129.2270-1-cf.natali@gmail.com> X-BeenThere: wireguard@lists.zx2c4.com X-Mailman-Version: 2.1.30rc1 Precedence: list List-Id: Development discussion of WireGuard List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: wireguard-bounces@lists.zx2c4.com Sender: "WireGuard" netdev@ - Original thread is at https://lore.kernel.org/wireguard/20220405212129.2270-1-cf.natali@gmail.com/ Hi Charles-François, On Tue, Apr 05, 2022 at 10:21:29PM +0100, Charles-Francois Natali wrote: > WireGuard currently uses round-robin to dispatch the handling of > packets, handling them on all online CPUs, including isolated ones > (isolcpus). > > This is unfortunate because it causes significant latency on isolated > CPUs - see e.g. below over 240 usec: > > kworker/47:1-2373323 [047] 243644.756405: funcgraph_entry: | > process_one_work() { kworker/47:1-2373323 [047] 243644.756406: > funcgraph_entry: | wg_packet_decrypt_worker() { [...] > kworker/47:1-2373323 [047] 243644.756647: funcgraph_exit: 0.591 us | } > kworker/47:1-2373323 [047] 243644.756647: funcgraph_exit: ! 242.655 us > | } > > Instead, restrict to non-isolated CPUs. Huh, interesting... I haven't seen this feature before. What's the intended use case? To never run _anything_ on those cores except processes you choose? To run some things but not intensive things? Is it sort of a RT-lite? I took a look in padata/pcrypt and it doesn't look like they're examining the housekeeping mask at all. Grepping for housekeeping_cpumask doesn't appear to show many results in things like workqueues, but rather in core scheduling stuff. So I'm not quite sure what to make of this patch. I suspect the thing to do might be to patch both wireguard and padata, and send a patch series to me, the padata people, and netdev@vger.kernel.org, and we can all hash this out together. Regarding your patch, is there a way to make that a bit more succinct, without introducing all of those helper functions? It seems awfully verbose for something that seems like a matter of replacing the online mask with the housekeeping mask. Jason