From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.zx2c4.com (lists.zx2c4.com [165.227.139.114]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AB685C27C4F for ; Thu, 13 Jun 2024 15:15:47 +0000 (UTC) Received: by lists.zx2c4.com (ZX2C4 Mail Server) with ESMTP id 4f6b014a; Thu, 13 Jun 2024 15:12:10 +0000 (UTC) Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lists.zx2c4.com (ZX2C4 Mail Server) with ESMTPS id 4948687d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Thu, 13 Jun 2024 15:12:08 +0000 (UTC) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6341761740; Thu, 13 Jun 2024 15:12:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B4A1C2BBFC; Thu, 13 Jun 2024 15:12:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1718291526; bh=Js8RVIcQWrK8SBzBs0EgFN3NPMVLrfZVEqHlgM+8ujY=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=PLFhTImnGBfruVfem7uoDdRQY9dt85NU9uz9NI1nWR1Tl8ZwZA2Rn8Oo+pStfyXsV EcyX+RV9PlAq8NGbBuCq1xDrwXHGIZbMJoy9lp/uxPIyy3x1VXFO0lRdU9guvTXWpK fWfovpAiSyeT2cI38X08BEvOMCHEP5MPP0MmNOwv9CguIjdgxn95t02Yg8rhsTjHrv JMsJdorpaWlfhjWGwd4su7F+sxA8gjgfARcKbZKcoULYJLtjoN0Cnd2hb1zWGCyIw6 EdYmQujOfzjZ7hZCGpSCTtFxQu7IFk8SsutSm4TeEzC3kDQYyJTjRTvS5rNl7PGpcJ CzwVznRaVDJlg== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id A66DDCE09E0; Thu, 13 Jun 2024 08:12:05 -0700 (PDT) Date: Thu, 13 Jun 2024 08:12:05 -0700 From: "Paul E. McKenney" To: "Jason A. Donenfeld" Cc: Jakub Kicinski , Julia Lawall , linux-block@vger.kernel.org, kernel-janitors@vger.kernel.org, bridge@lists.linux.dev, linux-trace-kernel@vger.kernel.org, Mathieu Desnoyers , kvm@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, "Naveen N. Rao" , Christophe Leroy , Nicholas Piggin , netdev@vger.kernel.org, wireguard@lists.zx2c4.com, linux-kernel@vger.kernel.org, ecryptfs@vger.kernel.org, Neil Brown , Olga Kornievskaia , Dai Ngo , Tom Talpey , linux-nfs@vger.kernel.org, linux-can@vger.kernel.org, Lai Jiangshan , netfilter-devel@vger.kernel.org, coreteam@netfilter.org, Vlastimil Babka Subject: Re: [PATCH 00/14] replace call_rcu by kfree_rcu for simple kmem_cache_free callback Message-ID: <6595ff2a-690e-4d6c-9be5-eb83f2df23fa@paulmck-laptop> References: <20240609082726.32742-1-Julia.Lawall@inria.fr> <20240612143305.451abf58@kernel.org> <08ee7eb2-8d08-4f1f-9c46-495a544b8c0e@paulmck-laptop> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-BeenThere: wireguard@lists.zx2c4.com X-Mailman-Version: 2.1.30rc1 Precedence: list List-Id: Development discussion of WireGuard List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: paulmck@kernel.org Errors-To: wireguard-bounces@lists.zx2c4.com Sender: "WireGuard" On Thu, Jun 13, 2024 at 04:11:52PM +0200, Jason A. Donenfeld wrote: > On Thu, Jun 13, 2024 at 05:46:11AM -0700, Paul E. McKenney wrote: > > How about a kmem_cache_destroy_rcu() that marks that specified cache > > for destruction, and then a kmem_cache_destroy_barrier() that waits? > > > > I took the liberty of adding your name to the Google document [1] and > > adding this section: > > Cool, though no need to make me yellow! No worries, Jakub is also colored yellow. People added tomorrow will be cyan if I follow my usual change-color ordering. ;-) > > > But then, if that mechanism generally works, we don't really need a new > > > function and we can just go with the first option of making > > > kmem_cache_destroy() asynchronously wait. It'll wait, as you described, > > > but then we adjust the tail of every kfree_rcu batch freeing cycle to > > > check if there are _still_ any old outstanding kmem_cache_destroy() > > > requests. If so, then we can splat and keep the old debugging info we > > > currently have for finding memleaks. > > > > The mechanism can always be sabotaged by memory-leak bugs on the part > > of the user of the kmem_cache structure in play, right? > > > > OK, but I see your point. I added this to the existing > > "kmem_cache_destroy() Lingers for kfree_rcu()" section: > > > > One way of preserving this debugging information is to splat if > > all of the slab’s memory has not been freed within a reasonable > > timeframe, perhaps the same 21 seconds that causes an RCU CPU > > stall warning. > > > > Does that capture it? > > Not quite what I was thinking. Your 21 seconds as a time-based thing I > guess could be fine. But I was mostly thinking: > > 1) kmem_cache_destroy() is called, but there are outstanding objects, so > it defers. > > 2) Sometime later, a kfree_rcu_work batch freeing operation runs. Or not, if there has been a leak and there happens to be no outstanding kfree_rcu() memory. > 3) At the end of this batch freeing, the kernel notices that the > kmem_cache whose destruction was previously deferred still has > outstanding objects and has not been destroyed. It can conclude that > there's thus been a memory leak. And the batch freeing can be replicated across CPUs, so it would be necessary to determine which was last to do this effective. Don't get me wrong, this can be done, but the performance/latency tradeoffs can be interesting. > In other words, instead of having to do this based on timers, you can > just have the batch freeing code ask, "did those pending kmem_cache > destructions get completed as a result of this last operation?" I agree that kfree_rcu_work-batch time is a good time to evaluate slab (and I have added this to the document), but I do not believe that it can completely replace timeouts. Thanx, Paul