From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2 autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 31336 invoked from network); 19 Sep 2022 18:16:12 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 19 Sep 2022 18:16:12 -0000 Received: (qmail 23622 invoked by uid 550); 19 Sep 2022 18:16:09 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 23590 invoked from network); 19 Sep 2022 18:16:08 -0000 Date: Mon, 19 Sep 2022 14:15:56 -0400 From: Rich Felker To: baiyang Cc: musl@lists.openwall.com Message-ID: <20220919181556.GT9709@brightrain.aerifal.cx> References: <2022091915532777412615@gmail.com> <20220919134319.GN9709@brightrain.aerifal.cx> <202209200132289145679@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <202209200132289145679@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) On Tue, Sep 20, 2022 at 01:32:31AM +0800, baiyang wrote: > Hi Rich, > > Thanks for your reply. > > > Unless you have an application that's explicitly using > > malloc_usable_size all over the place, it's highly unlikely that this > > is the cause of your real-world performance problems. > > 1. Yes, we have a real scenario where `malloc_usable_size` is called > frequently: we need to optimize the realloc experience. We add an > extra parameter to realloc - minimalCopyBytes: it represents the > actual size of data that needs to be copied after fallback to > malloc-copy-free mode. We will judge whether to call realloc or > complete malloc-memcpy-free by ourself based on factors such as the > size of the data that realloc needs to copy (obtained through > `malloc_usable_size`), the size that we actually need to copy when > we doing malloc-memcpy-free ourself (minimalCopyBytes) and the > chance of merging chunks (small blocks) or mremap (large blocks) in > the underlayer realloc. So, this is a real scenario, we need to call > `malloc_usable_size` frequently. Is there a reason you're relying on an unreliable and nonstandard function (malloc_usable_size) to do this rather than your program keeping track of its own knowledge of the allocated size? This is what the C language expects you to do. For example if you have a structure that contains a pointer to a dynamically sized buffer, normally you store the size in a size_t member right next to that pointer, allowing you to make these kind of decisions without having to probe anything. > 2. As I mentioned before, this isn't just a problem with > `malloc_usable_size`, since we actually include a full > `malloc_usable_size` procedure in both `realloc` and `free`, it > actually slows down The speed of other calls such as `free` and > `realloc`. So this problem actually slows down not only the > `malloc_usable_size` call itself, but also the realloc and free > calls. If this is affecting you too, that's a separate issue. But I can't tell from what you've reported so far whether you're just claiming this on a theoretical basis or whether you're actually experiencing unacceptable performance.