From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2 autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 11356 invoked from network); 10 Oct 2022 14:13:46 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 10 Oct 2022 14:13:46 -0000 Received: (qmail 3337 invoked by uid 550); 10 Oct 2022 14:13:38 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 3317 invoked from network); 10 Oct 2022 14:13:38 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665411210; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=SDnn1cUkc5/U1M6MEm1RIZIum47KrVXVH898t+z6cPU=; b=dEoDGFAwcre2baIMeLgDUpBBy4J50WyNqMREY/8VfZqC5z5fsTb2J6DJz/14sQkGB+iTFQ zFru2iYqyy2bLwbZOYHnX5hySBIcqSi5OGo31s0Nn8lo561prPPoQA3jB8AHzRGxAkp6zC CV8BXLOVxlMFCXNahaXMEmHRcGUvcMI= X-MC-Unique: f-c6OsZqNHiREkf7pWpOlw-1 From: Florian Weimer To: Siddhesh Poyarekar Cc: Rich Felker , baiyang , musl References: <2022091915532777412615@gmail.com> <20220919110829.GA2158779@port70.net> <874jx3h76u.fsf@oldenburg.str.redhat.com> <20220919134659.GO9709@brightrain.aerifal.cx> <874jx2phqm.fsf@oldenburg.str.redhat.com> Date: Mon, 10 Oct 2022 16:13:23 +0200 In-Reply-To: (Siddhesh Poyarekar's message of "Tue, 20 Sep 2022 09:54:43 -0400") Message-ID: <87lepnzrzw.fsf@oldenburg.str.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Subject: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) * Siddhesh Poyarekar: > On Tue, Sep 20, 2022 at 4:34 AM Florian Weimer wrote: >> The compiler needs to treat malloc_usable_size similar to realloc and >> just the size information for the buffer based on the return value from >> malloc_usable_size. This is admittedly harder to do than a comparable >> analysis for realloc if the compiler interprets the standard in such a >> way that after a successful realloc, any access to the original pointer >> value is undefined. >> >> malloc_usable_size is not actually *that* useful with allocators that do >> not have strict size classes because they do not over-allocate that >> much. For these allocators, it may be possible to increase the size of >> allocation significantly without moving it, but that is not reflected in >> the return value of malloc_usable_size at all. > > So the glibc manual does not document malloc_usable_size semantics > (which is weird since it is, well, a GNU extension!) I think we got it via dlmalloc, which says this: /* malloc_usable_size(void* p); Returns the number of bytes you can actually use in an allocated chunk, which may be more than you requested (although often not) due to alignment and minimum size constraints. You can use this many bytes without worrying about overwriting other allocated objects. This is not a particularly great programming practice. malloc_usable_size can be more useful in debugging and assertions, for example: p = malloc(n); assert(malloc_usable_size(p) >= 256); */ I don't think it's a GNU invention. The GNU malloc used to be Mike Haertel's malloc, as far as I can tell, and that didn't have the function. GNU malloc had a malloc_object_allocation_size function at one point, it seems, but I don't know if that was before it was a rebranded dlmalloc. The malloc subsystem wasn't part of the glibc CVS at the time. > Adding support for something that's already declared as bad > programming practice seems like a step backwards. Instead, I hope we > find a way to discourage active use of malloc_usable_size more > strongly. At least based on the systemd experience, the problem they > try to solve is that of glibc realloc being too slow for paths where > the reallocation should return the same block and that should be easy > to special-case. Is there any other valid reason to use > malloc_usable_size instead of simply using realloc? Do you know which case systemd tries to optimize? Increasing the allocation size or lowering it? Maybe we should define a non-copying version of realloc that if that's what programmers want. I've got a write-up somewhere what an a replacement for class-free allocators of jemalloc's xallocx function would look like. I think it has to involve arithmetic progressions (but no L-series), so it's not pretty. Anyway, if there is no real use case for malloc_usable_size with the current glibc realloc implementation, we should really deprecate it. Especially since C++ is moving into an incompatible direction. Thanks, Florian