From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2 autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 32420 invoked from network); 20 Sep 2022 17:34:38 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 20 Sep 2022 17:34:38 -0000 Received: (qmail 31981 invoked by uid 550); 20 Sep 2022 17:34:36 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 31961 invoked from network); 20 Sep 2022 17:34:35 -0000 Date: Tue, 20 Sep 2022 19:34:23 +0200 From: Szabolcs Nagy To: James Y Knight Cc: musl@lists.openwall.com, Florian Weimer , Rich Felker , baiyang Message-ID: <20220920173423.GE2158779@port70.net> Mail-Followup-To: James Y Knight , musl@lists.openwall.com, Florian Weimer , Rich Felker , baiyang References: <2022091915532777412615@gmail.com> <20220919110829.GA2158779@port70.net> <874jx3h76u.fsf@oldenburg.str.redhat.com> <20220919134659.GO9709@brightrain.aerifal.cx> <874jx2phqm.fsf@oldenburg.str.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) * James Y Knight [2022-09-20 12:59:00 -0400]: > On Tue, Sep 20, 2022 at 9:58 AM Siddhesh Poyarekar > wrote: > > > Adding support for something that's already declared as bad > > programming practice seems like a step backwards. Instead, I hope we > > find a way to discourage active use of malloc_usable_size more > > strongly. > > > BTW, if folks aren't aware, there is already work on the C++ side to expose > an API which lets you request a heap allocation of _at least_ the given > size, which rounds the actual size up in whatever way the allocator likes, > and returns the pointer and actual size allocated. With this API, you > declare an explicit intent that all of the memory -- up to the returned > size -- is valid to use without needing to go back to the allocator to ask > for more. > > The proposal is still making its way through the standardization process, > but hopefully it'll make it into the next version of C++ after C++23. (Of > course, that's not a sure thing until it happens.) Here's the doc, with > more rationale/etc: > https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p0901r9.html this does not seem to discuss how existing applications that override new() would cope with this. nor how existing implementations on top of c allocators would implement it (given that we just agreed that malloc_usable_size is not suitable for such use). nor how existing allocator tooling (interposers, profilers) would handle the new interface. > > Also, as noted in the doc, jemalloc experimentally implemented this > functionality in its non-standard API, via a function it called "smallocx" > -- though jemalloc hides the API so it can't be used by default. The API is > effectively: > typedef struct { void *ptr; size_t size; } smallocx_return_t; > smallocx_return_t smallocx(size_t size, int flags); > https://github.com/jemalloc/jemalloc/blob/a0734fd6ee326cd2059edbe4bca7092988a63684/src/jemalloc.c#L3414 > (That's consistent with jemalloc's other non-standard APIs, which stick > alignment/etc into a "flags" argument, but probably not suitable for a > more-standardized cross-implementation API) > > tcmalloc implements similar functionality, as well, with family of > functions named "tcmalloc_size_returning_operator_new": so there are already incompatible c apis, which means this should not be considered a viable proposal at this point. > > https://github.com/google/tcmalloc/blob/267aa2ec2817ab9d09b3fbb65ecb90193dd4348e/tcmalloc/malloc_extension.h#L549 > which of course also isn't a suitable API to support cross-implementation. > > If someone wants to push forward this area, IMO, it would be really great > to have an API exposing this functionality designed to be implemented in a > common way across libc malloc implementations -- and eventually added to > POSIX or C. this is done the wrong way around.