From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2 autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 25721 invoked from network); 24 Sep 2022 08:55:26 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 24 Sep 2022 08:55:26 -0000 Received: (qmail 9233 invoked by uid 550); 24 Sep 2022 08:55:23 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 8183 invoked from network); 24 Sep 2022 08:55:22 -0000 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:to:from:date:x-gm-message-state:from:to:cc:subject:date; bh=Zqs/D8WaQpMnp8lNiyfJ+99wihFwfMI2V1bnSBwPYR4=; b=rwnjkFjZURBMmmUrYYb9AHCQ47dwO+Epu/6/h6jkxkm6xCtKYpF9r0T4SK+5uLMTiB IxtcMCB8SiRUuoq92PSbf/LbOMaYQxHnaBNyEXak2oLjq2RGA+6QUb3o02S9Myt1vfkt ijc4uFsYBxhEduxgXnekoWFvaKk29G3KdEMSmvULDbCpmUQpXDp/fOgWM6L3XEoiltWO BoWPgnULrlIHRFOkbPqai1jXMplMgp7jhblN/D3S++mGdB5X8NChZAHX6449qokzt0UH 359m0LCO1G6l+p/wCAC5l92eqGsSBj+Rxdq33R/ggL3BHY6Svs/Fl0sTzl9G4dkjlzay XCLw== X-Gm-Message-State: ACrzQf0V1K9Sh7GDUZdskANFd7dSUQhnsr9tchFXrCr5I/w+QJcCCDjy 9eAusmVnGoojDcgU0hpVaP4= X-Google-Smtp-Source: AMsMyM6mNu7fVj5LUeUMRp8M8YZgVEqmOeqmDiiXI27DzD7+BIXrahOHEiSkXbtrg9NwxqXXfQlDDg== X-Received: by 2002:a17:902:e850:b0:178:64f:a57d with SMTP id t16-20020a170902e85000b00178064fa57dmr12768581plg.110.1664009710772; Sat, 24 Sep 2022 01:55:10 -0700 (PDT) Date: Sat, 24 Sep 2022 01:55:09 -0700 From: Fangrui Song To: James Y Knight , musl@lists.openwall.com, Florian Weimer , Rich Felker , baiyang Message-ID: <20220924085509.zftbujx224aam5hd@gmail.com> References: <2022091915532777412615@gmail.com> <20220919110829.GA2158779@port70.net> <874jx3h76u.fsf@oldenburg.str.redhat.com> <20220919134659.GO9709@brightrain.aerifal.cx> <874jx2phqm.fsf@oldenburg.str.redhat.com> <20220920173423.GE2158779@port70.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <20220920173423.GE2158779@port70.net> Subject: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) On 2022-09-20, Szabolcs Nagy wrote: >* James Y Knight [2022-09-20 12:59:00 -0400]: > >> On Tue, Sep 20, 2022 at 9:58 AM Siddhesh Poyarekar >> wrote: >> >> > Adding support for something that's already declared as bad >> > programming practice seems like a step backwards. Instead, I hope we >> > find a way to discourage active use of malloc_usable_size more >> > strongly. >> >> >> BTW, if folks aren't aware, there is already work on the C++ side to expose >> an API which lets you request a heap allocation of _at least_ the given >> size, which rounds the actual size up in whatever way the allocator likes, >> and returns the pointer and actual size allocated. With this API, you >> declare an explicit intent that all of the memory -- up to the returned >> size -- is valid to use without needing to go back to the allocator to ask >> for more. >> >> The proposal is still making its way through the standardization process, >> but hopefully it'll make it into the next version of C++ after C++23. (Of >> course, that's not a sure thing until it happens.) Here's the doc, with >> more rationale/etc: >> https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p0901r9.html > >this does not seem to discuss how existing applications >that override new() would cope with this. > >nor how existing implementations on top of c allocators >would implement it (given that we just agreed that >malloc_usable_size is not suitable for such use). > >nor how existing allocator tooling (interposers, profilers) >would handle the new interface. > >> >> Also, as noted in the doc, jemalloc experimentally implemented this >> functionality in its non-standard API, via a function it called "smallocx" >> -- though jemalloc hides the API so it can't be used by default. The API is >> effectively: >> typedef struct { void *ptr; size_t size; } smallocx_return_t; >> smallocx_return_t smallocx(size_t size, int flags); >> https://github.com/jemalloc/jemalloc/blob/a0734fd6ee326cd2059edbe4bca7092988a63684/src/jemalloc.c#L3414 >> (That's consistent with jemalloc's other non-standard APIs, which stick >> alignment/etc into a "flags" argument, but probably not suitable for a >> more-standardized cross-implementation API) >> >> tcmalloc implements similar functionality, as well, with family of >> functions named "tcmalloc_size_returning_operator_new": > >so there are already incompatible c apis, which means this >should not be considered a viable proposal at this point. Small addition: https://wg21.link/P0401R6 (allocate_at_least) has made it into C++23. https://reviews.llvm.org/D122877 libc++ has implemented it in the trivial way that just returns the user-requested size. >> https://github.com/google/tcmalloc/blob/267aa2ec2817ab9d09b3fbb65ecb90193dd4348e/tcmalloc/malloc_extension.h#L549 >> which of course also isn't a suitable API to support cross-implementation. >> >> If someone wants to push forward this area, IMO, it would be really great >> to have an API exposing this functionality designed to be implemented in a >> common way across libc malloc implementations -- and eventually added to >> POSIX or C. > >this is done the wrong way around.