From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,HTML_FONT_FACE_BAD,HTML_MESSAGE, MAILING_LIST_MULTI,NICE_REPLY_A,RCVD_IN_MSPIKE_H2, T_KAM_HTML_FONT_INVALID autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 20835 invoked from network); 20 Sep 2022 20:33:52 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 20 Sep 2022 20:33:52 -0000 Received: (qmail 23981 invoked by uid 550); 20 Sep 2022 20:33:49 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 23958 invoked from network); 20 Sep 2022 20:33:49 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:from:references:cc:to:content-language:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date; bh=vYyRyavszdwRTwGW5sD2+5UIxo/wvjtV9Ue9UHVFcJM=; b=R3pkWHQRNUcDDSCHXOBt47prML2Ibkp/U/kW3D55MUI6ttbkCURC+YmEmKuutXX8jH X4UQDWVEss27QTJt+/OYBWYup27GPcniTDqmr2vd2oZzx9txrH+M0+3sPNrwzFg8Xhm9 CZNJWFHE7NB9UMY+BVknfBGr293rxVdfU0kA6JnqCOXxAThXNgpFC0XFjDce1XIOCKKW vKv9A7dtxTWXCHAIR7LoWFrwHFb1w6VeJCn63SEk4n3VawskOBYUZsa5nzl4cFXiL/PP XptCYQe33J/UorWXeuLMUcl6U0PiDvc3I7Mil7ACHWsqHvjNAxaPsN5V4F5nYISV3v7F vIvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:from:references:cc:to:content-language:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date; bh=vYyRyavszdwRTwGW5sD2+5UIxo/wvjtV9Ue9UHVFcJM=; b=E/9Zzre6Lq6H+bF12lj0/ZywxxJaD6OUB8yOhKaWwfZpTm9y5fhbtsy3C9sJzRx3Ry 1mkdstCBCtKNdu1oj7q5HJfGorXxf6KFI9pA7RfTpLAcxI064UF5tYWhgO7UAT9v0O+R ZpWbqVz1tNsZOoBtFZ54BmQJHCCnk6fGBxFWg6TBUW7rJ5L2+9nk3WX2d/NV2kIdCaUF vXcU3VYKkLL4AVC6ayNxuCUD1dPssisRswZU8cxm2y/CeRr8iVOKSZBH8NCa9+TA8mAy YDrXjUihxpgmv9k5TwoxWw/n3XCefUtsytPR2oPts2rkBPfsWYv09DgvrIBPGxs2yptE Zq1w== X-Gm-Message-State: ACrzQf0PqQIgf04ahS0d+NHj6GFXyq8wpp7rpjkA0HyEq3qsKM2lkr0r q0q94Y5t0olt4VVKDtsIj4E4sMUKPWn/lqMr X-Google-Smtp-Source: AMsMyM6ZLhDDqfJxNGXxrizYT68xAkzMtfGlmn5WN9wOj5V9Ji4XOLrHfrEUows2IPFXGL8LFpNiUQ== X-Received: by 2002:adf:e10c:0:b0:225:3168:c261 with SMTP id t12-20020adfe10c000000b002253168c261mr14884234wrz.159.1663706017240; Tue, 20 Sep 2022 13:33:37 -0700 (PDT) Content-Type: multipart/alternative; boundary="------------HLjK9IMzkV3VggPjFSOYWj4X" Message-ID: Date: Tue, 20 Sep 2022 22:33:32 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.2.1 Content-Language: en-US To: musl@lists.openwall.com, baiyang Cc: Quentin Rameau , Florian Weimer References: <2022091915532777412615@gmail.com> <20220919110829.GA2158779@port70.net> <874jx3h76u.fsf@oldenburg.str.redhat.com> <20220919134659.GO9709@brightrain.aerifal.cx> <874jx2phqm.fsf@oldenburg.str.redhat.com> <2022092101393822582117@gmail.com> <20220920201244.4f40362e.quinq@fifth.space> <20220920181905.GR9709@brightrain.aerifal.cx> <2022092102351588157126@gmail.com> From: Gabriel Ravier In-Reply-To: <2022092102351588157126@gmail.com> Subject: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) This is a multi-part message in MIME format. --------------HLjK9IMzkV3VggPjFSOYWj4X Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 9/20/22 20:35, baiyang wrote: > > So what you're actually trying to do is more clear now. > > And this is something that you should do on the “application layer”, > > not expect the libc to magically taking care of this. > > Yes, we just do this at the application layer with the help of APIs > such as malloc_usable_size. You can look at the previous emails, isn't > that what we're discussing? > > > They want to know if realloc will resize the allocation in-place so > > the internal memcpy will not happen. > > AIUI, what they really need is not "usable_size", but "cost estimation > > for resizing allocation at pointer P to size S". Which I believe they > > try to deduce from malloc_usable_size. > > Yes, from malloc_usable_size, the size that the application layer > actually needs to copy, and other parameters like the OS, allocator, etc. > At this point it feels more and more like malloc basically can't fulfill your needs unless every implementation out there starts adding specialized methods for your application. In other words, it might be more productive for you to simply bring in a more agreeable allocator, instead of trying to get every single allocator out there to bow down to your needs. --------------HLjK9IMzkV3VggPjFSOYWj4X Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
On 9/20/22 20:35, baiyang wrote:
> So what you're actually trying to do is more clear now.
> And this is something that you should do on the “application layer”,
> not expect the libc to magically taking care of this.

Yes, we just do this at the application layer with the help of APIs such as malloc_usable_size. You can look at the previous emails, isn't that what we're discussing?

> They want to know if realloc will resize the allocation in-place so
> the internal memcpy will not happen.
> AIUI, what they really need is not "usable_size", but "cost estimation
> for resizing allocation at pointer P to size S". Which I believe they
> try to deduce from malloc_usable_size. 

Yes, from malloc_usable_size, the size that the application layer actually needs to copy, and other parameters like the OS, allocator, etc. 

At this point it feels more and more like malloc basically can't fulfill your needs unless every implementation out there starts adding specialized methods for your application. In other words, it might be more productive for you to simply bring in a more agreeable allocator, instead of trying to get every single allocator out there to bow down to your needs.

--------------HLjK9IMzkV3VggPjFSOYWj4X--