From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,HTML_FONT_FACE_BAD,HTML_MESSAGE, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,T_KAM_HTML_FONT_INVALID autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 596 invoked from network); 20 Sep 2022 17:39:53 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 20 Sep 2022 17:39:53 -0000 Received: (qmail 2016 invoked by uid 550); 20 Sep 2022 17:39:50 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 1986 invoked from network); 20 Sep 2022 17:39:49 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:mime-version:references:subject:cc:to:from:date:from:to :cc:subject:date; bh=RQD70xG+wb0vBCed2jLN1BLFMhXkKlOEeZroxOdxlAE=; b=RAaDI860ATdJ7XrGs3aD0uBaqIvXM3HAaQz6bJXGUbnjp/rT367m2MZ4z4apbpmRrG HhuHHgw0Ehp2p6yJu3sPoUgWbTcsxcOMKeoF3fXmaBLVSWxcMTaLRYHp+a4s0RgmEjJb nM9f8OOho3YwPRujMksYUqqZEOWk4UD2ISDdlLfTqsXaCzMyz0/mYUQuGif7VFcKf9Xn oz7vvcovqMf1QJLUQ15ULNtHlE7IBWiLLdtOH3rIa+veVpVkamr4auFeujP3PxI3H49M bqHRozi6UGE6CB7Nnv6y5Cj5ODUi73d0ZpzihRXzsYT04mKX+usXUgPNaEuznxFj3hxt FACQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=message-id:mime-version:references:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date; bh=RQD70xG+wb0vBCed2jLN1BLFMhXkKlOEeZroxOdxlAE=; b=WcT/oGfWhHshTqTUmYV8+co9fXFSb6mfQn8H1qpbGpPLYJa08JG3KuLi+wyP5vYdzR +fuik/rj8ycU1QOu9x1Gn7+Lv3VX6tttbRPegDf1O9KIrkjYqIvsVfOaY2wlxzCy7oM5 NQNFFgF8OQU+F+ScOmAOUueKqqImVGoXQeWTvsJKaeFO/JCnulywBUmeofHnqr68tRrv 1VZ62ZcTa86/Vqoq6kaZ0h6U92lsi45WLlOP7UsU9yVryuX7zvETEoJlhB2HvrerTodh mjntXztnHtpvfwCdl5MS6s8ZDqref+ITDDK0/AQfPuICF2B+TdLXmDyL6iyqpI8NCQge gS8g== X-Gm-Message-State: ACrzQf3ZjuEhU+DbaXtbgkZROz5RVsJOmYLYVEfyN4RksKLuZ+JqDq9F fdPWXSWHyfnmdu++zX+e3G0= X-Google-Smtp-Source: AMsMyM7yk5w5pJxO4P2n1gFhx+N/Zl0cV25dpUBBr6ojHLFaSdfJeIlv8IbCMg83/AJfZsKugj+/yw== X-Received: by 2002:a17:902:c40e:b0:178:4931:cc4d with SMTP id k14-20020a170902c40e00b001784931cc4dmr703092plk.97.1663695577520; Tue, 20 Sep 2022 10:39:37 -0700 (PDT) Date: Wed, 21 Sep 2022 01:39:39 +0800 From: baiyang To: "James Y Knight" , musl Cc: "Florian Weimer" , "Rich Felker" References: <2022091915532777412615@gmail.com>, <20220919110829.GA2158779@port70.net>, <874jx3h76u.fsf@oldenburg.str.redhat.com>, <20220919134659.GO9709@brightrain.aerifal.cx>, <874jx2phqm.fsf@oldenburg.str.redhat.com>, , X-Priority: 3 X-GUID: 19AE89E1-C99A-4A5B-A1FA-AD7DA13A1767 X-Has-Attach: no X-Mailer: Foxmail 7.2.23.116[cn] Mime-Version: 1.0 Message-ID: <2022092101393822582117@gmail.com> Content-Type: multipart/alternative; boundary="----=_001_NextPart216102362172_=----" Subject: Re: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) This is a multi-part message in MIME format. ------=_001_NextPart216102362172_=---- Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 PiB0Y21hbGxvYyBpbXBsZW1lbnRzIHNpbWlsYXIgZnVuY3Rpb25hbGl0eSwgYXMgd2VsbCwgd2l0 aCBmYW1pbHkgb2YgZnVuY3Rpb25zIG5hbWVkICJ0Y21hbGxvY19zaXplX3JldHVybmluZ19vcGVy YXRvcl9uZXciDQoNClRoaXMgaXMgYSBnb29kIHRoaW5nLCB3ZSBub3RpY2VkIGl0IGJlZm9yZS4g QnV0IHdlIGFjdHVhbGx5IG5lZWQgYSByZWFsbG9jIHRoYXQgKipjYW4gc3BlY2lmeSB0aGUgY29w eSByYW5nZSoqLg0KDQpBcyBpbiBvdXIgcHJldmlvdXMgZXhhbXBsZSwgc3VwcG9zZSB3ZSAidm9p ZCogcCBtYWxsb2MoMjAwS0IpIiBhbmQgIm1hbGxvY191c2FibGVfc2l6ZShwKSIgcmV0dXJucyAy NTZLQi4gRm9yIGFsbG9jYXRvcnMgc3VjaCBhcyB0Y21hbGxvYywgaWYgd2UgZG8gInJlYWxsb2Mo cCwgMzAwS0IpIiwgaXQgd2lsbCBhY3R1YWxseSBleGVjdXRlICJtYWxsb2MoMzAwS0IpK21lbWNw eSgqKjI1NktCKiopK2ZyZWUoMjU2S0IpIi4gQnV0IGF0IHRoaXMgdGltZSwgdGhlIGFjdHVhbCBi dXNpbmVzcyBvZiB0aGUgYXBwbGljYXRpb24gbGF5ZXIgb2Z0ZW4gb25seSBuZWVkcyB0byBjb3B5 IGEgc21hbGwgYW1vdW50IG9mIGNvbnRlbnQsIHN1Y2ggYXMgdGhlIGZpcnN0IDJLQiBvZiBkYXRh Lg0KDQpUaGVyZWZvcmUsIGl0IHdvdWxkIGJlIGdyZWF0IGlmIHRoZXJlIHdlcmUgYWRkaXRpb25h bCBwYXJhbWV0ZXJzIHRvIGluZm9ybSByZWFsbG9jIHRvIGp1c3QgZXhlY3V0ZSBtYWxsb2MoMzAw S0IpK21lbWNweSgqKjJLQioqKStmcmVlKDI1NktCKSB3aGVuIGRlZ2VuZXJhdGluZyBiYWNrIHRv IG1hbGxvYyttZW1jcHkrZnJlZS4NCiANCi0tDQoNCiAgIEJlc3QgUmVnYXJkcw0KICBCYWlZYW5n DQogIGJhaXlhbmdAZ21haWwuY29tDQogIGh0dHA6Ly9pLmJhaXkuY24NCioqKiogPCBFTkQgT0Yg RU1BSUwgPiAqKioqIA0KIA0KIA0KRnJvbTogSmFtZXMgWSBLbmlnaHQNCkRhdGU6IDIwMjItMDkt MjEgMDA6NTkNClRvOiBtdXNsDQpDQzogRmxvcmlhbiBXZWltZXI7IFJpY2ggRmVsa2VyOyBiYWl5 YW5nDQpTdWJqZWN0OiBSZTogW211c2xdIFRoZSBoZWFwIG1lbW9yeSBwZXJmb3JtYW5jZSAobWFs bG9jL2ZyZWUvcmVhbGxvYykgaXMgc2lnbmlmaWNhbnRseSBkZWdyYWRlZCBpbiBtdXNsIDEuMiAo Y29tcGFyZWQgdG8gMS4xKQ0KT24gVHVlLCBTZXAgMjAsIDIwMjIgYXQgOTo1OCBBTSBTaWRkaGVz aCBQb3lhcmVrYXIgPHNpZGRoZXNoQHJlZGhhdC5jb20+IHdyb3RlOg0KQWRkaW5nIHN1cHBvcnQg Zm9yIHNvbWV0aGluZyB0aGF0J3MgYWxyZWFkeSBkZWNsYXJlZCBhcyBiYWQNCnByb2dyYW1taW5n IHByYWN0aWNlIHNlZW1zIGxpa2UgYSBzdGVwIGJhY2t3YXJkcy4gIEluc3RlYWQsIEkgaG9wZSB3 ZQ0KZmluZCBhIHdheSB0byBkaXNjb3VyYWdlIGFjdGl2ZSB1c2Ugb2YgbWFsbG9jX3VzYWJsZV9z aXplIG1vcmUNCnN0cm9uZ2x5LiAgDQoNCkJUVywgaWYgZm9sa3MgYXJlbid0IGF3YXJlLCB0aGVy ZSBpcyBhbHJlYWR5IHdvcmsgb24gdGhlIEMrKyBzaWRlIHRvIGV4cG9zZSBhbiBBUEkgd2hpY2gg bGV0cyB5b3UgcmVxdWVzdCBhIGhlYXAgYWxsb2NhdGlvbiBvZiBfYXQgbGVhc3RfIHRoZSBnaXZl biBzaXplLCB3aGljaCByb3VuZHMgdGhlIGFjdHVhbCBzaXplIHVwIGluIHdoYXRldmVyIHdheSB0 aGUgYWxsb2NhdG9yIGxpa2VzLCBhbmQgcmV0dXJucyB0aGUgcG9pbnRlciBhbmQgYWN0dWFsIHNp emUgYWxsb2NhdGVkLiBXaXRoIHRoaXMgQVBJLCB5b3UgZGVjbGFyZSBhbiBleHBsaWNpdCBpbnRl bnQgdGhhdCBhbGwgb2YgdGhlIG1lbW9yeSAtLSB1cCB0byB0aGUgcmV0dXJuZWQgc2l6ZSAtLSBp cyB2YWxpZCB0byB1c2Ugd2l0aG91dCBuZWVkaW5nIHRvIGdvIGJhY2sgdG8gdGhlIGFsbG9jYXRv ciB0byBhc2sgZm9yIG1vcmUuDQoNClRoZSBwcm9wb3NhbCBpcyBzdGlsbCBtYWtpbmcgaXRzIHdh eSB0aHJvdWdoIHRoZSBzdGFuZGFyZGl6YXRpb24gcHJvY2VzcywgYnV0IGhvcGVmdWxseSBpdCds bCBtYWtlIGl0IGludG8gdGhlIG5leHQgdmVyc2lvbiBvZiBDKysgYWZ0ZXIgQysrMjMuICAoT2Yg Y291cnNlLCB0aGF0J3Mgbm90IGEgc3VyZSB0aGluZyB1bnRpbCBpdCBoYXBwZW5zLikgSGVyZSdz IHRoZSBkb2MsIHdpdGggbW9yZSByYXRpb25hbGUvZXRjOg0KICBodHRwczovL3d3dy5vcGVuLXN0 ZC5vcmcvanRjMS9zYzIyL3dnMjEvZG9jcy9wYXBlcnMvMjAyMi9wMDkwMXI5Lmh0bWwNCg0KQWxz bywgYXMgbm90ZWQgaW4gdGhlIGRvYywgamVtYWxsb2MgZXhwZXJpbWVudGFsbHkgaW1wbGVtZW50 ZWQgdGhpcyBmdW5jdGlvbmFsaXR5IGluIGl0cyBub24tc3RhbmRhcmQgQVBJLCB2aWEgYSBmdW5j dGlvbiBpdCBjYWxsZWQgInNtYWxsb2N4IiAtLSB0aG91Z2ggamVtYWxsb2MgaGlkZXMgdGhlIEFQ SSBzbyBpdCBjYW4ndCBiZSB1c2VkIGJ5IGRlZmF1bHQuIFRoZSBBUEkgaXMgZWZmZWN0aXZlbHk6 DQogIHR5cGVkZWYgc3RydWN0IHsgdm9pZCAqcHRyOyBzaXplX3Qgc2l6ZTsgfSBzbWFsbG9jeF9y ZXR1cm5fdDsNCiAgc21hbGxvY3hfcmV0dXJuX3Qgc21hbGxvY3goc2l6ZV90IHNpemUsIGludCBm bGFncyk7DQpodHRwczovL2dpdGh1Yi5jb20vamVtYWxsb2MvamVtYWxsb2MvYmxvYi9hMDczNGZk NmVlMzI2Y2QyMDU5ZWRiZTRiY2E3MDkyOTg4YTYzNjg0L3NyYy9qZW1hbGxvYy5jI0wzNDE0DQoo VGhhdCdzIGNvbnNpc3RlbnQgd2l0aCBqZW1hbGxvYydzIG90aGVyIG5vbi1zdGFuZGFyZCBBUElz LCB3aGljaCBzdGljayBhbGlnbm1lbnQvZXRjIGludG8gYSAiZmxhZ3MiIGFyZ3VtZW50LCBidXQg cHJvYmFibHkgbm90IHN1aXRhYmxlIGZvciBhIG1vcmUtc3RhbmRhcmRpemVkIGNyb3NzLWltcGxl bWVudGF0aW9uIEFQSSkNCg0KdGNtYWxsb2MgaW1wbGVtZW50cyBzaW1pbGFyIGZ1bmN0aW9uYWxp dHksIGFzIHdlbGwsIHdpdGggZmFtaWx5IG9mIGZ1bmN0aW9ucyBuYW1lZCAidGNtYWxsb2Nfc2l6 ZV9yZXR1cm5pbmdfb3BlcmF0b3JfbmV3IjoNCiAgaHR0cHM6Ly9naXRodWIuY29tL2dvb2dsZS90 Y21hbGxvYy9ibG9iLzI2N2FhMmVjMjgxN2FiOWQwOWIzZmJiNjVlY2I5MDE5M2RkNDM0OGUvdGNt YWxsb2MvbWFsbG9jX2V4dGVuc2lvbi5oI0w1NDkNCndoaWNoIG9mIGNvdXJzZSBhbHNvIGlzbid0 IGEgc3VpdGFibGUgQVBJIHRvIHN1cHBvcnQgY3Jvc3MtaW1wbGVtZW50YXRpb24uDQoNCklmIHNv bWVvbmUgd2FudHMgdG8gcHVzaCBmb3J3YXJkIHRoaXMgYXJlYSwgSU1PLCBpdCB3b3VsZCBiZSBy ZWFsbHkgZ3JlYXQgdG8gaGF2ZSBhbiBBUEkgZXhwb3NpbmcgdGhpcyBmdW5jdGlvbmFsaXR5IGRl c2lnbmVkIHRvIGJlIGltcGxlbWVudGVkIGluIGEgY29tbW9uIHdheSBhY3Jvc3MgbGliYyBtYWxs b2MgaW1wbGVtZW50YXRpb25zIC0tIGFuZCBldmVudHVhbGx5IGFkZGVkIHRvIFBPU0lYIG9yIEMu DQoNCg0K ------=_001_NextPart216102362172_=---- Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0A
> tcmalloc implem= ents similar functionality, as well, with family of functions named "tcmal= loc_size_returning_operator_new"
This is a good thing, w= e noticed it before. But we actually need a realloc that **can specify the= copy range**.

As in our previous example= , suppose we "void* p malloc(200KB)" and "malloc_usable_size(p)" returns 2= 56KB. For allocators such as tcmalloc, i= f we do "realloc(p, 300KB)", it will actually execute "malloc(300KB)+memcp= y(**256KB**)+free(256KB)". But at this time, t= he actual business of the application layer often only needs to copy a sma= ll amount of content, such as the first 2KB of data.

The= refore, it would be great if there were additional parameters to inform re= alloc to just execute malloc(300KB)+memcpy(**2KB**)+free(256KB) when degen= erating back to malloc+memcpy+free.
=0A
 
=0A
--

 <= img src=3D"http://baiy.cn/image/portrait_100px.jpg" border=3D"0">  Best Regards
  BaiYang
 =  baiyang@gmail.com
 =  http://i.baiy.cn
**** < END OF EMAIL > **= ** 
 
<= div> 
Date:&nbs= p;2022-09-21 00:59
To: musl
Subject: Re: [musl] The heap memory performance (malloc/free/= realloc) is significantly degraded in musl 1.2 (compared to 1.1)
On Tue, Sep 20, 2022 at 9:58 AM Siddhesh Poyarekar <siddhesh@redhat.com> wrote:
<= /div>
Adding support for something that's already declared as bad=
=0Aprogramming practice seems like a step backwards.  Instead, I = hope we
=0Afind a way to discourage active use of malloc_usable_size mo= re
=0Astrongly. 

BTW, if folks ar= en't aware, there is already work on the C++ side to expose an API which l= ets you request a heap allocation of _at least_ the given size, which roun= ds the actual size up in whatever way the allocator likes, and returns the= pointer and actual size allocated. With this API, you declare an explicit= intent that all of the memory -- up to the returned size -- is valid to u= se without needing to go back to the allocator to ask for more.
=
The proposal is still making its way through the standardiz= ation process, but hopefully it'll make it into the next version of C++ af= ter C++23.  (Of course, that's not a sure thing until it happens= .) Here's the doc, with more rationale/etc:
=

Also, as noted in the doc, jemalloc experimentally imp= lemented this functionality in its non-standard API, via a function it cal= led "smallocx" -- though jemalloc hides the API so it can't be used by def= ault. The API is effectively:
  typedef struct { void *ptr; size= _t size; } smallocx_return_t;
  smallocx_return_t smallocx(si= ze_t size, int flags);
(That's consistent= with jemalloc's other non-standard APIs, which stick alignment/etc i= nto a "flags" argument, but probably not suitable for a more-standardized = cross-implementation API)

tcmalloc implements sim= ilar functionality, as well, with family of functions named "tcmalloc_size= _returning_operator_new":
=
which of course also isn't a suitable API to support cross-implementa= tion.

If someone wants to push forward this = area, IMO, it would be really great to have an API exposing this functiona= lity designed to be implemented in a common way across libc malloc impleme= ntations -- and eventually added to POSIX or C.

=

=0A
=0A ------=_001_NextPart216102362172_=------