From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,HTML_FONT_FACE_BAD,HTML_MESSAGE, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,T_KAM_HTML_FONT_INVALID autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 31777 invoked from network); 20 Sep 2022 17:29:08 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 20 Sep 2022 17:29:08 -0000 Received: (qmail 28168 invoked by uid 550); 20 Sep 2022 17:29:05 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 28145 invoked from network); 20 Sep 2022 17:29:05 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:mime-version:references:subject:cc:to:from:date:from:to :cc:subject:date; bh=8bk8kQbEAYD+4dQbmp+J5GbYjwUjMwzAgrWIzC76eK4=; b=V+RwSMs1wBM1bzpfTL1wub0rhYsfp7oAnF7qK0LAButtW1e2NH70dwyZVHEIFKFa3Z /kEzjB/y2it5+WafGFFo9GZ7IwY7Va0Y/MDJtZY6+lUSQjwpEqTZAAI9kcvNlc6xkNig GELcTdHzKau5yPlYg9AqZ91IxW1WVTR+c7vEcEvSycHCGveDUdeKssCHajsZvpesS4/B AP/RWixw/W7jrssxlwZoBe6XoN6AN/wIrXJTQH8C/20SBy/rSe0ibevtSA3/5nne1Iqy KQbAa3SUbZtd7gc+K3o+2W/J9hIqudNof/p3RVa3AXPOI81zhwyMHTEsroREYJoek4Th 2J2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=message-id:mime-version:references:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date; bh=8bk8kQbEAYD+4dQbmp+J5GbYjwUjMwzAgrWIzC76eK4=; b=nsEiJ63XVSh9CivUgujQmOKWld1EHcJVhuWVvlw8hHVlksAnQpzD/ZoZFHbLO69tub nMw/zOLkoliBQBwUmxdpw2cjdEYA245KlAJpdIqmVxTf6Q98uqvJmRtGhPceWMqQbQ/H YONoWf+FpnnuG/LTc/NCjCUkJNVn5DHXGHjB5G2x1fP61Bz7mZSydgD1ZKrGEMDeoA8b QMu5wZHZXp8YqGycMfyglMI9cGjM/53s/s2h26r36tWSv4sagOFBf2wak+sTHi6bHhFo uFplqcVjehxgIQSQ31RFCjujw/eMUYh6PxJYheF3vVb7nUex2O3l3IZoi79qyxhDm2EC gE1Q== X-Gm-Message-State: ACrzQf3PaCbQDm8VX0IZ8gpfXq5y4djzrVBvCGZkf8H+RRP9MdSktKsZ yvly10Mv/vmkKxJ79cUc6Co= X-Google-Smtp-Source: AMsMyM7UbRaXx/yYIdeDRzCB6OEqDjhDYvmzs2Gk+e0jqprEVtIm3buXyjWf83PAMvDTqolhIC23TA== X-Received: by 2002:a17:90b:180d:b0:202:7cf6:9f9b with SMTP id lw13-20020a17090b180d00b002027cf69f9bmr5089402pjb.160.1663694932861; Tue, 20 Sep 2022 10:28:52 -0700 (PDT) Date: Wed, 21 Sep 2022 01:28:54 +0800 From: baiyang To: "Siddhesh Poyarekar" , "Florian Weimer" Cc: "Rich Felker" , musl References: <2022091915532777412615@gmail.com>, <20220919110829.GA2158779@port70.net>, <874jx3h76u.fsf@oldenburg.str.redhat.com>, <20220919134659.GO9709@brightrain.aerifal.cx>, <874jx2phqm.fsf@oldenburg.str.redhat.com>, X-Priority: 3 X-GUID: A4EFB90C-B859-4F22-AFFA-A80E4137BFC3 X-Has-Attach: no X-Mailer: Foxmail 7.2.23.116[cn] Mime-Version: 1.0 Message-ID: <202209210128529625329@gmail.com> Content-Type: multipart/alternative; boundary="----=_001_NextPart316105868543_=----" Subject: Re: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) This is a multi-part message in MIME format. ------=_001_NextPart316105868543_=---- Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 PiBJcyB0aGVyZSBhbnkgb3RoZXIgdmFsaWQgcmVhc29uIHRvIHVzZQ0KPiBtYWxsb2NfdXNhYmxl X3NpemUgaW5zdGVhZCBvZiBzaW1wbHkgdXNpbmcgcmVhbGxvYz8gDQoNClllcywgd2UgdXNlIGl0 IGFzIG9uZSBvZiB0aGUgcGFyYW1ldGVycyB0byBlc3RpbWF0ZSB0aGUgbWVtb3J5IGNvcHkgY29z dCB3aGVuIHJlYWxsb2MgZGVnZW5lcmF0ZXMgYmFjayB0byBtYWxsb2MrbWVtY3B5K2ZyZWUsIHJl ZmVyIHRvIHRoZSBwcmV2aW91cyBtYWlscy4gDQoNCi0tDQoNCiAgIEJlc3QgUmVnYXJkcw0KICBC YWlZYW5nDQogIGJhaXlhbmdAZ21haWwuY29tDQogIGh0dHA6Ly9pLmJhaXkuY24NCioqKiogPCBF TkQgT0YgRU1BSUwgPiAqKioqIA0KIA0KIA0KRnJvbTogU2lkZGhlc2ggUG95YXJla2FyDQpEYXRl OiAyMDIyLTA5LTIwIDIxOjU0DQpUbzogRmxvcmlhbiBXZWltZXINCkNDOiBSaWNoIEZlbGtlcjsg YmFpeWFuZzsgbXVzbA0KU3ViamVjdDogUmU6IFttdXNsXSBUaGUgaGVhcCBtZW1vcnkgcGVyZm9y bWFuY2UgKG1hbGxvYy9mcmVlL3JlYWxsb2MpIGlzIHNpZ25pZmljYW50bHkgZGVncmFkZWQgaW4g bXVzbCAxLjIgKGNvbXBhcmVkIHRvIDEuMSkNCk9uIFR1ZSwgU2VwIDIwLCAyMDIyIGF0IDQ6MzQg QU0gRmxvcmlhbiBXZWltZXIgPGZ3ZWltZXJAcmVkaGF0LmNvbT4gd3JvdGU6DQo+IFRoZSBjb21w aWxlciBuZWVkcyB0byB0cmVhdCBtYWxsb2NfdXNhYmxlX3NpemUgc2ltaWxhciB0byByZWFsbG9j IGFuZA0KPiBqdXN0IHRoZSBzaXplIGluZm9ybWF0aW9uIGZvciB0aGUgYnVmZmVyIGJhc2VkIG9u IHRoZSByZXR1cm4gdmFsdWUgZnJvbQ0KPiBtYWxsb2NfdXNhYmxlX3NpemUuICBUaGlzIGlzIGFk bWl0dGVkbHkgaGFyZGVyIHRvIGRvIHRoYW4gYSBjb21wYXJhYmxlDQo+IGFuYWx5c2lzIGZvciBy ZWFsbG9jIGlmIHRoZSBjb21waWxlciBpbnRlcnByZXRzIHRoZSBzdGFuZGFyZCBpbiBzdWNoIGEN Cj4gd2F5IHRoYXQgYWZ0ZXIgYSBzdWNjZXNzZnVsIHJlYWxsb2MsIGFueSBhY2Nlc3MgdG8gdGhl IG9yaWdpbmFsIHBvaW50ZXINCj4gdmFsdWUgaXMgdW5kZWZpbmVkLg0KPg0KPiBtYWxsb2NfdXNh YmxlX3NpemUgaXMgbm90IGFjdHVhbGx5ICp0aGF0KiB1c2VmdWwgd2l0aCBhbGxvY2F0b3JzIHRo YXQgZG8NCj4gbm90IGhhdmUgc3RyaWN0IHNpemUgY2xhc3NlcyBiZWNhdXNlIHRoZXkgZG8gbm90 IG92ZXItYWxsb2NhdGUgdGhhdA0KPiBtdWNoLiAgRm9yIHRoZXNlIGFsbG9jYXRvcnMsIGl0IG1h eSBiZSBwb3NzaWJsZSB0byBpbmNyZWFzZSB0aGUgc2l6ZSBvZg0KPiBhbGxvY2F0aW9uIHNpZ25p ZmljYW50bHkgd2l0aG91dCBtb3ZpbmcgaXQsIGJ1dCB0aGF0IGlzIG5vdCByZWZsZWN0ZWQgaW4N Cj4gdGhlIHJldHVybiB2YWx1ZSBvZiBtYWxsb2NfdXNhYmxlX3NpemUgYXQgYWxsLg0KIA0KU28g dGhlIGdsaWJjIG1hbnVhbCBkb2VzIG5vdCBkb2N1bWVudCBtYWxsb2NfdXNhYmxlX3NpemUgc2Vt YW50aWNzDQood2hpY2ggaXMgd2VpcmQgc2luY2UgaXQgaXMsIHdlbGwsIGEgR05VIGV4dGVuc2lv biEpIHNvIHRoZSBvbmx5DQpyZWZlcmVuY2UgdXNlcnMgaGF2ZSBpcyB0aGUgbWFuIHBhZ2UuICBU aGUgbWFuIHBhZ2UgYWxyZWFkeQ0KZGlzY291cmFnZXMgdXNlIG9mIG1hbGxvY191c2FibGVfc2l6 ZSB0byB3cml0ZSBiZXlvbmQgdGhlIGFsbG9jYXRlZA0Kc2l6ZSBpbiB0aGUgTk9URVMgc2VjdGlv bjoNCiANCiAgICBUaGUgIHZhbHVlICByZXR1cm5lZCBieSBtYWxsb2NfdXNhYmxlX3NpemUoKSBt YXkgYmUgZ3JlYXRlciB0aGFuDQp0aGUgcmVxdWVzdGVkIHNpemUgb2YgdGhlIGFsbG9jYXRpb24g YmVjYXVzZSBvZiBhbGlnbm1lbnQgYW5kIG1pbmltdW0NCnNpemUgY29uc3RyYWludHMuICBBbHRo b3VnaCB0aGUgZXhjZXNzIGJ5dGVzIGNhbiBiZSBvdmVyd3JpdHRlbiBieSB0aGUNCmFwcGxpY2F0 aW9uIHdpdGhvdXQgaWxsIGVmZmVjdHMsIHRoaXMgaXMgbm90IGdvb2QgcHJvZ3JhbW1pbmcNCnBy YWN0aWNlOiB0aGUgbnVtYmVyIG9mIGV4Y2VzcyBieXRlcyBpbiBhbiBhbGxvY2F0aW9uIGRlcGVu ZHMgb24gdGhlDQp1bmRlcmx5aW5nIGltcGxlbWVudGF0aW9uLg0KIA0KQWRkaW5nIHN1cHBvcnQg Zm9yIHNvbWV0aGluZyB0aGF0J3MgYWxyZWFkeSBkZWNsYXJlZCBhcyBiYWQNCnByb2dyYW1taW5n IHByYWN0aWNlIHNlZW1zIGxpa2UgYSBzdGVwIGJhY2t3YXJkcy4gIEluc3RlYWQsIEkgaG9wZSB3 ZQ0KZmluZCBhIHdheSB0byBkaXNjb3VyYWdlIGFjdGl2ZSB1c2Ugb2YgbWFsbG9jX3VzYWJsZV9z aXplIG1vcmUNCnN0cm9uZ2x5LiAgQXQgbGVhc3QgYmFzZWQgb24gdGhlIHN5c3RlbWQgZXhwZXJp ZW5jZSwgdGhlIHByb2JsZW0gdGhleQ0KdHJ5IHRvIHNvbHZlIGlzIHRoYXQgb2YgZ2xpYmMgcmVh bGxvYyBiZWluZyB0b28gc2xvdyBmb3IgcGF0aHMgd2hlcmUNCnRoZSByZWFsbG9jYXRpb24gc2hv dWxkIHJldHVybiB0aGUgc2FtZSBibG9jayBhbmQgdGhhdCBzaG91bGQgYmUgZWFzeQ0KdG8gc3Bl Y2lhbC1jYXNlLiAgSXMgdGhlcmUgYW55IG90aGVyIHZhbGlkIHJlYXNvbiB0byB1c2UNCm1hbGxv Y191c2FibGVfc2l6ZSBpbnN0ZWFkIG9mIHNpbXBseSB1c2luZyByZWFsbG9jPw0KIA0KU2lkDQog DQo= ------=_001_NextPart316105868543_=---- Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0A
> Is ther= e any other valid reason to use
> ma= lloc_usable_size instead of simply using realloc? 
=0A
Yes, we use it as one of the= parameters to estimate the memory copy cost when realloc degenerates back= to malloc+memcpy+free, refer to the previous mails. 

=0A
--<= td style=3D"font-size: 14px;">  Best Regards
  BaiYang
&nb= sp; baiyang@gmail.com=
  http://i.baiy.cn
<= /td>

&nbs= p;
**** < END OF= EMAIL > ****
 
 
 
From:&n= bsp;Siddhesh Poyarekar
Date: 2022-09-20 21:54
Subject: Re: [musl] The heap memory perform= ance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared= to 1.1)
On Tue, Sep 20, 2022 at 4:34 AM Floria= n Weimer <fweimer@redhat.com> wrote:
=0A
> The compiler = needs to treat malloc_usable_size similar to realloc and
=0A
>= just the size information for the buffer based on the return value from=0A
> malloc_usable_size.  This is admittedly harder to do= than a comparable
=0A
> analysis for realloc if the compiler = interprets the standard in such a
=0A
> way that after a succe= ssful realloc, any access to the original pointer
=0A
> value = is undefined.
=0A
>
=0A
> malloc_usable_size is no= t actually *that* useful with allocators that do
=0A
> not hav= e strict size classes because they do not over-allocate that
=0A
= > much.  For these allocators, it may be possible to increase the = size of
=0A
> allocation significantly without moving it, but = that is not reflected in
=0A
> the return value of malloc_usab= le_size at all.
=0A
 
=0A
So the glibc manual does = not document malloc_usable_size semantics
=0A
(which is weird sin= ce it is, well, a GNU extension!) so the only
=0A
reference users= have is the man page.  The man page already
=0A
discourages= use of malloc_usable_size to write beyond the allocated
=0A
size= in the NOTES section:
=0A
 
=0A
   = The  value  returned by malloc_usable_size() may be greater tha= n
=0A
the requested size of the allocation because of alignment a= nd minimum
=0A
size constraints.  Although the excess bytes = can be overwritten by the
=0A
application without ill effects, th= is is not good programming
=0A
practice: the number of excess byt= es in an allocation depends on the
=0A
underlying implementation.=
=0A
 
=0A
Adding support for something that's alre= ady declared as bad
=0A
programming practice seems like a step ba= ckwards.  Instead, I hope we
=0A
find a way to discourage ac= tive use of malloc_usable_size more
=0A
strongly.  At least = based on the systemd experience, the problem they
=0A
try to solv= e is that of glibc realloc being too slow for paths where
=0A
the= reallocation should return the same block and that should be easy
= =0A
to special-case.  Is there any other valid reason to use=0A
malloc_usable_size instead of simply using realloc?
=0A
=  
=0A
Sid
=0A
 
=0A
=0A= ------=_001_NextPart316105868543_=------