From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,HTML_FONT_FACE_BAD,HTML_MESSAGE, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,T_KAM_HTML_FONT_INVALID autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 1583 invoked from network); 19 Sep 2022 18:40:26 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 19 Sep 2022 18:40:26 -0000 Received: (qmail 3726 invoked by uid 550); 19 Sep 2022 18:40:23 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 3703 invoked from network); 19 Sep 2022 18:40:22 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:mime-version:references:subject:cc:to:from:date:from:to :cc:subject:date; bh=8lyZZa0oR/PUEIwxsmBcOZaRFsb5lKEXa42D+fQ3vRQ=; b=nz2c2f20zKF9I8uVus4K+oGugYqQ0Xn5wm14CjeX1tsN7yKyDRUrft1hzt9J15Jcb0 NSNjlDA8ydWI9659i1gYaKU/XEw02FRt8Z6nkhIsvAYf/tLBSpGUtkeITKPTEGX9Rn9O JeUIMfQl6kwIZJJPzAbXOdsPcXMlmgQMGkRrIhoNfpIiIWTGQvTcTsASkbsuWsc3RypT sgfQPAgkEc8mLTF40xM1WM64xb/jXbvV/Gsl20H6tE22Qw0C8XvGwA0J/NDaaV44p5+g UU0V3Ks10fftkQjgHM6nM0Y5hd730E1+ycX3GuXlPx+NKr4i4OnBDnNYngJRrtINC4WE 48tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=message-id:mime-version:references:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date; bh=8lyZZa0oR/PUEIwxsmBcOZaRFsb5lKEXa42D+fQ3vRQ=; b=7iQYyO/pohNPZ+0pSOj0tMMbleqvQP5/pQUrYXKjz0oIR+pNj53P7u7FHj5rVsSx9W nraWY3K9FxqeBoTJRGkq+l8feD/2UEfpCrTf4qgjtmba4GblGgFnqsSfutRdWZCIcbPT w/a4+qqq9yPU4kAql/0jDiKa5W0a91fEWoYnJiz2Z+c3j6PEho5Be/8hj3HmzeAJUUDi C843BBaL+633pQr1Lo0sYSP9wxpyK87n0TCad/ihA5OCUulMIj8LwMVNqIoAv5nSfoNc BfnwwNTkwpHuLz24ByDqmlzVC+GuwXGcnVakMObLzMSuv6Mb+k7u3KFBC021nGpPsj6E DPlw== X-Gm-Message-State: ACrzQf3gu6pBDHyBq1FBkNQoJPdNxSzRO8J0PbE8hlhNtDpPpiGqzJCs LTE0erIitkZWvTZI5JbVwFA= X-Google-Smtp-Source: AMsMyM7OyzJMxinYajkt7zC57kDOznxM7qU53q1GrE5SVBOYDk4pwZwHfakhW98KY8FPeRUHE4dgBQ== X-Received: by 2002:a17:902:d4c6:b0:177:ff9b:35d0 with SMTP id o6-20020a170902d4c600b00177ff9b35d0mr1061906plg.69.1663612810123; Mon, 19 Sep 2022 11:40:10 -0700 (PDT) Date: Tue, 20 Sep 2022 02:40:12 +0800 From: baiyang To: "Szabolcs Nagy" Cc: "James Y Knight" , musl , "Florian Weimer" References: <2022091915532777412615@gmail.com>, <20220919110829.GA2158779@port70.net>, <874jx3h76u.fsf@oldenburg.str.redhat.com>, <20220919134659.GO9709@brightrain.aerifal.cx>, , <2022092001404698842815@gmail.com>, <20220919181441.GC2158779@port70.net> X-Priority: 3 X-GUID: E5D49E70-9A46-4E0D-A7D4-14F99AEC3752 X-Has-Attach: no X-Mailer: Foxmail 7.2.23.116[cn] Mime-Version: 1.0 Message-ID: <2022092002395980764526@gmail.com> Content-Type: multipart/alternative; boundary="----=_001_NextPart771105884157_=----" Subject: Re: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) This is a multi-part message in MIME format. ------=_001_NextPart771105884157_=---- Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 PiBHZXRTaXplKHApIGlzIG5vdCB0aGUgZXhhY3Qgc2l6ZSAodGhhdCB0aGUgdXNlciBhbGxvY2F0 ZWQpIGJ1dCBhbiBpbnRlcm5hbCBzaXplICh3aGljaCBtYXkgYmUgbGFyZ2VyKQ0KDQpZZXMsIGFz IEkgbWVudGlvbmVkIGluIGFub3RoZXIgZW1haWwsIHdlIGp1c3QgbmVlZCB0aGlzICJpbnRlcm5h bCBzaXplIi4NCg0KVGhlIHZhbHVlIHJldHVybmVkIGJ5IG1hbGxvY191c2FibGVfc2l6ZSgpIG1h eSBiZSBncmVhdGVyIHRoYW4gdGhlIHJlcXVlc3RlZCBzaXplIG9mIHRoZSBhbGxvY2F0aW9uLg0K DQpBbHNvLCBJIGRvbid0IHRoaW5rIHRoZXJlIGlzIGFueSBhbWJpZ3VpdHkgaW4gdGhlIG1hbnVh bCBwYWdlcyBvZiBlYWNoIHBsYXRmb3JtIHJlZ2FyZGluZyB0aGlzICJpbnRlcm5hbCBzaXplIjog VGhlIHZhbHVlIHJldHVybmVkIGJ5IG1hbGxvY191c2FibGVfc2l6ZSgpIG1heSBiZSBncmVhdGVy IHRoYW4gdGhlIHJlcXVlc3RlZCBzaXplIG9mIHRoZSBhbGxvY2F0aW9uIC0tIHRoYXQncyBleGFj dGx5IHdoYXQgd2Ugd2FudC4NCiANCi0tDQoNCiAgIEJlc3QgUmVnYXJkcw0KICBCYWlZYW5nDQog IGJhaXlhbmdAZ21haWwuY29tDQogIGh0dHA6Ly9pLmJhaXkuY24NCioqKiogPCBFTkQgT0YgRU1B SUwgPiAqKioqIA0KIA0KIA0KRnJvbTogU3phYm9sY3MgTmFneQ0KRGF0ZTogMjAyMi0wOS0yMCAw MjoxNA0KVG86IGJhaXlhbmcNCkNDOiBKYW1lcyBZIEtuaWdodDsgbXVzbDsgRmxvcmlhbiBXZWlt ZXINClN1YmplY3Q6IFJlOiBSZTogW211c2xdIFRoZSBoZWFwIG1lbW9yeSBwZXJmb3JtYW5jZSAo bWFsbG9jL2ZyZWUvcmVhbGxvYykgaXMgc2lnbmlmaWNhbnRseSBkZWdyYWRlZCBpbiBtdXNsIDEu MiAoY29tcGFyZWQgdG8gMS4xKQ0KKiBiYWl5YW5nIDxiYWl5YW5nQGdtYWlsLmNvbT4gWzIwMjIt MDktMjAgMDE6NDA6NDggKzA4MDBdOg0KPiBJIGxvb2tlZCBhdCB0aGUgY29kZSBvZiB0Y21hbGxv YywgYnV0IEkgZGlkbid0IGZpbmQgYW55IG9mIHRoZSBwcm9ibGVtcyB5b3UgbWVudGlvbmVkIGlu IHRoZSBpbXBsZW1lbnRhdGlvbiBvZiBtYWxsb2NfdXNhYmxlX3NpemUgKHNlZTogaHR0cHM6Ly9n aXRodWIuY29tL2dvb2dsZS90Y21hbGxvYy9ibG9iLzkxNzliYjg4NDg0OGMzMDYxNjY2N2JhMTI5 YmNmOWFmZWUxMTRjMzIvdGNtYWxsb2MvdGNtYWxsb2MuY2MjTDEwOTkgKS4NCj4gDQo+IE9uIHRo ZSBjb250cmFyeSwgc2ltaWxhciB0byBtdXNsLCB0Y21hbGxvYyBhbHNvIGRpcmVjdGx5IHVzZXMg dGhlIHJldHVybiB2YWx1ZSBvZiBtYWxsb2NfdXNhYmxlX3NpemUgaW4gaXRzIHJlYWxsb2MgaW1w bGVtZW50YXRpb24gdG8gZGV0ZXJtaW5lIHdoZXRoZXIgbWVtb3J5IG5lZWRzIHRvIGJlIHJlYWxs b2NhdGVkOiBodHRwczovL2dpdGh1Yi5jb20vZ29vZ2xlL3RjbWFsbG9jL2Jsb2IvOTE3OWJiODg0 ODQ4YzMwNjE2NjY3YmExMjliY2Y5YWZlZTExNGMzMi90Y21hbGxvYy90Y21hbGxvYy5jYyNMMTQ5 OQ0KPiANCj4gSSB0aGluayB0aGlzIGlzIGVub3VnaCB0byBzaG93IHRoYXQgdGhlIHJldHVybiB2 YWx1ZSBvZiBtYWxsb2NfdXNhYmxlX3NpemUgaW4gdGNtYWxsb2MgaXMgYWNjdXJhdGUgYW5kIHJl bGlhYmxlLCBvdGhlcndpc2UgaXRzIG93biByZWFsbG9jIHdpbGwgY2F1c2UgYSBzZWdtZW50IGZh dWx0Lg0KIA0Kb2J2aW91c2x5IGludGVybmFsbHkgdGhlIGltcGxlbWVudGF0aW9uIGNhbiB1c2Ug dGhlIGludGVybmFsIGNodW5rIHNpemUuLi4NCiANCkdldFNpemUocCkgaXMgbm90IHRoZSBleGFj dCBzaXplICh0aGF0IHRoZSB1c2VyIGFsbG9jYXRlZCkgYnV0IGFuIGludGVybmFsDQpzaXplICh3 aGljaCBtYXkgYmUgbGFyZ2VyKSBhbmQgdGhhdCBtdXN0IG5vdCBiZSBleHBvc2VkICpvdXRzaWRl KiBvZiB0aGUNCm1hbGxvYyBpbXBsZW1lbnRhdGlvbiAob3RoZXIgdGhhbiBmb3IgZGlhZ25vc3Rp YyBwdXJwb3NlcykuDQogDQp5b3UgY2FuIGhhdmUgMiB2aWV3czoNCiANCigxKSB0Y21hbGxvYyBh bmQgamVtYWxsb2MgYXJlIGJ1Z2d5IGJlY2F1c2UgdGhleSBleHBvc2UgYW4gaW50ZXJuYWwNCiAg ICB0aGF0IG11c3Qgbm90IGJlIGV4cG9zZWQgKGJlY2F1ZXMgaXQgY2FuIGJyZWFrIHVzZXIgY29k ZSkuDQogDQooMikgdXNlciBjb2RlIGlzIGJ1Z2d5IGlmIGl0IHVzZXMgbWFsbG9jX3VzYWJsZV9z aXplIGZvciBhbnkgcHVycG9zZQ0KICAgIG90aGVyIHRoYW4gZGlhZ25vc3RpYy9zdGF0aXN0aWNz IChiZWNhdXNlIG90aGVyIHVzZXMgYXJlIGJyb2tlbg0KICAgIG9uIG1hbnkgaW1wbGVtZW50YXRp b25zKS4NCiANCmVpdGhlciB3YXkgdGhlIGJyb2tlbm5lc3MgeW91IHdhbnQgdG8gc3VwcG9ydCBp cyBhIHNlY3VyaXR5IGhhemFyZA0KYW5kIHlvdSBhcmUgbHVja3kgdGhhdCBtdXNsIHNhdmVzIHRo ZSBkYXk6IGl0IHdvcmtzIGhhcmQgbm90IHRvDQpleHBvc2UgaW50ZXJuYWwgc2l6ZXMgc28gdGhl IGNvZGUgeW91IHNlZW0gdG8gY2FyZSBhYm91dCBjYW4gb3BlcmF0ZQ0Kc2FmZWx5ICh3aGljaCBp cyBub3QgdHJ1ZSBvbiB0Y21hbGxvYyBhbmQgamVtYWxsb2M6IHRoZSBjb21waWxlcg0KbWF5IGJy ZWFrIHRoYXQgY29kZSkuDQo= ------=_001_NextPart771105884157_=---- Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable =0A
> GetSize(p) is not the exact size (that= the user allocated) but an internal = size (which may be larger)

Yes, as I mentioned in another email, we just need this "inter= nal size".

The value returned by malloc_usable_size() may be greate= r than the requested size of the allocation.
<= span style=3D"background-color: transparent;">
Also, I don't think t= here is any ambiguity in the manual pages of each platform regarding this = "internal size": The value returned by malloc_usable_size() may be greater= than the requested size of the allocation -- that's exactly what we want.
= =0A
 
=0A=
--
   Best Regards
 = BaiYang
  baiyang@gmail= .com
  http://i.baiy.cn<= br>
**** < EN= D OF EMAIL > **** 
 
 
D= ate: 2022-09-20 02:14
To: baiyang
Subject: Re: Re: [musl] The heap memory pe= rformance (malloc/free/realloc) is significantly degraded in musl 1.2 (com= pared to 1.1)
* baiyang <baiyang@gmail.com&g= t; [2022-09-20 01:40:48 +0800]:
=0A
> I looked at the code of = tcmalloc, but I didn't find any of the problems you mentioned in the imple= mentation of malloc_usable_size (see: https://github.com/google/tcmalloc/b= lob/9179bb884848c30616667ba129bcf9afee114c32/tcmalloc/tcmalloc.cc#L1099 ).=
=0A
>
=0A
> On the contrary, similar to musl, tc= malloc also directly uses the return value of malloc_usable_size in its re= alloc implementation to determine whether memory needs to be reallocated: = https://github.com/google/tcmalloc/blob/9179bb884848c30616667ba129bcf9afee= 114c32/tcmalloc/tcmalloc.cc#L1499
=0A
>
=0A
> I t= hink this is enough to show that the return value of malloc_usable_size in= tcmalloc is accurate and reliable, otherwise its own realloc will cause a= segment fault.
=0A
 
=0A
obviously internally the = implementation can use the internal chunk size...
=0A
 =0A
GetSize(p) is not the exact size (that the user allocated) but an= internal
=0A
size (which may be larger) and that must not be exp= osed *outside* of the
=0A
malloc implementation (other than for d= iagnostic purposes).
=0A
 
=0A
you can have 2 views= :
=0A
 
=0A
(1) tcmalloc and jemalloc are buggy bec= ause they expose an internal
=0A
    that must not= be exposed (becaues it can break user code).
=0A
 
=0A=
(2) user code is buggy if it uses malloc_usable_size for any purpose<= /div>=0A
    other than diagnostic/statistics (because = other uses are broken
=0A
    on many implementati= ons).
=0A
 
=0A
either way the brokenness you want = to support is a security hazard
=0A
and you are lucky that musl s= aves the day: it works hard not to
=0A
expose internal sizes so t= he code you seem to care about can operate
=0A
safely (which is n= ot true on tcmalloc and jemalloc: the compiler
=0A
may break that= code).
=0A
=0A ------=_001_NextPart771105884157_=------