From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2 autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 1432 invoked from network); 20 Sep 2022 08:34:16 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 20 Sep 2022 08:34:16 -0000 Received: (qmail 15531 invoked by uid 550); 20 Sep 2022 08:34:13 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 15500 invoked from network); 20 Sep 2022 08:34:12 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1663662841; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZRiXpKqYAY5hbvnEnsnO7JtSab/yddA+1scqK1hgZps=; b=XmJAVeV+k24VwB0meLbOR96cBwKM+K3ylJQq2u/kadftsGD60RRrsHdRCU8eyuYk3XtExh 6wV6MEVEKYzMUARo1VgeYUEpRpDLH7aXp1H4EgWEH4sYDwG80ErTaDVPrcAdUQpTf5qHrW WHT3WnuVrLs30FQiyP/JB+KSS4yfXKg= X-MC-Unique: WlA0jDaQNiSAOyfJ1JpElw-1 From: Florian Weimer To: Rich Felker Cc: baiyang , musl , Siddhesh Poyarekar References: <2022091915532777412615@gmail.com> <20220919110829.GA2158779@port70.net> <874jx3h76u.fsf@oldenburg.str.redhat.com> <20220919134659.GO9709@brightrain.aerifal.cx> Date: Tue, 20 Sep 2022 10:33:53 +0200 In-Reply-To: <20220919134659.GO9709@brightrain.aerifal.cx> (Rich Felker's message of "Mon, 19 Sep 2022 09:46:59 -0400") Message-ID: <874jx2phqm.fsf@oldenburg.str.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Subject: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1) * Rich Felker: > On Mon, Sep 19, 2022 at 02:36:41PM +0200, Florian Weimer wrote: >> * Szabolcs Nagy: >> >> > unlike musl those implementations don't return exact size nor have the >> > same security and memory fragmentation guarantees, so bad comparision. >> > >> > tcmalloc: >> > // Returns the actual number N of bytes reserved by tcmalloc for the pointer >> > // p. This number may be equal to or greater than the number of bytes >> > // requested when p was allocated. >> > // >> > // This function is just useful for statistics collection. The client must >> > // *not* read or write from the extra bytes that are indicated by this call. >> > >> > jemalloc: >> > The malloc_usable_size() function >> > returns the usable size of the allocation pointed to by >> > ptr. The return value may be larger than the size >> > that was requested during allocation. The >> > malloc_usable_size() function is not a >> > mechanism for in-place realloc(); rather >> > it is provided solely as a tool for introspection purposes. Any >> > discrepancy between the requested allocation size and the size reported >> > by malloc_usable_size() should not be >> > depended on, since such behavior is entirely implementation-dependent. >> >> These implementations are buggy or at least mis-documented. The >> interface contract is clearly that for that particular object, the extra >> bytes in the allocation are available for reading and writing. It is >> not guaranteed that the allocator will always provide the same number of >> extra bytes for the same requested size, but they must be there for the >> allocation being examined. It's even in the name of the function! > > I'm not sure I understand what you're saying, but the core problem > that really can't be solved is potential discrepancy between the > malloc implementation's idea of usable and the compiler's. For > example: > > char *p = malloc(1); > if (malloc_usable_size(p)>1) p[1] = 42; > > will cause a compiler that's actively detecting UB to abort the > program when malloc_usable_size returns a value larger than 1. The compiler needs to treat malloc_usable_size similar to realloc and just the size information for the buffer based on the return value from malloc_usable_size. This is admittedly harder to do than a comparable analysis for realloc if the compiler interprets the standard in such a way that after a successful realloc, any access to the original pointer value is undefined. malloc_usable_size is not actually *that* useful with allocators that do not have strict size classes because they do not over-allocate that much. For these allocators, it may be possible to increase the size of allocation significantly without moving it, but that is not reflected in the return value of malloc_usable_size at all. Thanks, Florian