Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1)

[Florian Weimer <fweimer@redhat.com>]

* Rich Felker:

> On Mon, Sep 19, 2022 at 02:36:41PM +0200, Florian Weimer wrote:
>> * Szabolcs Nagy:
>> 
>> > unlike musl those implementations don't return exact size nor have the
>> > same security and memory fragmentation guarantees, so bad comparision.
>> >
>> > tcmalloc:
>> >   // Returns the actual number N of bytes reserved by tcmalloc for the pointer
>> >   // p.  This number may be equal to or greater than the number of bytes
>> >   // requested when p was allocated.
>> >   //
>> >   // This function is just useful for statistics collection.  The client must
>> >   // *not* read or write from the extra bytes that are indicated by this call.
>> >
>> > jemalloc:
>> >       <para>The <function>malloc_usable_size()</function> function
>> >       returns the usable size of the allocation pointed to by
>> >       <parameter>ptr</parameter>.  The return value may be larger than the size
>> >       that was requested during allocation.  The
>> >       <function>malloc_usable_size()</function> function is not a
>> >       mechanism for in-place <function>realloc()</function>; rather
>> >       it is provided solely as a tool for introspection purposes.  Any
>> >       discrepancy between the requested allocation size and the size reported
>> >       by <function>malloc_usable_size()</function> should not be
>> >       depended on, since such behavior is entirely implementation-dependent.
>> 
>> These implementations are buggy or at least mis-documented.  The
>> interface contract is clearly that for that particular object, the extra
>> bytes in the allocation are available for reading and writing.  It is
>> not guaranteed that the allocator will always provide the same number of
>> extra bytes for the same requested size, but they must be there for the
>> allocation being examined.  It's even in the name of the function!
>
> I'm not sure I understand what you're saying, but the core problem
> that really can't be solved is potential discrepancy between the
> malloc implementation's idea of usable and the compiler's. For
> example:
>
> 	char *p = malloc(1);
> 	if (malloc_usable_size(p)>1) p[1] = 42;
>
> will cause a compiler that's actively detecting UB to abort the
> program when malloc_usable_size returns a value larger than 1.

The compiler needs to treat malloc_usable_size similar to realloc and
just the size information for the buffer based on the return value from
malloc_usable_size.  This is admittedly harder to do than a comparable
analysis for realloc if the compiler interprets the standard in such a
way that after a successful realloc, any access to the original pointer
value is undefined.

malloc_usable_size is not actually *that* useful with allocators that do
not have strict size classes because they do not over-allocate that
much.  For these allocators, it may be possible to increase the size of
allocation significantly without moving it, but that is not reflected in
the return value of malloc_usable_size at all.

Thanks,
Florian