Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1)

[James Y Knight <jyknight@google.com>]

[-- Attachment #1: Type: text/plain, Size: 3248 bytes --]

Indeed. RedHat mentioned that problem in their recent post about
_FORTIFY_SOURCE=3, here
https://developers.redhat.com/articles/2022/09/17/gccs-new-fortification-level

"""
_FORTIFY_SOURCE=3 revealed another pattern. Applications such as systemd
used malloc_usable_size to determine available space in objects and then
used the residual space. The glibc manual discourages this type of usage,
dictating that malloc_usable_size is for diagnostic purposes only. But
applications use the function as a hack to avoid reallocating buffers when
there is space in the underlying malloc chunk. The implementation of
malloc_usable_size needs to be fixed to return the allocated object size
instead of the chunk size in non-diagnostic use. Alternatively, another
solution is to deprecate the function. But that is a topic for discussion
by the glibc community.
"""

On Mon, Sep 19, 2022 at 9:47 AM Rich Felker <dalias@libc.org> wrote:

> On Mon, Sep 19, 2022 at 02:36:41PM +0200, Florian Weimer wrote:
> > * Szabolcs Nagy:
> >
> > > unlike musl those implementations don't return exact size nor have the
> > > same security and memory fragmentation guarantees, so bad comparision.
> > >
> > > tcmalloc:
> > >   // Returns the actual number N of bytes reserved by tcmalloc for the
> pointer
> > >   // p.  This number may be equal to or greater than the number of
> bytes
> > >   // requested when p was allocated.
> > >   //
> > >   // This function is just useful for statistics collection.  The
> client must
> > >   // *not* read or write from the extra bytes that are indicated by
> this call.
> > >
> > > jemalloc:
> > >       <para>The <function>malloc_usable_size()</function> function
> > >       returns the usable size of the allocation pointed to by
> > >       <parameter>ptr</parameter>.  The return value may be larger than
> the size
> > >       that was requested during allocation.  The
> > >       <function>malloc_usable_size()</function> function is not a
> > >       mechanism for in-place <function>realloc()</function>; rather
> > >       it is provided solely as a tool for introspection purposes.  Any
> > >       discrepancy between the requested allocation size and the size
> reported
> > >       by <function>malloc_usable_size()</function> should not be
> > >       depended on, since such behavior is entirely
> implementation-dependent.
> >
> > These implementations are buggy or at least mis-documented.  The
> > interface contract is clearly that for that particular object, the extra
> > bytes in the allocation are available for reading and writing.  It is
> > not guaranteed that the allocator will always provide the same number of
> > extra bytes for the same requested size, but they must be there for the
> > allocation being examined.  It's even in the name of the function!
>
> I'm not sure I understand what you're saying, but the core problem
> that really can't be solved is potential discrepancy between the
> malloc implementation's idea of usable and the compiler's. For
> example:
>
>         char *p = malloc(1);
>         if (malloc_usable_size(p)>1) p[1] = 42;
>
> will cause a compiler that's actively detecting UB to abort the
> program when malloc_usable_size returns a value larger than 1.
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 4150 bytes --]