Best Regards BaiYang baiyang@gmail.com http://i.baiy.cn |
From: James Y KnightDate: 2022-09-19 21:53To: muslCC: Florian Weimer; baiyangSubject: Re: [musl] The heap memory performance (malloc/free/realloc) is significantly degraded in musl 1.2 (compared to 1.1)Indeed. RedHat mentioned that problem in their recent post about _FORTIFY_SOURCE=3, here"""_FORTIFY_SOURCE=3 revealed another pattern. Applications such as systemd used malloc_usable_size to determine available space in objects and then used the residual space. The glibc manual discourages this type of usage, dictating that malloc_usable_size is for diagnostic purposes only. But applications use the function as a hack to avoid reallocating buffers when there is space in the underlying malloc chunk. The implementation of malloc_usable_size needs to be fixed to return the allocated object size instead of the chunk size in non-diagnostic use. Alternatively, another solution is to deprecate the function. But that is a topic for discussion by the glibc community.
"""On Mon, Sep 19, 2022 at 9:47 AM Rich Felker <dalias@libc.org> wrote:On Mon, Sep 19, 2022 at 02:36:41PM +0200, Florian Weimer wrote:
> * Szabolcs Nagy:
>
> > unlike musl those implementations don't return exact size nor have the
> > same security and memory fragmentation guarantees, so bad comparision.
> >
> > tcmalloc:
> > // Returns the actual number N of bytes reserved by tcmalloc for the pointer
> > // p. This number may be equal to or greater than the number of bytes
> > // requested when p was allocated.
> > //
> > // This function is just useful for statistics collection. The client must
> > // *not* read or write from the extra bytes that are indicated by this call.
> >
> > jemalloc:
> > <para>The <function>malloc_usable_size()</function> function
> > returns the usable size of the allocation pointed to by
> > <parameter>ptr</parameter>. The return value may be larger than the size
> > that was requested during allocation. The
> > <function>malloc_usable_size()</function> function is not a
> > mechanism for in-place <function>realloc()</function>; rather
> > it is provided solely as a tool for introspection purposes. Any
> > discrepancy between the requested allocation size and the size reported
> > by <function>malloc_usable_size()</function> should not be
> > depended on, since such behavior is entirely implementation-dependent.
>
> These implementations are buggy or at least mis-documented. The
> interface contract is clearly that for that particular object, the extra
> bytes in the allocation are available for reading and writing. It is
> not guaranteed that the allocator will always provide the same number of
> extra bytes for the same requested size, but they must be there for the
> allocation being examined. It's even in the name of the function!
I'm not sure I understand what you're saying, but the core problem
that really can't be solved is potential discrepancy between the
malloc implementation's idea of usable and the compiler's. For
example:
char *p = malloc(1);
if (malloc_usable_size(p)>1) p[1] = 42;
will cause a compiler that's actively detecting UB to abort the
program when malloc_usable_size returns a value larger than 1.
Rich