Re: tcmalloc compatibility

[Rich Felker <dalias@libc.org>]

On Tue, Apr 10, 2018 at 10:33:55PM +0200, Szabolcs Nagy wrote:
> * Florian Weimer <fw@deneb.enyo.de> [2018-04-10 19:53:46 +0200]:
> > We have some documentation nowadays:
> > 
> > <https://www.gnu.org/software/libc/manual/html_node/Replacing-malloc.html>
> > 
> > The remaining undocumented aspects concern cyclic dependencies, such
> > as the suitability of certain TLS models for implementing a custom
> > malloc, or using memory-allocating glibc functions such as fopen or
> > backtrace from the allocator itself.
> > 
> 
> it's not documented what the interposed malloc may or may not do.
> 
> cyclic dependency (because malloc calls a memory-allocating libc
> function) is one issue, but in general the libc can be in an
> inconsistent state at the point of an internal malloc call (e.g.
> some lock is held that should never be held when user code is
> running) and thus certain other libc functions may not operate
> as expected in an interposer.
> 
> documenting the set of libc apis that work as expected in an
> interposer is difficult.

I think limiting to AS-safe plus a small list of additional allowed
functions is reasonably easy to guarantee works.

> > In practice, malloc interposition works extremely well and causes few
> > issues due to interposition itself.  Obviously, there are bugs, but
> > most of them would still be bugs if the allocator was non-interposing.
> > (Examples are lots of initial-exec TLS data, and incorrect alignment
> > for allocations.)
> 
> even with the very small set of libc apis that a malloc
> interposer may want to use, there known (but undocumented)
> caveats:
> 
> - glibc dlsym calls calloc (bites anybody who tries to wrap
> calloc in the usual way)

Yes. While I think it's unreasonable that glibc calls calloc from
dlsym, I also think it's unreasonable to be calling dlsym from a
malloc replacement. I don't think I would include dlsym or other
heavy, AS-unsafe functions in an allowed-list for musl.

> - general dynamic tls may call malloc and uses global dl
> lock so using tls can cause recursion or deadlock.

Yes but this is a known glibc bug that's hopefully on the way to
being fixed. A robust/correct implementation (where TLS works in
signal handlers and without possibility of OOM-crashing) can't
allocate at access time.

> - using stdio for logging is problematic because of stdio
> locks and recursive malloc calls.

Yes.

> - malloc may get called before the first ctor and after
> the last dtor.

An implementation could avoid this, but yes, I agree, there should be
no contract that it won't.

> - malloc may be called on a thread with small stack
> (e.g. gai helper thread) so it should not have excessive
> stack usage.

Certainly increasing the default thread stack size could be a
requirement to use malloc replacements that have large stack usage,
but I tend to doubt that would happen. If malloc has large stack
usage, it's likely to be very slow.

> - glibc fork tries to reset the malloc state before
> calling the child atfork handler in an attempt to support
> non-as-safe fork handlers, code that relies on this can
> get broken with malloc interposition.

Interposed mallocs probably try to support this with pthread_atfork,
but state after MT-fork is permanently an AS context anyway.

> interposition only seems to work extremely well because
> users already fixed their code and things dont change
> much in this space.

Maybe. :)

Rich