Re: [musl] [PATCH v2] MT fork

[Rich Felker <dalias@libc.org>]

On Tue, Nov 10, 2020 at 07:52:17PM -0500, Rich Felker wrote:
> On Mon, Nov 09, 2020 at 05:23:21PM -0500, Rich Felker wrote:
> > On Mon, Nov 09, 2020 at 10:59:01PM +0100, Szabolcs Nagy wrote:
> > > (not ideal, since then interposers can't see all
> > > allocations, which some tools would like to see,
> > > but at least correct and robust. and it is annoying
> > > that we have to do all this extra work just because
> > > of mt-fork)
> > 
> > Yes. On the other hand if this were done more rigorously it would fix
> > the valgrind breakage of malloc during ldso startup..
> > 
> > > > The other pros of such an approach are stuff like making it so
> > > > application code doesn't get called as a callback from messy contexts
> > > > inside libc, e.g. with dynamic linker in inconsistent state. The major
> > > > con I see is that it precludes omitting the libc malloc entirely when
> > > > static linking, assuming you link any part of libc that uses malloc
> > > > internally. However, almost all such places only call malloc, not
> > > > free, so you'd just get the trivial bump allocator gratuitously
> > > > linked, rather than full mallocng or oldmalloc, except for dlerror
> > > > which shouldn't come up in static linked programs anyway.
> > > 
> > > i see.
> > > that sounds fine to me.
> > 
> > I'm still not sure it's fine, so I appreciate your input and anyone
> > else's who has spent some time thinking about this.
> 
> Here's a proposed first patch in series, getting rid of getdelim/stdio
> usage in ldso. I think that suffices to set the stage for adding
> __libc_malloc, __libc_free, __libc_calloc, __libc_realloc and having
> ldso use them.
> 
> To make this work, I think malloc needs to actually be a separate
> function wrapping __libc_malloc -- this is because __simple_malloc
> precludes the malloc symbol itself being weak. That's a very slight
> runtime cost, but has the benefit of eliminating the awful hack of
> relyin on link order to get __simple_malloc (bump allocator) to be
> chosen over full malloc. Now, the source file containing the bump
> allocator can define malloc as a call to __libc_malloc, and provide
> the bump allocator as the weak definition of __libc_malloc.
> mallocng/malloc.c would then provide the strong definition of
> __libc_malloc.
> 
> For the other functions, I think __libc_* can theoretically just be
> aliases for the public symbols, but this may (almost surely does)
> break valgrind, and it's messy to do at the source level, so perhaps
> they should be wrapped too. This should entirely prevent valgrind from
> interposing the libc-internal calls, thereby fixing the longstanding
> bug where it crashes by interposing them too early.

The __simple_malloc link logic for this actually turned out to be
somewhat complicated, so I want to document it here independent of
code before actually working out and posting the code draft:

The source file defining __simple_malloc (lite_malloc.c but I'll
probably rename it to malloc.c at some point) needs to also define
(the only definition of) __libc_malloc and a weak symbol for malloc,
which should be the only definition of malloc in libc.

The first ensures that any reference to __libc_malloc will cause this
file to be linked (rather than relying on link order or else having
the possibility of pulling in full malloc). The latter ensures that
any reference to malloc will cause this file to be linked, unless the
application already defined its own malloc, and weakness ensures they
won't clash if the application did.

Both __libc_malloc and (the weak) malloc function should just call
__libc_malloc_impl, which has a weak definition provided by
__simple_malloc and a strong definition if full malloc is linked (as a
result of referencing free, realloc, etc.). Here malloc could simply
be a weak alias for __libc_malloc, except that valgrind would then
munge __libc_malloc. The function is nothing but a tail call so
duplicating the code twice really doesn't hurt anyway.

Aside from the new __libc_malloc function, the above is really how
this always should have worked (to avoid the link order hack). Not
doing it was a matter of avoiding the wrapper function layer.

Rich