Re: Removing glibc from the musl .2 ABI

[Rich Felker <dalias@libc.org>]

On Wed, Jul 24, 2019 at 05:17:35PM +0200, Szabolcs Nagy wrote:
> * Rich Felker <dalias@libc.org> [2019-07-22 11:52:59 -0400]:
> > So, what I'd (tentatively; for discussion) like to do:
> > 
> > When ldso loads an application or shared library and detects that it's
> > glibc-linked (DT_NEEDED for libc.so.6), it both loads a gcompat
> > library instead *and* flags the dso as needing ABI-compat. The gcompat
> > library would be permanently RTLD_LOCAL, unable to be used for
> > resolving global symbols, since it would have to define symbols
> > conflicting with libc symbols names and with future directions of the
> > musl ABI.
> > 
> > Symbol lookups when relocating such a flagged dso would take place by
> > first processing gcompat (logically, adding it to the head of the dso
> > search list), then the normal symbol search order. The gcompat library
> > could also provide a replacement dlsym function, so that dlsym calls
> > from the glibc-linked DSO also follow this order, and a replacement
> > dlopen, so that dlopen of libc from the glibc-linked DSO would get the
> > gcompat module.
> > 
> > I'm not sure what mechanism gcompat would then use to make its own
> > references to the underlying real libc functions. This is something
> > we'd need to think about.
> 
> i'm not sure how gcompat would implement dlsym, if it's
> on top of the musl dlsym, then that needs to be accessible
> already (e.g. by exposing a __musl_dlsym alias) and can be
> used to do lookups in libc.so.

The same applies to any interface it would have to wrap due to
mismatched ABI. I can think of a couple potential ways:

1. Simply by referencing the symbol name directly. libgcompat would
not be a glibc dso, so ldso would not use it to resolve its own
symbols, and would end up finding them in libc. The only concern here
is whether the compiler or linker might do some kind of early binding
that makes the references circular and non-symbolic. I don't think the
linker can (can in the sense of "has standing to") do this, but
perhaps the compiler could optimize a call from a function named dlsym
to a function named dlsym to be local rather than going through the
GOT/PLT, since if it were interposed it would never be called to begin
with. However this wouldn't work if there were other aliases for the
same function, so it's probably not something the compiler could do.
gcompat could explicitly preclude it via something like:

void *__gcompat_dlsym(void *dso, const char *restrict name)
{
	...
	return dlsym(...);
}
weak_alias(__gcompat_dlsym, dlsym);

This works because the alias imposes a mandate that the definitions be
the same, and __gcompat_dlsym would be exported (thus not able to be
optimized out) and would necessarily have to honor interposition of
dlsym.

2. By having ldso remap symbol references *from* gcompat, so that
gcompat could refer to __libc_foo and have the reference get remapped
to plain foo.

I feel like option 2 is a nastier hack (especially on the musl side)
and not needed, since option 1 seems to work.

Rich