Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name

[Rich Felker <dalias@libc.org>]

On Tue, Jan 11, 2022 at 09:30:10PM +0000, Harmen Stoppels wrote:
> > Given that this question already surfaced twice and was denied just as
> > 
> 
> > often, I'm guessing no.
> 
> The fact that this is a common request could also imply it's behavior
> people expect. So far I've seen:
> 
> 1. Julia [1] splits binary dependencies into separate packages, so when
>    liba.so depends on libb.so, they live in a different dir, where
>    the absolute and relative paths are only known when the julia
>    interpreter has started, so neither rpaths or LD_LIBRARY_PATH can
>    be used.
> 
>    So they dlopen libb.so, and then dlopen liba.so in that
>    order, and then assume liba.so does not have to locate libb.so
>    again, because its soname is already seens before.
> 
>    The proposed workaround was: don't list libb.so in the
>    DT_NEEDED of liba.so (that is, if you're already doing the work of
>    the linker, you might as well not use the linker at all for locating
>    libs). However, being able to run executables shipped with julia
>    packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH
>    set properly)
> 2. The Nix / Guix / Spack people are trying to reduce startup time of
>    executables with many shared libraries (as well as fixing library
>    paths once and for all to keep executables run deterministically).
>    In Guix there's a blog post where they call this the "stat storm" [2],
>    and they solve it in a glibc patch: using context dependent ld.so.cache,
>    that is, a reverse mapping soname => library path.
>    In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED
>    in executables with absolute paths of all required libs (also
>    transient ones). This works fine, except on musl, where a dlopen by
>    soname will still do a search.

This could be solved much better by making an application-specific
directory full of symlinks to the libraries it uses and putting that
directory as the first thing in the program binary's rpath.

> 3. Wine is reported to rely on this earlier in the mailing list, but that
>    did not get any responses [3]. It reports the behavior is the same on
>    glibc, FreeBSD, NetBSD, and OpenBSD dynamic loaders; and musl is the
>    exception. 

There is some argument to be made for honoring SONAME as an action to
put the name into a dictonary for future dlopen to use, but having
this happen on libraries without any SONAME is really an anti-feature.
The point of dlopen with an explicit pathname is to load an explicit
file outside of the system library ecosystem, not to subvert the order
of path search for loading other libraries.

Rich