Re: [musl] is RUNPATH being incorrectly inherited inherited?

[Rich Felker <dalias@libc.org>]

On Wed, Jan 12, 2022 at 05:29:57PM -0800, Farid Zakaria wrote:
> Hello,
> 
> I'm observing a strange behavior and it seems to contradict what
> ld.so[1] says should be the case for RUNPATH, namely I am observing
> that musl's dynamic linker seems to use the DT_RUNPATH of the
> executable when searching for entries of its children.

I thought this was documented somewhere but maybe it's not. musl does
not have separate RPATH/RUNPATH behaviors. The clearly wrong part of
the legacy RPATH behavior (order relative to LD_LIBRARY_PATH) is not
followed, and the rationale stated at the time this was implemented in
musl was that the only justification for the old behavior there was
compatibility with existing binaries, which was not an issue for musl.

On the other hand, the glibc RUNPATH behavior of ignoring the
dependent DSO's (or main app's) RUNPATH when loading indirect
dependencies completely breaks the ability to set a RUNPATH in the
application to use a *set* of unmodified library binaries in a
particular application-specific path. I'm surprised you found an
instance where that's what you want, since as far as I could tell at
the time it was an unwanted change.

> > Using the directories specified in the DT_RUNPATH dynamic section
> > attribute of the binary if present. Such directories are searched
> > only to find those objects required by DT_NEEDED (direct
> > dependencies) entries and do not apply to those objects' children,
> > which must themselves have their own DT_RUNPATH entries. This is
> > unlike DT_RPATH, which is applied to searches for all children in
> > the dependency tree.
> 
> I've uploaded a Makefile[2] that you can use to follow along.
> 
> First let's build the binary. It's designed to depend on two shared
> libraries _libx.so_ and _liby.so_; _liby.so_ also depends on _libx.so_
> 
> Note: that I make the top-level _libx.so_ an absolute path using patchelf.
> 
> ```
> # build it
> $ make CC=musl-gcc
> 
> # let's inspect it to see what it looks like
> # here we can see the needed, including the one absolute
> $ patchelf --print-needed exe
> liby.so
> /home/fzakaria/code//c/libx.so
> libc.so
> 
> # here is the RUNPATH
> $ patchelf --print-rpath exe
> $ORIGIN/b:$ORIGIN/c
> 
> # an alternate query
> $ objdump -x ./exe | grep RUNPATH
>   RUNPATH              $ORIGIN/b:$ORIGIN/
> 
> # here is a dependent library
> $ patchelf --print-needed b/liby.so
> libx.so
> libc.so
> 
> # liby.so has no RUNPATH
> objdump -x b/liby.so| grep RUNPATH
> ```
> 
> I would expect this *to not work* since there is no way for liby.so to
> discover libx.so, especially given the previous correspondence on the
> mailing lists on how musl does not utilize soname for the cache.
> 
> However you can see that the linker is trying to resolve _libx.so_
> using the RUNPATH when it is loading _liby.so_
> 
> ```
> $ strace -e openat,stat,open ./exe
> open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/liby.so",
> O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/c/libx.so",
> O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/libx.so",
> O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or
> directory)
> open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/c/libx.so",
> O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> 24
> +++ exited with 0 +++
> ```
> 
> Harmen's libtree[3] already fails to find the library.
> ```
> $ libtree exe
> exe
> ├── libx.so [direct]
> └── .//b/liby.so [runpath]
>     └── libx.so not found
>         ┊ Paths considered
> ```
> 
> To me this means that the ld.so here is not respecting the description
> of not propagating the search paths to the children.

Yes, that's intentional. Is this something you want not to happen?

> FWIW, as per the other correspondence, if I remove part of the
> RUNPATH, then the binary fails to load even though _libx.so_ was
> already present.

This is expected given the current lack of SONAME processing because,
while it's present, it's not tracked as being "what you should get by
resolving the name libx.so". If we do adopt SONAME processing here it
would be found, and would necessarily preclude any path search from
happening since it would already be present. Presumably that's what
you would want to happen, right?

> 
> ```
> $ patchelf --print-rpath exe
> $ORIGIN/b
> 
> $ /exe
> Error loading shared library libx.so: No such file or directory
> (needed by /home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/liby.so)
> ```
> 
> Thank you.
> 
> [1] https://man7.org/linux/man-pages/man8/ld.so.8.html
> [2] https://gist.github.com/fzakaria/375f2c27f2db000369ec49a684ef8095
> [3] https://github.com/haampie/libtree

Thanks for the example and for finding something that needs better
documentation (even if just, for now, on the wiki under
https://wiki.musl-libc.org/functional-differences-from-glibc.html)

Rich