mailing list of musl libc
 help / color / mirror / code / Atom feed
From: Farid Zakaria <fmzakari@ucsc.edu>
To: Rich Felker <dalias@libc.org>
Cc: musl@lists.openwall.com, "Scogland, Tom" <scogland1@llnl.gov>,
	me@harmenstoppels.nl,  Carlos Maltzahn <carlosm@ucsc.edu>
Subject: Re: [musl] is RUNPATH being incorrectly inherited inherited?
Date: Wed, 12 Jan 2022 19:46:26 -0800	[thread overview]
Message-ID: <CAH4OOv4ba45gN67mgTuebxCh332nW-NeQxo5pQR+WWieDomQdw@mail.gmail.com> (raw)
In-Reply-To: <20220113032953.GX7074@brightrain.aerifal.cx>

Hi Rich,

Thank you for taking the time to read my email and responding.
I was worried it was a bit too pedantic in explanation and I had had
some trivial error in the setup.
(I am only beginning to gain deeper knowledge on linking)

On Wed, Jan 12, 2022 at 7:29 PM Rich Felker <dalias@libc.org> wrote:
>
> On Wed, Jan 12, 2022 at 05:29:57PM -0800, Farid Zakaria wrote:
> > Hello,
> >
> > I'm observing a strange behavior and it seems to contradict what
> > ld.so[1] says should be the case for RUNPATH, namely I am observing
> > that musl's dynamic linker seems to use the DT_RUNPATH of the
> > executable when searching for entries of its children.
>
> I thought this was documented somewhere but maybe it's not. musl does
> not have separate RPATH/RUNPATH behaviors. The clearly wrong part of
> the legacy RPATH behavior (order relative to LD_LIBRARY_PATH) is not
> followed, and the rationale stated at the time this was implemented in
> musl was that the only justification for the old behavior there was
> compatibility with existing binaries, which was not an issue for musl.
>
> On the other hand, the glibc RUNPATH behavior of ignoring the
> dependent DSO's (or main app's) RUNPATH when loading indirect
> dependencies completely breaks the ability to set a RUNPATH in the
> application to use a *set* of unmodified library binaries in a
> particular application-specific path. I'm surprised you found an
> instance where that's what you want, since as far as I could tell at
> the time it was an unwanted change.
>
> > > Using the directories specified in the DT_RUNPATH dynamic section
> > > attribute of the binary if present. Such directories are searched
> > > only to find those objects required by DT_NEEDED (direct
> > > dependencies) entries and do not apply to those objects' children,
> > > which must themselves have their own DT_RUNPATH entries. This is
> > > unlike DT_RPATH, which is applied to searches for all children in
> > > the dependency tree.
> >
> > I've uploaded a Makefile[2] that you can use to follow along.
> >
> > First let's build the binary. It's designed to depend on two shared
> > libraries _libx.so_ and _liby.so_; _liby.so_ also depends on _libx.so_
> >
> > Note: that I make the top-level _libx.so_ an absolute path using patchelf.
> >
> > ```
> > # build it
> > $ make CC=musl-gcc
> >
> > # let's inspect it to see what it looks like
> > # here we can see the needed, including the one absolute
> > $ patchelf --print-needed exe
> > liby.so
> > /home/fzakaria/code//c/libx.so
> > libc.so
> >
> > # here is the RUNPATH
> > $ patchelf --print-rpath exe
> > $ORIGIN/b:$ORIGIN/c
> >
> > # an alternate query
> > $ objdump -x ./exe | grep RUNPATH
> >   RUNPATH              $ORIGIN/b:$ORIGIN/
> >
> > # here is a dependent library
> > $ patchelf --print-needed b/liby.so
> > libx.so
> > libc.so
> >
> > # liby.so has no RUNPATH
> > objdump -x b/liby.so| grep RUNPATH
> > ```
> >
> > I would expect this *to not work* since there is no way for liby.so to
> > discover libx.so, especially given the previous correspondence on the
> > mailing lists on how musl does not utilize soname for the cache.
> >
> > However you can see that the linker is trying to resolve _libx.so_
> > using the RUNPATH when it is loading _liby.so_
> >
> > ```
> > $ strace -e openat,stat,open ./exe
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/liby.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/c/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or
> > directory)
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/c/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > 24
> > +++ exited with 0 +++
> > ```
> >
> > Harmen's libtree[3] already fails to find the library.
> > ```
> > $ libtree exe
> > exe
> > ├── libx.so [direct]
> > └── .//b/liby.so [runpath]
> >     └── libx.so not found
> >         ┊ Paths considered
> > ```
> >
> > To me this means that the ld.so here is not respecting the description
> > of not propagating the search paths to the children.
>
> Yes, that's intentional. Is this something you want not to happen?

This specific case, I only brought up after having read the manpage
for linux-ld/ld.so
on my quest to better understand dynamic linkers. It does seem that
other tools are making similar assumptions
however (libtree) when trying to replicate the search pattern employed
by a linker.

The rationale you mention about the ability to set a single top-level
RUNPATH is desirable however
I do find the delta between what's documented for ld.so to have been confusing.

I am as part of some research evaluating other alternatives to how to
specify link-paths and part of that
research direction involves more explicitly specifying whether a path
should/should not be inherited; in addition to a
few other ideas.

It's clear you are aware of this discrepancy and it was done so knowingly.
At least I know I was not mis-understanding the results of my setup :)

> > FWIW, as per the other correspondence, if I remove part of the
> > RUNPATH, then the binary fails to load even though _libx.so_ was
> > already present.
>
> This is expected given the current lack of SONAME processing because,
> while it's present, it's not tracked as being "what you should get by
> resolving the name libx.so". If we do adopt SONAME processing here it
> would be found, and would necessarily preclude any path search from
> happening since it would already be present. Presumably that's what
> you would want to happen, right?

Yes! This is more of an interesting quirk I find different between
glibc whose functionality
I think would make a positive addition. Happy to continue discussing
this issue on the other
mailing list emails :)

> >
> > ```
> > $ patchelf --print-rpath exe
> > $ORIGIN/b
> >
> > $ /exe
> > Error loading shared library libx.so: No such file or directory
> > (needed by /home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/liby.so)
> > ```
> >
> > Thank you.
> >
> > [1] https://man7.org/linux/man-pages/man8/ld.so.8.html
> > [2] https://gist.github.com/fzakaria/375f2c27f2db000369ec49a684ef8095
> > [3] https://github.com/haampie/libtree
>
> Thanks for the example and for finding something that needs better
> documentation (even if just, for now, on the wiki under
> https://wiki.musl-libc.org/functional-differences-from-glibc.html)
>
> Rich

On Wed, Jan 12, 2022 at 7:29 PM Rich Felker <dalias@libc.org> wrote:
>
> On Wed, Jan 12, 2022 at 05:29:57PM -0800, Farid Zakaria wrote:
> > Hello,
> >
> > I'm observing a strange behavior and it seems to contradict what
> > ld.so[1] says should be the case for RUNPATH, namely I am observing
> > that musl's dynamic linker seems to use the DT_RUNPATH of the
> > executable when searching for entries of its children.
>
> I thought this was documented somewhere but maybe it's not. musl does
> not have separate RPATH/RUNPATH behaviors. The clearly wrong part of
> the legacy RPATH behavior (order relative to LD_LIBRARY_PATH) is not
> followed, and the rationale stated at the time this was implemented in
> musl was that the only justification for the old behavior there was
> compatibility with existing binaries, which was not an issue for musl.
>
> On the other hand, the glibc RUNPATH behavior of ignoring the
> dependent DSO's (or main app's) RUNPATH when loading indirect
> dependencies completely breaks the ability to set a RUNPATH in the
> application to use a *set* of unmodified library binaries in a
> particular application-specific path. I'm surprised you found an
> instance where that's what you want, since as far as I could tell at
> the time it was an unwanted change.
>
> > > Using the directories specified in the DT_RUNPATH dynamic section
> > > attribute of the binary if present. Such directories are searched
> > > only to find those objects required by DT_NEEDED (direct
> > > dependencies) entries and do not apply to those objects' children,
> > > which must themselves have their own DT_RUNPATH entries. This is
> > > unlike DT_RPATH, which is applied to searches for all children in
> > > the dependency tree.
> >
> > I've uploaded a Makefile[2] that you can use to follow along.
> >
> > First let's build the binary. It's designed to depend on two shared
> > libraries _libx.so_ and _liby.so_; _liby.so_ also depends on _libx.so_
> >
> > Note: that I make the top-level _libx.so_ an absolute path using patchelf.
> >
> > ```
> > # build it
> > $ make CC=musl-gcc
> >
> > # let's inspect it to see what it looks like
> > # here we can see the needed, including the one absolute
> > $ patchelf --print-needed exe
> > liby.so
> > /home/fzakaria/code//c/libx.so
> > libc.so
> >
> > # here is the RUNPATH
> > $ patchelf --print-rpath exe
> > $ORIGIN/b:$ORIGIN/c
> >
> > # an alternate query
> > $ objdump -x ./exe | grep RUNPATH
> >   RUNPATH              $ORIGIN/b:$ORIGIN/
> >
> > # here is a dependent library
> > $ patchelf --print-needed b/liby.so
> > libx.so
> > libc.so
> >
> > # liby.so has no RUNPATH
> > objdump -x b/liby.so| grep RUNPATH
> > ```
> >
> > I would expect this *to not work* since there is no way for liby.so to
> > discover libx.so, especially given the previous correspondence on the
> > mailing lists on how musl does not utilize soname for the cache.
> >
> > However you can see that the linker is trying to resolve _libx.so_
> > using the RUNPATH when it is loading _liby.so_
> >
> > ```
> > $ strace -e openat,stat,open ./exe
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/liby.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/c/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or
> > directory)
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/c/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > 24
> > +++ exited with 0 +++
> > ```
> >
> > Harmen's libtree[3] already fails to find the library.
> > ```
> > $ libtree exe
> > exe
> > ├── libx.so [direct]
> > └── .//b/liby.so [runpath]
> >     └── libx.so not found
> >         ┊ Paths considered
> > ```
> >
> > To me this means that the ld.so here is not respecting the description
> > of not propagating the search paths to the children.
>
> Yes, that's intentional. Is this something you want not to happen?
>
> > FWIW, as per the other correspondence, if I remove part of the
> > RUNPATH, then the binary fails to load even though _libx.so_ was
> > already present.
>
> This is expected given the current lack of SONAME processing because,
> while it's present, it's not tracked as being "what you should get by
> resolving the name libx.so". If we do adopt SONAME processing here it
> would be found, and would necessarily preclude any path search from
> happening since it would already be present. Presumably that's what
> you would want to happen, right?
>
> >
> > ```
> > $ patchelf --print-rpath exe
> > $ORIGIN/b
> >
> > $ /exe
> > Error loading shared library libx.so: No such file or directory
> > (needed by /home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/liby.so)
> > ```
> >
> > Thank you.
> >
> > [1] https://man7.org/linux/man-pages/man8/ld.so.8.html
> > [2] https://gist.github.com/fzakaria/375f2c27f2db000369ec49a684ef8095
> > [3] https://github.com/haampie/libtree
>
> Thanks for the example and for finding something that needs better
> documentation (even if just, for now, on the wiki under
> https://wiki.musl-libc.org/functional-differences-from-glibc.html)
>
> Rich

  reply	other threads:[~2022-01-13  3:46 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-13  1:29 Farid Zakaria
2022-01-13  3:29 ` Rich Felker
2022-01-13  3:46   ` Farid Zakaria [this message]
2022-01-13 20:04     ` Tom Scogland

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAH4OOv4ba45gN67mgTuebxCh332nW-NeQxo5pQR+WWieDomQdw@mail.gmail.com \
    --to=fmzakari@ucsc.edu \
    --cc=carlosm@ucsc.edu \
    --cc=dalias@libc.org \
    --cc=me@harmenstoppels.nl \
    --cc=musl@lists.openwall.com \
    --cc=scogland1@llnl.gov \
    --subject='Re: [musl] is RUNPATH being incorrectly inherited inherited?' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).