mailing list of musl libc
 help / color / mirror / code / Atom feed
* [musl] Cannot dlopen() an already loaded shared library by its SONAME name
@ 2022-01-11 11:50 Ilia K
  2022-01-11 17:55 ` Markus Wichmann
  0 siblings, 1 reply; 6+ messages in thread
From: Ilia K @ 2022-01-11 11:50 UTC (permalink / raw)
  To: musl

[-- Attachment #1: Type: text/plain, Size: 1648 bytes --]

Hi!

It turns out that `dlopen()` in musl cannot find the already loaded shared
library using the library soname from the dynamic section, unlike glibc.
Here is a simple demo app:
```
# cat main.c
#include <dlfcn.h>
#include <stdio.h>

#define SOMELIB_DIR  "/root/dlopen_file_name_test/somelib/"
#define SOMELIB_NAME "libsomelib.so"

int main() {
    void* library_handle = dlopen(SOMELIB_DIR SOMELIB_NAME, RTLD_LAZY |
RTLD_LOCAL);
    printf("dlopen first load: %p\n", library_handle);

    {
        void* library_handle2 = dlopen(SOMELIB_DIR SOMELIB_NAME, RTLD_LAZY
| RTLD_LOCAL | RTLD_NOLOAD);
        printf("dlopen by file path: %p\n", library_handle2);
        if (library_handle2) dlclose(library_handle2);
    }

    {
        void* library_handle3 = dlopen(SOMELIB_NAME, RTLD_LAZY |
RTLD_LOCAL);  // RTLD_NOLOAD doesn't work either
        printf("dlopen by file name: %p %s\n", library_handle3,
library_handle3 ? NULL : dlerror());
        if (library_handle3) dlclose(library_handle3);
    }

    if (library_handle) dlclose(library_handle);
    return 0;
}
# cat somelib.c
int somelib_func() { return 0; }
# cat Makefile
all:
        mkdir -p somelib
        gcc -shared somelib.c -Wl,-soname,libsomelib.so -o
somelib/libsomelib.so
        gcc main.c -o main -ldl
```

Compile & run:
```
# make
mkdir -p somelib
gcc -shared somelib.c -Wl,-soname,libsomelib.so -o somelib/libsomelib.so
gcc main.c -o main -ldl
# ./main
dlopen first load: 0x7f06a28a4ca0
dlopen by file path: 0x7f06a28a4ca0
dlopen by file name: 0 Error loading shared library libsomelib.so: No such
file or directory
```

Do you have any plans to support it?

-- 
- Ilia

[-- Attachment #2: Type: text/html, Size: 2090 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name
  2022-01-11 11:50 [musl] Cannot dlopen() an already loaded shared library by its SONAME name Ilia K
@ 2022-01-11 17:55 ` Markus Wichmann
  2022-01-11 21:30   ` Harmen Stoppels
  0 siblings, 1 reply; 6+ messages in thread
From: Markus Wichmann @ 2022-01-11 17:55 UTC (permalink / raw)
  To: musl

On Tue, Jan 11, 2022 at 12:50:33PM +0100, Ilia K wrote:
> Hi!
>
> It turns out that `dlopen()` in musl cannot find the already loaded shared
> library using the library soname from the dynamic section, unlike glibc.
> [...]
> Do you have any plans to support it?
>

Given that this question already surfaced twice and was denied just as
often, I'm guessing no.

The reason for the behavior is that loading a library with explicit path
name is what you do with a plugin, and you don't necessarily want a
plugin's symbols to be visible to everyone. That's why a library loaded
by path name does not get a shortname set, and a shortname is what you
need to be able to find a library by just its name.

I would suggest always loading the library by its short name and setting
LD_LIBRARY_PATH if this is what you want. That behavior works
consistently across all libc implementations I know.

Ciao,
Markus

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name
  2022-01-11 17:55 ` Markus Wichmann
@ 2022-01-11 21:30   ` Harmen Stoppels
  2022-01-11 22:22     ` Rich Felker
  0 siblings, 1 reply; 6+ messages in thread
From: Harmen Stoppels @ 2022-01-11 21:30 UTC (permalink / raw)
  To: musl


[-- Attachment #1.1: Type: text/plain, Size: 2643 bytes --]

> Given that this question already surfaced twice and was denied just as
> 

> often, I'm guessing no.

The fact that this is a common request could also imply it's behavior
people expect. So far I've seen:

1. Julia [1] splits binary dependencies into separate packages, so when
   liba.so depends on libb.so, they live in a different dir, where
   the absolute and relative paths are only known when the julia
   interpreter has started, so neither rpaths or LD_LIBRARY_PATH can
   be used.

   So they dlopen libb.so, and then dlopen liba.so in that
   order, and then assume liba.so does not have to locate libb.so
   again, because its soname is already seens before.

   The proposed workaround was: don't list libb.so in the
   DT_NEEDED of liba.so (that is, if you're already doing the work of
   the linker, you might as well not use the linker at all for locating
   libs). However, being able to run executables shipped with julia
   packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH
   set properly)
2. The Nix / Guix / Spack people are trying to reduce startup time of
   executables with many shared libraries (as well as fixing library
   paths once and for all to keep executables run deterministically).
   In Guix there's a blog post where they call this the "stat storm" [2],
   and they solve it in a glibc patch: using context dependent ld.so.cache,
   that is, a reverse mapping soname => library path.
   In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED
   in executables with absolute paths of all required libs (also
   transient ones). This works fine, except on musl, where a dlopen by
   soname will still do a search.
3. Wine is reported to rely on this earlier in the mailing list, but that
   did not get any responses [3]. It reports the behavior is the same on
   glibc, FreeBSD, NetBSD, and OpenBSD dynamic loaders; and musl is the
   exception. 



[1] https://github.com/JuliaLang/julia/issues/40556
[2] https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/
[3] https://www.openwall.com/lists/musl/2021/12/16/1


> The reason for the behavior is that loading a library with explicit path
> 

> name is what you do with a plugin, and you don't necessarily want a
> 

> plugin's symbols to be visible to everyone. That's why a library loaded
> 

> by path name does not get a shortname set, and a shortname is what you
> 

> need to be able to find a library by just its name.

To me it seems very uncommon that two libraries with the same soname are not
supposed to be the same library.

[-- Attachment #1.2: publickey - me@harmenstoppels.nl - 0xFD537C88.asc --]
[-- Type: application/pgp-keys, Size: 1815 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 509 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name
  2022-01-11 21:30   ` Harmen Stoppels
@ 2022-01-11 22:22     ` Rich Felker
  2022-01-12 14:52       ` Harmen Stoppels
  0 siblings, 1 reply; 6+ messages in thread
From: Rich Felker @ 2022-01-11 22:22 UTC (permalink / raw)
  To: Harmen Stoppels; +Cc: musl

On Tue, Jan 11, 2022 at 09:30:10PM +0000, Harmen Stoppels wrote:
> > Given that this question already surfaced twice and was denied just as
> > 
> 
> > often, I'm guessing no.
> 
> The fact that this is a common request could also imply it's behavior
> people expect. So far I've seen:
> 
> 1. Julia [1] splits binary dependencies into separate packages, so when
>    liba.so depends on libb.so, they live in a different dir, where
>    the absolute and relative paths are only known when the julia
>    interpreter has started, so neither rpaths or LD_LIBRARY_PATH can
>    be used.
> 
>    So they dlopen libb.so, and then dlopen liba.so in that
>    order, and then assume liba.so does not have to locate libb.so
>    again, because its soname is already seens before.
> 
>    The proposed workaround was: don't list libb.so in the
>    DT_NEEDED of liba.so (that is, if you're already doing the work of
>    the linker, you might as well not use the linker at all for locating
>    libs). However, being able to run executables shipped with julia
>    packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH
>    set properly)
> 2. The Nix / Guix / Spack people are trying to reduce startup time of
>    executables with many shared libraries (as well as fixing library
>    paths once and for all to keep executables run deterministically).
>    In Guix there's a blog post where they call this the "stat storm" [2],
>    and they solve it in a glibc patch: using context dependent ld.so.cache,
>    that is, a reverse mapping soname => library path.
>    In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED
>    in executables with absolute paths of all required libs (also
>    transient ones). This works fine, except on musl, where a dlopen by
>    soname will still do a search.

This could be solved much better by making an application-specific
directory full of symlinks to the libraries it uses and putting that
directory as the first thing in the program binary's rpath.

> 3. Wine is reported to rely on this earlier in the mailing list, but that
>    did not get any responses [3]. It reports the behavior is the same on
>    glibc, FreeBSD, NetBSD, and OpenBSD dynamic loaders; and musl is the
>    exception. 

There is some argument to be made for honoring SONAME as an action to
put the name into a dictonary for future dlopen to use, but having
this happen on libraries without any SONAME is really an anti-feature.
The point of dlopen with an explicit pathname is to load an explicit
file outside of the system library ecosystem, not to subvert the order
of path search for loading other libraries.

Rich

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name
  2022-01-11 22:22     ` Rich Felker
@ 2022-01-12 14:52       ` Harmen Stoppels
  2022-01-12 16:58         ` Farid Zakaria
  0 siblings, 1 reply; 6+ messages in thread
From: Harmen Stoppels @ 2022-01-12 14:52 UTC (permalink / raw)
  To: Rich Felker; +Cc: musl


[-- Attachment #1.1: Type: text/plain, Size: 3272 bytes --]

> >
> > 1.  Julia [1] splits binary dependencies into separate packages, so when
> >     liba.so depends on libb.so, they live in a different dir, where
> >     the absolute and relative paths are only known when the julia
> >     interpreter has started, so neither rpaths or LD_LIBRARY_PATH can
> >     be used.
> >     So they dlopen libb.so, and then dlopen liba.so in that
> >     order, and then assume liba.so does not have to locate libb.so
> >     again, because its soname is already seens before.
> >     The proposed workaround was: don't list libb.so in the
> >     DT_NEEDED of liba.so (that is, if you're already doing the work of
> >     the linker, you might as well not use the linker at all for locating
> >     libs). However, being able to run executables shipped with julia
> >     packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH
> >     set properly)
> >
> > 2.  The Nix / Guix / Spack people are trying to reduce startup time of
> >     executables with many shared libraries (as well as fixing library
> >     paths once and for all to keep executables run deterministically).
> >     In Guix there's a blog post where they call this the "stat storm" [2],
> >     and they solve it in a glibc patch: using context dependent ld.so.cache,
> >     that is, a reverse mapping soname => library path.
> >     In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED
> >     in executables with absolute paths of all required libs (also
> >     transient ones). This works fine, except on musl, where a dlopen by
> >     soname will still do a search.
>
> This could be solved much better by making an application-specific
> directory full of symlinks to the libraries it uses and putting that
> directory as the first thing in the program binary's rpath.

So the proposal is basically to replicate an application-specific
ld.so.cache in the filesystem? Create a dir per ELF file, put its
path in the RPATH, fill the dir with symlinks from soname => library.

One problem with this is that $ORIGIN starts behaving differently
compared to ld.so.cache. $ORIGIN is now relative to the directory
of the symlink, not to the realpath of library. So if a library
of a dependent package dlopen's a library in its own prefix by soname,
relying on an rpath of say $ORIGIN/plugins, this will fail if the
symlink dir is just a flat file list.

So it means you'd effectively need to merge the prefixes, and this
won't fly in Nix / Spack.

Also it won't be a manageable solution for Julia, cause they require immutability
of each prefix (and if they knew the relative path ahead of time, they
wouldn't need this at all).

> but having this happen on libraries without any SONAME is really an anti-feature.

What do you mean? I think we're talking only about libraries that have a SONAME,
do you mean dlopen-by-soname?

So, do I understand correctly that loading a lib by path & putting its soname in
a dict, so that future libs opened by soname can early exit is fine?

But loading a lib by soname first, and then by path, and upon opening it
happens to have a soname seen before, but is a different file (st_dev/st_ino),
then it should continue with this lib, not early exit?

[-- Attachment #1.2: publickey - me@harmenstoppels.nl - 0xFD537C88.asc --]
[-- Type: application/pgp-keys, Size: 1815 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 509 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name
  2022-01-12 14:52       ` Harmen Stoppels
@ 2022-01-12 16:58         ` Farid Zakaria
  0 siblings, 0 replies; 6+ messages in thread
From: Farid Zakaria @ 2022-01-12 16:58 UTC (permalink / raw)
  To: musl; +Cc: Rich Felker

FWIW, I think we all agree that the discussion is involving libraries
that have a _soname_ and utilizing that as the cache key mechanism.

The other approach utilizing a symlink directory doesn't work with is
how I am trying to freeze dependencies using
https://github.com/fzakaria/shrinkwrap
The tool basically lifts all NEEDED dependencies from the whole
closure to the top level executable and requires them at absolute path
and does by trying to modify only
the executable ELF file.  Doing a symlink approach is a _lot more_
heavy handed to try and achieve the same effect.

On Wed, Jan 12, 2022 at 6:52 AM Harmen Stoppels <me@harmenstoppels.nl> wrote:
>
> > >
> > > 1.  Julia [1] splits binary dependencies into separate packages, so when
> > >     liba.so depends on libb.so, they live in a different dir, where
> > >     the absolute and relative paths are only known when the julia
> > >     interpreter has started, so neither rpaths or LD_LIBRARY_PATH can
> > >     be used.
> > >     So they dlopen libb.so, and then dlopen liba.so in that
> > >     order, and then assume liba.so does not have to locate libb.so
> > >     again, because its soname is already seens before.
> > >     The proposed workaround was: don't list libb.so in the
> > >     DT_NEEDED of liba.so (that is, if you're already doing the work of
> > >     the linker, you might as well not use the linker at all for locating
> > >     libs). However, being able to run executables shipped with julia
> > >     packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH
> > >     set properly)
> > >
> > > 2.  The Nix / Guix / Spack people are trying to reduce startup time of
> > >     executables with many shared libraries (as well as fixing library
> > >     paths once and for all to keep executables run deterministically).
> > >     In Guix there's a blog post where they call this the "stat storm" [2],
> > >     and they solve it in a glibc patch: using context dependent ld.so.cache,
> > >     that is, a reverse mapping soname => library path.
> > >     In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED
> > >     in executables with absolute paths of all required libs (also
> > >     transient ones). This works fine, except on musl, where a dlopen by
> > >     soname will still do a search.
> >
> > This could be solved much better by making an application-specific
> > directory full of symlinks to the libraries it uses and putting that
> > directory as the first thing in the program binary's rpath.
>
> So the proposal is basically to replicate an application-specific
> ld.so.cache in the filesystem? Create a dir per ELF file, put its
> path in the RPATH, fill the dir with symlinks from soname => library.
>
> One problem with this is that $ORIGIN starts behaving differently
> compared to ld.so.cache. $ORIGIN is now relative to the directory
> of the symlink, not to the realpath of library. So if a library
> of a dependent package dlopen's a library in its own prefix by soname,
> relying on an rpath of say $ORIGIN/plugins, this will fail if the
> symlink dir is just a flat file list.
>
> So it means you'd effectively need to merge the prefixes, and this
> won't fly in Nix / Spack.
>
> Also it won't be a manageable solution for Julia, cause they require immutability
> of each prefix (and if they knew the relative path ahead of time, they
> wouldn't need this at all).
>
> > but having this happen on libraries without any SONAME is really an anti-feature.
>
> What do you mean? I think we're talking only about libraries that have a SONAME,
> do you mean dlopen-by-soname?
>
> So, do I understand correctly that loading a lib by path & putting its soname in
> a dict, so that future libs opened by soname can early exit is fine?
>
> But loading a lib by soname first, and then by path, and upon opening it
> happens to have a soname seen before, but is a different file (st_dev/st_ino),
> then it should continue with this lib, not early exit?

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2022-01-12 16:58 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-11 11:50 [musl] Cannot dlopen() an already loaded shared library by its SONAME name Ilia K
2022-01-11 17:55 ` Markus Wichmann
2022-01-11 21:30   ` Harmen Stoppels
2022-01-11 22:22     ` Rich Felker
2022-01-12 14:52       ` Harmen Stoppels
2022-01-12 16:58         ` Farid Zakaria

Code repositories for project(s) associated with this inbox:

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).