* [musl] Cannot dlopen() an already loaded shared library by its SONAME name @ 2022-01-11 11:50 Ilia K 2022-01-11 17:55 ` Markus Wichmann 0 siblings, 1 reply; 6+ messages in thread From: Ilia K @ 2022-01-11 11:50 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 1648 bytes --] Hi! It turns out that `dlopen()` in musl cannot find the already loaded shared library using the library soname from the dynamic section, unlike glibc. Here is a simple demo app: ``` # cat main.c #include <dlfcn.h> #include <stdio.h> #define SOMELIB_DIR "/root/dlopen_file_name_test/somelib/" #define SOMELIB_NAME "libsomelib.so" int main() { void* library_handle = dlopen(SOMELIB_DIR SOMELIB_NAME, RTLD_LAZY | RTLD_LOCAL); printf("dlopen first load: %p\n", library_handle); { void* library_handle2 = dlopen(SOMELIB_DIR SOMELIB_NAME, RTLD_LAZY | RTLD_LOCAL | RTLD_NOLOAD); printf("dlopen by file path: %p\n", library_handle2); if (library_handle2) dlclose(library_handle2); } { void* library_handle3 = dlopen(SOMELIB_NAME, RTLD_LAZY | RTLD_LOCAL); // RTLD_NOLOAD doesn't work either printf("dlopen by file name: %p %s\n", library_handle3, library_handle3 ? NULL : dlerror()); if (library_handle3) dlclose(library_handle3); } if (library_handle) dlclose(library_handle); return 0; } # cat somelib.c int somelib_func() { return 0; } # cat Makefile all: mkdir -p somelib gcc -shared somelib.c -Wl,-soname,libsomelib.so -o somelib/libsomelib.so gcc main.c -o main -ldl ``` Compile & run: ``` # make mkdir -p somelib gcc -shared somelib.c -Wl,-soname,libsomelib.so -o somelib/libsomelib.so gcc main.c -o main -ldl # ./main dlopen first load: 0x7f06a28a4ca0 dlopen by file path: 0x7f06a28a4ca0 dlopen by file name: 0 Error loading shared library libsomelib.so: No such file or directory ``` Do you have any plans to support it? -- - Ilia [-- Attachment #2: Type: text/html, Size: 2090 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name 2022-01-11 11:50 [musl] Cannot dlopen() an already loaded shared library by its SONAME name Ilia K @ 2022-01-11 17:55 ` Markus Wichmann 2022-01-11 21:30 ` Harmen Stoppels 0 siblings, 1 reply; 6+ messages in thread From: Markus Wichmann @ 2022-01-11 17:55 UTC (permalink / raw) To: musl On Tue, Jan 11, 2022 at 12:50:33PM +0100, Ilia K wrote: > Hi! > > It turns out that `dlopen()` in musl cannot find the already loaded shared > library using the library soname from the dynamic section, unlike glibc. > [...] > Do you have any plans to support it? > Given that this question already surfaced twice and was denied just as often, I'm guessing no. The reason for the behavior is that loading a library with explicit path name is what you do with a plugin, and you don't necessarily want a plugin's symbols to be visible to everyone. That's why a library loaded by path name does not get a shortname set, and a shortname is what you need to be able to find a library by just its name. I would suggest always loading the library by its short name and setting LD_LIBRARY_PATH if this is what you want. That behavior works consistently across all libc implementations I know. Ciao, Markus ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name 2022-01-11 17:55 ` Markus Wichmann @ 2022-01-11 21:30 ` Harmen Stoppels 2022-01-11 22:22 ` Rich Felker 0 siblings, 1 reply; 6+ messages in thread From: Harmen Stoppels @ 2022-01-11 21:30 UTC (permalink / raw) To: musl [-- Attachment #1.1: Type: text/plain, Size: 2643 bytes --] > Given that this question already surfaced twice and was denied just as > > often, I'm guessing no. The fact that this is a common request could also imply it's behavior people expect. So far I've seen: 1. Julia [1] splits binary dependencies into separate packages, so when liba.so depends on libb.so, they live in a different dir, where the absolute and relative paths are only known when the julia interpreter has started, so neither rpaths or LD_LIBRARY_PATH can be used. So they dlopen libb.so, and then dlopen liba.so in that order, and then assume liba.so does not have to locate libb.so again, because its soname is already seens before. The proposed workaround was: don't list libb.so in the DT_NEEDED of liba.so (that is, if you're already doing the work of the linker, you might as well not use the linker at all for locating libs). However, being able to run executables shipped with julia packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH set properly) 2. The Nix / Guix / Spack people are trying to reduce startup time of executables with many shared libraries (as well as fixing library paths once and for all to keep executables run deterministically). In Guix there's a blog post where they call this the "stat storm" [2], and they solve it in a glibc patch: using context dependent ld.so.cache, that is, a reverse mapping soname => library path. In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED in executables with absolute paths of all required libs (also transient ones). This works fine, except on musl, where a dlopen by soname will still do a search. 3. Wine is reported to rely on this earlier in the mailing list, but that did not get any responses [3]. It reports the behavior is the same on glibc, FreeBSD, NetBSD, and OpenBSD dynamic loaders; and musl is the exception. [1] https://github.com/JuliaLang/julia/issues/40556 [2] https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/ [3] https://www.openwall.com/lists/musl/2021/12/16/1 > The reason for the behavior is that loading a library with explicit path > > name is what you do with a plugin, and you don't necessarily want a > > plugin's symbols to be visible to everyone. That's why a library loaded > > by path name does not get a shortname set, and a shortname is what you > > need to be able to find a library by just its name. To me it seems very uncommon that two libraries with the same soname are not supposed to be the same library. [-- Attachment #1.2: publickey - me@harmenstoppels.nl - 0xFD537C88.asc --] [-- Type: application/pgp-keys, Size: 1815 bytes --] [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 509 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name 2022-01-11 21:30 ` Harmen Stoppels @ 2022-01-11 22:22 ` Rich Felker 2022-01-12 14:52 ` Harmen Stoppels 0 siblings, 1 reply; 6+ messages in thread From: Rich Felker @ 2022-01-11 22:22 UTC (permalink / raw) To: Harmen Stoppels; +Cc: musl On Tue, Jan 11, 2022 at 09:30:10PM +0000, Harmen Stoppels wrote: > > Given that this question already surfaced twice and was denied just as > > > > > often, I'm guessing no. > > The fact that this is a common request could also imply it's behavior > people expect. So far I've seen: > > 1. Julia [1] splits binary dependencies into separate packages, so when > liba.so depends on libb.so, they live in a different dir, where > the absolute and relative paths are only known when the julia > interpreter has started, so neither rpaths or LD_LIBRARY_PATH can > be used. > > So they dlopen libb.so, and then dlopen liba.so in that > order, and then assume liba.so does not have to locate libb.so > again, because its soname is already seens before. > > The proposed workaround was: don't list libb.so in the > DT_NEEDED of liba.so (that is, if you're already doing the work of > the linker, you might as well not use the linker at all for locating > libs). However, being able to run executables shipped with julia > packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH > set properly) > 2. The Nix / Guix / Spack people are trying to reduce startup time of > executables with many shared libraries (as well as fixing library > paths once and for all to keep executables run deterministically). > In Guix there's a blog post where they call this the "stat storm" [2], > and they solve it in a glibc patch: using context dependent ld.so.cache, > that is, a reverse mapping soname => library path. > In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED > in executables with absolute paths of all required libs (also > transient ones). This works fine, except on musl, where a dlopen by > soname will still do a search. This could be solved much better by making an application-specific directory full of symlinks to the libraries it uses and putting that directory as the first thing in the program binary's rpath. > 3. Wine is reported to rely on this earlier in the mailing list, but that > did not get any responses [3]. It reports the behavior is the same on > glibc, FreeBSD, NetBSD, and OpenBSD dynamic loaders; and musl is the > exception. There is some argument to be made for honoring SONAME as an action to put the name into a dictonary for future dlopen to use, but having this happen on libraries without any SONAME is really an anti-feature. The point of dlopen with an explicit pathname is to load an explicit file outside of the system library ecosystem, not to subvert the order of path search for loading other libraries. Rich ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name 2022-01-11 22:22 ` Rich Felker @ 2022-01-12 14:52 ` Harmen Stoppels 2022-01-12 16:58 ` Farid Zakaria 0 siblings, 1 reply; 6+ messages in thread From: Harmen Stoppels @ 2022-01-12 14:52 UTC (permalink / raw) To: Rich Felker; +Cc: musl [-- Attachment #1.1: Type: text/plain, Size: 3272 bytes --] > > > > 1. Julia [1] splits binary dependencies into separate packages, so when > > liba.so depends on libb.so, they live in a different dir, where > > the absolute and relative paths are only known when the julia > > interpreter has started, so neither rpaths or LD_LIBRARY_PATH can > > be used. > > So they dlopen libb.so, and then dlopen liba.so in that > > order, and then assume liba.so does not have to locate libb.so > > again, because its soname is already seens before. > > The proposed workaround was: don't list libb.so in the > > DT_NEEDED of liba.so (that is, if you're already doing the work of > > the linker, you might as well not use the linker at all for locating > > libs). However, being able to run executables shipped with julia > > packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH > > set properly) > > > > 2. The Nix / Guix / Spack people are trying to reduce startup time of > > executables with many shared libraries (as well as fixing library > > paths once and for all to keep executables run deterministically). > > In Guix there's a blog post where they call this the "stat storm" [2], > > and they solve it in a glibc patch: using context dependent ld.so.cache, > > that is, a reverse mapping soname => library path. > > In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED > > in executables with absolute paths of all required libs (also > > transient ones). This works fine, except on musl, where a dlopen by > > soname will still do a search. > > This could be solved much better by making an application-specific > directory full of symlinks to the libraries it uses and putting that > directory as the first thing in the program binary's rpath. So the proposal is basically to replicate an application-specific ld.so.cache in the filesystem? Create a dir per ELF file, put its path in the RPATH, fill the dir with symlinks from soname => library. One problem with this is that $ORIGIN starts behaving differently compared to ld.so.cache. $ORIGIN is now relative to the directory of the symlink, not to the realpath of library. So if a library of a dependent package dlopen's a library in its own prefix by soname, relying on an rpath of say $ORIGIN/plugins, this will fail if the symlink dir is just a flat file list. So it means you'd effectively need to merge the prefixes, and this won't fly in Nix / Spack. Also it won't be a manageable solution for Julia, cause they require immutability of each prefix (and if they knew the relative path ahead of time, they wouldn't need this at all). > but having this happen on libraries without any SONAME is really an anti-feature. What do you mean? I think we're talking only about libraries that have a SONAME, do you mean dlopen-by-soname? So, do I understand correctly that loading a lib by path & putting its soname in a dict, so that future libs opened by soname can early exit is fine? But loading a lib by soname first, and then by path, and upon opening it happens to have a soname seen before, but is a different file (st_dev/st_ino), then it should continue with this lib, not early exit? [-- Attachment #1.2: publickey - me@harmenstoppels.nl - 0xFD537C88.asc --] [-- Type: application/pgp-keys, Size: 1815 bytes --] [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 509 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name 2022-01-12 14:52 ` Harmen Stoppels @ 2022-01-12 16:58 ` Farid Zakaria 0 siblings, 0 replies; 6+ messages in thread From: Farid Zakaria @ 2022-01-12 16:58 UTC (permalink / raw) To: musl; +Cc: Rich Felker FWIW, I think we all agree that the discussion is involving libraries that have a _soname_ and utilizing that as the cache key mechanism. The other approach utilizing a symlink directory doesn't work with is how I am trying to freeze dependencies using https://github.com/fzakaria/shrinkwrap The tool basically lifts all NEEDED dependencies from the whole closure to the top level executable and requires them at absolute path and does by trying to modify only the executable ELF file. Doing a symlink approach is a _lot more_ heavy handed to try and achieve the same effect. On Wed, Jan 12, 2022 at 6:52 AM Harmen Stoppels <me@harmenstoppels.nl> wrote: > > > > > > > 1. Julia [1] splits binary dependencies into separate packages, so when > > > liba.so depends on libb.so, they live in a different dir, where > > > the absolute and relative paths are only known when the julia > > > interpreter has started, so neither rpaths or LD_LIBRARY_PATH can > > > be used. > > > So they dlopen libb.so, and then dlopen liba.so in that > > > order, and then assume liba.so does not have to locate libb.so > > > again, because its soname is already seens before. > > > The proposed workaround was: don't list libb.so in the > > > DT_NEEDED of liba.so (that is, if you're already doing the work of > > > the linker, you might as well not use the linker at all for locating > > > libs). However, being able to run executables shipped with julia > > > packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH > > > set properly) > > > > > > 2. The Nix / Guix / Spack people are trying to reduce startup time of > > > executables with many shared libraries (as well as fixing library > > > paths once and for all to keep executables run deterministically). > > > In Guix there's a blog post where they call this the "stat storm" [2], > > > and they solve it in a glibc patch: using context dependent ld.so.cache, > > > that is, a reverse mapping soname => library path. > > > In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED > > > in executables with absolute paths of all required libs (also > > > transient ones). This works fine, except on musl, where a dlopen by > > > soname will still do a search. > > > > This could be solved much better by making an application-specific > > directory full of symlinks to the libraries it uses and putting that > > directory as the first thing in the program binary's rpath. > > So the proposal is basically to replicate an application-specific > ld.so.cache in the filesystem? Create a dir per ELF file, put its > path in the RPATH, fill the dir with symlinks from soname => library. > > One problem with this is that $ORIGIN starts behaving differently > compared to ld.so.cache. $ORIGIN is now relative to the directory > of the symlink, not to the realpath of library. So if a library > of a dependent package dlopen's a library in its own prefix by soname, > relying on an rpath of say $ORIGIN/plugins, this will fail if the > symlink dir is just a flat file list. > > So it means you'd effectively need to merge the prefixes, and this > won't fly in Nix / Spack. > > Also it won't be a manageable solution for Julia, cause they require immutability > of each prefix (and if they knew the relative path ahead of time, they > wouldn't need this at all). > > > but having this happen on libraries without any SONAME is really an anti-feature. > > What do you mean? I think we're talking only about libraries that have a SONAME, > do you mean dlopen-by-soname? > > So, do I understand correctly that loading a lib by path & putting its soname in > a dict, so that future libs opened by soname can early exit is fine? > > But loading a lib by soname first, and then by path, and upon opening it > happens to have a soname seen before, but is a different file (st_dev/st_ino), > then it should continue with this lib, not early exit? ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-01-12 16:58 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-01-11 11:50 [musl] Cannot dlopen() an already loaded shared library by its SONAME name Ilia K 2022-01-11 17:55 ` Markus Wichmann 2022-01-11 21:30 ` Harmen Stoppels 2022-01-11 22:22 ` Rich Felker 2022-01-12 14:52 ` Harmen Stoppels 2022-01-12 16:58 ` Farid Zakaria
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).