mailing list of musl libc
 help / color / mirror / code / Atom feed
From: Harmen Stoppels <me@harmenstoppels.nl>
To: musl@lists.openwall.com
Subject: Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name
Date: Tue, 11 Jan 2022 21:30:10 +0000	[thread overview]
Message-ID: <D8YsUB86oW8DejmzNJUGU-fcHh7ogVUXNvhGD7HB-ayqu2jdEPf_HbaiqNkRaiVbWNpYXMXHCiiqUFXd4cS2J2foq4xvd-jyrOTtGUHpXDY=@harmenstoppels.nl> (raw)
In-Reply-To: <20220111175541.GB2046@voyager>


[-- Attachment #1.1: Type: text/plain, Size: 2643 bytes --]

> Given that this question already surfaced twice and was denied just as
> 

> often, I'm guessing no.

The fact that this is a common request could also imply it's behavior
people expect. So far I've seen:

1. Julia [1] splits binary dependencies into separate packages, so when
   liba.so depends on libb.so, they live in a different dir, where
   the absolute and relative paths are only known when the julia
   interpreter has started, so neither rpaths or LD_LIBRARY_PATH can
   be used.

   So they dlopen libb.so, and then dlopen liba.so in that
   order, and then assume liba.so does not have to locate libb.so
   again, because its soname is already seens before.

   The proposed workaround was: don't list libb.so in the
   DT_NEEDED of liba.so (that is, if you're already doing the work of
   the linker, you might as well not use the linker at all for locating
   libs). However, being able to run executables shipped with julia
   packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH
   set properly)
2. The Nix / Guix / Spack people are trying to reduce startup time of
   executables with many shared libraries (as well as fixing library
   paths once and for all to keep executables run deterministically).
   In Guix there's a blog post where they call this the "stat storm" [2],
   and they solve it in a glibc patch: using context dependent ld.so.cache,
   that is, a reverse mapping soname => library path.
   In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED
   in executables with absolute paths of all required libs (also
   transient ones). This works fine, except on musl, where a dlopen by
   soname will still do a search.
3. Wine is reported to rely on this earlier in the mailing list, but that
   did not get any responses [3]. It reports the behavior is the same on
   glibc, FreeBSD, NetBSD, and OpenBSD dynamic loaders; and musl is the
   exception. 



[1] https://github.com/JuliaLang/julia/issues/40556
[2] https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/
[3] https://www.openwall.com/lists/musl/2021/12/16/1


> The reason for the behavior is that loading a library with explicit path
> 

> name is what you do with a plugin, and you don't necessarily want a
> 

> plugin's symbols to be visible to everyone. That's why a library loaded
> 

> by path name does not get a shortname set, and a shortname is what you
> 

> need to be able to find a library by just its name.

To me it seems very uncommon that two libraries with the same soname are not
supposed to be the same library.

[-- Attachment #1.2: publickey - me@harmenstoppels.nl - 0xFD537C88.asc --]
[-- Type: application/pgp-keys, Size: 1815 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 509 bytes --]

  reply	other threads:[~2022-01-11 21:30 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-11 11:50 Ilia K
2022-01-11 17:55 ` Markus Wichmann
2022-01-11 21:30   ` Harmen Stoppels [this message]
2022-01-11 22:22     ` Rich Felker
2022-01-12 14:52       ` Harmen Stoppels
2022-01-12 16:58         ` Farid Zakaria

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='D8YsUB86oW8DejmzNJUGU-fcHh7ogVUXNvhGD7HB-ayqu2jdEPf_HbaiqNkRaiVbWNpYXMXHCiiqUFXd4cS2J2foq4xvd-jyrOTtGUHpXDY=@harmenstoppels.nl' \
    --to=me@harmenstoppels.nl \
    --cc=musl@lists.openwall.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).