mailing list of musl libc
 help / color / mirror / code / Atom feed
* [musl] Satisfying DT_NEEDED from previous dlopens with explicit path
       [not found] <1639611583.3fo4bm7va3.none.ref@localhost>
@ 2021-12-16  1:02 ` Alex Xu (Hello71)
  0 siblings, 0 replies; only message in thread
From: Alex Xu (Hello71) @ 2021-12-16  1:02 UTC (permalink / raw)
  To: musl

Hi,

Recently, Wine made a change that resulted in launching of any program 
to fail on musl. Specifically, it stopped ignoring the failure of dlsym 
on a NULL pointer which was returned from dlopen because library 
ws2_32.so could not be loaded because it has DT_NEEDED on ntdll.so which 
is not located in the system library paths but was previously 
successfully dlopened with an explicit path.

Currently, glibc, FreeBSD, NetBSD, and OpenBSD dynamic loaders allow 
this to succeed by satisfying DT_NEEDED dependencies if a previously 
dlopened library has the required DT_SONAME. I tested glibc and FreeBSD 
manually and inferred behavior from the NetBSD and OpenBSD sources.

FreeBSD: https://github.com/freebsd/freebsd-src/commit/0eb88f20298d056bf09b52ec2d84d3662b8fd152
NetBSD: https://github.com/NetBSD/src/commit/57fae3de47d379ffe8e38009a68e0b511e17cd5a
OpenBSD: https://github.com/openbsd/src/commit/0af06aaf0d91e6672e6f5dba4afd8f02e314f164

Windows requires all DLL names to be unique across the process. 
Using LoadLibrary with an explicit path will return a handle to an 
existing loaded library with the same filename if one exists. 
https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order: 
"If a DLL with the same module name is already loaded in memory, the 
system checks only for redirection and a manifest before resolving to 
the loaded DLL, no matter which directory it is in. The system does not 
search for the DLL."

Mac OS X/macOS uses "install_name" instead of DT_SONAME. This is 
normally a full path to the library, but can also use the placeholder 
@rpath. I believe this refers to the path by which it was found. Using 
this mechanism, plugins with the same name as a system library can be 
used as long as RTLD_LOCAL is specified in dlopen flags.

Haiku appears to satisfy DT_NEEDED dependencies based on the DT_SONAMEs 
of existing loaded libraries, and the original basenames for libraries 
which didn't contain DT_SONAME.

The behavior of satisfying DT_NEEDED using DT_SONAME is known to be used 
by Julia, OpenJDK, and Wine. Julia is currently broken on musl 
(https://github.com/JuliaLang/julia/issues/40556), OpenJDK works around 
it using LD_LIBRARY_PATH, and I believe Wine was always broken but the 
issue was silently ignored for most programs.

As I see it, the following options are available (or not):

0. musl cannot implement install_name because ELF doesn't have it. The 
   Windows behavior is bad and should not be implemented.
1. musl can implement DT_SONAME search, matching other ELF loaders.
2. Some applications may be able to use DT_RPATH or DT_RUNPATH with 
   $ORIGIN or $ORIGIN-relative paths. However, my understanding is that 
   Julia cannot do so, as the relative paths contain a value which is 
   not known until runtime.
3. The applications can use LD_LIBRARY_PATH. This doesn't work for the 
   same reason.
4. The applications could forgo DT_NEEDED entirely. The main downside of 
   this is that it would no longer be possible to use --no-undefined/-z 
   defs to diagnose missing symbols at compile time. It would also not 
   be possible to detect missing dependencies at runtime if they consist 
   entirely of weak symbols, but this seems unlikely.

Considering these factors, I believe musl should implement DT_SONAME 
search. A patch to implement it is available at 
https://github.com/JuliaLang/julia/issues/40556#issuecomment-825126303, 
but I think it would be simpler and use slightly less memory to re-use 
the existing shortname member which is currently unused for libraries 
loaded by explicit path.

Thanks,
Alex.

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2021-12-16  1:02 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <1639611583.3fo4bm7va3.none.ref@localhost>
2021-12-16  1:02 ` [musl] Satisfying DT_NEEDED from previous dlopens with explicit path Alex Xu (Hello71)

Code repositories for project(s) associated with this inbox:

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).