From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 20493 invoked from network); 16 Dec 2021 01:02:51 -0000 Received: from mother.openwall.net (195.42.179.200) by inbox.vuxu.org with ESMTPUTF8; 16 Dec 2021 01:02:51 -0000 Received: (qmail 23857 invoked by uid 550); 16 Dec 2021 01:02:48 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 23822 invoked from network); 16 Dec 2021 01:02:48 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.ca; s=s2048; t=1639616554; bh=ldszT8nzjO+J51cU6rcIkhyaGdHYAPA7Y15UscLVFF8=; h=Date:From:Subject:To:References:From:Subject:Reply-To; b=fh6U2kP/CVcBIsUOGhC8JpnDXd4GYbwTPdbxxmKAJN6lmTNw+8Eda4CGo/4Q2kMdrto8d7Eu2AUV4ZrmivvimDt3WzcMl4fWcLkqV/E26tduxeIZ3BQek5ZStHjJCfFEPVAtSYQpQsHDoIM7FoM5kA0cryt2ZWmOIgY+mvmN0DbYYaCjK1cIuHoca0+KFAbUIUtF18pzSRFyGJWlp337Mg1sMPjG2kym8XdA7A/CwaNyJmye9igckP6jjhKOF0oLUxYndAmubh+tXyVTI4D6h5hwzdVXknYPxEKDS2M3mjD6RIByBPNU+h+PLi9j00iMZGQqG6nCBIHMcyKYBi+epQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1639616554; bh=DS43RaI9qM81+brstkLP9NQUrEotUgvuKYJ+QSZFVBl=; h=X-Sonic-MF:Date:From:Subject:To:From:Subject; b=tmvnro7Uu07Afgnm6hsmnEcU8u7oHBOpzGB+CTHfF03SyG0mLmooGcOu6UFbxFtD/XdMFNvnCgo/f4TTcBwIzfj+zroDfLIl0vs0WB/2Ea77Lod0X5IvJnSDl0OBCu8qkduB5IPHr8aTPuBkQl+JzUtkW/nMoC/D5PMEq8DMw4QQ84uWMb2YBiTRnoItpu8qVC7JHzFX11bBY/FpYMUrLBpxTf8nOL4S9aK1SDzJj52N6i0rMIvIgKasTs016zxx4CdiLEF5j5mHX+vSM97e7zYCR1Ij6c7DPTq8bdaT61Tq32Y/fwFSyvpkut7e3+R+4jyN9xs1fmiurFhZHdFpRA== X-YMail-OSG: t6WKHeoVM1m_adcv0HEFyE84OuDYTjen74sr3_GGsJccY5T0xLkaz65b09vfH6g M4Cezvk8AakdxIBD6WqY7p2XJhaNbzYRKCq1jDXxTPlmS451wB5KpFiv0Ug2l00Q1gFQL.iL.azQ 9QwZcomAN2zFrsdFveeP_a3aftSVuDV0MlRQYIG0oddOdLI.DpKzuH3M_tq96pPcEHIb5hIGGwrP AT6Bi1njaioDQlQl7z._qTc13v.1dVcCsHn6j93ZpkQgWPq1YYx3qjkxaiZsWAnGA4LRVAOoWZgH _g6wc3MmUc1vOcOwWC4mJu8Awix5zARHtj8PCfUz91MPPSC.czzXsa1bB8BPc_2QOE43sdIIJBaO Z.9PugcavGfT9XfjTw4IjnPFmfBFmrZpsDqkXI5TTH6kiboNlRtpBMTswS7OEkrY3Ko.P2H8jR9J qXUrA0KLgkNusuZ58as0QJRknuCyIx6Gh2ZEjk5aTrooQPQprWkb3v.XU3xnkGB3e1lQ2SO5KiPL HIAnmjL_v.680gf1fCzaYaxh6TO9D6hHGSVy_QmMf0SJ64r1IayUUNff_9mjax4AHDwMvdMlyEZl G2ruuCSIPkfLSewAc7zhudP3iuKJKWQTfP6cBLFXrB_et_GQURadTQG4Sfvs7Vw6Zkn.zavY6QWo KrHYtuLBEW1yLxKFWg4bqbzXsjumpWn0nZHIDeK8UPEecLDVNaCDgzuoNj8jfvnGVCQZGnblM2xo VXRM6vFznYvE05F81BuNDIibmkjGgmFHBWxS7UdsOjVZb116rgZVEtS1g6SAL.sYPvzNqFfGetji 3jycfn2o4QiP0rJ1l4q9ZUdf5T2ltWDQsKQxq4oEWOJkG.XKoi7qOPZ4brWM3bxlZOD4whYz_xXU rVb7l8fkT2pK0MQhaPMM__.p.TkM2Cdcla8fcsbtak6bou0W1T67xOnBuPUAGuVbfD0vUDa_sln2 7SxS_6_9CEysZ6TTnr3D9bWBfSYQ5xxcRjWoxXjQfGfwQ6pRoYe.6EouFX4T2i3zZ0bANkmrJSr3 iNJL4elF8uNK0NOBYq4g5IE.L7XMm67jkU5OAEHfFuGjebm4tP.NxMkF_8FdTGWr1YmmtWxKDQe1 C1e3OdUlsxPqAdQtbmrnUsKrDs77.plsqbJ1JWkxsLsR_i_64Hh0bt6QU8YuclPOGQwYtoXuT421 WL8TcjI7P4q27BA7pZWMe8qOoXVyX.j7Qie1PTSzHERkWupRomWlDpiqIJ.itcVZcwbKzSj_QF04 Zn1WIZDLiWW2AnRJ9lkTmz.S2D9uXzwikvWbKXvuxBIpjHDpefm0lJSMvhxZqSZyG0zL5lRfO1V7 CrAgl_XOhoel1Wtgdnzik3b.cdcnQHIJyjCLDGjspjxurSN0wao0vvyEfvsmSoQHTtPKJkRB4fYR zzc_gems5ydREUtFP6Wn3hZPudlRr2bv89VRRgKSMXLgczZ2tAwC5nP3rwN2lV66ilMaNS_0y0oj w0lEcR4krkPtNg2ylgcg7NpzKB6LkOBmwzb0Xp_oK6guB8mqkMBrafPiekDWoUrKW7_.xO5OBbWI QW.o3rdNZzzvNxX7ZM3wNvbfQ.u_Kz0H.kUm_37SwEM7m15Q9q1o.1XJ_1CPpFwpFlRae0w3QnMG TVQIXdK5D8MpvMERyORG.6MOM0NFD7ZMRhsJ9yVi99ESYIr3z4iMVMJsyFtWW3FRoVcsvlx7Idw5 Xq926.SGzOTb1_0kCma19g660fgDeziUn1zADem6QjX9oD99ozBAW6pWsIihx_3LEipzwsnXkkec sS8b5xHkhLTHMMya51ze2FFFduZUdYTBpr_ejtKElQMjZpD_bpNYJHDIToJT5N3HfzcDgKP.IIYj JchrNt1LzpFrY4X1zVpNHhBxUlEE4CfXA0BdSoCg19mkUIx9NiZEXozTdZIrZyWARCsU3j36ofdZ iASpwGPyr9y8mMRKQE4_3vByqmBxGd01Q4fMSDhqBZvuaHA2RBDZEVctN5252P.Mef5wg8R4yhs. CwbHopvWQ4WmuoPQqYdqociJLRetkNpI9ReIrWcYr21z4m8duI0Uj89ff6bch3jcGnY8.ZDX6dzs Y9GBPmEJDt0n0Ssupa1Jre_q2f.ULR_7np0UmimIPzd41All5D1BTBx8oUpf9i5cWe4vV9EnIqWv au4jmBMwGQdq6MyHlZn8pjyUDnGg4CPt_aF9LJ7h0zikXAfe2tP_W6yIYoS43F4B0._XNwyVbxhU lUOSNIE5TXr3XY4t7ljMSaJs.bd2U31rkvoLa9Aaq79VrrffdBQ-- X-Sonic-MF: Date: Wed, 15 Dec 2021 20:02:29 -0500 From: "Alex Xu (Hello71)" To: musl@lists.openwall.com MIME-Version: 1.0 Message-Id: <1639611583.3fo4bm7va3.none@localhost> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable References: <1639611583.3fo4bm7va3.none.ref@localhost> X-Mailer: WebService/1.1.19415 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo Subject: [musl] Satisfying DT_NEEDED from previous dlopens with explicit path Hi, Recently, Wine made a change that resulted in launching of any program=20 to fail on musl. Specifically, it stopped ignoring the failure of dlsym=20 on a NULL pointer which was returned from dlopen because library=20 ws2_32.so could not be loaded because it has DT_NEEDED on ntdll.so which=20 is not located in the system library paths but was previously=20 successfully dlopened with an explicit path. Currently, glibc, FreeBSD, NetBSD, and OpenBSD dynamic loaders allow=20 this to succeed by satisfying DT_NEEDED dependencies if a previously=20 dlopened library has the required DT_SONAME. I tested glibc and FreeBSD=20 manually and inferred behavior from the NetBSD and OpenBSD sources. FreeBSD: https://github.com/freebsd/freebsd-src/commit/0eb88f20298d056bf09b= 52ec2d84d3662b8fd152 NetBSD: https://github.com/NetBSD/src/commit/57fae3de47d379ffe8e38009a68e0b= 511e17cd5a OpenBSD: https://github.com/openbsd/src/commit/0af06aaf0d91e6672e6f5dba4afd= 8f02e314f164 Windows requires all DLL names to be unique across the process.=20 Using LoadLibrary with an explicit path will return a handle to an=20 existing loaded library with the same filename if one exists.=20 https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-se= arch-order:=20 "If a DLL with the same module name is already loaded in memory, the=20 system checks only for redirection and a manifest before resolving to=20 the loaded DLL, no matter which directory it is in. The system does not=20 search for the DLL." Mac OS X/macOS uses "install_name" instead of DT_SONAME. This is=20 normally a full path to the library, but can also use the placeholder=20 @rpath. I believe this refers to the path by which it was found. Using=20 this mechanism, plugins with the same name as a system library can be=20 used as long as RTLD_LOCAL is specified in dlopen flags. Haiku appears to satisfy DT_NEEDED dependencies based on the DT_SONAMEs=20 of existing loaded libraries, and the original basenames for libraries=20 which didn't contain DT_SONAME. The behavior of satisfying DT_NEEDED using DT_SONAME is known to be used=20 by Julia, OpenJDK, and Wine. Julia is currently broken on musl=20 (https://github.com/JuliaLang/julia/issues/40556), OpenJDK works around=20 it using LD_LIBRARY_PATH, and I believe Wine was always broken but the=20 issue was silently ignored for most programs. As I see it, the following options are available (or not): 0. musl cannot implement install_name because ELF doesn't have it. The=20 Windows behavior is bad and should not be implemented. 1. musl can implement DT_SONAME search, matching other ELF loaders. 2. Some applications may be able to use DT_RPATH or DT_RUNPATH with=20 $ORIGIN or $ORIGIN-relative paths. However, my understanding is that=20 Julia cannot do so, as the relative paths contain a value which is=20 not known until runtime. 3. The applications can use LD_LIBRARY_PATH. This doesn't work for the=20 same reason. 4. The applications could forgo DT_NEEDED entirely. The main downside of=20 this is that it would no longer be possible to use --no-undefined/-z=20 defs to diagnose missing symbols at compile time. It would also not=20 be possible to detect missing dependencies at runtime if they consist=20 entirely of weak symbols, but this seems unlikely. Considering these factors, I believe musl should implement DT_SONAME=20 search. A patch to implement it is available at=20 https://github.com/JuliaLang/julia/issues/40556#issuecomment-825126303,=20 but I think it would be simpler and use slightly less memory to re-use=20 the existing shortname member which is currently unused for libraries=20 loaded by explicit path. Thanks, Alex.