From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 27295 invoked from network); 11 Jan 2022 21:30:26 -0000 Received: from mother.openwall.net (195.42.179.200) by inbox.vuxu.org with ESMTPUTF8; 11 Jan 2022 21:30:26 -0000 Received: (qmail 31757 invoked by uid 550); 11 Jan 2022 21:30:24 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 30701 invoked from network); 11 Jan 2022 21:30:23 -0000 Date: Tue, 11 Jan 2022 21:30:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=harmenstoppels.nl; s=protonmail; t=1641936611; bh=x0YjKUeCQvhDuq5VxD2NX49rZ7v2p9k9GTDz4MVuBWE=; h=Date:To:From:Reply-To:Subject:Message-ID:In-Reply-To:References: From:To:Cc; b=WupHN+2ozK+3AG/xEkcXnG91FHt6b6d9WTDPTq6ICgZbmR5pj8dOZzOAq+vMtmVbB j3hHuWSe6yJaUMutsLuE0zl6+CyWMp0KMCxuf0+BnC3nSVke6b1YNYyuEmCLWcD4ES 2xYYa/73pYQT5dV+yhMzptCHQX9Ji9ZmFZkRDi6g= To: musl@lists.openwall.com From: Harmen Stoppels Message-ID: In-Reply-To: <20220111175541.GB2046@voyager> References: <20220111175541.GB2046@voyager> MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg=pgp-sha256; boundary="------7b0821854570ddd4a46edb326d436edd62cbcaf83108aef3014993f0e7832110"; charset=utf-8 Subject: Re: [musl] Cannot dlopen() an already loaded shared library by its SONAME name This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------7b0821854570ddd4a46edb326d436edd62cbcaf83108aef3014993f0e7832110 Content-Type: multipart/mixed;boundary=---------------------d3c686bec94b0acb78024982b3fed9a5 -----------------------d3c686bec94b0acb78024982b3fed9a5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain;charset=utf-8 > Given that this question already surfaced twice and was denied just as > = > often, I'm guessing no. The fact that this is a common request could also imply it's behavior people expect. So far I've seen: 1. Julia [1] splits binary dependencies into separate packages, so when liba.so depends on libb.so, they live in a different dir, where the absolute and relative paths are only known when the julia interpreter has started, so neither rpaths or LD_LIBRARY_PATH can be used. So they dlopen libb.so, and then dlopen liba.so in that order, and then assume liba.so does not have to locate libb.so again, because its soname is already seens before. The proposed workaround was: don't list libb.so in the DT_NEEDED of liba.so (that is, if you're already doing the work of the linker, you might as well not use the linker at all for locating libs). However, being able to run executables shipped with julia packages would still be nice (e.g. a subprocess with LD_LIBRARY_PATH set properly) 2. The Nix / Guix / Spack people are trying to reduce startup time of executables with many shared libraries (as well as fixing library paths once and for all to keep executables run deterministically). In Guix there's a blog post where they call this the "stat storm" [2], and they solve it in a glibc patch: using context dependent ld.so.cache= , that is, a reverse mapping soname =3D> library path. In Nix the proposal to fix the "stat storm" is to replace DT_NEEDED in executables with absolute paths of all required libs (also transient ones). This works fine, except on musl, where a dlopen by soname will still do a search. 3. Wine is reported to rely on this earlier in the mailing list, but that did not get any responses [3]. It reports the behavior is the same on glibc, FreeBSD, NetBSD, and OpenBSD dynamic loaders; and musl is the exception. = [1] https://github.com/JuliaLang/julia/issues/40556 [2] https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-= cache/ [3] https://www.openwall.com/lists/musl/2021/12/16/1 > The reason for the behavior is that loading a library with explicit path > = > name is what you do with a plugin, and you don't necessarily want a > = > plugin's symbols to be visible to everyone. That's why a library loaded > = > by path name does not get a shortname set, and a shortname is what you > = > need to be able to find a library by just its name. To me it seems very uncommon that two libraries with the same soname are n= ot supposed to be the same library. -----------------------d3c686bec94b0acb78024982b3fed9a5 Content-Type: application/pgp-keys; filename="publickey - me@harmenstoppels.nl - 0xFD537C88.asc"; name="publickey - me@harmenstoppels.nl - 0xFD537C88.asc" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="publickey - me@harmenstoppels.nl - 0xFD537C88.asc"; name="publickey - me@harmenstoppels.nl - 0xFD537C88.asc" LS0tLS1CRUdJTiBQR1AgUFVCTElDIEtFWSBCTE9DSy0tLS0tDQpWZXJzaW9uOiBPcGVuUEdQLmpz IHY0LjEwLjEwDQpDb21tZW50OiBodHRwczovL29wZW5wZ3Bqcy5vcmcNCg0KeHNCTkJGblJhMUlC Q0FET3JaRXJVVGNDakphbTNkekcrUzBhdGNkS2FGcDJ0cm9HNXRoS3RaaUt1b0dEDQpUZllpT1VF RER1d3VGWS8xYkF2Zi9YeWYxcVhlNXI2ZFM0Vk1QYlhvaWptNUxaZmNmdTNlV0tzKytnVzMNCnY3 SDQxTlVsNEVramNqVzZ6NURZeUpGQnRqaEVWYzh3dk1jV0ZDcnA1ZjRyeXFtaGk1MlVZMnFvS0Ry VQ0KSmQ2bDc4c256a0diUmw1VzNJVG5FTHdNN1dTV1R5NG44ZWl0ZEQ2Q2xEVk94akR4ZUhBd1dX b3pNb3R6DQpHZ0N3ZWJaaDFXaG1VcjQ1UkM0RC9DSk0yZHBrc3Ezb241cHBIRTR4Vk5FUmkvWk9V dmxDTUZvLzA4WDUNCkRPamZOaHc1OWRkUnJQc2xTbExRTS91bFpNV0FKckcwaDJTQjd3djk4ODFD bnZ1MVdUM28xSTV0QUJFQg0KQUFITksyMWxRR2hoY20xbGJuTjBiM0J3Wld4ekxtNXNJRHh0WlVC b1lYSnRaVzV6ZEc5d2NHVnNjeTV1DQpiRDdDd0g4RUVBRUlBQ2tGQWxuUmExUUdDd2tIQ0FNQ0NS Q05IT0Z2WmZtRWdnUVZDQW9DQXhZQ0FRSVoNCkFRSWJBd0llQVFBS0NSQ05IT0Z2WmZtRWduZTZD QUMzdjBzNUdiSmhYRFJmQWJrV1ZMSUNSb2hFYlZSbw0KMjZBZUErL1hMeUpmYU1lOGFaQUxGUFdp bktVU3B0RjU3YlhCdCtUYnZ0QmZyOHdaTURkTkcycjRTQTd3DQpOcVp0L0R2SzZDbGU3R3JuZmVE WXY2OWpPVCt5cnhoOERzUjNXenpBdllXUVdVTUxEVUQ2OUNIQkgyNWQNCjNsQVViUU5aODBDT1dh UmI4MWtVbVNSY3d6M25vV08vK3Q5L0l1TkxHdG9PbnBqaCthZW1nNUFtWjlYNQ0KdE54UGZxNzhD N3ptQ1c0Ynp1RXZBczc3S09ld2dkT0VaYWs5QldMMWlBSXJLWk5LU3NXN3ZXSTFSMXY1DQptTUZu STY1OFVadW1hRjVJaThLNmxTTStNRDZrVCt4aEFkRDNoTmJGbTd2cDdtZ1pGMHpadmR1c0FMYUoN Ci9Gam9GUHpBa3BkQ3pzQk5CRm5SYTFJQkNBRFhEQ0RmY2FXTndGRnA3K01LNExVWVM5RkxUNG4r akpFdw0KM0o2c2VVY2JDM25MMjJwQ3lXL0xkN1RXOXlsTDlpczgzbExVOGwwbVJvc1lnaG9aS1kx akREUk1Da1kvDQprKyt6S3h1QkpmSHdCZktjZm9nNWZCdTgvaGNSbVlBUUgrZ2ZnK0Z1T3JNeHBV OTQ2bGxBWnVCQnlJeHUNCkM0WnArSVJOOUxuWjFwVE9TRWo2ZlhZc2xXNmxGcmpFbXcxUWwxTFgw TUI5d2lHWGs3aHhSTEJpUzluaQ0KRGtRMFFtU0YyOE82R3V0ZCt6VlJhcmFJeTJQeFovVjQ0cjV6 NVI0NU0wUHlRdVQxVTN1QVd6NitqUEMzDQpPY0FrTlBTL0xDWGR2TXpOa3FBeFJCYnVxdlR3QmJN aE5hNmR0L0V5ZStjQXpYQUs2YkFOeFlqL2s2TWYNCnduSmh1WWVqQUJFQkFBSEN3R2tFR0FFSUFC TUZBbG5SYTFVSkVJMGM0VzlsK1lTQ0Foc01BQW9KRUkwYw0KNFc5bCtZU0NPd0lJQUpQYTdCN25s N1pGY0xiRXVaOU1ZeGNpaCthU0U5Qkt2S3VZaFFYajdkdllEUEFaDQp6MUI4bEJNRUZsNy9PS3hM OUpzQ0QwcngvZWFRaldBTUhFQTFJUlhDSzRrYUppVk54SU9vQVRzeVpLVXINCnE4S2s5Z3pwZWtM WXl5T08zUlRYUkxYUW1EMHp1YjRsS01GdmpvRzBTbGtMNUFubVBtYXZiVXhqcnorMg0KUUhwZFB1 QTZ6VndXME5yMXZ2V1RVdmtpYkhUUktwUTlzdEtsV3JTOWpZaTU0MDVJaXR2c2N0MEQ5Y2ZwDQpp ZVdJckk0aHd3NjZQQmJ2OE1MQWNKZi9pamVPWURQY0FxYzAzMTZOVkhBSTdpYm5MVWNnYUhFNHAx VU4NCmZEVzEzd29vRWR1bktkbkE5c1RuTHhtZmwwWnJJc0FHenRTa0RDT3NlWDg1ZHViVHBvc1M2 MFk9DQo9T0x1ZQ0KLS0tLS1FTkQgUEdQIFBVQkxJQyBLRVkgQkxPQ0stLS0tLQ0K -----------------------d3c686bec94b0acb78024982b3fed9a5-- --------7b0821854570ddd4a46edb326d436edd62cbcaf83108aef3014993f0e7832110 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: ProtonMail wsBzBAEBCAAGBQJh3fbVACEJEI0c4W9l+YSCFiEE/VN8iJjfG7NRPUUojRzh b2X5hII4+Af9EDYBxYQrRQRjI+Oj2Uu9gTeDf6ncH3ExjRz48DJ/T5RjfK5Z e6075Oe25r+9jh63tR4FdGgmxz1GGojQiDE8Zxe0j27KntFJ7MBlRBHU/PcG jgZ6EMHcjogtH/BjRg0pN0PVyZlIxq2Zu7NfPiHNudNp3lJ3UG1RtSrxiHyA badvrpsZTBAEMVlMuLGmlloVKAxyc08dOoFxLbaA6XGYSmurUhhafDmxX065 RR1V9JtNv8dzL61R90n/jWCeAo/kg3j+CLLvXCdHbDusXhbFn8ZE5F9KFmle mS7VSQpcCLyug6o6iJ+juYgVJYUKYs8UM81iS7O+maH4o+tsyp0B4g== =raGa -----END PGP SIGNATURE----- --------7b0821854570ddd4a46edb326d436edd62cbcaf83108aef3014993f0e7832110--