From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,T_SCC_BODY_TEXT_LINE,URIBL_BLACK autolearn=no autolearn_force=no version=3.4.4 Received: (qmail 10024 invoked from network); 20 Aug 2022 09:43:24 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 20 Aug 2022 09:43:24 -0000 Received: (qmail 19993 invoked by uid 550); 20 Aug 2022 09:43:22 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 19958 invoked from network); 20 Aug 2022 09:43:21 -0000 Date: Sat, 20 Aug 2022 11:43:08 +0200 From: Szabolcs Nagy To: Colin Cross Cc: musl@lists.openwall.com, Ryan Prichard Message-ID: <20220820094308.GK1320090@port70.net> Mail-Followup-To: Colin Cross , musl@lists.openwall.com, Ryan Prichard References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: Subject: Re: [musl] Running musl executables without a preinstalled dynamic linker * Colin Cross [2022-08-15 14:35:33 -0700]: > I would like to distribute dynamic binaries built against musl to > systems that do not have the musl dynamic linker installed in any > known location (e.g. /lib/ld-musl-$ARCH.so.1). I have two prototypes > that enable this, and I=E2=80=99d like to gauge whether either is somethi= ng > that would be of interest to check in to musl, or whether it would be > something we should keep in our project. >=20 > The first solution is based on the embedded linker we use to test > bionic libc on non-Android systems. The dynamic linker is compiled as > usual, then the resulting elf file is embedded as raw data into > Scrt1.o and the PT_INTERP section removed. The entry point is changed > to point to a trampoline that modifies AT_BASE, AT_ENTRY and AT_PHDR > to simulate how the kernel would initialize them if the dynamic linker > was mapped separately by the kernel instead of as part of the main > executable, and then jumps to the dynamic linker. >=20 > This embedded linker solution works relatively well, except that the > dynamic linker=E2=80=99s elf sections are inside the main executable=E2= =80=99s elf > sections, which can break reasonable assumptions. For example, musl=E2= =80=99s > dladdr fails to find symbols in the embedded linker, and gdb has > trouble finding debug information from the linker. Musl=E2=80=99s reuse = of > libc.so as the linker means that these problems apply to everything in > libc.so, and also increases the size of every binary by including all > of libc.so. >=20 > These problems with the embedded linker could be somewhat mitigated by > splitting the dynamic linker out of libc.so when using the embedded > linker. That requires compiling the ldso sources against a statically > linked libc.a, tweaking some of the initialization, and forwarding the > dl* calls from libc.so to the separate linker. The changes are > relatively small, but result in a pretty big difference in musl=E2=80=99s > internals with and without the embedded linker that may be hard to > maintain. >=20 that breaks atomic update of the libc and introduces libc internal abi. (i.e. bad for long term security and maintainability) > The second solution we call =E2=80=9Crelinterp=E2=80=9D. It was original= ly designed > by Ryan Prichard as a standalone trampoline that could be used with > musl, glibc or bionic, but I=E2=80=99ve more tightly integrated it with m= usl > in order to reuse CRTJMP for architecture portability and some of > musl=E2=80=99s string functions to reduce the size of the code. It uses a > similar trampoline in Scrt1.o, but with a much larger implementation > that reads DT_RUNPATH to construct a path to the dynamic linker that > is relative to the executable. It then maps the dynamic linker as the > kernel would, modifies AT_BASE, AT_ENTRY and AT_PHDR, and jumps to the > dynamic linker. >=20 i think this is a better approach. i would not use Scrt1.o though, the same toolchain should be usable for normal linking and relinterp linking, just use a different name like Xcrt1.o. > The current prototype of relinterp is tricky to compile, as it > requires using -fvisibility=3Dhidden and ld -r partial linking to build > a Scrt1.o file that uses some of the src/string/*.c sources without > any relocations, and then objcopy =E2=80=93keep-global-symbol to hide the > string symbols. It=E2=80=99s only useful if DT_RUNPATH contains $ORIGIN = so > that the dynamic linker can be distributed alongside the executable, > so it is probably never going to be suitable for setuid binaries. >=20 > If relinterp were going to be included with musl I=E2=80=99d refactor it = to > reuse the __dls* bootstrapping from dynlink.c so that it can link > against libc.a and not worry about avoiding any relocations. >=20 i would make Xcrt1.o self-contained and size optimized: it only runs at start up, this is a different requirement from the -O3 build of normal string functions. and then there is no dependency on libc internals (which may have various instrumentations that does not work in Xcrt1.o). > An alternative solution to these two would be to distribute statically > linked binaries, which precludes the use of dlopen, or to wrap every > executable in a shell script that runs the dynamic linker directly. >=20 i think it is possible to support static linking such that if dlopen is linked then the entire libc gets linked into the main exe with libc apis exported. then dlopen can work from an otherwise static exe. (may not be easy to implement in practice though) > Do either of these prototypes seem interesting enough to clean up and > post as upstream patches, or should I keep them as a side project that > I can bolt on to musl with minimal invasive changes? >=20 > Colin