From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, URIBL_BLACK autolearn=no autolearn_force=no version=3.4.4 Received: (qmail 18746 invoked from network); 27 Sep 2022 04:33:52 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 27 Sep 2022 04:33:52 -0000 Received: (qmail 28236 invoked by uid 550); 27 Sep 2022 04:33:49 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 28192 invoked from network); 27 Sep 2022 04:33:48 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=android.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=LIp02TsiNvcfISWMfgeURZ17Bc8eWOGDna35nzUZL00=; b=u+ry9g3daTg9cH+XbGvEAwMPmAKfT6ZdWPhHeP6NGLB8T+7aRpgjvXsfvpDypAopM+ vtpIcdWXm1oSkWYBOIB7drl9EP4GmwspAbQysg2Utv4AayO4oBT9r9xgi3dluRyh3pzF pt+oB5P3S7ymfh4RItdQd9ZBLPtfeovtnQRKL15kFE0j+yNyaR10cCmw61OLx7SJTipy mE0etoEfMS/Hj9xs8/XMxgHe5kYWSBk0kcAY9bZyDbITTT9Wu3Mufxl/maHbO+jPG4IM hg1XLlRJ6nUww2F88uL4T/PqWfknZKA0eX1xEq+d4iHAQab1iRFsK2TWYywN4pY+K20V 0MQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=LIp02TsiNvcfISWMfgeURZ17Bc8eWOGDna35nzUZL00=; b=sfWZXX9yG26us8mNCFTePd9JqcNN2aQ53SoPA6hLnvu2RpUsslbz1gcf0N6tQN2CKj G3BU1UnFRS0znyRmbjPImAsW859EG/QCAnnMtjRu1iO/UPwGOADXT9TZMhV2QgTrhmz9 2A4bTputehelh/oVO4nuzZgeQtelBsx3s9tLwm+mqm7eDt0Wptkqe2zN3nrVlF7R7oas CZMNYnfgwbt/gJPOSCFGs3Exv8M9PSGwV6MSUzuLiVWU6hCpcJLXlepF4Iq+q1aDSnVy aNc4ZN7hnL0R71mxHaFxvbXU6NjmYEoWdATPaC9oV9zSEdySip4DZnkk1C+D77i79wo3 oFHg== X-Gm-Message-State: ACrzQf2jHRDNKAjVDagQ4dwKHLdrqyE2ZpaTh1xT8moozIXvpPE1aoA3 +KKFVcvM7ZPbovcNz/YHThK+zk5g3XoCg6+mnPVjMQ== X-Google-Smtp-Source: AMsMyM7+K6OcF/u11STdwBYnc9RUdxnJ5GMmSw+NNO2vYRgtJnx+TiAVCxQ1FqHNpRK8fT1VeaEMA6/5myuSDsiv7aE= X-Received: by 2002:a92:3652:0:b0:2df:4133:787 with SMTP id d18-20020a923652000000b002df41330787mr11696171ilf.39.1664253215920; Mon, 26 Sep 2022 21:33:35 -0700 (PDT) MIME-Version: 1.0 References: <20220820094308.GK1320090@port70.net> <20220823081802.GM1320090@port70.net> <20220926230241.GF9709@brightrain.aerifal.cx> In-Reply-To: From: Colin Cross Date: Mon, 26 Sep 2022 21:33:22 -0700 Message-ID: To: Rich Felker Cc: musl@lists.openwall.com, Ryan Prichard , Rodger Combs Content-Type: multipart/mixed; boundary="000000000000c0c9d605e9a12580" Subject: Re: [musl] Running musl executables without a preinstalled dynamic linker --000000000000c0c9d605e9a12580 Content-Type: text/plain; charset="UTF-8" On Mon, Sep 26, 2022 at 4:12 PM Colin Cross wrote: > > On Mon, Sep 26, 2022 at 4:02 PM Rich Felker wrote: > > > > On Mon, Sep 26, 2022 at 03:42:01PM -0700, Colin Cross wrote: > > > On Mon, Sep 26, 2022 at 3:38 PM Colin Cross wrote: > > > > > > > > On Tue, Aug 23, 2022 at 1:18 AM Szabolcs Nagy wrote: > > > > > > > > > > * Colin Cross [2022-08-22 17:22:06 -0700]: > > > > > > On Sat, Aug 20, 2022 at 2:43 AM Szabolcs Nagy wrote: > > > > > > > i would not use Scrt1.o though, the same toolchain should be > > > > > > > usable for normal linking and relinterp linking, just use a > > > > > > > different name like Xcrt1.o. > > > > > > > > > > > > Is there some way to get gcc/clang to use Xcrt1.o without using > > > > > > -nostdlib and passing all the crtbegin/end objects manually? > > > > > > > > > > this requires compiler changes (new cmdline flag) but then i think > > > > > the code is upstreamable. > > > > > > > > I've used relinterp.o for now, and selected instead of Scrt1.o in > > > > musl-gcc.specs and ld.musl-clang. > > > > > > > > > > > > > > > > i would make Xcrt1.o self-contained and size optimized: it only > > > > > > > runs at start up, this is a different requirement from the -O3 > > > > > > > build of normal string functions. and then there is no dependency > > > > > > > on libc internals (which may have various instrumentations that > > > > > > > does not work in Xcrt1.o). > > > > > > > > > > > > Doesn't this same logic apply to most of the code in dynlink.c? My > > > > > > main worry with a self contained implementation is that it requires > > > > > > reimplementations of various string functions that are easy to get > > > > > > wrong. The current prototype reuses the C versions of musl's string > > > > > > functions, but implements its own syscall wrappers to avoid > > > > > > interactions with musl internals like errno. > > > > > > > > > > dynlink is in libc.so so it can use code from there. > > > > > > > > > > but moving libc code into the executable has different constraints. > > > > > so you will have to make random decisions that string functions are > > > > > in but errno is out, wrt which libc internal makes sense in the exe. > > > > > > > > > > i would just keep a separate implementation (or at least compile > > > > > the code separately). string functions are easy to implement if > > > > > you dont try to optimize them imo. then you have full control over > > > > > what is going on in the exe entry code. > > > > > > > > I left the reimplementations of string functions and syscalls as > > > > suggested. Patch attached. > > > > > From 0df460188b95f79272003bd0e5c12bceb2a3c25f Mon Sep 17 00:00:00 2001 > > > From: Colin Cross > > > Date: Thu, 22 Sep 2022 19:14:01 -0700 > > > Subject: [PATCH] Add entry point to find dynamic loader relative to the > > > executable > > > > > > Distributing binaries built against musl to systems that don't already > > > have musl is problematic due to the hardcoded absolute path to the > > > dynamic loader (e.g. /lib/ld-musl-$ARCH.so.1) in the PT_INTERP header. > > > This patch adds a feature to avoid the problem by leaving out PT_INTERP > > > and replacing Scrt1.o with an entry point that can find the dynamic > > > loader using DT_RUNPATH or LD_LIBRARY_PATH. > > > > > > The entry point is in crt/relinterp.c. It uses auxval to get the > > > program headers and find the load address of the binary, then > > > searches LD_LIBRARY_PATH or DT_RUNPATH for the dynamic loader. > > > Once found, it mmaps the loader similar to the way the kernel > > > does when PT_INTERP is set. The musl loader uses PT_INTERP to set > > > the path to the loader in the shared library info exported to the > > > debugger, so relinterp creates a copy of the program headers > > > with the PT_INTERP entry added pointing to the found location of > > > the dynamic loader. It updates AT_BASE to point to the address > > > of the dynamic loader, then jumps to the loaders entry point. > > > > > > The dynamic loader then loads shared libraries and handles > > > relocations before jumping to the executable's entry point, which is > > > the entry point in relinterp.c again. Relinterp detects that > > > relocations have been performed and calls __libc_start_main, the > > > same way Scrt1.o would have. > > > > > > Since relinterp runs before relocations have been performed it has > > > to avoid referecing any libc functions. That means reimplementing > > > the few syscalls and string functions that it uses, and avoiding > > > implicit calls to memcpy and memset that may be inserted by the > > > compiler. > > > > > > Enabling relinterp is handled in the spec file for gcc and in > > > the linker script for clang via a -relinterp argument. > > > > > > Normally gdb and lldb look for a symbol named "_dl_debug_state" in > > > the interpreter to get notified when the dynamic loader has modified > > > the list of shared libraries. When using relinterp the debugger is > > > not aware of the interpreter (at process launch PT_INTERP is unset > > > and auxv AT_BASE is 0) so it doesn't know where to look for the symbol. > > > > > > They fall back to looking in the executable, so we can provide a symbol > > > in relinterp.c for it to find. The dynamic loader is then modified > > > to also find the symbol in the exectuable and to call it from its own > > > _dl_debug_state function. > > > > > > The same tests in libc_test pass with or without LDFLAGS += -relinterp > > > with both musl-gcc and musl-clang. > > > > > > Ryan Prichard (rprichard@google.com) authored the original prototype > > > of relinterp. > > > > Have you looked at https://www.openwall.com/lists/musl/2020/03/29/9 > > where this has already been done? It's not upstream but my > > understanding is that the author has been using it successfully for a > > long time, and it's been through some review and as I recall was at > > least close to acceptable for upstream. > > > > Rich > > No, I had not seen that, and it looks to have identical functionality. > I'll experiment with switching to it. dcrt1 seems to be a perfect drop in replacement for this. Quick testing shows it works for my needs on x86_64, x86, aarch64 and arm architectures. I've attached a minor patch that fixes some unused variable and label warnings. --000000000000c0c9d605e9a12580 Content-Type: text/x-patch; charset="US-ASCII"; name="0001-Fix-unused-variable-and-label-warnings-in-dcrt1.c.patch" Content-Disposition: attachment; filename="0001-Fix-unused-variable-and-label-warnings-in-dcrt1.c.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_l8jpblln0 RnJvbSBlZjZkYWQ1YjE1OGEwYTdhY2JlZDZiNjk3OTk2NGNkYzIyOGJjNzlmIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBDb2xpbiBDcm9zcyA8Y2Nyb3NzQGFuZHJvaWQuY29tPgpEYXRl OiBNb24sIDI2IFNlcCAyMDIyIDIxOjI2OjIwIC0wNzAwClN1YmplY3Q6IFtQQVRDSF0gRml4IHVu dXNlZCB2YXJpYWJsZSBhbmQgbGFiZWwgd2FybmluZ3MgaW4gZGNydDEuYwoKQ2hhbmdlLUlkOiBJ NTRhZGQ0NWYwNTI1MDQ2YzcyZTYyOTEzOTI1NzE0NzVkMzE2NDA4OQotLS0KIGNydC9kY3J0MS5j ICAgIHwgNSAtLS0tLQogbGRzby9kbHN0YXJ0LmMgfCAyICsrCiAyIGZpbGVzIGNoYW5nZWQsIDIg aW5zZXJ0aW9ucygrKSwgNSBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9jcnQvZGNydDEuYyBi L2NydC9kY3J0MS5jCmluZGV4IDVlODQ0MDQ4Li5lYjFkNjY1MiAxMDA2NDQKLS0tIGEvY3J0L2Rj cnQxLmMKKysrIGIvY3J0L2RjcnQxLmMKQEAgLTExNiw5ICsxMTYsNiBAQCBzdGF0aWMgY2hhciAq Y3J0X2dldGVudihjb25zdCBjaGFyICpuYW1lLCBjaGFyICoqZW52aXJvbikKIAogc3RhdGljIGlu bGluZSB2b2lkICptYXBfbGlicmFyeShpbnQgZmQpCiB7Ci0Jdm9pZCAqYWxsb2NhdGVkX2J1Zj0w OwotCXNpemVfdCBhbGxvY2F0ZWRfYnVmX3NpemU7Ci0Jc2l6ZV90IHBoc2l6ZTsKIAlzaXplX3Qg YWRkcl9taW49U0laRV9NQVgsIGFkZHJfbWF4PTA7CiAJc2l6ZV90IHRoaXNfbWluLCB0aGlzX21h eDsKIAlvZmZfdCBvZmZfc3RhcnQgPSAwOwpAQCAtMjE5LDcgKzIxNiw2IEBAIHN0YXRpYyBzaXpl X3QgZmluZF9saW5rZXIoY2hhciAqb3V0YnVmLCBzaXplX3QgYnVmc2l6ZSwgY29uc3QgY2hhciAq dGhpc19wYXRoLCBzCiB7CiAJY29uc3QgY2hhciAqcGF0aHNbMl07IC8vIGVudnBhdGgsIHJwYXRo L3J1bnBhdGgKIAlzaXplX3QgaTsKLQlpbnQgZmQ7CiAKIAkvLyBJbiB0aGUgc3VpZC9zZWN1cmUg Y2FzZSwgc2tpcCBldmVyeXRoaW5nIGFuZCB1c2UgdGhlIGZpeGVkIHBhdGgKIAlpZiAoc2VjdXJl KQpAQCAtMjkzLDcgKzI4OSw2IEBAIGhpZGRlbiBfTm9yZXR1cm4gdm9pZCBfX2RsczIodW5zaWdu ZWQgY2hhciAqYmFzZSwgc2l6ZV90ICpwKQogCWNoYXIgKiphcmd2ID0gKHZvaWQgKikocCsxKTsK IAlpbnQgZmQ7CiAJaW50IHNlY3VyZTsKLQlpbnQgcHJvdCA9IFBST1RfUkVBRDsKIAlFaGRyICps b2FkZXJfaGRyOwogCVBoZHIgKm5ld19oZHI7CiAJdm9pZCAqZW50cnk7CmRpZmYgLS1naXQgYS9s ZHNvL2Rsc3RhcnQuYyBiL2xkc28vZGxzdGFydC5jCmluZGV4IDQ5ZTZhOTkyLi5kODIxODIxMSAx MDA2NDQKLS0tIGEvbGRzby9kbHN0YXJ0LmMKKysrIGIvbGRzby9kbHN0YXJ0LmMKQEAgLTE2Myw3 ICsxNjMsOSBAQCBoaWRkZW4gdm9pZCBfZGxzdGFydF9jKHNpemVfdCAqc3AsIHNpemVfdCAqZHlu dikKICNlbmRpZgogCiAJc3RhZ2UyX2Z1bmMgZGxzMjsKKyNpZmRlZiBETF9ETkkKIHNraXBfcmVs b2NzOgorI2VuZGlmCiAJR0VURlVOQ1NZTSgmZGxzMiwgX19kbHMyLCBiYXNlK2R5bltEVF9QTFRH T1RdKTsKIAlkbHMyKCh2b2lkICopYmFzZSwgc3ApOwogfQotLSAKMi4zNy4zLjk5OC5nNTc3ZTU5 MTQzZi1nb29nCgo= --000000000000c0c9d605e9a12580--