From: Colin Cross <ccross@android.com>
To: Rich Felker <dalias@libc.org>
Cc: musl@lists.openwall.com, Ryan Prichard <rprichard@google.com>,
Rodger Combs <rodger.combs@gmail.com>
Subject: Re: [musl] Running musl executables without a preinstalled dynamic linker
Date: Mon, 26 Sep 2022 21:33:22 -0700 [thread overview]
Message-ID: <CAMbhsRRVFTK9e6cyY27MOtYMn397Qej5zMT6imRMT2SXfnPPKg@mail.gmail.com> (raw)
In-Reply-To: <CAMbhsRRwojnxAJuMqhHLaU3weKaT4Q0_Tk=fai47_e5b9V_GOA@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 6425 bytes --]
On Mon, Sep 26, 2022 at 4:12 PM Colin Cross <ccross@android.com> wrote:
>
> On Mon, Sep 26, 2022 at 4:02 PM Rich Felker <dalias@libc.org> wrote:
> >
> > On Mon, Sep 26, 2022 at 03:42:01PM -0700, Colin Cross wrote:
> > > On Mon, Sep 26, 2022 at 3:38 PM Colin Cross <ccross@android.com> wrote:
> > > >
> > > > On Tue, Aug 23, 2022 at 1:18 AM Szabolcs Nagy <nsz@port70.net> wrote:
> > > > >
> > > > > * Colin Cross <ccross@android.com> [2022-08-22 17:22:06 -0700]:
> > > > > > On Sat, Aug 20, 2022 at 2:43 AM Szabolcs Nagy <nsz@port70.net> wrote:
> > > > > > > i would not use Scrt1.o though, the same toolchain should be
> > > > > > > usable for normal linking and relinterp linking, just use a
> > > > > > > different name like Xcrt1.o.
> > > > > >
> > > > > > Is there some way to get gcc/clang to use Xcrt1.o without using
> > > > > > -nostdlib and passing all the crtbegin/end objects manually?
> > > > >
> > > > > this requires compiler changes (new cmdline flag) but then i think
> > > > > the code is upstreamable.
> > > >
> > > > I've used relinterp.o for now, and selected instead of Scrt1.o in
> > > > musl-gcc.specs and ld.musl-clang.
> > > >
> > > > >
> > > > > > > i would make Xcrt1.o self-contained and size optimized: it only
> > > > > > > runs at start up, this is a different requirement from the -O3
> > > > > > > build of normal string functions. and then there is no dependency
> > > > > > > on libc internals (which may have various instrumentations that
> > > > > > > does not work in Xcrt1.o).
> > > > > >
> > > > > > Doesn't this same logic apply to most of the code in dynlink.c? My
> > > > > > main worry with a self contained implementation is that it requires
> > > > > > reimplementations of various string functions that are easy to get
> > > > > > wrong. The current prototype reuses the C versions of musl's string
> > > > > > functions, but implements its own syscall wrappers to avoid
> > > > > > interactions with musl internals like errno.
> > > > >
> > > > > dynlink is in libc.so so it can use code from there.
> > > > >
> > > > > but moving libc code into the executable has different constraints.
> > > > > so you will have to make random decisions that string functions are
> > > > > in but errno is out, wrt which libc internal makes sense in the exe.
> > > > >
> > > > > i would just keep a separate implementation (or at least compile
> > > > > the code separately). string functions are easy to implement if
> > > > > you dont try to optimize them imo. then you have full control over
> > > > > what is going on in the exe entry code.
> > > >
> > > > I left the reimplementations of string functions and syscalls as
> > > > suggested. Patch attached.
> >
> > > From 0df460188b95f79272003bd0e5c12bceb2a3c25f Mon Sep 17 00:00:00 2001
> > > From: Colin Cross <ccross@android.com>
> > > Date: Thu, 22 Sep 2022 19:14:01 -0700
> > > Subject: [PATCH] Add entry point to find dynamic loader relative to the
> > > executable
> > >
> > > Distributing binaries built against musl to systems that don't already
> > > have musl is problematic due to the hardcoded absolute path to the
> > > dynamic loader (e.g. /lib/ld-musl-$ARCH.so.1) in the PT_INTERP header.
> > > This patch adds a feature to avoid the problem by leaving out PT_INTERP
> > > and replacing Scrt1.o with an entry point that can find the dynamic
> > > loader using DT_RUNPATH or LD_LIBRARY_PATH.
> > >
> > > The entry point is in crt/relinterp.c. It uses auxval to get the
> > > program headers and find the load address of the binary, then
> > > searches LD_LIBRARY_PATH or DT_RUNPATH for the dynamic loader.
> > > Once found, it mmaps the loader similar to the way the kernel
> > > does when PT_INTERP is set. The musl loader uses PT_INTERP to set
> > > the path to the loader in the shared library info exported to the
> > > debugger, so relinterp creates a copy of the program headers
> > > with the PT_INTERP entry added pointing to the found location of
> > > the dynamic loader. It updates AT_BASE to point to the address
> > > of the dynamic loader, then jumps to the loaders entry point.
> > >
> > > The dynamic loader then loads shared libraries and handles
> > > relocations before jumping to the executable's entry point, which is
> > > the entry point in relinterp.c again. Relinterp detects that
> > > relocations have been performed and calls __libc_start_main, the
> > > same way Scrt1.o would have.
> > >
> > > Since relinterp runs before relocations have been performed it has
> > > to avoid referecing any libc functions. That means reimplementing
> > > the few syscalls and string functions that it uses, and avoiding
> > > implicit calls to memcpy and memset that may be inserted by the
> > > compiler.
> > >
> > > Enabling relinterp is handled in the spec file for gcc and in
> > > the linker script for clang via a -relinterp argument.
> > >
> > > Normally gdb and lldb look for a symbol named "_dl_debug_state" in
> > > the interpreter to get notified when the dynamic loader has modified
> > > the list of shared libraries. When using relinterp the debugger is
> > > not aware of the interpreter (at process launch PT_INTERP is unset
> > > and auxv AT_BASE is 0) so it doesn't know where to look for the symbol.
> > >
> > > They fall back to looking in the executable, so we can provide a symbol
> > > in relinterp.c for it to find. The dynamic loader is then modified
> > > to also find the symbol in the exectuable and to call it from its own
> > > _dl_debug_state function.
> > >
> > > The same tests in libc_test pass with or without LDFLAGS += -relinterp
> > > with both musl-gcc and musl-clang.
> > >
> > > Ryan Prichard (rprichard@google.com) authored the original prototype
> > > of relinterp.
> >
> > Have you looked at https://www.openwall.com/lists/musl/2020/03/29/9
> > where this has already been done? It's not upstream but my
> > understanding is that the author has been using it successfully for a
> > long time, and it's been through some review and as I recall was at
> > least close to acceptable for upstream.
> >
> > Rich
>
> No, I had not seen that, and it looks to have identical functionality.
> I'll experiment with switching to it.
dcrt1 seems to be a perfect drop in replacement for this. Quick
testing shows it works for my needs on x86_64, x86, aarch64 and arm
architectures. I've attached a minor patch that fixes some unused
variable and label warnings.
[-- Attachment #2: 0001-Fix-unused-variable-and-label-warnings-in-dcrt1.c.patch --]
[-- Type: text/x-patch, Size: 1607 bytes --]
From ef6dad5b158a0a7acbed6b6979964cdc228bc79f Mon Sep 17 00:00:00 2001
From: Colin Cross <ccross@android.com>
Date: Mon, 26 Sep 2022 21:26:20 -0700
Subject: [PATCH] Fix unused variable and label warnings in dcrt1.c
Change-Id: I54add45f0525046c72e6291392571475d3164089
---
crt/dcrt1.c | 5 -----
ldso/dlstart.c | 2 ++
2 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/crt/dcrt1.c b/crt/dcrt1.c
index 5e844048..eb1d6652 100644
--- a/crt/dcrt1.c
+++ b/crt/dcrt1.c
@@ -116,9 +116,6 @@ static char *crt_getenv(const char *name, char **environ)
static inline void *map_library(int fd)
{
- void *allocated_buf=0;
- size_t allocated_buf_size;
- size_t phsize;
size_t addr_min=SIZE_MAX, addr_max=0;
size_t this_min, this_max;
off_t off_start = 0;
@@ -219,7 +216,6 @@ static size_t find_linker(char *outbuf, size_t bufsize, const char *this_path, s
{
const char *paths[2]; // envpath, rpath/runpath
size_t i;
- int fd;
// In the suid/secure case, skip everything and use the fixed path
if (secure)
@@ -293,7 +289,6 @@ hidden _Noreturn void __dls2(unsigned char *base, size_t *p)
char **argv = (void *)(p+1);
int fd;
int secure;
- int prot = PROT_READ;
Ehdr *loader_hdr;
Phdr *new_hdr;
void *entry;
diff --git a/ldso/dlstart.c b/ldso/dlstart.c
index 49e6a992..d8218211 100644
--- a/ldso/dlstart.c
+++ b/ldso/dlstart.c
@@ -163,7 +163,9 @@ hidden void _dlstart_c(size_t *sp, size_t *dynv)
#endif
stage2_func dls2;
+#ifdef DL_DNI
skip_relocs:
+#endif
GETFUNCSYM(&dls2, __dls2, base+dyn[DT_PLTGOT]);
dls2((void *)base, sp);
}
--
2.37.3.998.g577e59143f-goog
next prev parent reply other threads:[~2022-09-27 4:33 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-15 21:35 Colin Cross
2022-08-20 9:43 ` Szabolcs Nagy
2022-08-23 0:22 ` Colin Cross
2022-08-23 8:18 ` Szabolcs Nagy
2022-09-26 22:38 ` Colin Cross
2022-09-26 22:42 ` Colin Cross
2022-09-26 23:02 ` Rich Felker
2022-09-26 23:12 ` Colin Cross
2022-09-27 4:33 ` Colin Cross [this message]
2022-09-27 5:09 ` Ridley Combs
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAMbhsRRVFTK9e6cyY27MOtYMn397Qej5zMT6imRMT2SXfnPPKg@mail.gmail.com \
--to=ccross@android.com \
--cc=dalias@libc.org \
--cc=musl@lists.openwall.com \
--cc=rodger.combs@gmail.com \
--cc=rprichard@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/musl/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).