RTLD_LAZY deferred symbol binding

RTLD_LAZY deferred symbol binding

[Scherbatiy Alexander]

Hello,

musl libc release 1.1.17 has new feature [1]:
- RTLD_LAZY deferred symbol binding, functionally equivalent to lazy binding

The lazy bindings section [2] gives more details on it:
Newer versions of musl implement “deferred binding” in place of lazy binding, whereby binding is deferred until a subsequent dlopen call that introduces new symbols, rather than at the point of the function call.

It is still is not clear for me what is a difference between of lazy and deferred binding.

I wrote a simple example there a shared library with an unresolved symbols is loaded by dlopen with RTLD_LAZY option (source code is at the end of the email).
It works on my Ubuntu desktop but fails on Alpine linux  3.10.3 with musl libc  1.1.22 (x86_64) with message:
"dlopen failed: Error relocating bin/shared/libshared_lib.so: unresolved_function: symbol not found"

What is a good example that can show how the new "deferred symbol binding" feature works so it fails before muls libc 1.1.17 and starts working after it? 

[1] https://git.musl-libc.org/cgit/musl/tree/WHATSNEW
[2] https://wiki.musl-libc.org/functional-differences-from-glibc.html

Thanks,
Alexander.

Loading a shared library with unresolved symbols example:

--- include/resolved_lib.h ---
void resolved_function();

--- include/unresolved_lib.h ---
void unresolved_function();

--- include/shared_lib.h ---
void call_resolved_function();
void call_unresolved_function();

--- src/resolved_impl.c ---
#include <stdio.h>
#include "resolved_lib.h"

void resolved_function() {
    printf("call resolved function.\n");
}

---  src/shared_lib.c ---
#include "shared_lib.h"
#include "resolved_lib.h"
#include "unresolved_lib.h"

void call_resolved_function() {
    resolved_function();
}

void call_unresolved_function() {
    unresolved_function();
}

--- src/main.c ---
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>

void call_resolved_function_dynamic() {

    const char *lib_path = "bin/shared/libshared_lib.so";
    void (*call_resolved_function)(void);

    void *handle = dlopen(lib_path, RTLD_LAZY);

    if (!handle) {
        fprintf(stderr, "dlopen failed: %s\n", dlerror());
        exit(EXIT_FAILURE);
    }

    *(void **) (&call_resolved_function) = dlsym(handle, "call_resolved_function");

    char *error = dlerror();
    if (error != NULL)  {
        fprintf(stderr, "%s\n", error);
        exit(EXIT_FAILURE);
    }

    (*call_resolved_function)();
    dlclose(handle);
}

int main(int argc, char* argv[]) {
    printf("call main.\n");
    call_resolved_function_dynamic();
}

--- ---
# build sources
gcc -c -fPIC src/resolved_impl.c -Iinclude -o bin/shared/resolved_impl.o
gcc -c -fPIC src/shared_lib.c    -Iinclude -o bin/shared/shared_lib.o
gcc -shared bin/shared/shared_lib.o bin/shared/resolved_impl.o -Iinclude -o bin/shared/libshared_lib.so
gcc -c src/main.c                -Iinclude -o bin/main.o
gcc bin/main.o -ldl   -o bin/main

# run
export LD_LIBRARY_PATH=./bin/shared
./bin/main
--- ---

Re: RTLD_LAZY deferred symbol binding

[Szabolcs Nagy]

* Scherbatiy Alexander <alexander.scherbatiy@bell-sw.com> [2019-12-11 13:09:36 +0300]:
> # build sources
> gcc -c -fPIC src/resolved_impl.c -Iinclude -o bin/shared/resolved_impl.o
> gcc -c -fPIC src/shared_lib.c    -Iinclude -o bin/shared/shared_lib.o
> gcc -shared bin/shared/shared_lib.o bin/shared/resolved_impl.o -Iinclude -o bin/shared/libshared_lib.so
^^^^^^^^^^^^^^^

you need to pass -Wl,-z,lazy (and verify it with readelf -d )
because alpine (and various other distros) defaults to -z now
(and then libc obviously cant do lazy binding no matter what
you specified in dlopen).

Re: RTLD_LAZY deferred symbol binding

[Scherbatiy Alexander]

Thank you. It works.

I looked at the ld help on linux Alpine and it shows
>  ld --help
>  -z lazy                     Mark object lazy runtime binding (default)

Should the lazy option be used by default or the documentation needs to be updated?

Thanks,
Alexander.

11.12.2019, 13:35, "Szabolcs Nagy" <nsz@port70.net>:
> * Scherbatiy Alexander <alexander.scherbatiy@bell-sw.com> [2019-12-11 13:09:36 +0300]:
>>  # build sources
>>  gcc -c -fPIC src/resolved_impl.c -Iinclude -o bin/shared/resolved_impl.o
>>  gcc -c -fPIC src/shared_lib.c -Iinclude -o bin/shared/shared_lib.o
>>  gcc -shared bin/shared/shared_lib.o bin/shared/resolved_impl.o -Iinclude -o bin/shared/libshared_lib.so
>
> ^^^^^^^^^^^^^^^
>
> you need to pass -Wl,-z,lazy (and verify it with readelf -d )
> because alpine (and various other distros) defaults to -z now
> (and then libc obviously cant do lazy binding no matter what
> you specified in dlopen).

Re: RTLD_LAZY deferred symbol binding

[Szabolcs Nagy]

* Scherbatiy Alexander <alexander.scherbatiy@bell-sw.com> [2019-12-11 14:53:43 +0300]:
> Thank you. It works.
> 
> I looked at the ld help on linux Alpine and it shows
> > ld --help
> > -z lazy                     Mark object lazy runtime binding (default)
> 
> Should the lazy option be used by default or the documentation needs to be updated?

that's standard binutils documentation and it's correct:
i think alpine patches gcc to pass -z now, not ld (and
i think gentoo hardened does the same).

i guess alpine could maintain a toolchain documentation
where it describes the changes compared to upstream
defaults, but you can check their patches
https://git.alpinelinux.org/aports/tree/main/gcc

> 
> Thanks,
> Alexander.
> 
> 
> 11.12.2019, 13:35, "Szabolcs Nagy" <nsz@port70.net>:
> > * Scherbatiy Alexander <alexander.scherbatiy@bell-sw.com> [2019-12-11 13:09:36 +0300]:
> >>  # build sources
> >>  gcc -c -fPIC src/resolved_impl.c -Iinclude -o bin/shared/resolved_impl.o
> >>  gcc -c -fPIC src/shared_lib.c -Iinclude -o bin/shared/shared_lib.o
> >>  gcc -shared bin/shared/shared_lib.o bin/shared/resolved_impl.o -Iinclude -o bin/shared/libshared_lib.so
> >
> > ^^^^^^^^^^^^^^^
> >
> > you need to pass -Wl,-z,lazy (and verify it with readelf -d )
> > because alpine (and various other distros) defaults to -z now
> > (and then libc obviously cant do lazy binding no matter what
> > you specified in dlopen).

Re: RTLD_LAZY deferred symbol binding

[Rich Felker]

On Wed, Dec 11, 2019 at 02:55:48PM +0300, Scherbatiy Alexander wrote:
> Thank you. It works.
> 
> I looked at the ld help on linux Alpine and it shows
> >  ld --help
> >  -z lazy                     Mark object lazy runtime binding (default)
> 
> Should the lazy option be used by default

No, it's really a bug to be relying on this behavior, as it precludes
much-wanted hardening and is not well-defined per the supported
standards. Deferred binding was added for the sake of a very small but
important and stubborn set of software (i.e. Xorg) that was depending
on lazy binding and where musl-based dists were needing messy hacks to
make it work without (explicit load order of modules in xorg.conf).
The software that needs this behavior can be built with -z lazy as
needed. This is not unique to musl; it applies to hardened glibc
toolchains/distros where bindnow is default, too.

> or the documentation needs to be updated?

That's the documentation for ld, which applies if you invoke ld as a
command. That's not how you link hosted software. It's linked by
invoking $(CC) (e.g. gcc) which may be the part passing -z now; I
forget. If the default is actually changed in ld, then yes ld should
be patched to print the default it was configured with accurately (and
I would think such a patch would be acceptable for upstream).

Rich