Re: dlopen'ing glibc linked libraries

[Szabolcs Nagy <nsz@port70.net>]

* Szabolcs Nagy <nsz@port70.net> [2014-01-21 19:17:27 +0100]:
> it may be possible to solve a lot of compatibility issues with
> preloading an extra lib next to musl, i experimented with running
> applications with
> 
> export LD_LIBRARY_PATH=/usr/lib:/usr/lib/i386-linux-gnu:/lib/i386-linux-gnu
> export LD_PRELOAD=./libcompat.so
> /lib/ld-musl-i386.so /usr/bin/something

i did further experiments, the issues i found so far:

- brk pointer is sometimes at the wrong place at start up
so malloc fails (kernel bug, rarely happens here)

- some applications read /proc/self/exe to determine the
path to their binary (to start other instances with
different arguments) but that will point to the loader

- some applications use gcc's closure extension so they
need executable stack (the kernel maps the loader with
non-executable stack)

these are all rare and can be worked around by modifying
the interpreter in the binary using patchelf (or setting
up a namespace where /lib/ld-linux.so.2 points to musl)

- binaries compiled without _FILE_OFFSET_BITS=64 are not
compatible with musl on i386 (on my debian stable system
there are more than 200 binaries and libraries which use
readdir, __[fl]xstat instead of readdir64, __[fl]xstat64)
(this is probably the biggest obstacle for compatibility)

- programs using sbrk fail (affects programs that try to
implement their own malloc (eg bash) or programs using the
boehm gc) (this will go away when musl's alloc is updated)

- i had to add many symbols (>300) most of them are trivial
wrappers, but some of them would be hard to do correctly
(the implementation internals like __pthread_register_cancel
and the messy nss/nscd/rpc apis are the most painful ones)