Re: [musl] Supporting multilib LD_LIBRARY_PATHs

[Alexey Izbyshev <izbyshev@ispras.ru>]

On 2023-02-08 03:49, Rich Felker wrote:
> On Tue, Feb 07, 2023 at 02:58:37PM -0800, Colin Cross wrote:
>> I'm hitting an issue where some test infrastructure is setting
>> LD_LIBRARY_PATH to a list that contains both 32-bit and 64-bit
>> libraries because it is unsure whether the code under test is going to
>> execute 32-bit or 64-bit processes or both.  When using musl the
>> dynamic loader takes the first library with a matching name and then
>> fails to load it if it is for the wrong elf class.
>> 
>> The attached patch verifies the elf machine and class when searching
>> the path list, continuing the search if a valid elf header with an
>> incorrect machine or class is found.
> 
> While it requires some consideration to ensure that this yields safe &
> consistent behavior, I think it at least admits that; I haven't
> checked the actual code, but conceptually, it should be equivalent to
> treating finding a mismatched-arch library as a conclusive result
> whose behavior is searching the remainder of the search path.
> 
> I'm a little bit skeptical of the motivation though. In general, it's
> not safe to just set LD_LIBRARY_PATH and run programs that might
> invoke other programs, since the math might contain outdated or
> mismatched libraries relative to what those other programs might want.
> On a system with multiple libcs present, the libraries found could
> even be for the wrong one. Really, LD_LIBRARY_PATH should just be set
> for invoking a single program (or family of binaries) that ships with
> its own versions of libraries or when you're overriding certain
> libraries for it, etc. This is contrary to how the environment works,
> and one reason it's probably better to use ldso --library-path=...
> rather than LD_LIBRARY_PATH when overriding libraries for a particular
> program invocation.
> 
It's sometimes convenient to use LD_PRELOAD in conjunction with 
LD_LIBRARY_PATH to override libc functions in a specific process tree 
containing binaries for different architectures, e.g. 32-bit/64-bit on 
x86 systems or native/foreign (run via qemu-user). I don't know how to 
achieve that with current musl other than by overriding 
exec*/posix_spawn functions and manually checking the architecture of 
the binary (and not forgetting to process "#!" for scripts), so I'd be 
glad to see the proposed functionality in musl.

In general, it would still not work on systems with multiple libcs, but 
I'd expect the mixed-libc case to be much rarer than the mixed-arch one.

Alexey