[musl] Search path for shared libs

[musl] Search path for shared libs

[Kai Peter]

Hi,

I did run in an issue with the search path of shared objects. I'm on a 
real custom system, so the pathes below are correct. There is no glibc 
at all, the gcc was modified to build with the loader in /qlnx/lib. curl 
was build with shared openssl libs as a dependency for git. Then I issue

$ ldd /misc/bin/curl
         /qlnx/lib/ld-musl-x86_64.so.1 (0x7f1e1df97000)
         libcurl.so.4 => /misc/lib/libcurl.so.4 (0x7f1e1df0f000)
         libssl.so.1.1 => /libs/lib/libssl.so.1.1 (0x7f1e1de79000)
         libcrypto.so.1.1 => /libs/lib/libcrypto.so.1.1 (0x7f1e1dbb0000)
         libz.so.1 => /qlnx/lib/libz.so.1 (0x7f1e1db93000)
         libc.so => /qlnx/lib/ld-musl-x86_64.so.1 (0x7f1e1df97000)
         libzstd.so.1 => /qlnx/lib/libzstd.so.1 (0x7f1e1da81000)

All fine, pathes are correct and working. Then:

$ ldd /misc/lib/libcurl.so.4 2>&1 | grep -v reloc
         ldd (0x7f122b691000)
Error loading shared library libssl.so.1.1: No such file or directory 
(needed by /misc/lib/libcurl.so.4)
Error loading shared library libcrypto.so.1.1: No such file or directory 
(needed by /misc/lib/libcurl.so.4)
         libzstd.so.1 => /qlnx/lib/libzstd.so.1 (0x7f122b4f7000)
         libz.so.1 => /qlnx/lib/libz.so.1 (0x7f122b4da000)
         libc.so => ldd (0x7f122b691000)

As a workaround I can create symlinks of the missing ssl libs in 
/qlnx/lib. The ld-musl-x86_64.path contains all pathes and it is read. 
So the question is: Why does libcurl.so.4 not find libssl/libcrypto? Any 
insight?

Kai

Re: [musl] Search path for shared libs

[Rich Felker]

On Wed, Apr 20, 2022 at 09:15:43AM +0200, Kai Peter wrote:
> Hi,
> 
> I did run in an issue with the search path of shared objects. I'm on
> a real custom system, so the pathes below are correct. There is no
> glibc at all, the gcc was modified to build with the loader in
> /qlnx/lib. curl was build with shared openssl libs as a dependency
> for git. Then I issue
> 
> $ ldd /misc/bin/curl
>         /qlnx/lib/ld-musl-x86_64.so.1 (0x7f1e1df97000)
>         libcurl.so.4 => /misc/lib/libcurl.so.4 (0x7f1e1df0f000)
>         libssl.so.1.1 => /libs/lib/libssl.so.1.1 (0x7f1e1de79000)
>         libcrypto.so.1.1 => /libs/lib/libcrypto.so.1.1 (0x7f1e1dbb0000)
>         libz.so.1 => /qlnx/lib/libz.so.1 (0x7f1e1db93000)
>         libc.so => /qlnx/lib/ld-musl-x86_64.so.1 (0x7f1e1df97000)
>         libzstd.so.1 => /qlnx/lib/libzstd.so.1 (0x7f1e1da81000)
> 
> All fine, pathes are correct and working. Then:
> 
> $ ldd /misc/lib/libcurl.so.4 2>&1 | grep -v reloc
>         ldd (0x7f122b691000)
> Error loading shared library libssl.so.1.1: No such file or
> directory (needed by /misc/lib/libcurl.so.4)
> Error loading shared library libcrypto.so.1.1: No such file or
> directory (needed by /misc/lib/libcurl.so.4)
>         libzstd.so.1 => /qlnx/lib/libzstd.so.1 (0x7f122b4f7000)
>         libz.so.1 => /qlnx/lib/libz.so.1 (0x7f122b4da000)
>         libc.so => ldd (0x7f122b691000)
> 
> As a workaround I can create symlinks of the missing ssl libs in
> /qlnx/lib. The ld-musl-x86_64.path contains all pathes and it is
> read. So the question is: Why does libcurl.so.4 not find
> libssl/libcrypto? Any insight?

Is the ldd you're invoking a symlink to ldso? If so, that's deprecated
usage which has problems using the right path (because the pathname
comes from what was used to invoke ldd) and you should replace it with
a shell script:

#!/bin/sh
exec /qlnx/lib/ld-musl-x86_64.so.1 --list "$@"

Then ldso will have the right base pathname to look for its .path file
relative to.

Re: [musl] Search path for shared libs

[Kai Peter]

On 2022-04-20 15:12, Rich Felker wrote:
> On Wed, Apr 20, 2022 at 09:15:43AM +0200, Kai Peter wrote:
>> Hi,
>> 
>> I did run in an issue with the search path of shared objects. I'm on
>> a real custom system, so the pathes below are correct. There is no
>> glibc at all, the gcc was modified to build with the loader in
>> /qlnx/lib. curl was build with shared openssl libs as a dependency
>> for git. Then I issue
>> 
>> $ ldd /misc/bin/curl
>>         /qlnx/lib/ld-musl-x86_64.so.1 (0x7f1e1df97000)
>>         libcurl.so.4 => /misc/lib/libcurl.so.4 (0x7f1e1df0f000)
>>         libssl.so.1.1 => /libs/lib/libssl.so.1.1 (0x7f1e1de79000)
>>         libcrypto.so.1.1 => /libs/lib/libcrypto.so.1.1 
>> (0x7f1e1dbb0000)
>>         libz.so.1 => /qlnx/lib/libz.so.1 (0x7f1e1db93000)
>>         libc.so => /qlnx/lib/ld-musl-x86_64.so.1 (0x7f1e1df97000)
>>         libzstd.so.1 => /qlnx/lib/libzstd.so.1 (0x7f1e1da81000)
>> 
>> All fine, pathes are correct and working. Then:
>> 
>> $ ldd /misc/lib/libcurl.so.4 2>&1 | grep -v reloc
>>         ldd (0x7f122b691000)
>> Error loading shared library libssl.so.1.1: No such file or
>> directory (needed by /misc/lib/libcurl.so.4)
>> Error loading shared library libcrypto.so.1.1: No such file or
>> directory (needed by /misc/lib/libcurl.so.4)
>>         libzstd.so.1 => /qlnx/lib/libzstd.so.1 (0x7f122b4f7000)
>>         libz.so.1 => /qlnx/lib/libz.so.1 (0x7f122b4da000)
>>         libc.so => ldd (0x7f122b691000)
>> 
>> As a workaround I can create symlinks of the missing ssl libs in
>> /qlnx/lib. The ld-musl-x86_64.path contains all pathes and it is
>> read. So the question is: Why does libcurl.so.4 not find
>> libssl/libcrypto? Any insight?
> 
> Is the ldd you're invoking a symlink to ldso? If so, that's deprecated
> usage which has problems using the right path (because the pathname
> comes from what was used to invoke ldd) and you should replace it with
> a shell script:
> 
> #!/bin/sh
> exec /qlnx/lib/ld-musl-x86_64.so.1 --list "$@"
> 
> Then ldso will have the right base pathname to look for its .path file
> relative to.

Hi,

ldd is a symlink to ../lib/ld-musl-x86_64.so.1. However, I found a 
solution in another thread (where I accidentally replied to). I did 
remember that I changed the sys_path in dynlink.c. At this time I didn't 
know all lib pathes. I did add the missing ones and all is working fine 
for me. The root cause was that libcurl was used by git and didn't  find 
libssl. This is working now. As suggested I have replaced the link with 
the script too.

Kai