[musl] dlopen() issue

[musl] dlopen() issue

[Vince Fleming]

[-- Attachment #1: Type: text/plain, Size: 1000 bytes --]

Hi all;

I seem to be having a problem with dynamically loading a library.   It's
intermittent; it will succeed and work fine, then on the next run it'll
give "libaio: cannot open shared object file: No such file or directory",
or "Dynamic loading not supported", which appear to come from dlerror() in
musl, not the application I'm porting to musl.

Has anyone else seen such behavior?

-- 
Vince Fleming
Director, Solutions & Services, WekaIO
M 848-220-0041* ­*E vince@weka.io* ­*W https://www.weka.io/
<https://www.weka.io/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>*
­* * ­*
<https://www.weka.io/promo/the-state-of-ai-and-analytics-infrastructure-2021/>
*Join
our webinar:*The State of AI and Analytics Infrastructure 2021
<https://www.weka.io/promo/the-state-of-ai-and-analytics-infrastructure-2021/>
<https://www.weka.io/press-releases/weka-named-to-fast-companys-annual-list-of-the-worlds-most-innovative-companies-for-2021/>

[-- Attachment #2: Type: text/html, Size: 5488 bytes --]

Re: [musl] dlopen() issue

[Rich Felker]

On Thu, Sep 23, 2021 at 05:47:37PM -0400, Vince Fleming wrote:
> Hi all;
> 
> I seem to be having a problem with dynamically loading a library.   It's
> intermittent; it will succeed and work fine, then on the next run it'll
> give "libaio: cannot open shared object file: No such file or directory",
> or "Dynamic loading not supported", which appear to come from dlerror() in
> musl, not the application I'm porting to musl.
> 
> Has anyone else seen such behavior?

The error means your program is static linked. It's not clear why
you're not seeing it other times; presumably you're rebuilding (or at
least relinking) the program between runs, or else running different
versions of it each time.

Rich

Re: [musl] dlopen() issue

[Vince Fleming]

[-- Attachment #1: Type: text/plain, Size: 2384 bytes --]

Yes, the statically-linked program is calling dlopen() to dynamically-link
in a library.   This works fine with glibc.   The library in question is
libaio, which I've also built with musl.

And, oddly, I am not rebuilding between runs - I'm executing the same
binary over and over again.  The program is a disk benchmark (fio).   The
errors are very random, so it's not dependent on the input to fio, which is
nearly identical anyway (reads vs writes with all other parameters the
same, for example).  I have a wrapper that executes fio 6 times.  Repeating
the wrapper run produces errors from dlopen() randomly - sometimes the
first fio fails, other times it works.  Same for 2nd through 6th runs.  It
randomly produces a "no such file", other times the "dynamic loading not
supported".

I'm puzzled by the randomness of it, and suspect that the cause is outside
of dlopen() but don't know where to start looking...  any ideas?

On Thu, Sep 23, 2021 at 7:09 PM Rich Felker <dalias@libc.org> wrote:

> On Thu, Sep 23, 2021 at 05:47:37PM -0400, Vince Fleming wrote:
> > Hi all;
> >
> > I seem to be having a problem with dynamically loading a library.   It's
> > intermittent; it will succeed and work fine, then on the next run it'll
> > give "libaio: cannot open shared object file: No such file or directory",
> > or "Dynamic loading not supported", which appear to come from dlerror()
> in
> > musl, not the application I'm porting to musl.
> >
> > Has anyone else seen such behavior?
>
> The error means your program is static linked. It's not clear why
> you're not seeing it other times; presumably you're rebuilding (or at
> least relinking) the program between runs, or else running different
> versions of it each time.
>
> Rich
>


-- 
Vince Fleming
Director, Solutions & Services, WekaIO
M 848-220-0041* ­*E vince@weka.io* ­*W https://www.weka.io/
<https://www.weka.io/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>*
­* * ­*
<https://www.weka.io/promo/the-state-of-ai-and-analytics-infrastructure-2021/>
*Join
our webinar:*The State of AI and Analytics Infrastructure 2021
<https://www.weka.io/promo/the-state-of-ai-and-analytics-infrastructure-2021/>
<https://www.weka.io/press-releases/weka-named-to-fast-companys-annual-list-of-the-worlds-most-innovative-companies-for-2021/>

[-- Attachment #2: Type: text/html, Size: 7393 bytes --]

Re: [musl] dlopen() issue

[Rich Felker]

On Fri, Sep 24, 2021 at 08:12:33AM -0400, Vince Fleming wrote:
> Yes, the statically-linked program is calling dlopen() to dynamically-link
> in a library.   This works fine with glibc.   The library in question is
> libaio, which I've also built with musl.

This is expected to fail with "dynamic loading not supported" as the
dlerror error message. There is no dynamic loader for static linked
programs. Doing it is theoretically possible, but has a lot of
limitations. glibc just ignores these rather than even trying to do it
right, leading to a situation where it's unsafe if the glibc version
available at runtime differs from the one you static-linked with,
among other things. If we add it in some point in musl, it will need
to be done in a way that doesn't have this problem.

If you want to ship a self-contained program that needs dynamic
loading, the easiest way that works at present is shipping
ld-musl-$(ARCH).so.1, a dynamic binary, and a wrapper script that
invokes ld-musl explicitly to load the dynamic binary. There's a
proposal that will make an easier way in the future too but it's not
upstream yet.

> And, oddly, I am not rebuilding between runs - I'm executing the same
> binary over and over again.  The program is a disk benchmark (fio).   The
> errors are very random, so it's not dependent on the input to fio, which is
> nearly identical anyway (reads vs writes with all other parameters the
> same, for example).  I have a wrapper that executes fio 6 times.  Repeating
> the wrapper run produces errors from dlopen() randomly - sometimes the
> first fio fails, other times it works.  Same for 2nd through 6th runs.  It
> randomly produces a "no such file", other times the "dynamic loading not
> supported".

I can't think of any explanation for this except that you're running
different binaries or doing something that clobbers error state or
displays the wrong error message (e.g. by inspecting errno first and
printing strerror(errno) rather than dlerror()). There is no way it is
"working sometimes" because there is no code here to do dynamic
loading.

> I'm puzzled by the randomness of it, and suspect that the cause is outside
> of dlopen() but don't know where to start looking...  any ideas?

Running under strace would show you better what's happening.


> On Thu, Sep 23, 2021 at 7:09 PM Rich Felker <dalias@libc.org> wrote:
> 
> > On Thu, Sep 23, 2021 at 05:47:37PM -0400, Vince Fleming wrote:
> > > Hi all;
> > >
> > > I seem to be having a problem with dynamically loading a library.   It's
> > > intermittent; it will succeed and work fine, then on the next run it'll
> > > give "libaio: cannot open shared object file: No such file or directory",
> > > or "Dynamic loading not supported", which appear to come from dlerror()
> > in
> > > musl, not the application I'm porting to musl.
> > >
> > > Has anyone else seen such behavior?
> >
> > The error means your program is static linked. It's not clear why
> > you're not seeing it other times; presumably you're rebuilding (or at
> > least relinking) the program between runs, or else running different
> > versions of it each time.
> >
> > Rich
> >
> 
> 
> -- 
> Vince Fleming
> Director, Solutions & Services, WekaIO
> M 848-220-0041* ­*E vince@weka.io* ­*W https://www.weka.io/
> <https://www.weka.io/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>*
> ­* * ­*
> <https://www.weka.io/promo/the-state-of-ai-and-analytics-infrastructure-2021/>
> *Join
> our webinar:*The State of AI and Analytics Infrastructure 2021
> <https://www.weka.io/promo/the-state-of-ai-and-analytics-infrastructure-2021/>
> <https://www.weka.io/press-releases/weka-named-to-fast-companys-annual-list-of-the-worlds-most-innovative-companies-for-2021/>