Re: [musl] dynamic linker is capturing "reserved" library names erroneously

[Christopher Sean Morrison <brlcad@mac.com>]

[-- Attachment #1: Type: text/plain, Size: 5309 bytes --]


> On Jul 28, 2022, at 5:18 AM, Szabolcs Nagy <nsz@port70.net> wrote:
> 
> * Christopher Sean Morrison <brlcad@mac.com> [2022-07-27 19:06:24 -0400]:
>> First consideration, the code seems to take a position that those library names are somehow universally reserved and I believe that to be incorrect.
> 
> the way libraries are looked up is not specified in detail and
> the standard gives the permission to implementations to decide.

This is my understanding as well.  That’s why I characterized it as incorrect with respect to the existing implementation identifying them as “reserved”, as well as with regard to the simplified example provided.  Posix does not reserve them, the compiler+linker behaves as expected, but it’s musl’s dynamic linker implementation that currently does not behave as I would intuitively expect.

> some library names are special, but in conforming environment
> this is only observable via the c99 tool so that's where this
> is specified, the language does not know anything about libraries.

It's observable when an application is run, and posix certainly describes a multitude of aspects of runtime behavior.  The 2017 update and precursors could have called out such reservations, and they do not.  By that same logic, I could just as well create a ‘librt’ library using fort77.  That’s still in posix, but its man page only calls out undefined behavior on the ‘libf.a’ library.  In that case, the runtime behavior would still be incorrect…  Those pages really only referred to just those specific tools, their behavior, and really has no bearing on resolving symbols at runtime — it’s entirely musl’s dynamic linker prerogative.

> i believe the reason musl has to special case the names in ld.so

> is to be able to load libraries linked against glibc. another
> reason is to support looking up symbols in librt etc via dlsym.

That is why I proposed a solution of deferring the capture until after searching, so it will both satisfy that desire to resolve symbols for when -lm, -lr, -lpthread, etc are specified while also supporting the general case where such libraries actually do exist and are intended to be used.
 
>> For background context, I maintain BRL-CAD, a large open source CAD system that has been in development for over 40 years.
>> BRL-CAD’s flagship API with thousands of integrations around the world is the “librt” ray tracing library.
> 
> this would not work eg on older glibc if one of your dependencies
> used clock_gettime. there is a reason why the c99 tool spec says
> that the behaviour is unspecified if you have another librt in the
> lookup path, this is a fragile setup.

With all due respect, our experience in this regard does not reflect your statement.  We’ve been able to robustly support compilation and deployment across old versions of glibc without issue or fragility… for decades.  If anything, the glibc environments have always been some of the most robust and reliable resolution behavior (SGI IRIX and SYSV Solaris on the other hand…).  Most issues are mitigated by utilizing an isolated installation root (e.g., /usr/X11R6).

> i think if you want to avoid porting work then you should fix this
> name collision on your side.

Thank you for your advice, but I'd reject that on principle alone.  That advice also seems to be discounting the significant aforementioned costs.  It’d be a great way to lose customers and 3rd party integrations.  Porting is trivial in comparison and has been managed just fine for a very long time.  We’re good.

I suspect you’d be similarly disagreeable if you’d used ‘nsz@’ for 10+ years and were suggested to ‘fix’ it, i.e., change yours just because an external (non-critically impacting) hosting provider (e.g., Yahoo) felt 4 chars was plenty or decided to reserve ’nsz’ for their own convenience, even if some notion of a standard were involved.  Technically trivial, but changing would affect friends, family, banking, social media accounts, and (for sake of comparative argument) 2000 other sites you use regularly, including many government agencies that require paper-mailing any changes.  It just wouldn’t be worth the effort and you might even feel like you shouldn’t have to change it.  Of course, ymmv.

> of course musl can also be fixed to be more friendly to such usage,
> but it's not obvious how to tell if a librt.so is a user library or
> part of libc (or glibc in case glibc abi compat is used).

I’ll note that the compiler has no problem sorting it out consistently.  In fact it did so, or there would have been missing symbol errors during library or executable linking.  It’s really the dynamic linker using different logic that is causing this issue.  I suspect it’d work just fine to eliminate the reserved code block altogether for binaries compiled on a given system (not sure about cross-compiled).  That’s why I suggested the path of "if there’s a library listed in the ldd table that exists in the prescribed rpath directory or in the system's linker search path, that should be used".  Then it can fall back or not.  ld-musl is already deafult loaded, so it will typically just be missing symbols that weren't loaded. 

Cheers!
Sean


[-- Attachment #2: Type: text/html, Size: 7149 bytes --]