[musl] sysconf(_SC_NPROCESSORS_CONF) broken.

[musl] sysconf(_SC_NPROCESSORS_CONF) broken.

[Rob Landley]

NPROCESSORS_CONF is supposed to show total processors, NPROCESSORS_ONLN shows
available processors using the tasket mask sched_getaffinity()).

Musl is (uniquely) using the getaffinity() version for both. Neither glibc nor
bionic have that bug.

Test: my laptop has 4 processors:

$ taskset 7 nproc
3
$ taskset 7 nproc --all
4

With musl, both show 3.

Rob

[musl] Re: sysconf(_SC_NPROCESSORS_CONF) broken.

[Rob Landley]

On 7/27/22 01:28, Rob Landley wrote:
> NPROCESSORS_CONF is supposed to show total processors, NPROCESSORS_ONLN shows
> available processors using the tasket mask sched_getaffinity()).
> 
> Musl is (uniquely) using the getaffinity() version for both. Neither glibc nor
> bionic have that bug.
> 
> Test: my laptop has 4 processors:
> 
> $ taskset 7 nproc
> 3
> $ taskset 7 nproc --all
> 4
> 
> With musl, both show 3.

P.S. According to strace, devuan's nprocs --all is reading sysfs:

openat(AT_FDCWD, "/sys/devices/system/cpu",
O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
getdents64(3, /* 22 entries */, 32768)  = 656
getdents64(3, /* 0 entries */, 32768)   = 0

At a guess, counting the cpu[0-9]* entries? I looked at bionic's source and it
had a comment that x86 can use /proc/cpuinfo but arm only shows "available"
processors there, not total processors...

Rob

Re: [musl] Re: sysconf(_SC_NPROCESSORS_CONF) broken.

[Szabolcs Nagy]

* Rob Landley <rob@landley.net> [2022-07-27 01:34:41 -0500]:

> On 7/27/22 01:28, Rob Landley wrote:
> > NPROCESSORS_CONF is supposed to show total processors, NPROCESSORS_ONLN shows
> > available processors using the tasket mask sched_getaffinity()).
> > 
> > Musl is (uniquely) using the getaffinity() version for both. Neither glibc nor
> > bionic have that bug.
> > 
> > Test: my laptop has 4 processors:
> > 
> > $ taskset 7 nproc
> > 3
> > $ taskset 7 nproc --all
> > 4
> > 
> > With musl, both show 3.
> 
> P.S. According to strace, devuan's nprocs --all is reading sysfs:
> 
> openat(AT_FDCWD, "/sys/devices/system/cpu",
> O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
> fstat(3, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
> getdents64(3, /* 22 entries */, 32768)  = 656
> getdents64(3, /* 0 entries */, 32768)   = 0
> 
> At a guess, counting the cpu[0-9]* entries? I looked at bionic's source and it
> had a comment that x86 can use /proc/cpuinfo but arm only shows "available"
> processors there, not total processors...

i think the most recent thread on the topic:
https://www.openwall.com/lists/musl/2021/07/06/2

an older discussion is at
https://www.openwall.com/lists/musl/2019/03/15/5

Re: [musl] Re: sysconf(_SC_NPROCESSORS_CONF) broken.

[Jonathan Rajotte-Julien]

Hi,

> 
> i think the most recent thread on the topic:
> https://www.openwall.com/lists/musl/2021/07/06/2
> 
> an older discussion is at
> https://www.openwall.com/lists/musl/2019/03/15/5


FIY, we recently had to revisit our handling of all this.

See [1].

Which seems to be similar to what Vincent Donnefort, from [2], proposed.

A follow up to unify our approach is also available here[3].

[1] https://github.com/lttng/lttng-ust/commit/66dbdc3448a77043d0fd59f47b17e77a8d59fddb
[2] https://www.openwall.com/lists/musl/2021/07/06/2
[3] https://review.lttng.org/c/lttng-ust/+/8619

Cheers

Re: [musl] Re: sysconf(_SC_NPROCESSORS_CONF) broken.

[enh]

interesting... in case anyone's wondering why bionic didn't use
/sys/devices/system/cpu/possible, i didn't know it existed.

i'm pretty tempted to move us over to that. not because of the
distinction between "present" and "possible" -- i'm assuming over here
in mobile those two should always be the same anyway -- but because i
can just share code with "online" and do slightly less work than the
current directory scan...

On Wed, Jul 27, 2022 at 4:00 PM Jonathan Rajotte-Julien
<jonathan.rajotte-julien@efficios.com> wrote:
>
> Hi,
>
> >
> > i think the most recent thread on the topic:
> > https://www.openwall.com/lists/musl/2021/07/06/2
> >
> > an older discussion is at
> > https://www.openwall.com/lists/musl/2019/03/15/5
>
>
> FIY, we recently had to revisit our handling of all this.
>
> See [1].
>
> Which seems to be similar to what Vincent Donnefort, from [2], proposed.
>
> A follow up to unify our approach is also available here[3].
>
> [1] https://github.com/lttng/lttng-ust/commit/66dbdc3448a77043d0fd59f47b17e77a8d59fddb
> [2] https://www.openwall.com/lists/musl/2021/07/06/2
> [3] https://review.lttng.org/c/lttng-ust/+/8619
>
> Cheers

Re: [musl] Re: sysconf(_SC_NPROCESSORS_CONF) broken.

[enh]

done: https://android-review.googlesource.com/c/platform/bionic/+/2168844

what's the worst that could happen?


On Thu, Jul 28, 2022 at 10:27 AM enh <enh@google.com> wrote:
>
> interesting... in case anyone's wondering why bionic didn't use
> /sys/devices/system/cpu/possible, i didn't know it existed.
>
> i'm pretty tempted to move us over to that. not because of the
> distinction between "present" and "possible" -- i'm assuming over here
> in mobile those two should always be the same anyway -- but because i
> can just share code with "online" and do slightly less work than the
> current directory scan...
>
> On Wed, Jul 27, 2022 at 4:00 PM Jonathan Rajotte-Julien
> <jonathan.rajotte-julien@efficios.com> wrote:
> >
> > Hi,
> >
> > >
> > > i think the most recent thread on the topic:
> > > https://www.openwall.com/lists/musl/2021/07/06/2
> > >
> > > an older discussion is at
> > > https://www.openwall.com/lists/musl/2019/03/15/5
> >
> >
> > FIY, we recently had to revisit our handling of all this.
> >
> > See [1].
> >
> > Which seems to be similar to what Vincent Donnefort, from [2], proposed.
> >
> > A follow up to unify our approach is also available here[3].
> >
> > [1] https://github.com/lttng/lttng-ust/commit/66dbdc3448a77043d0fd59f47b17e77a8d59fddb
> > [2] https://www.openwall.com/lists/musl/2021/07/06/2
> > [3] https://review.lttng.org/c/lttng-ust/+/8619
> >
> > Cheers

Re: [musl] Re: sysconf(_SC_NPROCESSORS_CONF) broken.

[NRK]

On Fri, Jul 29, 2022 at 11:42:19AM -0700, enh wrote:
> done: https://android-review.googlesource.com/c/platform/bionic/+/2168844

	if (getline(&line, &allocated_size, fp) != -1) {
	  cpu_count = GetCpuCountFromString(line);
	  free(line);
	}

Shouldn't the buffer be `free`-ed regardless or weather getline fails or
not? The getline(3) manpage suggests so:

> If *lineptr is set to NULL before the call, then getline() will
> allocate a buffer for storing the line. This buffer should be freed by
> the user program even if getline() failed.

- NRK

Re: [musl] Re: sysconf(_SC_NPROCESSORS_CONF) broken.

[enh]

On Fri, Jul 29, 2022 at 11:53 AM NRK <nrk@disroot.org> wrote:
>
> On Fri, Jul 29, 2022 at 11:42:19AM -0700, enh wrote:
> > done: https://android-review.googlesource.com/c/platform/bionic/+/2168844
>
>         if (getline(&line, &allocated_size, fp) != -1) {
>           cpu_count = GetCpuCountFromString(line);
>           free(line);
>         }
>
> Shouldn't the buffer be `free`-ed regardless or weather getline fails or
> not? The getline(3) manpage suggests so:

yeah, i almost changed that in passing (because i worried that someone
would copy & paste for a loop, where you don't want to free() on every
iteration) but assumed (incorrectly!) that earlier me had checked.

fixed now, thanks:
https://android-review.googlesource.com/c/platform/bionic/+/2170542

> > If *lineptr is set to NULL before the call, then getline() will
> > allocate a buffer for storing the line. This buffer should be freed by
> > the user program even if getline() failed.

it looks like it's not entirely theoretical either --- a quick look at
bionic's [OpenBSD] getdelim() implementation shows that it can return
failure on the first call but leaving buf non-null (you'd need the
first allocation to succeed, and need a second allocation that fails;
which -- along with the fact -- is probably why i don't remember ever
seeing this in practice).

i'll send a patch to man7.org to mention this in the ERRORS section,
because i didn't see it up top when i looked before getting to the
part of your mail with a ctrl-f'able quote :-)

> - NRK

Re: [musl] Re: sysconf(_SC_NPROCESSORS_CONF) broken.

[enh]

[-- Attachment #1: Type: text/plain, Size: 1727 bytes --]

On Tue, Jul 26, 2022 at 11:27 PM Rob Landley <rob@landley.net> wrote:

> On 7/27/22 01:28, Rob Landley wrote:
> > NPROCESSORS_CONF is supposed to show total processors, NPROCESSORS_ONLN
> shows
> > available processors using the tasket mask sched_getaffinity()).
> >
> > Musl is (uniquely) using the getaffinity() version for both. Neither
> glibc nor
> > bionic have that bug.
> >
> > Test: my laptop has 4 processors:
> >
> > $ taskset 7 nproc
> > 3
> > $ taskset 7 nproc --all
> > 4
> >
> > With musl, both show 3.
>
> P.S. According to strace, devuan's nprocs --all is reading sysfs:
>
> openat(AT_FDCWD, "/sys/devices/system/cpu",
> O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
> fstat(3, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
> getdents64(3, /* 22 entries */, 32768)  = 656
> getdents64(3, /* 0 entries */, 32768)   = 0
>
> At a guess, counting the cpu[0-9]* entries? I looked at bionic's source
> and it
> had a comment that x86 can use /proc/cpuinfo but arm only shows "available"
> processors there, not total processors...
>

yeah, bionic uses /sys/devices/system/cpu and
/sys/devices/system/cpu/online and we haven't had to touch that code since
we switched to doing it that way.

note that my claim in that comment about x86 may not be strictly correct.
i'm not sure i'd ever seen an x86 cpu powering down cores at that time, and
i'm not sure i have since either, but i assume that -- if such a thing is
possible -- they'd look the same as arm? i'd guess the comment should
actually say something like "on systems where cores are hotplugged", and
that this probably can happen on intel too (i'm guessing their recent
big.LITTLE equivalent will have brought this kind of thing to their world
too).


> Rob
>

[-- Attachment #2: Type: text/html, Size: 2430 bytes --]

[musl] dynamic linker is capturing "reserved" library names erroneously

[Christopher Sean Morrison]

[-- Attachment #1: Type: text/plain, Size: 4766 bytes --]


The gist of this bug report / change request is best demonstrated by the following code setup:

$ cat >> librt.cpp
#include <stdio.h>
void foo(void) { printf("hello\n"); }
$ cat >> test.cpp
#include <stdio.h>
int main(int ac, char *av[]) { extern void foo(void); foo(); return 0; }
$ g++ -shared -o librt.so librt.cpp
$ g++ -o test test.cpp -L. -lrt -Wl,-rpath=.
$ ldd test
	/lib/ld-musl-x86_64.so.1 (0x7f35cf52a000)
	librt.so => /lib/ld-musl-x86_64.so.1 (0x7f35cf52a000)
	libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f35cf52a000)
Error relocating test: _Z3foov: symbol not found
$ ./test
Error relocating ./test: _Z3foov: symbol not found

In brief, ld appears to be capturing the resolution of librt as being satisfied by ld-musl-x86_64 at runtime despite the -L resolving correctly at compilation time (and rpath is ignored).  This naturally results in runtime symbol not found message(s) for any symbols in such a named library.  The offending code in musl appears to be https://git.musl-libc.org/cgit/musl/tree/ldso/dynlink.c#n1011 <https://git.musl-libc.org/cgit/musl/tree/ldso/dynlink.c#n1011>

First consideration, the code seems to take a position that those library names are somehow universally reserved and I believe that to be incorrect.  There was mention of reserved lib names long ago specifically regarding the behavior of the “c89”  and “c99” C compiler CLI in their posix.1 man pages, but those did not carry over to prescribing dynamic library behavior, C++ behavior, or other any other aspects of the standard as far as I’m aware.  Perhaps a citation can be provided, but I was unable to find a relevant mention of ‘rt’, “librt”, “-lr”, etc, (or any of the other libs from line 1011) in the latest version of the standard.

Second consideration, the block’s preceding comment seems to document the primary intention as being to reduce app porting burden where -lm, -lc, and friends have long-since been embedded in build systems.  Given those libraries are all combined in musl’s libc implementation, that seems reasonable.  Automatically binding requests for -lm to musl’s libc, for example, certainly makes sense to ease porting.

Given those two considerations, I would suggest + request that the library resolution behavior be changed so that they are not captured reserved names but merely a fallback when normal searching would otherwise result in not-found.  That is, move the logic in dynlink.c so that it happens later, after typical searching as needed.  That way, the encoded rpath and libraries specified at link time will be respected, and build systems specifying -lm will still automatically resolve to musl’s libc (unless there really is a libm).

My expectation is that user applications should be able to specify any library name (even libc), link against it, and resolve to it at runtime as in the example above.  The gnu and bsd dynamic linkers do not make any presumption or restriction on library name resolution (at least not any more afaik), even with regards to auto-linking standard libraries.  Changing this behavior would also address a number of related reports I came across regarding this issue (e.g., search "error relocating" "symbol not found" musl <https://www.google.com/search?q=%22error+relocating%22+%22symbol+not+found%22+musl&client=safari&rls=en&biw=1413&bih=1095&sxsrf=ALiCzsbL2SRfDzU58gJPcpF0YXZM1Q2WwQ%3A1658961343157&ei=v73hYqOYCY2s5NoP8rmEwA8&ved=0ahUKEwjj87fMkJr5AhUNFlkFHfIcAfgQ4dUDCA0&uact=5&oq=%22error+relocating%22+%22symbol+not+found%22+musl&gs_lcp=Cgdnd3Mtd2l6EAMyBggAEB4QFjIFCAAQhgMyBQgAEIYDMgUIABCGAzoFCAAQgAQ6BggAEB4QB0oECEEYAUoECEYYAFD-A1i2HWD8HWgBcAB4AIABsQGIAdAEkgEDNC4ymAEAoAEBwAEB&sclient=gws-wiz>).

For background context, I maintain BRL-CAD, a large open source CAD system that has been in development for over 40 years.  BRL-CAD’s flagship API with thousands of integrations around the world is the “librt” ray tracing library.  With dev going all the way back to 1983, it predates both ANSI C and POSIX.  Renaming isn’t likely anytime soon as it would be exceptionally cost-prohibitive and impacts so many other codes in production use.  We’ve maintained BRL-CAD’s portability across dozens of architectures, operating systems, and compilation environments over the years, and librt conflicts are not new.  What’s unique/novel here is musl’s runtime linker override behavior that hasn’t been seen for quite some time.  I found workarounds we can employ, but hopefully behavior can be improved to benefit future musl development.

Thank you for everyone’s efforts on an alternative to the established, and thank you for consideration of this issue.

Cheers!
Sean Morrsion
BRL-CAD


[-- Attachment #2: Type: text/html, Size: 9566 bytes --]

Re: [musl] dynamic linker is capturing "reserved" library names erroneously

[Szabolcs Nagy]

* Christopher Sean Morrison <brlcad@mac.com> [2022-07-27 19:06:24 -0400]:
> First consideration, the code seems to take a position that those library names are somehow universally reserved and I believe that to be incorrect.

the way libraries are looked up is not specified in detail and
the standard gives the permission to implementations to decide.

some library names are special, but in conforming environment
this is only observable via the c99 tool so that's where this
is specified, the language does not know anything about libraries.

one exception is dlopen which has

  If file contains a <slash> character, the file argument is used
  as the pathname for the file. Otherwise, file is used in an
  implementation-defined manner to yield a pathname.

so if you want reliable behaviour you need a path with /
e.g. in your example ./librt.so would have worked fine.

i believe the reason musl has to special case the names in ld.so
is to be able to load libraries linked against glibc. another
reason is to support looking up symbols in librt etc via dlsym.

> For background context, I maintain BRL-CAD, a large open source CAD system that has been in development for over 40 years.
> BRL-CAD’s flagship API with thousands of integrations around the world is the “librt” ray tracing library.

this would not work eg on older glibc if one of your dependencies
used clock_gettime. there is a reason why the c99 tool spec says
that the behaviour is unspecified if you have another librt in the
lookup path, this is a fragile setup.

i think if you want to avoid porting work then you should fix this
name collision on your side.

of course musl can also be fixed to be more friendly to such usage,
but it's not obvious how to tell if a librt.so is a user library or
part of libc (or glibc in case glibc abi compat is used).

Re: [musl] dynamic linker is capturing "reserved" library names erroneously

[Christopher Sean Morrison]

[-- Attachment #1: Type: text/plain, Size: 5309 bytes --]


> On Jul 28, 2022, at 5:18 AM, Szabolcs Nagy <nsz@port70.net> wrote:
> 
> * Christopher Sean Morrison <brlcad@mac.com> [2022-07-27 19:06:24 -0400]:
>> First consideration, the code seems to take a position that those library names are somehow universally reserved and I believe that to be incorrect.
> 
> the way libraries are looked up is not specified in detail and
> the standard gives the permission to implementations to decide.

This is my understanding as well.  That’s why I characterized it as incorrect with respect to the existing implementation identifying them as “reserved”, as well as with regard to the simplified example provided.  Posix does not reserve them, the compiler+linker behaves as expected, but it’s musl’s dynamic linker implementation that currently does not behave as I would intuitively expect.

> some library names are special, but in conforming environment
> this is only observable via the c99 tool so that's where this
> is specified, the language does not know anything about libraries.

It's observable when an application is run, and posix certainly describes a multitude of aspects of runtime behavior.  The 2017 update and precursors could have called out such reservations, and they do not.  By that same logic, I could just as well create a ‘librt’ library using fort77.  That’s still in posix, but its man page only calls out undefined behavior on the ‘libf.a’ library.  In that case, the runtime behavior would still be incorrect…  Those pages really only referred to just those specific tools, their behavior, and really has no bearing on resolving symbols at runtime — it’s entirely musl’s dynamic linker prerogative.

> i believe the reason musl has to special case the names in ld.so

> is to be able to load libraries linked against glibc. another
> reason is to support looking up symbols in librt etc via dlsym.

That is why I proposed a solution of deferring the capture until after searching, so it will both satisfy that desire to resolve symbols for when -lm, -lr, -lpthread, etc are specified while also supporting the general case where such libraries actually do exist and are intended to be used.
 
>> For background context, I maintain BRL-CAD, a large open source CAD system that has been in development for over 40 years.
>> BRL-CAD’s flagship API with thousands of integrations around the world is the “librt” ray tracing library.
> 
> this would not work eg on older glibc if one of your dependencies
> used clock_gettime. there is a reason why the c99 tool spec says
> that the behaviour is unspecified if you have another librt in the
> lookup path, this is a fragile setup.

With all due respect, our experience in this regard does not reflect your statement.  We’ve been able to robustly support compilation and deployment across old versions of glibc without issue or fragility… for decades.  If anything, the glibc environments have always been some of the most robust and reliable resolution behavior (SGI IRIX and SYSV Solaris on the other hand…).  Most issues are mitigated by utilizing an isolated installation root (e.g., /usr/X11R6).

> i think if you want to avoid porting work then you should fix this
> name collision on your side.

Thank you for your advice, but I'd reject that on principle alone.  That advice also seems to be discounting the significant aforementioned costs.  It’d be a great way to lose customers and 3rd party integrations.  Porting is trivial in comparison and has been managed just fine for a very long time.  We’re good.

I suspect you’d be similarly disagreeable if you’d used ‘nsz@’ for 10+ years and were suggested to ‘fix’ it, i.e., change yours just because an external (non-critically impacting) hosting provider (e.g., Yahoo) felt 4 chars was plenty or decided to reserve ’nsz’ for their own convenience, even if some notion of a standard were involved.  Technically trivial, but changing would affect friends, family, banking, social media accounts, and (for sake of comparative argument) 2000 other sites you use regularly, including many government agencies that require paper-mailing any changes.  It just wouldn’t be worth the effort and you might even feel like you shouldn’t have to change it.  Of course, ymmv.

> of course musl can also be fixed to be more friendly to such usage,
> but it's not obvious how to tell if a librt.so is a user library or
> part of libc (or glibc in case glibc abi compat is used).

I’ll note that the compiler has no problem sorting it out consistently.  In fact it did so, or there would have been missing symbol errors during library or executable linking.  It’s really the dynamic linker using different logic that is causing this issue.  I suspect it’d work just fine to eliminate the reserved code block altogether for binaries compiled on a given system (not sure about cross-compiled).  That’s why I suggested the path of "if there’s a library listed in the ldd table that exists in the prescribed rpath directory or in the system's linker search path, that should be used".  Then it can fall back or not.  ld-musl is already deafult loaded, so it will typically just be missing symbols that weren't loaded. 

Cheers!
Sean


[-- Attachment #2: Type: text/html, Size: 7149 bytes --]

Re: [musl] dynamic linker is capturing "reserved" library names erroneously

[Rich Felker]

On Thu, Jul 28, 2022 at 04:39:07PM -0400, Christopher Sean Morrison wrote:
> 
> > On Jul 28, 2022, at 5:18 AM, Szabolcs Nagy <nsz@port70.net> wrote:
> > 
> > * Christopher Sean Morrison <brlcad@mac.com> [2022-07-27 19:06:24 -0400]:
> >> First consideration, the code seems to take a position that those
> >> library names are somehow universally reserved and I believe that
> >> to be incorrect.

As others have noted, the behavior of -lrt is defined as linking the
standard library for realtime interfaces, not a user-defined library
by that name. The specification goes so far as to say, regarding use
of -L to try to override that:

  If a directory specified by a -L option contains files with names
  starting with any of the strings "libc.", "libl.", "libpthread.",
  "libm.", "librt.", "libtrace.", "libxnet.", or "liby.", the results
  are unspecified.

You may think it seems like it should be okay to use the name "librt"
if you're not linking the standard -lrt, but imagine what would happen
if you were on an implementation where some of the standard functions
were defined in a discrete librt.so rather than all integrated in
libc.so like musl does: any *other* library that got pulled in
(possibly even one of the standard ones) that depends on librt would
end up getting that DT_NEEDED reference resolved to your library by
the same name, rather than to the standard one, and you would end up
with runtime link errors resolving the missing symbols.

I'm really surprised you haven't run into any problems with this clash
before. Surely I'd think someone would have tried to use clock_gettime
(the only modern way to get the current system clock time) in a
program that also needs to use your library, and on many historical
implementations, including glibc up until recently, you couldn't get
clock_gettime without -lrt (this the horrible syscall() hacks all over
the place to use the syscall directly).

> > i believe the reason musl has to special case the names in ld.so
> 
> > is to be able to load libraries linked against glibc. another
> > reason is to support looking up symbols in librt etc via dlsym.
> 
> That is why I proposed a solution of deferring the capture until
> after searching, so it will both satisfy that desire to resolve
> symbols for when -lm, -lr, -lpthread, etc are specified while also
> supporting the general case where such libraries actually do exist
> and are intended to be used.

This would cause very bad things to happen if someone copied the glibc
versions of those libraries around with a glibc-linked program they
were trying to use with ABI-compat, not aware that they were actually
part of glibc and not third-party libraries the application needed.

There is something of a long-term direction to decouple the ABI-compat
stuff from musl, and I'm not sure if it would make sense to unreserve
the names at the same time. A proposal to do this, like any proposal
for supporting nonstandard functionality that could have unforseen
consequences, would need to involve research into what those
consequences might be, if any. It might end up being okay to do
something like first hardening protection against loading glibc-linked
libraries by those names (assuming they're the corresponding parts of
the standard library from glibc) and then doing like you said, only
using them as fallbacks after search.

Rich

Re: [musl] dynamic linker is capturing "reserved" library names erroneously

[Christopher Sean Morrison]

Hi Rich,

> On Jul 28, 2022, at 8:07 PM, Rich Felker <dalias@libc.org> wrote:
> 
> As others have noted, the behavior of -lrt is defined as linking the standard library for realtime interfaces, not a user-defined library by that name.

Others have not stated that or I would have called the specific claim into scrutiny.  Unless my junk mail filter is being aggressive, there’s been only one reply prior.  As to the claim, though, that behavior is only mentioned in regards to the c99 utility and its compile-time link editor resolution.  There is no issue raised with regards to the c99 utility or compile-time symbol resolution.

Both gcc and clang indeed resolve symbols on musl at compile time without issue and consistent with their behavior on other platforms.  My example showed that intentionally.  The dynamic linker is what’s at issue and the standard says absolutely nothing about it from what I can see.

> The specification goes so far as to say, regarding use of -L to try to override that:
> 
>  If a directory specified by a -L option contains files with names
>  starting with any of the strings "libc.", "libl.", "libpthread.",
>  "libm.", "librt.", "libtrace.", "libxnet.", or "liby.", the results
>  are unspecified.

Again, you've quoted the ‘c99’ man page, yes?  As already mentioned, that page does not prescribe or imply any behavior on the dynamic linker or runtime resolution.  If defines how the c99 utility is to behave.  Moreover, if that’s to be taken as universal, it’s inconsistent with the ‘fort77’ man page that only calls out -L with a libf.a as being unspecified.  My example could just as well created librt using fort77 and still observed the same result.  The dynamic linker cannot satisfy both man pages without being inconsistent with one of them.   

> You may think it seems like it should be okay to use the name “librt" if you're not linking the standard -lrt

I do, and it’s been done successfully for over 40 years now across nearly every desktop and server platform in existence… The magnitude and implications of that timespan and portability exposure really seems to get lost on folks.  :)

> but imagine what would happen if you were on an implementation where some of the standard functions were defined in a discrete librt.so rather than all integrated in libc.so like musl does:

We don’t have to imagine as that was a common library configuration through the late 90’s and early 00’s.  As already mentioned, most issues are managed through path ordering (or one is intentionally overriding a library).

> This would cause very bad things to happen if someone copied the glibc versions of those libraries around with a glibc-linked program they were trying to use with ABI-compat, not aware that they were actually part of glibc and not third-party libraries the application needed.

This is a valid concern, though I would expect such a glibc-linked program to have an rpath set such that it’s not simply defaulting to DT_RUNPATH, ldconfig, or default paths.  Modern versions of GNU ld in fact require rpaths be set for relocation to work.  I think you’ve also described a situation where someone system-installed rather carelessly — installing an overriding librt/libm/libc/etc into /usr/lib is going to result in bad things happening on just about any platform and is not what’s being suggested.  Bad things will happen today on just about any Linux system if you copy an incompatible glibc from one system to another and install it in a location where it takes precedence.

It seems like the conflict in play is wanting glibc-compiled applications to just work when run on musl, hence the main reason why it appears to aggressively resolve the various libs at runtime regardless of the set rpath.  If you removed the dynamic override outright then one couldn’t just copy an executable over and expect it to work (would need to be recompiled or have their glibc installed for rpath to resolve), but that’s true today on other platforms.  It would behave like GNU ld.  It’s just convenient that glibc has been mostly ABI-compatible for a while, but that’s certainly not always the case and shouldn’t ever be assumed (imho).

> There is something of a long-term direction to decouple the ABI-compat stuff from musl, and I'm not sure if it would make sense to unreserve the names at the same time. A proposal to do this, like any proposal for supporting nonstandard functionality that could have unforseen consequences, would need to involve research into what those consequences might be, if any. It might end up being okay to do something like first hardening protection against loading glibc-linked libraries by those names (assuming they're the corresponding parts of the standard library from glibc) and then doing like you said, only using them as fallbacks after search.

I certainly wasn’t expecting this to be a quick and easy can of worms, but appreciate the consideration.  I maintain a simple expectation (these days) that users should be able to compile and link against libraries named anything, hence the simple program and library example.  The example demonstrably works on linux, mac windows, hpux, *bsd, haiku, etc.. and with good reason and implications.  This was mostly about opening a dialog for feedback.  Thanks for receiving and responding to the report.

Cheers!
Sean

p.s. for anyone finding this thread in the future couple workarounds are LD_PRELOAD or specify full paths when linking. for cmake, set CMP0060 to NEW.