Symbol versioning approximation trips on compat symbols

Symbol versioning approximation trips on compat symbols

[Florian Weimer]

On what appears to be current Alpine Linux (musl-1.1.19-r10), the
following reproducer

######################################################################
cat > symver.c <<EOF
void
compat_function (void)
{
}
__asm__ (".symver compat_function,compat_function@SYMVER");

void
call_compat_function (void)
{
  return compat_function ();
}
EOF

echo "SYMVER { };" > symver.map

cat > main.c <<EOF
extern void call_compat_function (void);

int
main (void)
{
  call_compat_function ();
}
EOF

gcc -fpic -shared -o symver.so -Wl,--version-script=symver.map symver.c
gcc -Wl,--rpath=. -o main main.c symver.so
######################################################################

fails with:

$ ./main
Error relocating ./symver.so: compat_function: symbol not found

The problem is the compatibility symbol (one @ instead of @@).  The
dynamic linker is supposed to ignore the difference between the two, the
default vs non-default version only matters to the link editor when
processing an undefined symbol without a symbol version.

In my case, I do not need symbol interposition and therefore can work
around this, but I wonder if there is some sort of approved compile-time
or link-time check to detect this issue.  Unfortunately, the Alpine
Linux toolchain (and part of the system) is built *with* symbol
versioning support, so this does not appear to be straightforward.

The actual application does not need to make the symbol interposable, so
I can use a hidden alias within the DSO for PLT avoidance (and more
configure checks to disable all this on targets which do not support
*that*).

Thanks,
Florian

Re: Symbol versioning approximation trips on compat symbols

[Rich Felker]

On Mon, Jan 21, 2019 at 06:57:53PM +0100, Florian Weimer wrote:
> On what appears to be current Alpine Linux (musl-1.1.19-r10), the
> following reproducer
> 
> ######################################################################
> cat > symver.c <<EOF
> void
> compat_function (void)
> {
> }
> __asm__ (".symver compat_function,compat_function@SYMVER");
> 
> void
> call_compat_function (void)
> {
>   return compat_function ();
> }
> EOF
> 
> echo "SYMVER { };" > symver.map
> 
> cat > main.c <<EOF
> extern void call_compat_function (void);
> 
> int
> main (void)
> {
>   call_compat_function ();
> }
> EOF
> 
> gcc -fpic -shared -o symver.so -Wl,--version-script=symver.map symver.c
> gcc -Wl,--rpath=. -o main main.c symver.so
> ######################################################################
> 
> fails with:
> 
> $ ./main
> Error relocating ./symver.so: compat_function: symbol not found
> 
> The problem is the compatibility symbol (one @ instead of @@).  The
> dynamic linker is supposed to ignore the difference between the two, the
> default vs non-default version only matters to the link editor when
> processing an undefined symbol without a symbol version.
> 
> In my case, I do not need symbol interposition and therefore can work
> around this, but I wonder if there is some sort of approved compile-time
> or link-time check to detect this issue.  Unfortunately, the Alpine
> Linux toolchain (and part of the system) is built *with* symbol
> versioning support, so this does not appear to be straightforward.
> 
> The actual application does not need to make the symbol interposable, so
> I can use a hidden alias within the DSO for PLT avoidance (and more
> configure checks to disable all this on targets which do not support
> *that*).

The same issue came up before with libgcc defining and referencing a
non-default-version symbol for some weird compatibility hack. I don't
remember the details but Szabolcs Nagy was involved in investigating
and might. In any case, the root cause is that musl's dynamic linker
does not support symbol versioning; for the sake of being able to load
libraries that were build with versioning, it always resolves a symbol
to the "latest"/default version, the same as ld would do at link time.
Normally this is the right thing as long as you don't actually have
things that were linked against an old incompatible version, but it
also breaks explicit linking to a particular version as in your
example above.

The right fix is probably to add support for symbol version matching
in the dynamic linker. Unfortunately this involves some extra logic in
the extreme hot paths, so it's hard to make the cost unobservably low,
and last I checked some members of the community were opposed to it on
ideological grounds. If there's a good need for it (and I think just
avoiding silent breakage of third-party libs using versioning and
intending for it to work is a fairly good one already), support can be
added, but doing it without negative impact is a pretty big task.

Rich

Re: Symbol versioning approximation trips on compat symbols

[u-uy74]

On Wed, Jan 23, 2019 at 08:43:40PM -0500, Rich Felker wrote:
> The same issue came up before with libgcc defining and referencing a
> non-default-version symbol for some weird compatibility hack. I don't
> remember the details but Szabolcs Nagy was involved in investigating
> and might. In any case, the root cause is that musl's dynamic linker
> does not support symbol versioning; for the sake of being able to load
> libraries that were build with versioning, it always resolves a symbol
> to the "latest"/default version, the same as ld would do at link time.
> Normally this is the right thing as long as you don't actually have
> things that were linked against an old incompatible version, but it
> also breaks explicit linking to a particular version as in your
> example above.
> 
> The right fix is probably to add support for symbol version matching
> in the dynamic linker. Unfortunately this involves some extra logic in
> the extreme hot paths, so it's hard to make the cost unobservably low,
> and last I checked some members of the community were opposed to it on
> ideological grounds. If there's a good need for it (and I think just

IMVHO symbol versioning is basically aimed to hide the complexity of
the evolution of libraries, to make certain usage cases "just work".

OTOH it does not reduce the complexity under the hood but rather adds some
extra of it. That's why I see its impact as double negative, postulating
more complex tools and reducing the capacity of fellow integrators to
analyze their systems.

> avoiding silent breakage of third-party libs using versioning and
> intending for it to work is a fairly good one already), support can be
> added, but doing it without negative impact is a pretty big task.

This cost itself (of adding the support) means that some other useful
changes in musl will be missing at _every_ given time in the future
development. Is this workaround, for the sake of libraries who do unusual
hacks, really so important?

Rune

Re: Symbol versioning approximation trips on compat symbols

[Florian Weimer]

* u-uy:

> IMVHO symbol versioning is basically aimed to hide the complexity of
> the evolution of libraries, to make certain usage cases "just work".
>
> OTOH it does not reduce the complexity under the hood but rather adds some
> extra of it. That's why I see its impact as double negative, postulating
> more complex tools and reducing the capacity of fellow integrators to
> analyze their systems.

That's certainly a valid position to take.  But the reality is that all
toolchains that target musl have symbol versioning support built into
them today, just not in the dynamic linker.  It would be far easier
unrelated upstreams if the toolchain simply did not support symbol
versioning at all (because the dynamic linker does not support it, so
it's not usable in each place where it is required anyway).  Then a
simple check for .symver support in the assembler would tell us whether
we can symbol versionining or not.

I have no idea how this current support is supposed to work,
particularly with cross-compiling.  Does everyone have to write
configure hacks for *-musl targets to disable symbol versioning support?
That goes against the mantra of testing what you use (and avoid
hard-coded knowledge about specific versions/targets).

Thanks,
Florian

Re: Symbol versioning approximation trips on compat symbols

[Szabolcs Nagy]

* Rich Felker <dalias@libc.org> [2019-01-23 20:43:40 -0500]:
> On Mon, Jan 21, 2019 at 06:57:53PM +0100, Florian Weimer wrote:
> > On what appears to be current Alpine Linux (musl-1.1.19-r10), the
> > following reproducer
> > 
> > ######################################################################
> > cat > symver.c <<EOF
> > void
> > compat_function (void)
> > {
> > }
> > __asm__ (".symver compat_function,compat_function@SYMVER");
> > 
> > void
> > call_compat_function (void)
> > {
> >   return compat_function ();
> > }
> > EOF
> > 
> > echo "SYMVER { };" > symver.map
> > 
> > cat > main.c <<EOF
> > extern void call_compat_function (void);
> > 
> > int
> > main (void)
> > {
> >   call_compat_function ();
> > }
> > EOF
> > 
> > gcc -fpic -shared -o symver.so -Wl,--version-script=symver.map symver.c
> > gcc -Wl,--rpath=. -o main main.c symver.so
> > ######################################################################
> > 
> > fails with:
> > 
> > $ ./main
> > Error relocating ./symver.so: compat_function: symbol not found
> > 
> > The problem is the compatibility symbol (one @ instead of @@).  The
> > dynamic linker is supposed to ignore the difference between the two, the
> > default vs non-default version only matters to the link editor when
> > processing an undefined symbol without a symbol version.
> > 
> > In my case, I do not need symbol interposition and therefore can work
> > around this, but I wonder if there is some sort of approved compile-time
> > or link-time check to detect this issue.  Unfortunately, the Alpine
> > Linux toolchain (and part of the system) is built *with* symbol
> > versioning support, so this does not appear to be straightforward.
> > 
> > The actual application does not need to make the symbol interposable, so
> > I can use a hidden alias within the DSO for PLT avoidance (and more
> > configure checks to disable all this on targets which do not support
> > *that*).
> 
> The same issue came up before with libgcc defining and referencing a
> non-default-version symbol for some weird compatibility hack. I don't
> remember the details but Szabolcs Nagy was involved in investigating
> and might. In any case, the root cause is that musl's dynamic linker
> does not support symbol versioning; for the sake of being able to load
> libraries that were build with versioning, it always resolves a symbol
> to the "latest"/default version, the same as ld would do at link time.
> Normally this is the right thing as long as you don't actually have
> things that were linked against an old incompatible version, but it
> also breaks explicit linking to a particular version as in your
> example above.

the libgcc_s.so.1 issue on x86 was an extern object symbol (used by
function-multi-versioning ifuncs) that is initialized by a ctor.

but it turned out to be broken (ifunc resolvers may run before
relocations for the extern object are processed), so the symbol
was removed (moved to libgcc.a), but a compat symbol (@) was
kept around and the ctor of libgcc_s.so.1 still references it.

so musl cannot load libgcc_s.so.1 since there is no default
version (@@) of the symbol. (in a musl based gcc this is fixed:
there is no ifunc support or compat issue anyway
https://gcc.gnu.org/ml/gcc-patches/2016-11/msg01125.html
you would expect that it's enough to build gcc for
--disable-gnu-indirect-function or --disable-symvers
but even with those the compat symbol is there...)

i wonder what is the use-case for using a compat symbol without
introducing a new default version for the symbol in general?

> 
> The right fix is probably to add support for symbol version matching
> in the dynamic linker. Unfortunately this involves some extra logic in
> the extreme hot paths, so it's hard to make the cost unobservably low,
> and last I checked some members of the community were opposed to it on
> ideological grounds. If there's a good need for it (and I think just
> avoiding silent breakage of third-party libs using versioning and
> intending for it to work is a fairly good one already), support can be
> added, but doing it without negative impact is a pretty big task.
> 
> Rich

Re: Symbol versioning approximation trips on compat symbols

[Florian Weimer]

* Szabolcs Nagy:

> but it turned out to be broken (ifunc resolvers may run before
> relocations for the extern object are processed), so the symbol
> was removed (moved to libgcc.a), but a compat symbol (@) was
> kept around and the ctor of libgcc_s.so.1 still references it.

I assume we cannot use a hidden alias to eliminate the symbolic
reference because it is a data symbol, so copy relocations are a
possibility and with the hidden alias, the constructor would update the
wrong object?

> i wonder what is the use-case for using a compat symbol without
> introducing a new default version for the symbol in general?

A compat symbol can be implemented by a different library, and
applications can link to the symbol and get it from the new library.

Mostly it is just for documenting intent, marking the symbol as
deprecated.  It's still relatively easy to link to the symbol from new
binaries (even without resorting to dlvsym).

Thanks,
Florian

Re: Symbol versioning approximation trips on compat symbols

[Rich Felker]

On Thu, Jan 24, 2019 at 12:18:12PM +0100, Florian Weimer wrote:
> * Szabolcs Nagy:
> 
> > but it turned out to be broken (ifunc resolvers may run before
> > relocations for the extern object are processed), so the symbol
> > was removed (moved to libgcc.a), but a compat symbol (@) was
> > kept around and the ctor of libgcc_s.so.1 still references it.
> 
> I assume we cannot use a hidden alias to eliminate the symbolic
> reference because it is a data symbol, so copy relocations are a
> possibility and with the hidden alias, the constructor would update the
> wrong object?

I believe we had a perfectly workable alternate solution using
something like that, but someone on the upstream side wasn't willing
to change it.

> > i wonder what is the use-case for using a compat symbol without
> > introducing a new default version for the symbol in general?
> 
> A compat symbol can be implemented by a different library, and
> applications can link to the symbol and get it from the new library.
> 
> Mostly it is just for documenting intent, marking the symbol as
> deprecated.  It's still relatively easy to link to the symbol from new
> binaries (even without resorting to dlvsym).

For what it's worth, compat symbols would have let us remove symbols
that shouldn't have been put in musl, such as lchmod (which confuses
broken apps which wrongly expect that, if it exists, it should work)
and perhaps all the LFS64 symbols (which should be present for
binary-compat but not at ld-time so that configure doesn't detect and
try to use them). However, doing it that way would put a symver table
in musl libc.so, resulting in lots of other behaviors we almost surely
don't want. It does shed some light on possible motivations, though, I
think.

Rich

Re: Symbol versioning approximation trips on compat symbols

[u-uy74]

On Sat, Jan 26, 2019 at 11:04:24PM -0500, Rich Felker wrote:
> For what it's worth, compat symbols would have let us remove symbols
> that shouldn't have been put in musl, such as lchmod (which confuses
> broken apps which wrongly expect that, if it exists, it should work)

For what my integrator perspective is worth, exposing brokenness instead
of catering for it is a Good Thing.

Feature detection is one of the typical areas being messed up, in numerous
softwares.

Thanks musl for exposing the pitfalls and forcing upstreams think better.

Rune

Re: Symbol versioning approximation trips on compat symbols

[Florian Weimer]

* u-uy:

> On Sat, Jan 26, 2019 at 11:04:24PM -0500, Rich Felker wrote:
>> For what it's worth, compat symbols would have let us remove symbols
>> that shouldn't have been put in musl, such as lchmod (which confuses
>> broken apps which wrongly expect that, if it exists, it should work)
>
> For what my integrator perspective is worth, exposing brokenness instead
> of catering for it is a Good Thing.
>
> Feature detection is one of the typical areas being messed up, in numerous
> softwares.
>
> Thanks musl for exposing the pitfalls and forcing upstreams think better.

As I explained, precisely that is not the case because the main musl
user, Alpine Linux, builds musl with a broken toolchain that is not
properly targeted to musl's features.  I think the official musl
instructions have the same result.

So far, no one has presented a compelling way how to test for symbol
versioning support.

Thanks,
Florian

Re: Symbol versioning approximation trips on compat symbols

[Timo Teras]

On Mon, 28 Jan 2019 07:34:25 +0100
Florian Weimer <fweimer@redhat.com> wrote:

> * u-uy:
> 
> > On Sat, Jan 26, 2019 at 11:04:24PM -0500, Rich Felker wrote:  
> >> For what it's worth, compat symbols would have let us remove
> >> symbols that shouldn't have been put in musl, such as lchmod
> >> (which confuses broken apps which wrongly expect that, if it
> >> exists, it should work)  
> >
> > For what my integrator perspective is worth, exposing brokenness
> > instead of catering for it is a Good Thing.
> >
> > Feature detection is one of the typical areas being messed up, in
> > numerous softwares.
> >
> > Thanks musl for exposing the pitfalls and forcing upstreams think
> > better.  
> 
> As I explained, precisely that is not the case because the main musl
> user, Alpine Linux, builds musl with a broken toolchain that is not
> properly targeted to musl's features.  I think the official musl
> instructions have the same result.

I remember trying to build tool chain with symbol versioning disabled.
However, it did not work out.

It's long time ago, and I don't remember exacts (if it was build
failure, or run time failure). Could be that the symbol version
extension stop working failing build of various software.

Or maybe it was that the libgcc builds with symbol versioning
enabled/disabled are not compatible, so we would also lose glibc
compatibility if we disabled.

But what I remember that it did not work at all. Perhaps it was just
broken gcc version or something like that.

In any case, I came to conclusion that toolchain with symbol versions
disabled on linux did not fell like first class citizen.

Or maybe I just messed up something.

Has anyone built toolchain with symbol versions disabled from recent
binutils+gcc and used that to build a non-trivial amount of software?
Say 100+ packages?

Timo

Re: Symbol versioning approximation trips on compat symbols

[Szabolcs Nagy]

* Timo Teras <timo.teras@iki.fi> [2019-01-28 11:17:32 +0200]:
> 
> Has anyone built toolchain with symbol versions disabled from recent
> binutils+gcc and used that to build a non-trivial amount of software?
> Say 100+ packages?

i tried to build at least gcc out of curiosity,
the build succeeds, but libgcc_s.so.1 have symvers anyway
(other target libs like libstdc++.so.6 are fine).

Re: Symbol versioning approximation trips on compat symbols

[Szabolcs Nagy]

* Florian Weimer <fweimer@redhat.com> [2019-01-28 07:34:25 +0100]:
> So far, no one has presented a compelling way how to test for symbol
> versioning support.

if you don't care about cross compilation then i think the example
you gave is a valid test.

(OT?) Re: [musl] Symbol versioning approximation trips on compat symbols

[u-uy74]

On Mon, Jan 28, 2019 at 07:34:25AM +0100, Florian Weimer wrote:
> * u-uy:
> 
> > On Sat, Jan 26, 2019 at 11:04:24PM -0500, Rich Felker wrote:
> >> For what it's worth, compat symbols would have let us remove symbols
> >> that shouldn't have been put in musl, such as lchmod (which confuses
> >> broken apps which wrongly expect that, if it exists, it should work)
> >
> > For what my integrator perspective is worth, exposing brokenness instead
> > of catering for it is a Good Thing.

> So far, no one has presented a compelling way how to test for symbol
> versioning support.

My comment apparently fell out of context. Sorry for that.

It was about applications who make undue assumptions (and as a consequence
use misdirected tests). Exposing the failures of those assumptions is
vital to be able to make it better.

If there is a feature which is hard or impossible to test for, like
symbol versioning, it means that the applications may _have_ to rely on
an explicit build flag telling whether to use it.

IOW it is for the most part not a technical problem but rather a problem
of awareness among application developers.

Rune

Re: (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols

[Markus Wichmann]

On Mon, Jan 28, 2019 at 02:08:13PM +0100, u-uy74@aetey.se wrote:
> If there is a feature which is hard or impossible to test for, like
> symbol versioning, it means that the applications may _have_ to rely on
> an explicit build flag telling whether to use it.
> 

Why is symbol versioning hard to test for, again? You make a library
containing two versions of the same function, one returns one, the other
two. The one that returns one is the default symbol. The test
application tries to link against the version that returns two. If that
works, symbol versioning is supported.

Or do you have to take care of toolchains without shared linking support
as well? Is there such a thing as static libs with symbol versioning?

Ciao,
Markus

Re: (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols

[Rich Felker]

On Mon, Jan 28, 2019 at 02:08:13PM +0100, u-uy74@aetey.se wrote:
> On Mon, Jan 28, 2019 at 07:34:25AM +0100, Florian Weimer wrote:
> > * u-uy:
> > 
> > > On Sat, Jan 26, 2019 at 11:04:24PM -0500, Rich Felker wrote:
> > >> For what it's worth, compat symbols would have let us remove symbols
> > >> that shouldn't have been put in musl, such as lchmod (which confuses
> > >> broken apps which wrongly expect that, if it exists, it should work)
> > >
> > > For what my integrator perspective is worth, exposing brokenness instead
> > > of catering for it is a Good Thing.
> 
> > So far, no one has presented a compelling way how to test for symbol
> > versioning support.
> 
> My comment apparently fell out of context. Sorry for that.
> 
> It was about applications who make undue assumptions (and as a consequence
> use misdirected tests). Exposing the failures of those assumptions is
> vital to be able to make it better.
> 
> If there is a feature which is hard or impossible to test for, like
> symbol versioning, it means that the applications may _have_ to rely on
> an explicit build flag telling whether to use it.
> 
> IOW it is for the most part not a technical problem but rather a problem
> of awareness among application developers.

It wouldn't be hard to test for if toolchains had been consistent with
musl capabilities all along. They have not been. So we have a
situation where the valid build-time tests indicate support, but
runtime silently lacks it. I don't think this is a good situation to
have created, and it's not fixable by telling the toolchains "no, we
don't actually have that" because there are and will be deployed
toolchains that don't know/honor that for a long time.

Fortunately, it mostly doesn't matter since the main intended usage
for versioning is to link to the current/default version symbols,
assuming your apps are all up-to-date with respect to your libs (and
libs wrt each other). But I still think honoring version bindings in
ldso is the right course of action.

Rich

Re: (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols

[Rich Felker]

On Mon, Jan 28, 2019 at 04:22:45PM +0100, Markus Wichmann wrote:
> On Mon, Jan 28, 2019 at 02:08:13PM +0100, u-uy74@aetey.se wrote:
> > If there is a feature which is hard or impossible to test for, like
> > symbol versioning, it means that the applications may _have_ to rely on
> > an explicit build flag telling whether to use it.
> 
> Why is symbol versioning hard to test for, again? You make a library
> containing two versions of the same function, one returns one, the other
> two. The one that returns one is the default symbol. The test
> application tries to link against the version that returns two. If that
> works, symbol versioning is supported.

That's not a test for symbol versioning. At best that's a test for an
odd sort of bug under the assumption that symbol versioning is
basically supported.

If you're trying to identify the current situation with musl, what
you'd instead need to be testing for is failure to resolve a
*non-default* version at runtime when that's what should happen
according to the symbol versioning rules. The key here is that this is
a runtime test, making it invalid in the eyes of a large portion of
the community (myself included) who consider a test that can't work
when cross-compiling to be an invalid configure-time test.

Rich

Re: Symbol versioning approximation trips on compat symbols

[A. Wilcox]

[-- Attachment #1.1: Type: text/plain, Size: 727 bytes --]

On 01/28/19 00:34, Florian Weimer wrote:
> As I explained, precisely that is not the case because the main musl
> user, Alpine Linux, builds musl with a broken toolchain that is not
> properly targeted to musl's features.  I think the official musl
> instructions have the same result.


*ahem* Adélie?  OpenWrt?  Void?  midipix?  morpheus?  Emscripten?

Alpine hardly has a monopoly on usage of musl.

Of course, since almost all of us follow the patches and configuration
of m-c-m, that's a moot point.  But the idea that "musl == Alpine" is a
harmful one, especially as the community continues to grow.

Best,
--arw


-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Symbol versioning approximation trips on compat symbols

[Matias Fonzo]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

On Mon, 28 Jan 2019 15:57:54 -0600
"A. Wilcox" <awilfox@adelielinux.org> wrote:

> On 01/28/19 00:34, Florian Weimer wrote:
> > As I explained, precisely that is not the case because the main musl
> > user, Alpine Linux, builds musl with a broken toolchain that is not
> > properly targeted to musl's features.  I think the official musl
> > instructions have the same result.  
> 
> 
> *ahem* Adélie?  OpenWrt?  Void?  midipix?  morpheus?  Emscripten?
> 
> Alpine hardly has a monopoly on usage of musl.
> 
> Of course, since almost all of us follow the patches and configuration
> of m-c-m, that's a moot point.  But the idea that "musl == Alpine" is
> a harmful one, especially as the community continues to grow.
> 

It should be nice if musl.cc (if this is community-musl page) could
list all the distributions based on Musl (including Dragora,
Sabotage, ...) and all the cross-compiler projects, etc.  :-)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: Symbol versioning approximation trips on compat symbols

[Zach van Rijn]

---- On Mon, 28 Jan 2019 17:52:25 -0500 Matias Fonzo <selk@dragora.org> wrote ---- 
 > It should be nice if musl.cc (if this is community-musl page) could 
 > list all the distributions based on Musl (including Dragora, 
 > Sabotage, ...) and all the cross-compiler projects, etc.  :-) 
 >  
 > 

The focus of musl.cc is exclusively musl cross-compilation toolchains,
therefore "musl-based Linux distributions" is off-topic.

The official community-based musl site is the Wiki [1];
this might be a more appropriate venue for what you're seeking,
and it already does exactly what you're seeking.

As does the Wikipedia page [2].

It might be good to mention musl on the Dragora Wikipedia page [3], too.
Currently the only page I can find which mentions musl is [4] even though
several distros have been using musl for the better part of the 2010s.


ZV


[1]: https://wiki.musl-libc.org/projects-using-musl.html

[2]: https://en.wikipedia.org/wiki/Musl#Use

[3]: https://en.wikipedia.org/wiki/Dragora_GNU/Linux-Libre

[4]: http://dragora.org/repo.fsl/wiki?name=Cross+compilers

Re: Symbol versioning approximation trips on compat symbols

[A. Wilcox]

[-- Attachment #1.1: Type: text/plain, Size: 908 bytes --]

On 01/28/19 17:12, Zach van Rijn wrote:
> The official community-based musl site is the Wiki [1];
> this might be a more appropriate venue for what you're seeking,
> and it already does exactly what you're seeking.
> 
> 
> [1]: https://wiki.musl-libc.org/projects-using-musl.html


That page is also a bit of a mess.  I would clean it up but I don't know
if I have the time.

It'd be nice if it had a divide between "experimental" distros and
"production" distros, that is ones that intended to be run on a
workstation/server vs those that are intended to be poked and prodded
and used for research.  Also, "last activity" would be nice, so you can
tell what's maintained.

(It'd be cool for Adélie, and probably Alpine, if the list were
alphabetised, too ;) - but I won't push it.)

Best,
--arw

-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Symbol versioning approximation trips on compat symbols

[Rich Felker]

On Mon, Jan 28, 2019 at 05:41:10PM -0600, A. Wilcox wrote:
> On 01/28/19 17:12, Zach van Rijn wrote:
> > The official community-based musl site is the Wiki [1];
> > this might be a more appropriate venue for what you're seeking,
> > and it already does exactly what you're seeking.
> > 
> > 
> > [1]: https://wiki.musl-libc.org/projects-using-musl.html
> 
> 
> That page is also a bit of a mess.  I would clean it up but I don't know
> if I have the time.
> 
> It'd be nice if it had a divide between "experimental" distros and
> "production" distros, that is ones that intended to be run on a
> workstation/server vs those that are intended to be poked and prodded
> and used for research.  Also, "last activity" would be nice, so you can
> tell what's maintained.
> 
> (It'd be cool for Adélie, and probably Alpine, if the list were
> alphabetised, too ;) - but I won't push it.)

I'm not opposed to either idea, but I'm not sure how we'd define the
experimental vs production distinction. Some dists, especially
source-based ones like Sabotage, lack the selection of packages users
might expect from a production desktop or server distro, but they can
still be very suitable for production use in embedded or in small or
"more traditional" servers without all kinds of modern bloat.

Maybe it would make sense to suggest a few comparable well-known
distros for each one ("Debian-like", "Gentoo-like", "Buildroot-like",
etc.)?

Rich

Re: Symbol versioning approximation trips on compat symbols

[A. Wilcox]

[-- Attachment #1.1: Type: text/plain, Size: 2192 bytes --]

On 01/28/19 17:47, Rich Felker wrote:
> On Mon, Jan 28, 2019 at 05:41:10PM -0600, A. Wilcox wrote:
>> On 01/28/19 17:12, Zach van Rijn wrote:
>>> The official community-based musl site is the Wiki [1];
>>> this might be a more appropriate venue for what you're seeking,
>>> and it already does exactly what you're seeking.
>>>
>>>
>>> [1]: https://wiki.musl-libc.org/projects-using-musl.html
>>
>>
>> That page is also a bit of a mess.  I would clean it up but I don't know
>> if I have the time.
>>
>> It'd be nice if it had a divide between "experimental" distros and
>> "production" distros, that is ones that intended to be run on a
>> workstation/server vs those that are intended to be poked and prodded
>> and used for research.  Also, "last activity" would be nice, so you can
>> tell what's maintained.
>>
>> (It'd be cool for Adélie, and probably Alpine, if the list were
>> alphabetised, too ;) - but I won't push it.)
> 
> I'm not opposed to either idea, but I'm not sure how we'd define the
> experimental vs production distinction. Some dists, especially
> source-based ones like Sabotage, lack the selection of packages users
> might expect from a production desktop or server distro, but they can
> still be very suitable for production use in embedded or in small or
> "more traditional" servers without all kinds of modern bloat.
> 
> Maybe it would make sense to suggest a few comparable well-known
> distros for each one ("Debian-like", "Gentoo-like", "Buildroot-like",
> etc.)?
> 
> Rich 


Perhaps it would be easier to categorise into "research" and "general use".

Research: bootstrap-linux, oasis, Snowflake
General use: Adélie, Alpine, Bedrock, CLFS/ELFS, Exherbo, januslinux,
LightCube, morpheus, Sabotage, TAZ, Void

None of them are Debian-likes.  (I might actually be personally insulted
if you said Adélie was a Debian-like; we do not use Apt, and we actually
care about Libre Software instead of just paying lipservice to it.)

Also, I thought Dragora shipped an actual musl version.  Maybe I'm mistaken.

Best,
--arw

-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols

[u-uy74]

On Mon, Jan 28, 2019 at 10:29:12AM -0500, Rich Felker wrote:
> On Mon, Jan 28, 2019 at 02:08:13PM +0100, u-uy74@aetey.se wrote:
> > If there is a feature which is hard or impossible to test for, like
> > symbol versioning, it means that the applications may _have_ to rely on
> > an explicit build flag telling whether to use it.

> It wouldn't be hard to test for if toolchains had been consistent with
> musl capabilities all along. They have not been. So we have a
> situation where the valid build-time tests indicate support, but
> runtime silently lacks it. I don't think this is a good situation to

What is the value (the "validity"?) of a build-time test if it
does not help to distinguish how to build a usable binary?
This seems to be testing for A when we want to know B.

> Fortunately, it mostly doesn't matter since the main intended usage
> for versioning is to link to the current/default version symbols,
> assuming your apps are all up-to-date with respect to your libs (and
> libs wrt each other). But I still think honoring version bindings in
> ldso is the right course of action.

I assume you'll find a good balance between the cost/complexity and
usefulness.

(Musl is by far the best libc I ever worked with on Linux, among others
omitting a number of regrettably popular misfeatures. Kudos.)

Rune

Re: Symbol versioning approximation trips on compat symbols

[Matias Fonzo]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]

On Mon, 28 Jan 2019 18:12:28 -0500
Zach van Rijn <me@zv.io> wrote:

> ---- On Mon, 28 Jan 2019 17:52:25 -0500 Matias Fonzo
> <selk@dragora.org> wrote ---- 
>  > It should be nice if musl.cc (if this is community-musl page)
>  > could list all the distributions based on Musl (including Dragora, 
>  > Sabotage, ...) and all the cross-compiler projects, etc.  :-) 
>  >  
>  >   
> 
> The focus of musl.cc is exclusively musl cross-compilation toolchains,
> therefore "musl-based Linux distributions" is off-topic.
> 
> The official community-based musl site is the Wiki [1];
> this might be a more appropriate venue for what you're seeking,
> and it already does exactly what you're seeking.

But to contribute you need an account in Github, plus send a Pull
Request, it would be better if this is more direct, like the usage of
Oddmuse[1].

[1] https://oddmuse.org

> As does the Wikipedia page [2].
> 
> It might be good to mention musl on the Dragora Wikipedia page [3],
> too. Currently the only page I can find which mentions musl is [4]
> even though several distros have been using musl for the better part
> of the 2010s.
> 

I don't think so.  The information there is out of date, and I haven't
be able to edit the most part of the Wikipedia page.  Last time that
I've tried it is like you can't edit a great portion ...

Anyway, I have pending to update the official website.


Regards,
Matías


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: Symbol versioning approximation trips on compat symbols

[Matias Fonzo]

[-- Attachment #1: Type: text/plain, Size: 464 bytes --]

On Mon, 28 Jan 2019 21:22:30 -0600
"A. Wilcox" <awilfox@adelielinux.org> wrote:
> [..]
> 
> Also, I thought Dragora shipped an actual musl version.  Maybe I'm
> mistaken.

Latest published version is the version 3.0-alpha2, which has musl
1.1.20+.  The current one is from the commit
"de7dc1318f493184b20f7661bc12b1829b957b67" which is a few commits
behind of 1.1.21.  The distro is under development up to reach the
stable version, for general use.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols

[Rich Felker]

On Tue, Jan 29, 2019 at 06:13:05PM +0100, u-uy74@aetey.se wrote:
> On Mon, Jan 28, 2019 at 10:29:12AM -0500, Rich Felker wrote:
> > On Mon, Jan 28, 2019 at 02:08:13PM +0100, u-uy74@aetey.se wrote:
> > > If there is a feature which is hard or impossible to test for, like
> > > symbol versioning, it means that the applications may _have_ to rely on
> > > an explicit build flag telling whether to use it.
> 
> > It wouldn't be hard to test for if toolchains had been consistent with
> > musl capabilities all along. They have not been. So we have a
> > situation where the valid build-time tests indicate support, but
> > runtime silently lacks it. I don't think this is a good situation to
> 
> What is the value (the "validity"?) of a build-time test if it
> does not help to distinguish how to build a usable binary?
> This seems to be testing for A when we want to know B.

It's valid in that it works assuming the relevant components are
following the reasonable rules/spec for what to do, which was designed
to work in such a manner that it's safely detectable at configure
time. In particular the tooling should only be offering symver support
on targets that support it. We screwed up by not suppressing that in
gcc a long time ago, but I think we probably would have hit more
package-level breakage (that would have led to fruitless arguments and
had negative impact on ability to work with upstreams) if lack of
support stopped packages at build time rather than just silently
breaking later if mismatched lib versions are use.

Rich