* Symbol versioning approximation trips on compat symbols @ 2019-01-21 17:57 Florian Weimer 2019-01-24 1:43 ` Rich Felker 0 siblings, 1 reply; 26+ messages in thread From: Florian Weimer @ 2019-01-21 17:57 UTC (permalink / raw) To: musl On what appears to be current Alpine Linux (musl-1.1.19-r10), the following reproducer ###################################################################### cat > symver.c <<EOF void compat_function (void) { } __asm__ (".symver compat_function,compat_function@SYMVER"); void call_compat_function (void) { return compat_function (); } EOF echo "SYMVER { };" > symver.map cat > main.c <<EOF extern void call_compat_function (void); int main (void) { call_compat_function (); } EOF gcc -fpic -shared -o symver.so -Wl,--version-script=symver.map symver.c gcc -Wl,--rpath=. -o main main.c symver.so ###################################################################### fails with: $ ./main Error relocating ./symver.so: compat_function: symbol not found The problem is the compatibility symbol (one @ instead of @@). The dynamic linker is supposed to ignore the difference between the two, the default vs non-default version only matters to the link editor when processing an undefined symbol without a symbol version. In my case, I do not need symbol interposition and therefore can work around this, but I wonder if there is some sort of approved compile-time or link-time check to detect this issue. Unfortunately, the Alpine Linux toolchain (and part of the system) is built *with* symbol versioning support, so this does not appear to be straightforward. The actual application does not need to make the symbol interposable, so I can use a hidden alias within the DSO for PLT avoidance (and more configure checks to disable all this on targets which do not support *that*). Thanks, Florian ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-21 17:57 Symbol versioning approximation trips on compat symbols Florian Weimer @ 2019-01-24 1:43 ` Rich Felker 2019-01-24 9:28 ` u-uy74 2019-01-24 11:09 ` Szabolcs Nagy 0 siblings, 2 replies; 26+ messages in thread From: Rich Felker @ 2019-01-24 1:43 UTC (permalink / raw) To: musl On Mon, Jan 21, 2019 at 06:57:53PM +0100, Florian Weimer wrote: > On what appears to be current Alpine Linux (musl-1.1.19-r10), the > following reproducer > > ###################################################################### > cat > symver.c <<EOF > void > compat_function (void) > { > } > __asm__ (".symver compat_function,compat_function@SYMVER"); > > void > call_compat_function (void) > { > return compat_function (); > } > EOF > > echo "SYMVER { };" > symver.map > > cat > main.c <<EOF > extern void call_compat_function (void); > > int > main (void) > { > call_compat_function (); > } > EOF > > gcc -fpic -shared -o symver.so -Wl,--version-script=symver.map symver.c > gcc -Wl,--rpath=. -o main main.c symver.so > ###################################################################### > > fails with: > > $ ./main > Error relocating ./symver.so: compat_function: symbol not found > > The problem is the compatibility symbol (one @ instead of @@). The > dynamic linker is supposed to ignore the difference between the two, the > default vs non-default version only matters to the link editor when > processing an undefined symbol without a symbol version. > > In my case, I do not need symbol interposition and therefore can work > around this, but I wonder if there is some sort of approved compile-time > or link-time check to detect this issue. Unfortunately, the Alpine > Linux toolchain (and part of the system) is built *with* symbol > versioning support, so this does not appear to be straightforward. > > The actual application does not need to make the symbol interposable, so > I can use a hidden alias within the DSO for PLT avoidance (and more > configure checks to disable all this on targets which do not support > *that*). The same issue came up before with libgcc defining and referencing a non-default-version symbol for some weird compatibility hack. I don't remember the details but Szabolcs Nagy was involved in investigating and might. In any case, the root cause is that musl's dynamic linker does not support symbol versioning; for the sake of being able to load libraries that were build with versioning, it always resolves a symbol to the "latest"/default version, the same as ld would do at link time. Normally this is the right thing as long as you don't actually have things that were linked against an old incompatible version, but it also breaks explicit linking to a particular version as in your example above. The right fix is probably to add support for symbol version matching in the dynamic linker. Unfortunately this involves some extra logic in the extreme hot paths, so it's hard to make the cost unobservably low, and last I checked some members of the community were opposed to it on ideological grounds. If there's a good need for it (and I think just avoiding silent breakage of third-party libs using versioning and intending for it to work is a fairly good one already), support can be added, but doing it without negative impact is a pretty big task. Rich ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-24 1:43 ` Rich Felker @ 2019-01-24 9:28 ` u-uy74 2019-01-24 10:11 ` Florian Weimer 2019-01-24 11:09 ` Szabolcs Nagy 1 sibling, 1 reply; 26+ messages in thread From: u-uy74 @ 2019-01-24 9:28 UTC (permalink / raw) To: musl On Wed, Jan 23, 2019 at 08:43:40PM -0500, Rich Felker wrote: > The same issue came up before with libgcc defining and referencing a > non-default-version symbol for some weird compatibility hack. I don't > remember the details but Szabolcs Nagy was involved in investigating > and might. In any case, the root cause is that musl's dynamic linker > does not support symbol versioning; for the sake of being able to load > libraries that were build with versioning, it always resolves a symbol > to the "latest"/default version, the same as ld would do at link time. > Normally this is the right thing as long as you don't actually have > things that were linked against an old incompatible version, but it > also breaks explicit linking to a particular version as in your > example above. > > The right fix is probably to add support for symbol version matching > in the dynamic linker. Unfortunately this involves some extra logic in > the extreme hot paths, so it's hard to make the cost unobservably low, > and last I checked some members of the community were opposed to it on > ideological grounds. If there's a good need for it (and I think just IMVHO symbol versioning is basically aimed to hide the complexity of the evolution of libraries, to make certain usage cases "just work". OTOH it does not reduce the complexity under the hood but rather adds some extra of it. That's why I see its impact as double negative, postulating more complex tools and reducing the capacity of fellow integrators to analyze their systems. > avoiding silent breakage of third-party libs using versioning and > intending for it to work is a fairly good one already), support can be > added, but doing it without negative impact is a pretty big task. This cost itself (of adding the support) means that some other useful changes in musl will be missing at _every_ given time in the future development. Is this workaround, for the sake of libraries who do unusual hacks, really so important? Rune ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-24 9:28 ` u-uy74 @ 2019-01-24 10:11 ` Florian Weimer 0 siblings, 0 replies; 26+ messages in thread From: Florian Weimer @ 2019-01-24 10:11 UTC (permalink / raw) To: u-uy74; +Cc: musl * u-uy: > IMVHO symbol versioning is basically aimed to hide the complexity of > the evolution of libraries, to make certain usage cases "just work". > > OTOH it does not reduce the complexity under the hood but rather adds some > extra of it. That's why I see its impact as double negative, postulating > more complex tools and reducing the capacity of fellow integrators to > analyze their systems. That's certainly a valid position to take. But the reality is that all toolchains that target musl have symbol versioning support built into them today, just not in the dynamic linker. It would be far easier unrelated upstreams if the toolchain simply did not support symbol versioning at all (because the dynamic linker does not support it, so it's not usable in each place where it is required anyway). Then a simple check for .symver support in the assembler would tell us whether we can symbol versionining or not. I have no idea how this current support is supposed to work, particularly with cross-compiling. Does everyone have to write configure hacks for *-musl targets to disable symbol versioning support? That goes against the mantra of testing what you use (and avoid hard-coded knowledge about specific versions/targets). Thanks, Florian ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-24 1:43 ` Rich Felker 2019-01-24 9:28 ` u-uy74 @ 2019-01-24 11:09 ` Szabolcs Nagy 2019-01-24 11:18 ` Florian Weimer 1 sibling, 1 reply; 26+ messages in thread From: Szabolcs Nagy @ 2019-01-24 11:09 UTC (permalink / raw) To: musl * Rich Felker <dalias@libc.org> [2019-01-23 20:43:40 -0500]: > On Mon, Jan 21, 2019 at 06:57:53PM +0100, Florian Weimer wrote: > > On what appears to be current Alpine Linux (musl-1.1.19-r10), the > > following reproducer > > > > ###################################################################### > > cat > symver.c <<EOF > > void > > compat_function (void) > > { > > } > > __asm__ (".symver compat_function,compat_function@SYMVER"); > > > > void > > call_compat_function (void) > > { > > return compat_function (); > > } > > EOF > > > > echo "SYMVER { };" > symver.map > > > > cat > main.c <<EOF > > extern void call_compat_function (void); > > > > int > > main (void) > > { > > call_compat_function (); > > } > > EOF > > > > gcc -fpic -shared -o symver.so -Wl,--version-script=symver.map symver.c > > gcc -Wl,--rpath=. -o main main.c symver.so > > ###################################################################### > > > > fails with: > > > > $ ./main > > Error relocating ./symver.so: compat_function: symbol not found > > > > The problem is the compatibility symbol (one @ instead of @@). The > > dynamic linker is supposed to ignore the difference between the two, the > > default vs non-default version only matters to the link editor when > > processing an undefined symbol without a symbol version. > > > > In my case, I do not need symbol interposition and therefore can work > > around this, but I wonder if there is some sort of approved compile-time > > or link-time check to detect this issue. Unfortunately, the Alpine > > Linux toolchain (and part of the system) is built *with* symbol > > versioning support, so this does not appear to be straightforward. > > > > The actual application does not need to make the symbol interposable, so > > I can use a hidden alias within the DSO for PLT avoidance (and more > > configure checks to disable all this on targets which do not support > > *that*). > > The same issue came up before with libgcc defining and referencing a > non-default-version symbol for some weird compatibility hack. I don't > remember the details but Szabolcs Nagy was involved in investigating > and might. In any case, the root cause is that musl's dynamic linker > does not support symbol versioning; for the sake of being able to load > libraries that were build with versioning, it always resolves a symbol > to the "latest"/default version, the same as ld would do at link time. > Normally this is the right thing as long as you don't actually have > things that were linked against an old incompatible version, but it > also breaks explicit linking to a particular version as in your > example above. the libgcc_s.so.1 issue on x86 was an extern object symbol (used by function-multi-versioning ifuncs) that is initialized by a ctor. but it turned out to be broken (ifunc resolvers may run before relocations for the extern object are processed), so the symbol was removed (moved to libgcc.a), but a compat symbol (@) was kept around and the ctor of libgcc_s.so.1 still references it. so musl cannot load libgcc_s.so.1 since there is no default version (@@) of the symbol. (in a musl based gcc this is fixed: there is no ifunc support or compat issue anyway https://gcc.gnu.org/ml/gcc-patches/2016-11/msg01125.html you would expect that it's enough to build gcc for --disable-gnu-indirect-function or --disable-symvers but even with those the compat symbol is there...) i wonder what is the use-case for using a compat symbol without introducing a new default version for the symbol in general? > > The right fix is probably to add support for symbol version matching > in the dynamic linker. Unfortunately this involves some extra logic in > the extreme hot paths, so it's hard to make the cost unobservably low, > and last I checked some members of the community were opposed to it on > ideological grounds. If there's a good need for it (and I think just > avoiding silent breakage of third-party libs using versioning and > intending for it to work is a fairly good one already), support can be > added, but doing it without negative impact is a pretty big task. > > Rich ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-24 11:09 ` Szabolcs Nagy @ 2019-01-24 11:18 ` Florian Weimer 2019-01-27 4:04 ` Rich Felker 0 siblings, 1 reply; 26+ messages in thread From: Florian Weimer @ 2019-01-24 11:18 UTC (permalink / raw) To: musl * Szabolcs Nagy: > but it turned out to be broken (ifunc resolvers may run before > relocations for the extern object are processed), so the symbol > was removed (moved to libgcc.a), but a compat symbol (@) was > kept around and the ctor of libgcc_s.so.1 still references it. I assume we cannot use a hidden alias to eliminate the symbolic reference because it is a data symbol, so copy relocations are a possibility and with the hidden alias, the constructor would update the wrong object? > i wonder what is the use-case for using a compat symbol without > introducing a new default version for the symbol in general? A compat symbol can be implemented by a different library, and applications can link to the symbol and get it from the new library. Mostly it is just for documenting intent, marking the symbol as deprecated. It's still relatively easy to link to the symbol from new binaries (even without resorting to dlvsym). Thanks, Florian ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-24 11:18 ` Florian Weimer @ 2019-01-27 4:04 ` Rich Felker 2019-01-27 9:36 ` u-uy74 0 siblings, 1 reply; 26+ messages in thread From: Rich Felker @ 2019-01-27 4:04 UTC (permalink / raw) To: musl On Thu, Jan 24, 2019 at 12:18:12PM +0100, Florian Weimer wrote: > * Szabolcs Nagy: > > > but it turned out to be broken (ifunc resolvers may run before > > relocations for the extern object are processed), so the symbol > > was removed (moved to libgcc.a), but a compat symbol (@) was > > kept around and the ctor of libgcc_s.so.1 still references it. > > I assume we cannot use a hidden alias to eliminate the symbolic > reference because it is a data symbol, so copy relocations are a > possibility and with the hidden alias, the constructor would update the > wrong object? I believe we had a perfectly workable alternate solution using something like that, but someone on the upstream side wasn't willing to change it. > > i wonder what is the use-case for using a compat symbol without > > introducing a new default version for the symbol in general? > > A compat symbol can be implemented by a different library, and > applications can link to the symbol and get it from the new library. > > Mostly it is just for documenting intent, marking the symbol as > deprecated. It's still relatively easy to link to the symbol from new > binaries (even without resorting to dlvsym). For what it's worth, compat symbols would have let us remove symbols that shouldn't have been put in musl, such as lchmod (which confuses broken apps which wrongly expect that, if it exists, it should work) and perhaps all the LFS64 symbols (which should be present for binary-compat but not at ld-time so that configure doesn't detect and try to use them). However, doing it that way would put a symver table in musl libc.so, resulting in lots of other behaviors we almost surely don't want. It does shed some light on possible motivations, though, I think. Rich ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-27 4:04 ` Rich Felker @ 2019-01-27 9:36 ` u-uy74 2019-01-28 6:34 ` Florian Weimer 0 siblings, 1 reply; 26+ messages in thread From: u-uy74 @ 2019-01-27 9:36 UTC (permalink / raw) To: musl On Sat, Jan 26, 2019 at 11:04:24PM -0500, Rich Felker wrote: > For what it's worth, compat symbols would have let us remove symbols > that shouldn't have been put in musl, such as lchmod (which confuses > broken apps which wrongly expect that, if it exists, it should work) For what my integrator perspective is worth, exposing brokenness instead of catering for it is a Good Thing. Feature detection is one of the typical areas being messed up, in numerous softwares. Thanks musl for exposing the pitfalls and forcing upstreams think better. Rune ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-27 9:36 ` u-uy74 @ 2019-01-28 6:34 ` Florian Weimer 2019-01-28 9:17 ` Timo Teras ` (3 more replies) 0 siblings, 4 replies; 26+ messages in thread From: Florian Weimer @ 2019-01-28 6:34 UTC (permalink / raw) To: u-uy74; +Cc: musl * u-uy: > On Sat, Jan 26, 2019 at 11:04:24PM -0500, Rich Felker wrote: >> For what it's worth, compat symbols would have let us remove symbols >> that shouldn't have been put in musl, such as lchmod (which confuses >> broken apps which wrongly expect that, if it exists, it should work) > > For what my integrator perspective is worth, exposing brokenness instead > of catering for it is a Good Thing. > > Feature detection is one of the typical areas being messed up, in numerous > softwares. > > Thanks musl for exposing the pitfalls and forcing upstreams think better. As I explained, precisely that is not the case because the main musl user, Alpine Linux, builds musl with a broken toolchain that is not properly targeted to musl's features. I think the official musl instructions have the same result. So far, no one has presented a compelling way how to test for symbol versioning support. Thanks, Florian ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-28 6:34 ` Florian Weimer @ 2019-01-28 9:17 ` Timo Teras 2019-01-28 11:33 ` Szabolcs Nagy 2019-01-28 12:40 ` Szabolcs Nagy ` (2 subsequent siblings) 3 siblings, 1 reply; 26+ messages in thread From: Timo Teras @ 2019-01-28 9:17 UTC (permalink / raw) To: Florian Weimer; +Cc: musl, u-uy74 On Mon, 28 Jan 2019 07:34:25 +0100 Florian Weimer <fweimer@redhat.com> wrote: > * u-uy: > > > On Sat, Jan 26, 2019 at 11:04:24PM -0500, Rich Felker wrote: > >> For what it's worth, compat symbols would have let us remove > >> symbols that shouldn't have been put in musl, such as lchmod > >> (which confuses broken apps which wrongly expect that, if it > >> exists, it should work) > > > > For what my integrator perspective is worth, exposing brokenness > > instead of catering for it is a Good Thing. > > > > Feature detection is one of the typical areas being messed up, in > > numerous softwares. > > > > Thanks musl for exposing the pitfalls and forcing upstreams think > > better. > > As I explained, precisely that is not the case because the main musl > user, Alpine Linux, builds musl with a broken toolchain that is not > properly targeted to musl's features. I think the official musl > instructions have the same result. I remember trying to build tool chain with symbol versioning disabled. However, it did not work out. It's long time ago, and I don't remember exacts (if it was build failure, or run time failure). Could be that the symbol version extension stop working failing build of various software. Or maybe it was that the libgcc builds with symbol versioning enabled/disabled are not compatible, so we would also lose glibc compatibility if we disabled. But what I remember that it did not work at all. Perhaps it was just broken gcc version or something like that. In any case, I came to conclusion that toolchain with symbol versions disabled on linux did not fell like first class citizen. Or maybe I just messed up something. Has anyone built toolchain with symbol versions disabled from recent binutils+gcc and used that to build a non-trivial amount of software? Say 100+ packages? Timo ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-28 9:17 ` Timo Teras @ 2019-01-28 11:33 ` Szabolcs Nagy 0 siblings, 0 replies; 26+ messages in thread From: Szabolcs Nagy @ 2019-01-28 11:33 UTC (permalink / raw) To: musl; +Cc: Florian Weimer, u-uy74 * Timo Teras <timo.teras@iki.fi> [2019-01-28 11:17:32 +0200]: > > Has anyone built toolchain with symbol versions disabled from recent > binutils+gcc and used that to build a non-trivial amount of software? > Say 100+ packages? i tried to build at least gcc out of curiosity, the build succeeds, but libgcc_s.so.1 have symvers anyway (other target libs like libstdc++.so.6 are fine). ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-28 6:34 ` Florian Weimer 2019-01-28 9:17 ` Timo Teras @ 2019-01-28 12:40 ` Szabolcs Nagy 2019-01-28 13:08 ` (OT?) Re: [musl] " u-uy74 2019-01-28 21:57 ` A. Wilcox 3 siblings, 0 replies; 26+ messages in thread From: Szabolcs Nagy @ 2019-01-28 12:40 UTC (permalink / raw) To: musl; +Cc: u-uy74, Florian Weimer * Florian Weimer <fweimer@redhat.com> [2019-01-28 07:34:25 +0100]: > So far, no one has presented a compelling way how to test for symbol > versioning support. if you don't care about cross compilation then i think the example you gave is a valid test. ^ permalink raw reply [flat|nested] 26+ messages in thread
* (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols 2019-01-28 6:34 ` Florian Weimer 2019-01-28 9:17 ` Timo Teras 2019-01-28 12:40 ` Szabolcs Nagy @ 2019-01-28 13:08 ` u-uy74 2019-01-28 15:22 ` Markus Wichmann 2019-01-28 15:29 ` Rich Felker 2019-01-28 21:57 ` A. Wilcox 3 siblings, 2 replies; 26+ messages in thread From: u-uy74 @ 2019-01-28 13:08 UTC (permalink / raw) To: Florian Weimer; +Cc: musl On Mon, Jan 28, 2019 at 07:34:25AM +0100, Florian Weimer wrote: > * u-uy: > > > On Sat, Jan 26, 2019 at 11:04:24PM -0500, Rich Felker wrote: > >> For what it's worth, compat symbols would have let us remove symbols > >> that shouldn't have been put in musl, such as lchmod (which confuses > >> broken apps which wrongly expect that, if it exists, it should work) > > > > For what my integrator perspective is worth, exposing brokenness instead > > of catering for it is a Good Thing. > So far, no one has presented a compelling way how to test for symbol > versioning support. My comment apparently fell out of context. Sorry for that. It was about applications who make undue assumptions (and as a consequence use misdirected tests). Exposing the failures of those assumptions is vital to be able to make it better. If there is a feature which is hard or impossible to test for, like symbol versioning, it means that the applications may _have_ to rely on an explicit build flag telling whether to use it. IOW it is for the most part not a technical problem but rather a problem of awareness among application developers. Rune ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols 2019-01-28 13:08 ` (OT?) Re: [musl] " u-uy74 @ 2019-01-28 15:22 ` Markus Wichmann 2019-01-28 15:34 ` Rich Felker 2019-01-28 15:29 ` Rich Felker 1 sibling, 1 reply; 26+ messages in thread From: Markus Wichmann @ 2019-01-28 15:22 UTC (permalink / raw) To: musl On Mon, Jan 28, 2019 at 02:08:13PM +0100, u-uy74@aetey.se wrote: > If there is a feature which is hard or impossible to test for, like > symbol versioning, it means that the applications may _have_ to rely on > an explicit build flag telling whether to use it. > Why is symbol versioning hard to test for, again? You make a library containing two versions of the same function, one returns one, the other two. The one that returns one is the default symbol. The test application tries to link against the version that returns two. If that works, symbol versioning is supported. Or do you have to take care of toolchains without shared linking support as well? Is there such a thing as static libs with symbol versioning? Ciao, Markus ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols 2019-01-28 15:22 ` Markus Wichmann @ 2019-01-28 15:34 ` Rich Felker 0 siblings, 0 replies; 26+ messages in thread From: Rich Felker @ 2019-01-28 15:34 UTC (permalink / raw) To: musl On Mon, Jan 28, 2019 at 04:22:45PM +0100, Markus Wichmann wrote: > On Mon, Jan 28, 2019 at 02:08:13PM +0100, u-uy74@aetey.se wrote: > > If there is a feature which is hard or impossible to test for, like > > symbol versioning, it means that the applications may _have_ to rely on > > an explicit build flag telling whether to use it. > > Why is symbol versioning hard to test for, again? You make a library > containing two versions of the same function, one returns one, the other > two. The one that returns one is the default symbol. The test > application tries to link against the version that returns two. If that > works, symbol versioning is supported. That's not a test for symbol versioning. At best that's a test for an odd sort of bug under the assumption that symbol versioning is basically supported. If you're trying to identify the current situation with musl, what you'd instead need to be testing for is failure to resolve a *non-default* version at runtime when that's what should happen according to the symbol versioning rules. The key here is that this is a runtime test, making it invalid in the eyes of a large portion of the community (myself included) who consider a test that can't work when cross-compiling to be an invalid configure-time test. Rich ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols 2019-01-28 13:08 ` (OT?) Re: [musl] " u-uy74 2019-01-28 15:22 ` Markus Wichmann @ 2019-01-28 15:29 ` Rich Felker 2019-01-29 17:13 ` u-uy74 1 sibling, 1 reply; 26+ messages in thread From: Rich Felker @ 2019-01-28 15:29 UTC (permalink / raw) To: musl On Mon, Jan 28, 2019 at 02:08:13PM +0100, u-uy74@aetey.se wrote: > On Mon, Jan 28, 2019 at 07:34:25AM +0100, Florian Weimer wrote: > > * u-uy: > > > > > On Sat, Jan 26, 2019 at 11:04:24PM -0500, Rich Felker wrote: > > >> For what it's worth, compat symbols would have let us remove symbols > > >> that shouldn't have been put in musl, such as lchmod (which confuses > > >> broken apps which wrongly expect that, if it exists, it should work) > > > > > > For what my integrator perspective is worth, exposing brokenness instead > > > of catering for it is a Good Thing. > > > So far, no one has presented a compelling way how to test for symbol > > versioning support. > > My comment apparently fell out of context. Sorry for that. > > It was about applications who make undue assumptions (and as a consequence > use misdirected tests). Exposing the failures of those assumptions is > vital to be able to make it better. > > If there is a feature which is hard or impossible to test for, like > symbol versioning, it means that the applications may _have_ to rely on > an explicit build flag telling whether to use it. > > IOW it is for the most part not a technical problem but rather a problem > of awareness among application developers. It wouldn't be hard to test for if toolchains had been consistent with musl capabilities all along. They have not been. So we have a situation where the valid build-time tests indicate support, but runtime silently lacks it. I don't think this is a good situation to have created, and it's not fixable by telling the toolchains "no, we don't actually have that" because there are and will be deployed toolchains that don't know/honor that for a long time. Fortunately, it mostly doesn't matter since the main intended usage for versioning is to link to the current/default version symbols, assuming your apps are all up-to-date with respect to your libs (and libs wrt each other). But I still think honoring version bindings in ldso is the right course of action. Rich ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols 2019-01-28 15:29 ` Rich Felker @ 2019-01-29 17:13 ` u-uy74 2019-01-30 14:57 ` Rich Felker 0 siblings, 1 reply; 26+ messages in thread From: u-uy74 @ 2019-01-29 17:13 UTC (permalink / raw) To: musl On Mon, Jan 28, 2019 at 10:29:12AM -0500, Rich Felker wrote: > On Mon, Jan 28, 2019 at 02:08:13PM +0100, u-uy74@aetey.se wrote: > > If there is a feature which is hard or impossible to test for, like > > symbol versioning, it means that the applications may _have_ to rely on > > an explicit build flag telling whether to use it. > It wouldn't be hard to test for if toolchains had been consistent with > musl capabilities all along. They have not been. So we have a > situation where the valid build-time tests indicate support, but > runtime silently lacks it. I don't think this is a good situation to What is the value (the "validity"?) of a build-time test if it does not help to distinguish how to build a usable binary? This seems to be testing for A when we want to know B. > Fortunately, it mostly doesn't matter since the main intended usage > for versioning is to link to the current/default version symbols, > assuming your apps are all up-to-date with respect to your libs (and > libs wrt each other). But I still think honoring version bindings in > ldso is the right course of action. I assume you'll find a good balance between the cost/complexity and usefulness. (Musl is by far the best libc I ever worked with on Linux, among others omitting a number of regrettably popular misfeatures. Kudos.) Rune ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: (OT?) Re: [musl] Symbol versioning approximation trips on compat symbols 2019-01-29 17:13 ` u-uy74 @ 2019-01-30 14:57 ` Rich Felker 0 siblings, 0 replies; 26+ messages in thread From: Rich Felker @ 2019-01-30 14:57 UTC (permalink / raw) To: musl On Tue, Jan 29, 2019 at 06:13:05PM +0100, u-uy74@aetey.se wrote: > On Mon, Jan 28, 2019 at 10:29:12AM -0500, Rich Felker wrote: > > On Mon, Jan 28, 2019 at 02:08:13PM +0100, u-uy74@aetey.se wrote: > > > If there is a feature which is hard or impossible to test for, like > > > symbol versioning, it means that the applications may _have_ to rely on > > > an explicit build flag telling whether to use it. > > > It wouldn't be hard to test for if toolchains had been consistent with > > musl capabilities all along. They have not been. So we have a > > situation where the valid build-time tests indicate support, but > > runtime silently lacks it. I don't think this is a good situation to > > What is the value (the "validity"?) of a build-time test if it > does not help to distinguish how to build a usable binary? > This seems to be testing for A when we want to know B. It's valid in that it works assuming the relevant components are following the reasonable rules/spec for what to do, which was designed to work in such a manner that it's safely detectable at configure time. In particular the tooling should only be offering symver support on targets that support it. We screwed up by not suppressing that in gcc a long time ago, but I think we probably would have hit more package-level breakage (that would have led to fruitless arguments and had negative impact on ability to work with upstreams) if lack of support stopped packages at build time rather than just silently breaking later if mismatched lib versions are use. Rich ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-28 6:34 ` Florian Weimer ` (2 preceding siblings ...) 2019-01-28 13:08 ` (OT?) Re: [musl] " u-uy74 @ 2019-01-28 21:57 ` A. Wilcox 2019-01-28 22:52 ` Matias Fonzo 3 siblings, 1 reply; 26+ messages in thread From: A. Wilcox @ 2019-01-28 21:57 UTC (permalink / raw) To: musl [-- Attachment #1.1: Type: text/plain, Size: 727 bytes --] On 01/28/19 00:34, Florian Weimer wrote: > As I explained, precisely that is not the case because the main musl > user, Alpine Linux, builds musl with a broken toolchain that is not > properly targeted to musl's features. I think the official musl > instructions have the same result. *ahem* Adélie? OpenWrt? Void? midipix? morpheus? Emscripten? Alpine hardly has a monopoly on usage of musl. Of course, since almost all of us follow the patches and configuration of m-c-m, that's a moot point. But the idea that "musl == Alpine" is a harmful one, especially as the community continues to grow. Best, --arw -- A. Wilcox (awilfox) Project Lead, Adélie Linux https://www.adelielinux.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-28 21:57 ` A. Wilcox @ 2019-01-28 22:52 ` Matias Fonzo 2019-01-28 23:12 ` Zach van Rijn 0 siblings, 1 reply; 26+ messages in thread From: Matias Fonzo @ 2019-01-28 22:52 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 937 bytes --] On Mon, 28 Jan 2019 15:57:54 -0600 "A. Wilcox" <awilfox@adelielinux.org> wrote: > On 01/28/19 00:34, Florian Weimer wrote: > > As I explained, precisely that is not the case because the main musl > > user, Alpine Linux, builds musl with a broken toolchain that is not > > properly targeted to musl's features. I think the official musl > > instructions have the same result. > > > *ahem* Adélie? OpenWrt? Void? midipix? morpheus? Emscripten? > > Alpine hardly has a monopoly on usage of musl. > > Of course, since almost all of us follow the patches and configuration > of m-c-m, that's a moot point. But the idea that "musl == Alpine" is > a harmful one, especially as the community continues to grow. > It should be nice if musl.cc (if this is community-musl page) could list all the distributions based on Musl (including Dragora, Sabotage, ...) and all the cross-compiler projects, etc. :-) [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 949 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-28 22:52 ` Matias Fonzo @ 2019-01-28 23:12 ` Zach van Rijn 2019-01-28 23:41 ` A. Wilcox 2019-01-29 19:31 ` Matias Fonzo 0 siblings, 2 replies; 26+ messages in thread From: Zach van Rijn @ 2019-01-28 23:12 UTC (permalink / raw) To: musl ---- On Mon, 28 Jan 2019 17:52:25 -0500 Matias Fonzo <selk@dragora.org> wrote ---- > It should be nice if musl.cc (if this is community-musl page) could > list all the distributions based on Musl (including Dragora, > Sabotage, ...) and all the cross-compiler projects, etc. :-) > > The focus of musl.cc is exclusively musl cross-compilation toolchains, therefore "musl-based Linux distributions" is off-topic. The official community-based musl site is the Wiki [1]; this might be a more appropriate venue for what you're seeking, and it already does exactly what you're seeking. As does the Wikipedia page [2]. It might be good to mention musl on the Dragora Wikipedia page [3], too. Currently the only page I can find which mentions musl is [4] even though several distros have been using musl for the better part of the 2010s. ZV [1]: https://wiki.musl-libc.org/projects-using-musl.html [2]: https://en.wikipedia.org/wiki/Musl#Use [3]: https://en.wikipedia.org/wiki/Dragora_GNU/Linux-Libre [4]: http://dragora.org/repo.fsl/wiki?name=Cross+compilers ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-28 23:12 ` Zach van Rijn @ 2019-01-28 23:41 ` A. Wilcox 2019-01-28 23:47 ` Rich Felker 2019-01-29 19:31 ` Matias Fonzo 1 sibling, 1 reply; 26+ messages in thread From: A. Wilcox @ 2019-01-28 23:41 UTC (permalink / raw) To: musl [-- Attachment #1.1: Type: text/plain, Size: 908 bytes --] On 01/28/19 17:12, Zach van Rijn wrote: > The official community-based musl site is the Wiki [1]; > this might be a more appropriate venue for what you're seeking, > and it already does exactly what you're seeking. > > > [1]: https://wiki.musl-libc.org/projects-using-musl.html That page is also a bit of a mess. I would clean it up but I don't know if I have the time. It'd be nice if it had a divide between "experimental" distros and "production" distros, that is ones that intended to be run on a workstation/server vs those that are intended to be poked and prodded and used for research. Also, "last activity" would be nice, so you can tell what's maintained. (It'd be cool for Adélie, and probably Alpine, if the list were alphabetised, too ;) - but I won't push it.) Best, --arw -- A. Wilcox (awilfox) Project Lead, Adélie Linux https://www.adelielinux.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-28 23:41 ` A. Wilcox @ 2019-01-28 23:47 ` Rich Felker 2019-01-29 3:22 ` A. Wilcox 0 siblings, 1 reply; 26+ messages in thread From: Rich Felker @ 2019-01-28 23:47 UTC (permalink / raw) To: musl On Mon, Jan 28, 2019 at 05:41:10PM -0600, A. Wilcox wrote: > On 01/28/19 17:12, Zach van Rijn wrote: > > The official community-based musl site is the Wiki [1]; > > this might be a more appropriate venue for what you're seeking, > > and it already does exactly what you're seeking. > > > > > > [1]: https://wiki.musl-libc.org/projects-using-musl.html > > > That page is also a bit of a mess. I would clean it up but I don't know > if I have the time. > > It'd be nice if it had a divide between "experimental" distros and > "production" distros, that is ones that intended to be run on a > workstation/server vs those that are intended to be poked and prodded > and used for research. Also, "last activity" would be nice, so you can > tell what's maintained. > > (It'd be cool for Adélie, and probably Alpine, if the list were > alphabetised, too ;) - but I won't push it.) I'm not opposed to either idea, but I'm not sure how we'd define the experimental vs production distinction. Some dists, especially source-based ones like Sabotage, lack the selection of packages users might expect from a production desktop or server distro, but they can still be very suitable for production use in embedded or in small or "more traditional" servers without all kinds of modern bloat. Maybe it would make sense to suggest a few comparable well-known distros for each one ("Debian-like", "Gentoo-like", "Buildroot-like", etc.)? Rich ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-28 23:47 ` Rich Felker @ 2019-01-29 3:22 ` A. Wilcox 2019-01-29 19:40 ` Matias Fonzo 0 siblings, 1 reply; 26+ messages in thread From: A. Wilcox @ 2019-01-29 3:22 UTC (permalink / raw) To: musl [-- Attachment #1.1: Type: text/plain, Size: 2192 bytes --] On 01/28/19 17:47, Rich Felker wrote: > On Mon, Jan 28, 2019 at 05:41:10PM -0600, A. Wilcox wrote: >> On 01/28/19 17:12, Zach van Rijn wrote: >>> The official community-based musl site is the Wiki [1]; >>> this might be a more appropriate venue for what you're seeking, >>> and it already does exactly what you're seeking. >>> >>> >>> [1]: https://wiki.musl-libc.org/projects-using-musl.html >> >> >> That page is also a bit of a mess. I would clean it up but I don't know >> if I have the time. >> >> It'd be nice if it had a divide between "experimental" distros and >> "production" distros, that is ones that intended to be run on a >> workstation/server vs those that are intended to be poked and prodded >> and used for research. Also, "last activity" would be nice, so you can >> tell what's maintained. >> >> (It'd be cool for Adélie, and probably Alpine, if the list were >> alphabetised, too ;) - but I won't push it.) > > I'm not opposed to either idea, but I'm not sure how we'd define the > experimental vs production distinction. Some dists, especially > source-based ones like Sabotage, lack the selection of packages users > might expect from a production desktop or server distro, but they can > still be very suitable for production use in embedded or in small or > "more traditional" servers without all kinds of modern bloat. > > Maybe it would make sense to suggest a few comparable well-known > distros for each one ("Debian-like", "Gentoo-like", "Buildroot-like", > etc.)? > > Rich Perhaps it would be easier to categorise into "research" and "general use". Research: bootstrap-linux, oasis, Snowflake General use: Adélie, Alpine, Bedrock, CLFS/ELFS, Exherbo, januslinux, LightCube, morpheus, Sabotage, TAZ, Void None of them are Debian-likes. (I might actually be personally insulted if you said Adélie was a Debian-like; we do not use Apt, and we actually care about Libre Software instead of just paying lipservice to it.) Also, I thought Dragora shipped an actual musl version. Maybe I'm mistaken. Best, --arw -- A. Wilcox (awilfox) Project Lead, Adélie Linux https://www.adelielinux.org [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-29 3:22 ` A. Wilcox @ 2019-01-29 19:40 ` Matias Fonzo 0 siblings, 0 replies; 26+ messages in thread From: Matias Fonzo @ 2019-01-29 19:40 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 464 bytes --] On Mon, 28 Jan 2019 21:22:30 -0600 "A. Wilcox" <awilfox@adelielinux.org> wrote: > [..] > > Also, I thought Dragora shipped an actual musl version. Maybe I'm > mistaken. Latest published version is the version 3.0-alpha2, which has musl 1.1.20+. The current one is from the commit "de7dc1318f493184b20f7661bc12b1829b957b67" which is a few commits behind of 1.1.21. The distro is under development up to reach the stable version, for general use. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 949 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Symbol versioning approximation trips on compat symbols 2019-01-28 23:12 ` Zach van Rijn 2019-01-28 23:41 ` A. Wilcox @ 2019-01-29 19:31 ` Matias Fonzo 1 sibling, 0 replies; 26+ messages in thread From: Matias Fonzo @ 2019-01-29 19:31 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 1449 bytes --] On Mon, 28 Jan 2019 18:12:28 -0500 Zach van Rijn <me@zv.io> wrote: > ---- On Mon, 28 Jan 2019 17:52:25 -0500 Matias Fonzo > <selk@dragora.org> wrote ---- > > It should be nice if musl.cc (if this is community-musl page) > > could list all the distributions based on Musl (including Dragora, > > Sabotage, ...) and all the cross-compiler projects, etc. :-) > > > > > > The focus of musl.cc is exclusively musl cross-compilation toolchains, > therefore "musl-based Linux distributions" is off-topic. > > The official community-based musl site is the Wiki [1]; > this might be a more appropriate venue for what you're seeking, > and it already does exactly what you're seeking. But to contribute you need an account in Github, plus send a Pull Request, it would be better if this is more direct, like the usage of Oddmuse[1]. [1] https://oddmuse.org > As does the Wikipedia page [2]. > > It might be good to mention musl on the Dragora Wikipedia page [3], > too. Currently the only page I can find which mentions musl is [4] > even though several distros have been using musl for the better part > of the 2010s. > I don't think so. The information there is out of date, and I haven't be able to edit the most part of the Wikipedia page. Last time that I've tried it is like you can't edit a great portion ... Anyway, I have pending to update the official website. Regards, Matías [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 949 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2019-01-30 14:57 UTC | newest] Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-01-21 17:57 Symbol versioning approximation trips on compat symbols Florian Weimer 2019-01-24 1:43 ` Rich Felker 2019-01-24 9:28 ` u-uy74 2019-01-24 10:11 ` Florian Weimer 2019-01-24 11:09 ` Szabolcs Nagy 2019-01-24 11:18 ` Florian Weimer 2019-01-27 4:04 ` Rich Felker 2019-01-27 9:36 ` u-uy74 2019-01-28 6:34 ` Florian Weimer 2019-01-28 9:17 ` Timo Teras 2019-01-28 11:33 ` Szabolcs Nagy 2019-01-28 12:40 ` Szabolcs Nagy 2019-01-28 13:08 ` (OT?) Re: [musl] " u-uy74 2019-01-28 15:22 ` Markus Wichmann 2019-01-28 15:34 ` Rich Felker 2019-01-28 15:29 ` Rich Felker 2019-01-29 17:13 ` u-uy74 2019-01-30 14:57 ` Rich Felker 2019-01-28 21:57 ` A. Wilcox 2019-01-28 22:52 ` Matias Fonzo 2019-01-28 23:12 ` Zach van Rijn 2019-01-28 23:41 ` A. Wilcox 2019-01-28 23:47 ` Rich Felker 2019-01-29 3:22 ` A. Wilcox 2019-01-29 19:40 ` Matias Fonzo 2019-01-29 19:31 ` Matias Fonzo
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).