* Adjustments to roadmap @ 2015-08-28 2:43 Rich Felker 2015-08-28 3:38 ` Khem Raj ` (3 more replies) 0 siblings, 4 replies; 29+ messages in thread From: Rich Felker @ 2015-08-28 2:43 UTC (permalink / raw) To: musl Hey everyone, Between travel for a CII meeting last month (from which there's exciting stuff I still need to follow up on, especially tlsify), work, and making personal time to enjoy the summer, I've gotten rather behind on musl releases. I think I'm caught up with everything important for 1.1.11 though, and plan to release it ASAP. During this period of release delay, I haven't been updating the roadmap on the wiki, but lots of new ideas have come up for enhancing musl that may be appropriate to prioritize somewhere in the next few release cycles. I'd like to share some of them here and get feedback on what the community/users would like to see: ---------------------------------------------------------------------- 1. Unifying static/shared libc .o files. Right now, all the files in musl are built twice, once as .o for inclusion in libc.a, and once as ".lo" (my naming convention) for inclusion in libc.so. This of course requires twice the time and space to build, and also has semi-gratuitously separate code paths that need to be maintained. I would like to eliminate that. One of the obvious big reasons for building libc.so's files separately is that they need to be PIC. But for static-PIE support, which musl now has and which I'm trying to get upstream in GCC/binutils, libc.a needs to be built as PIC too. I did some tests, and the size increase of libc.a by using PIC is about 1-3% on "important" archs and still only 4-6% on some of the worst ones. I don't have figures on performance cost, though, and it's harder to measure. If we do get rid of the dual-build, it will still be possible to build a non-PIC libc.a via something like --disable-pic, which would imply --disable-shared. It would just be a separate run of the build, which is somewhat inconvenient now but much less so if we add out-of-tree build support. And of course getting rid of the double-build in the makefile would simplify it and make developing out-of-tree build support easier. I have more detailed notes on this topic I'll post later. 2. Rewriting malloc It came to my attention during this last release cycle that musl's strategies for avoiding contention in malloc were leading to excessive heap growth/fragmentation, contrary to my explicit intent. This was the issue Timo Teräs reported for which I made a mitigation as commit c3761622e8168b0c6453637ac82e70b09af3e8e9. I don't yet have a proposed long-term direction for malloc. Some brief testing determined that the fine-grained locking we have now, despite being theoretically of dubious benefit, actually does help; switching to a single lock resulted in significant performance regressions. In addition, I've lost a good deal of faith in the whole "dlmalloc" style heap-management approach. Worst-case fragmentation is expressed by the fact that allocations as small as M/N can fail, where M is the "total free memory" (total minus used) and N is the number of live allocations. It should be possible to achieve much better bounds by avoiding excessive mixing of different-sized chunks, and there are known highly-inefficient ways to do this, but I haven't found any good ones. What I'd like to do at the same time is eliminate most or all of the need for merging of free chunks -- this is the main bottleneck to concurrency. 3. Symbol versioning Right now musl's dynamic linker ignores symbol versioning information except to select the "current" version of a symbol the same way ld does. This is sufficient for most things, if you don't want to support old library ABIs and mixed (i.e. broken) ABIs, but it breaks some hacks GCC is doing in their target libs (libgcc_s.so, ...), and some setups that arguably "should" work, especially for C++ apps. What's probably worse is that you get silent breakage when an app is trying to use symbol versioning, expecting it to work, and it doesn't. I don't think the status quo is a reasonable option. We should either teach GCC that musl targets don't support symbol versioning, and make sure apps/libs' build systems detect this, or we should make them work. My leaning is towards the latter. This is not an endorsement of symbol versioning. It's a poor approximation of the best possible solution to a problem that's not fully solvable, and it DOES introduce the possibility of silent breakage in most places where it's used, but there are ways to use it safely for some limited purposes, and supporting these seems to be the path of least headache. 4. Dynamic linker library chain changes There is some ambiguity of global symbol definition priority when a library is first loaded as RTLD_LOCAL then later moved to the global namespace with RTLD_GLOBAL, but basically I think what musl is doing right now is "wrong" short of an official interpretation of the standard to the contrary. If this is resolved, I'd like to rework how musl does its linked list of shared libraries. Right now there's just one list with a flag for each library indicating whether it's in the global namespace or not. This forces searches for symbols to step through libraries that are not relevant, and also forces TLS initialization for new threads to step through all the libraries even if only a few actually have TLS. What I'd like to do is have 3 or more separate lists: one for all the libraries (for gdb and dl_iterate_phdr to use), one representing the global namespace, and one for just the libraries with TLS. This trades a very small amount of memory in 'struct dso' for significant performance improvements in programs with large numbers of libraries. ---------------------------------------------------------------------- In addition to the above, there are all of the existing roadmap items on the wiki which are open for discussion of how they should be prioritized. The big projects are roughly: - Atomics refactorization/deduplication - Bits refactorization/deduplication - Out-of-tree builds - LC_COLLATE - IDN - Advanced glibc ABI-compat features in dynamic linker - Documentation Apologies for the slow progress lately. Don't worry though, there's still lots more good stuff to come for musl. Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 2:43 Adjustments to roadmap Rich Felker @ 2015-08-28 3:38 ` Khem Raj 2015-08-28 4:21 ` Rich Felker 2015-08-28 7:24 ` u-wsnj ` (2 subsequent siblings) 3 siblings, 1 reply; 29+ messages in thread From: Khem Raj @ 2015-08-28 3:38 UTC (permalink / raw) To: musl On Thu, Aug 27, 2015 at 7:43 PM, Rich Felker <dalias@libc.org> wrote: > Hey everyone, > > Between travel for a CII meeting last month (from which there's > exciting stuff I still need to follow up on, especially tlsify), work, > and making personal time to enjoy the summer, I've gotten rather > behind on musl releases. I think I'm caught up with everything > important for 1.1.11 though, and plan to release it ASAP. > > During this period of release delay, I haven't been updating the > roadmap on the wiki, but lots of new ideas have come up for enhancing > musl that may be appropriate to prioritize somewhere in the next few > release cycles. I'd like to share some of them here and get feedback > on what the community/users would like to see: > > ---------------------------------------------------------------------- > > 1. Unifying static/shared libc .o files. > > Right now, all the files in musl are built twice, once as .o for > inclusion in libc.a, and once as ".lo" (my naming convention) for .lo is conflicting with libtool generated objects btw. > inclusion in libc.so. This of course requires twice the time and space > to build, and also has semi-gratuitously separate code paths that need > to be maintained. I would like to eliminate that. > > One of the obvious big reasons for building libc.so's files separately > is that they need to be PIC. But for static-PIE support, which musl > now has and which I'm trying to get upstream in GCC/binutils, libc.a > needs to be built as PIC too. I did some tests, and the size increase > of libc.a by using PIC is about 1-3% on "important" archs and still > only 4-6% on some of the worst ones. I don't have figures on > performance cost, though, and it's harder to measure. > > If we do get rid of the dual-build, it will still be possible to build > a non-PIC libc.a via something like --disable-pic, which would imply > --disable-shared. It would just be a separate run of the build, which > is somewhat inconvenient now but much less so if we add out-of-tree > build support. And of course getting rid of the double-build in the > makefile would simplify it and make developing out-of-tree build > support easier. This seems a good proposal. We need some data on arches like mips > > I have more detailed notes on this topic I'll post later. > > 2. Rewriting malloc > > It came to my attention during this last release cycle that musl's > strategies for avoiding contention in malloc were leading to excessive > heap growth/fragmentation, contrary to my explicit intent. This was > the issue Timo Teräs reported for which I made a mitigation as commit > c3761622e8168b0c6453637ac82e70b09af3e8e9. > > I don't yet have a proposed long-term direction for malloc. Some brief > testing determined that the fine-grained locking we have now, despite > being theoretically of dubious benefit, actually does help; switching > to a single lock resulted in significant performance regressions. > > In addition, I've lost a good deal of faith in the whole "dlmalloc" > style heap-management approach. Worst-case fragmentation is expressed > by the fact that allocations as small as M/N can fail, where M is the > "total free memory" (total minus used) and N is the number of live > allocations. It should be possible to achieve much better bounds by > avoiding excessive mixing of different-sized chunks, and there are > known highly-inefficient ways to do this, but I haven't found any good > ones. > > What I'd like to do at the same time is eliminate most or all of the > need for merging of free chunks -- this is the main bottleneck to > concurrency. > > 3. Symbol versioning > > Right now musl's dynamic linker ignores symbol versioning information > except to select the "current" version of a symbol the same way ld > does. This is sufficient for most things, if you don't want to support > old library ABIs and mixed (i.e. broken) ABIs, but it breaks some > hacks GCC is doing in their target libs (libgcc_s.so, ...), and some > setups that arguably "should" work, especially for C++ apps. What's > probably worse is that you get silent breakage when an app is trying > to use symbol versioning, expecting it to work, and it doesn't. > > I don't think the status quo is a reasonable option. We should either > teach GCC that musl targets don't support symbol versioning, and make > sure apps/libs' build systems detect this, or we should make them > work. My leaning is towards the latter. we can detect using triplets so apps should be able to conflgure for musl. I think symbol versioning could be a good thing for musl for backward compatibilty with itself who knows what future holds for musl, we might have to deal with own past. but it should be a build time option, for glibc if we disable versioned symbols code size reduces significantly. > > This is not an endorsement of symbol versioning. It's a poor > approximation of the best possible solution to a problem that's not > fully solvable, and it DOES introduce the possibility of silent > breakage in most places where it's used, but there are ways to use it > safely for some limited purposes, and supporting these seems to be the > path of least headache. > > 4. Dynamic linker library chain changes > > There is some ambiguity of global symbol definition priority when a > library is first loaded as RTLD_LOCAL then later moved to the global > namespace with RTLD_GLOBAL, but basically I think what musl is doing > right now is "wrong" short of an official interpretation of the > standard to the contrary. > > If this is resolved, I'd like to rework how musl does its linked list > of shared libraries. Right now there's just one list with a flag for > each library indicating whether it's in the global namespace or not. > This forces searches for symbols to step through libraries that are > not relevant, and also forces TLS initialization for new threads to > step through all the libraries even if only a few actually have TLS. > > What I'd like to do is have 3 or more separate lists: one for all the > libraries (for gdb and dl_iterate_phdr to use), one representing the > global namespace, and one for just the libraries with TLS. This trades > a very small amount of memory in 'struct dso' for significant > performance improvements in programs with large numbers of libraries. > > ---------------------------------------------------------------------- > > In addition to the above, there are all of the existing roadmap items > on the wiki which are open for discussion of how they should be > prioritized. The big projects are roughly: > > - Atomics refactorization/deduplication > - Bits refactorization/deduplication > - Out-of-tree builds > - LC_COLLATE > - IDN > - Advanced glibc ABI-compat features in dynamic linker > - Documentation how about nommu ? > > Apologies for the slow progress lately. Don't worry though, there's > still lots more good stuff to come for musl. Some goals around code size might be interesting too. > > Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 3:38 ` Khem Raj @ 2015-08-28 4:21 ` Rich Felker 2015-08-28 6:57 ` Вася Бойцов 2015-08-28 12:16 ` Justin Cormack 0 siblings, 2 replies; 29+ messages in thread From: Rich Felker @ 2015-08-28 4:21 UTC (permalink / raw) To: musl On Thu, Aug 27, 2015 at 08:38:16PM -0700, Khem Raj wrote: > On Thu, Aug 27, 2015 at 7:43 PM, Rich Felker <dalias@libc.org> wrote: > > Hey everyone, > > > > Between travel for a CII meeting last month (from which there's > > exciting stuff I still need to follow up on, especially tlsify), work, > > and making personal time to enjoy the summer, I've gotten rather > > behind on musl releases. I think I'm caught up with everything > > important for 1.1.11 though, and plan to release it ASAP. > > > > During this period of release delay, I haven't been updating the > > roadmap on the wiki, but lots of new ideas have come up for enhancing > > musl that may be appropriate to prioritize somewhere in the next few > > release cycles. I'd like to share some of them here and get feedback > > on what the community/users would like to see: > > > > ---------------------------------------------------------------------- > > > > 1. Unifying static/shared libc .o files. > > > > Right now, all the files in musl are built twice, once as .o for > > inclusion in libc.a, and once as ".lo" (my naming convention) for > > ..lo is conflicting with libtool generated objects btw. Yes. Since we're not using libtool that doesn't matter but it may be confusing to some people who see it. > > inclusion in libc.so. This of course requires twice the time and space > > to build, and also has semi-gratuitously separate code paths that need > > to be maintained. I would like to eliminate that. > > > > One of the obvious big reasons for building libc.so's files separately > > is that they need to be PIC. But for static-PIE support, which musl > > now has and which I'm trying to get upstream in GCC/binutils, libc.a > > needs to be built as PIC too. I did some tests, and the size increase > > of libc.a by using PIC is about 1-3% on "important" archs and still > > only 4-6% on some of the worst ones. I don't have figures on > > performance cost, though, and it's harder to measure. > > > > If we do get rid of the dual-build, it will still be possible to build > > a non-PIC libc.a via something like --disable-pic, which would imply > > --disable-shared. It would just be a separate run of the build, which > > is somewhat inconvenient now but much less so if we add out-of-tree > > build support. And of course getting rid of the double-build in the > > makefile would simplify it and make developing out-of-tree build > > support easier. > > This seems a good proposal. We need some data on arches like mips MIPS is always semi-PIC anyway, so the size increase is fairly minimal there -- about 3%. Actually I'm a bit puzzled why it's even that much. > > 3. Symbol versioning > > > > Right now musl's dynamic linker ignores symbol versioning information > > except to select the "current" version of a symbol the same way ld > > does. This is sufficient for most things, if you don't want to support > > old library ABIs and mixed (i.e. broken) ABIs, but it breaks some > > hacks GCC is doing in their target libs (libgcc_s.so, ...), and some > > setups that arguably "should" work, especially for C++ apps. What's > > probably worse is that you get silent breakage when an app is trying > > to use symbol versioning, expecting it to work, and it doesn't. > > > > I don't think the status quo is a reasonable option. We should either > > teach GCC that musl targets don't support symbol versioning, and make > > sure apps/libs' build systems detect this, or we should make them > > work. My leaning is towards the latter. > > we can detect using triplets so apps should be able to conflgure for musl. > I think symbol versioning could be a good thing for musl for backward > compatibilty with itself > who knows what future holds for musl, we might have to deal with own > past. but it should be a build time > option, for glibc if we disable versioned symbols code size reduces > significantly. Symbol versioning is generally not a valid approach to libc ABI problems, because it only addresses the ABI between a consumer of the library and that library, not the ABIs between multiple consumers of the library (e.g. an app and another library that both use types from libc). It's also broken in that the version is bound at link-time, but what matters is the version code was using at compile-time. This distinction screws up the behavior for static linking completely. So I think we can confidently say versioning of libc symbols does not have any place in musl. (Also, if libc.so had a version table, it would break glibc ABI-compat completely since the version strings would not match the caller's references. If we support version tables, libc.so actually needs to ensure at runtime that any symbol table it has gets ignored, since it might inadvertently pick up versions from the compiler runtime libgcc.a/etc. in the future.) > > In addition to the above, there are all of the existing roadmap items > > on the wiki which are open for discussion of how they should be > > prioritized. The big projects are roughly: > > > > - Atomics refactorization/deduplication > > - Bits refactorization/deduplication > > - Out-of-tree builds > > - LC_COLLATE > > - IDN > > - Advanced glibc ABI-compat features in dynamic linker > > - Documentation > > how about nommu ? I think the 1.1.11 release will cover most of the generic NOMMU support considerations, but we could add: 5. FDPIC entry point and dynamic linker code. 6. Support for building as pure thumb2 code for ARM Cortex-M. Did you have other NOMMU needs in mind? > > Apologies for the slow progress lately. Don't worry though, there's > > still lots more good stuff to come for musl. > > Some goals around code size might be interesting too. Anything particular in mind? Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 4:21 ` Rich Felker @ 2015-08-28 6:57 ` Вася Бойцов 2015-08-28 12:47 ` Rich Felker 2015-08-28 12:16 ` Justin Cormack 1 sibling, 1 reply; 29+ messages in thread From: Вася Бойцов @ 2015-08-28 6:57 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 6244 bytes --] Hello, I would like to know what is the current progress and ideas on PAM with LDAP calls what is the roadmap for this feature and workaround for using it now? On Fri, Aug 28, 2015 at 7:21 AM, Rich Felker <dalias@libc.org> wrote: > On Thu, Aug 27, 2015 at 08:38:16PM -0700, Khem Raj wrote: > > On Thu, Aug 27, 2015 at 7:43 PM, Rich Felker <dalias@libc.org> wrote: > > > Hey everyone, > > > > > > Between travel for a CII meeting last month (from which there's > > > exciting stuff I still need to follow up on, especially tlsify), work, > > > and making personal time to enjoy the summer, I've gotten rather > > > behind on musl releases. I think I'm caught up with everything > > > important for 1.1.11 though, and plan to release it ASAP. > > > > > > During this period of release delay, I haven't been updating the > > > roadmap on the wiki, but lots of new ideas have come up for enhancing > > > musl that may be appropriate to prioritize somewhere in the next few > > > release cycles. I'd like to share some of them here and get feedback > > > on what the community/users would like to see: > > > > > > ---------------------------------------------------------------------- > > > > > > 1. Unifying static/shared libc .o files. > > > > > > Right now, all the files in musl are built twice, once as .o for > > > inclusion in libc.a, and once as ".lo" (my naming convention) for > > > > ..lo is conflicting with libtool generated objects btw. > > Yes. Since we're not using libtool that doesn't matter but it may be > confusing to some people who see it. > > > > inclusion in libc.so. This of course requires twice the time and space > > > to build, and also has semi-gratuitously separate code paths that need > > > to be maintained. I would like to eliminate that. > > > > > > One of the obvious big reasons for building libc.so's files separately > > > is that they need to be PIC. But for static-PIE support, which musl > > > now has and which I'm trying to get upstream in GCC/binutils, libc.a > > > needs to be built as PIC too. I did some tests, and the size increase > > > of libc.a by using PIC is about 1-3% on "important" archs and still > > > only 4-6% on some of the worst ones. I don't have figures on > > > performance cost, though, and it's harder to measure. > > > > > > If we do get rid of the dual-build, it will still be possible to build > > > a non-PIC libc.a via something like --disable-pic, which would imply > > > --disable-shared. It would just be a separate run of the build, which > > > is somewhat inconvenient now but much less so if we add out-of-tree > > > build support. And of course getting rid of the double-build in the > > > makefile would simplify it and make developing out-of-tree build > > > support easier. > > > > This seems a good proposal. We need some data on arches like mips > > MIPS is always semi-PIC anyway, so the size increase is fairly minimal > there -- about 3%. Actually I'm a bit puzzled why it's even that much. > > > > 3. Symbol versioning > > > > > > Right now musl's dynamic linker ignores symbol versioning information > > > except to select the "current" version of a symbol the same way ld > > > does. This is sufficient for most things, if you don't want to support > > > old library ABIs and mixed (i.e. broken) ABIs, but it breaks some > > > hacks GCC is doing in their target libs (libgcc_s.so, ...), and some > > > setups that arguably "should" work, especially for C++ apps. What's > > > probably worse is that you get silent breakage when an app is trying > > > to use symbol versioning, expecting it to work, and it doesn't. > > > > > > I don't think the status quo is a reasonable option. We should either > > > teach GCC that musl targets don't support symbol versioning, and make > > > sure apps/libs' build systems detect this, or we should make them > > > work. My leaning is towards the latter. > > > > we can detect using triplets so apps should be able to conflgure for > musl. > > I think symbol versioning could be a good thing for musl for backward > > compatibilty with itself > > who knows what future holds for musl, we might have to deal with own > > past. but it should be a build time > > option, for glibc if we disable versioned symbols code size reduces > > significantly. > > Symbol versioning is generally not a valid approach to libc ABI > problems, because it only addresses the ABI between a consumer of the > library and that library, not the ABIs between multiple consumers of > the library (e.g. an app and another library that both use types from > libc). It's also broken in that the version is bound at link-time, but > what matters is the version code was using at compile-time. This > distinction screws up the behavior for static linking completely. > > So I think we can confidently say versioning of libc symbols does not > have any place in musl. (Also, if libc.so had a version table, it > would break glibc ABI-compat completely since the version strings > would not match the caller's references. If we support version tables, > libc.so actually needs to ensure at runtime that any symbol table it > has gets ignored, since it might inadvertently pick up versions from > the compiler runtime libgcc.a/etc. in the future.) > > > > In addition to the above, there are all of the existing roadmap items > > > on the wiki which are open for discussion of how they should be > > > prioritized. The big projects are roughly: > > > > > > - Atomics refactorization/deduplication > > > - Bits refactorization/deduplication > > > - Out-of-tree builds > > > - LC_COLLATE > > > - IDN > > > - Advanced glibc ABI-compat features in dynamic linker > > > - Documentation > > > > how about nommu ? > > I think the 1.1.11 release will cover most of the generic NOMMU > support considerations, but we could add: > > 5. FDPIC entry point and dynamic linker code. > > 6. Support for building as pure thumb2 code for ARM Cortex-M. > > Did you have other NOMMU needs in mind? > > > > Apologies for the slow progress lately. Don't worry though, there's > > > still lots more good stuff to come for musl. > > > > Some goals around code size might be interesting too. > > Anything particular in mind? > > Rich > -- Respectfully, Boytsov Vasiliy [-- Attachment #2: Type: text/html, Size: 7933 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 6:57 ` Вася Бойцов @ 2015-08-28 12:47 ` Rich Felker 0 siblings, 0 replies; 29+ messages in thread From: Rich Felker @ 2015-08-28 12:47 UTC (permalink / raw) To: musl On Fri, Aug 28, 2015 at 09:57:56AM +0300, Вася Бойцов wrote: > Hello, I would like to know what is the current progress and ideas on PAM > with LDAP calls what is the roadmap for this feature and workaround for > using it now? If you mean passwd/group db lookup (that's separate from PAM which has nothing to do with libc functionality), then on the libc side it should be complete. However an outside daemon providing query service on the nscd socket is needed. Everything seems to work if you have glibc's nscd doing the work, but somebody still needs to write an LDAP bridge (or general bridge that can use nss modules or something). While it's not code internal to musl, having such a third-party daemon available is on the 1.2.0 goals. Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 4:21 ` Rich Felker 2015-08-28 6:57 ` Вася Бойцов @ 2015-08-28 12:16 ` Justin Cormack 1 sibling, 0 replies; 29+ messages in thread From: Justin Cormack @ 2015-08-28 12:16 UTC (permalink / raw) To: musl On 28 August 2015 at 05:21, Rich Felker <dalias@libc.org> wrote: >> > >> > 1. Unifying static/shared libc .o files. >> > >> > Right now, all the files in musl are built twice, once as .o for >> > inclusion in libc.a, and once as ".lo" (my naming convention) for >> >> ..lo is conflicting with libtool generated objects btw. > > Yes. Since we're not using libtool that doesn't matter but it may be > confusing to some people who see it. NetBSD uses .pico for this. Justin ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 2:43 Adjustments to roadmap Rich Felker 2015-08-28 3:38 ` Khem Raj @ 2015-08-28 7:24 ` u-wsnj 2015-08-28 11:39 ` Рысь 2015-08-30 5:18 ` Рысь 2015-08-31 7:09 ` Adjustments to roadmap Rich Felker 3 siblings, 1 reply; 29+ messages in thread From: u-wsnj @ 2015-08-28 7:24 UTC (permalink / raw) To: musl On Thu, Aug 27, 2015 at 10:43:48PM -0400, Rich Felker wrote: > 3. Symbol versioning > > Right now musl's dynamic linker ignores symbol versioning information > except to select the "current" version of a symbol the same way ld > does. This is sufficient for most things, if you don't want to support > old library ABIs and mixed (i.e. broken) ABIs, but it breaks some > hacks GCC is doing in their target libs (libgcc_s.so, ...), and some > setups that arguably "should" work, especially for C++ apps. What's > probably worse is that you get silent breakage when an app is trying > to use symbol versioning, expecting it to work, and it doesn't. The symbol versioning is hardly the only or most straightforward or most efficient way to manage ABI contracts between multiple parties. I feel that it emerged to mitigate problems created by traditional deployment practices. "Traditional" implies "formed in the past, given differing prerequisites", aka suboptimal. I our deployments symbol versioning is fully redundant, I'm happy not having to deal with it with musl. > I don't think the status quo is a reasonable option. We should either > teach GCC that musl targets don't support symbol versioning, and make > sure apps/libs' build systems detect this, or we should make them > work. My leaning is towards the latter. I'd rather prefer the former. Otherwise supporting an approach chosen for unrelated reasons somewhere else imposes a certain complexity cost on musl and on any packager/integrator who does not need versioning. Rune ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 7:24 ` u-wsnj @ 2015-08-28 11:39 ` Рысь 2015-08-28 17:18 ` Rich Felker 0 siblings, 1 reply; 29+ messages in thread From: Рысь @ 2015-08-28 11:39 UTC (permalink / raw) To: musl On Fri, 28 Aug 2015 09:24:22 +0200 u-wsnj@aetey.se wrote: > > I don't think the status quo is a reasonable option. We should > > either teach GCC that musl targets don't support symbol versioning, > > and make sure apps/libs' build systems detect this, or we should > > make them work. My leaning is towards the latter. > > I'd rather prefer the former. Otherwise supporting an approach chosen > for unrelated reasons somewhere else imposes a certain complexity > cost on musl and on any packager/integrator who does not need > versioning. > > Rune > I agree with this too. What are the real reasons raised to reconsider support for symbol versioning? How other not glibc build environments currently handle them? ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 11:39 ` Рысь @ 2015-08-28 17:18 ` Rich Felker 2015-08-30 4:21 ` Рысь 0 siblings, 1 reply; 29+ messages in thread From: Rich Felker @ 2015-08-28 17:18 UTC (permalink / raw) To: musl On Fri, Aug 28, 2015 at 06:39:32PM +0700, Рысь wrote: > On Fri, 28 Aug 2015 09:24:22 +0200 > u-wsnj@aetey.se wrote: > > > > I don't think the status quo is a reasonable option. We should > > > either teach GCC that musl targets don't support symbol versioning, > > > and make sure apps/libs' build systems detect this, or we should > > > make them work. My leaning is towards the latter. > > > > I'd rather prefer the former. Otherwise supporting an approach chosen > > for unrelated reasons somewhere else imposes a certain complexity > > cost on musl and on any packager/integrator who does not need > > versioning. > > > > Rune > > > > I agree with this too. What are the real reasons raised to reconsider > support for symbol versioning? How other not glibc build environments > currently handle them? The real reasons are what I said: - Internal use by some software of versioned symbols, e.g. in libgcc. I would not call this as packaging/integration issue because it's not solved by using matching library versions at runtime. It requires internal changes that are not formulatic but require individual consideration for any software affected to develop and alternate solution. - Presumed (I haven't tested this) breakage using C++ programs built with different GCC versions (and maybe even just different -std options?) with the latest libstdc++. Symbol versioning actually can't make this work when there are multiple C++ shared libraries in the same program built with different libstdc++ versions, but people generally know C++ is ABI-hell and static link most of all of the C++ code, only using dynamic linking for C libraries, and in that case symbol versioning _should_ be sufficient to make it work safely. Unlike the first issue, this issue _could_ be addressed at he packaging/integration level by making different SONAMEs for different libstdc++ versions/ABIs, or different library paths for different apps, and I'm aware that Rune is doing something like this, but I don't think it's reasonable to tell users they have to deviate heavily from upstream GCC intent to make this work. - Silent breakage in various libraries that use symbol versioning and break their ABIs. As I've said before, symbol versioning does NOT solve this problem correctly or safely, but the chance of nasty, dangerous blow-up is much higher when apps/libs _think_ they have symbol versioning at their disposal but it doesn't actually work. This issue (unlike the above 2) could probably be fixed by teaching the toolchain that musl does not support versioned symbols, but I have no real-world information on how many libs would fallback to some other solution (SONAMEs?) vs simply refusing to build. As I've said I'm confident in my existing conclusion that symbol versioning was a misfeature and that it's not something musl should use. But however dubious it is, it's a dynamic-linking behavior that a fair amount of software expects should work. The only potential cost of supporting it is some impact on dynamic linking time, which I'll of course work to minimize if we support it. Code size impact is negligible IMO. BTW there is also one use for symbol versioning that's semi-legitimate and not related to mixing ABIS; I believe glibc makes use of this. If you have a family of related shared libraries and want to provide some symbols for private interfaces between them, you can use a non-default version. This prevents the symbol from being linkable by ld without specifying a specific version (e.g. so configure scripts won't detect the presence of the symbol). I would rather this feature have never existed, but it does exist, and software that uses it is not doing anything sketchy with mixing of incompatible ABIs via versioning; it's just using the mechanism to achieve a different result. Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 17:18 ` Rich Felker @ 2015-08-30 4:21 ` Рысь 2015-08-30 4:46 ` Rich Felker 0 siblings, 1 reply; 29+ messages in thread From: Рысь @ 2015-08-30 4:21 UTC (permalink / raw) To: musl On Fri, 28 Aug 2015 13:18:18 -0400 Rich Felker <dalias@libc.org> wrote: > On Fri, Aug 28, 2015 at 06:39:32PM +0700, Рысь wrote: > > On Fri, 28 Aug 2015 09:24:22 +0200 > > u-wsnj@aetey.se wrote: > > > > > > I don't think the status quo is a reasonable option. We should > > > > either teach GCC that musl targets don't support symbol > > > > versioning, and make sure apps/libs' build systems detect this, > > > > or we should make them work. My leaning is towards the latter. > > > > > > I'd rather prefer the former. Otherwise supporting an approach > > > chosen for unrelated reasons somewhere else imposes a certain > > > complexity cost on musl and on any packager/integrator who does > > > not need versioning. > > > > > > Rune > > > > > > > I agree with this too. What are the real reasons raised to > > reconsider support for symbol versioning? How other not glibc build > > environments currently handle them? > > The real reasons are what I said: > > - Internal use by some software of versioned symbols, e.g. in libgcc. > I would not call this as packaging/integration issue because it's > not solved by using matching library versions at runtime. It > requires internal changes that are not formulatic but require > individual consideration for any software affected to develop and > alternate solution. > > - Presumed (I haven't tested this) breakage using C++ programs built > with different GCC versions (and maybe even just different -std > options?) with the latest libstdc++. Symbol versioning actually > can't make this work when there are multiple C++ shared libraries in > the same program built with different libstdc++ versions, but people > generally know C++ is ABI-hell and static link most of all of the > C++ code, only using dynamic linking for C libraries, and in that > case symbol versioning _should_ be sufficient to make it work > safely. Unlike the first issue, this issue _could_ be addressed at > he packaging/integration level by making different SONAMEs for > different libstdc++ versions/ABIs, or different library paths for > different apps, and I'm aware that Rune is doing something like > this, but I don't think it's reasonable to tell users they have to > deviate heavily from upstream GCC intent to make this work. > > - Silent breakage in various libraries that use symbol versioning and > break their ABIs. As I've said before, symbol versioning does NOT > solve this problem correctly or safely, but the chance of nasty, > dangerous blow-up is much higher when apps/libs _think_ they have > symbol versioning at their disposal but it doesn't actually work. > This issue (unlike the above 2) could probably be fixed by teaching > the toolchain that musl does not support versioned symbols, but I > have no real-world information on how many libs would fallback to > some other solution (SONAMEs?) vs simply refusing to build. > > As I've said I'm confident in my existing conclusion that symbol > versioning was a misfeature and that it's not something musl should > use. But however dubious it is, it's a dynamic-linking behavior that a > fair amount of software expects should work. The only potential cost > of supporting it is some impact on dynamic linking time, which I'll of > course work to minimize if we support it. Code size impact is > negligible IMO. > > BTW there is also one use for symbol versioning that's semi-legitimate > and not related to mixing ABIS; I believe glibc makes use of this. If > you have a family of related shared libraries and want to provide some > symbols for private interfaces between them, you can use a non-default > version. This prevents the symbol from being linkable by ld without > specifying a specific version (e.g. so configure scripts won't detect > the presence of the symbol). I would rather this feature have never > existed, but it does exist, and software that uses it is not doing > anything sketchy with mixing of incompatible ABIs via versioning; it's > just using the mechanism to achieve a different result. > > Rich Don't you think that the snowball effect will be later in future so much amplified by this decision so that it will not be even able to be patched out? Well, then, that's why I prefer to use old stable versions which do not suffer from these virtual problems. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-30 4:21 ` Рысь @ 2015-08-30 4:46 ` Rich Felker 2015-08-30 5:13 ` Рысь 0 siblings, 1 reply; 29+ messages in thread From: Rich Felker @ 2015-08-30 4:46 UTC (permalink / raw) To: musl On Sun, Aug 30, 2015 at 11:21:28AM +0700, Рысь wrote: > Don't you think that the snowball effect will be later in future so much > amplified by this decision so that it will not be even able to be > patched out? Well, then, that's why I prefer to use old stable versions > which do not suffer from these virtual problems. No, I don't, mainly because I don't ascribe that level of self-importance to musl. :-) People who are using misguided features like symbol versioning are not making the decision to do so based on whether or not musl supports them. And there are already good reasons to argue against use of symbol versioning aside from "it doesn't work with musl" such as how it interacts with static linking. As for your concern about "being unable to patch it out", if you're running correct/matching library versions for your apps so they don't need old symbol versions, the worst that should happen from "turning back off" version support in musl ldso would be that libgcc_s breaks (due to the internal use of a non-public version); then you just go and patch libgcc_s not to do this. If on the other hand you _do_ have mismatched library versions such that old symbol versions are needed, then right now you've got silent breakage with musl; if we supported versions and then you went and patched that out, you'd just end up back where we are now, but not with any new breakage. Does that adequately address your concerns? Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-30 4:46 ` Rich Felker @ 2015-08-30 5:13 ` Рысь 2015-08-30 5:30 ` Rich Felker 0 siblings, 1 reply; 29+ messages in thread From: Рысь @ 2015-08-30 5:13 UTC (permalink / raw) To: musl On Sun, 30 Aug 2015 00:46:54 -0400 Rich Felker <dalias@libc.org> wrote: > On Sun, Aug 30, 2015 at 11:21:28AM +0700, Рысь wrote: > > Don't you think that the snowball effect will be later in future so > > much amplified by this decision so that it will not be even able to > > be patched out? Well, then, that's why I prefer to use old stable > > versions which do not suffer from these virtual problems. > > No, I don't, mainly because I don't ascribe that level of > self-importance to musl. :-) People who are using misguided features > like symbol versioning are not making the decision to do so based on > whether or not musl supports them. And there are already good reasons > to argue against use of symbol versioning aside from "it doesn't work > with musl" such as how it interacts with static linking. > > As for your concern about "being unable to patch it out", if you're > running correct/matching library versions for your apps so they don't > need old symbol versions, the worst that should happen from "turning > back off" version support in musl ldso would be that libgcc_s breaks > (due to the internal use of a non-public version); then you just go > and patch libgcc_s not to do this. > > If on the other hand you _do_ have mismatched library versions such > that old symbol versions are needed, then right now you've got silent > breakage with musl; if we supported versions and then you went and > patched that out, you'd just end up back where we are now, but not > with any new breakage. > > Does that adequately address your concerns? > > Rich The snowball effect I referred to is that your silent decision will amplify in future by allowing third parties to use symvers more and more in their code, then it will be so widespread it will be impossible to run anything without them. Instead, you and community as an alternative to glibc should teach them not to do. This is not a purely technical talk, sorry. You can't refere to pure technical reasons here. Why you want to be clean and straight in the area where hacks are in general use? For myself I already did everything in my systems to be sure that there is no such treat as "symbol versioning": patching binutils ld not to accept them, then patching every thing which silently requires them (alarmed by build failures in this way of course). I also do not use recent gcc, so I don't know which problems have arised again and why now again "libgcc_s breaks" (and why I don't even have it in my systems, both "desktop" and "server" configurations which include C++ complex code like qt4). I do care about future musl compatibility with existing code. All that new stuff is great and I tested 1.1.10 works on my systems without breakage, but when I hear "symbol versioning" that causes some fear despite I think it's only local hacks to support some gcc madness again (right?). I did fight with glibc symvers long time before to just forget about this always arising treat. Personally, I think symvers is a child of hell and always should be avoided and projects who use them should be teached about dangers. I also understand people who just don't know what "symbol versioning" is, but musl usage is mainly embedded plus those ideal desktops and alike built by those who know Linux more beyond "how to build kernel". And Linux itself never was a system which can be universally plugged in everywhere (or we'd have a non-configurable kernel). And how other non-glibc build environments solve libgcc_s breakage? Or they dead already? ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-30 5:13 ` Рысь @ 2015-08-30 5:30 ` Rich Felker 2015-08-30 9:00 ` Szabolcs Nagy 2015-08-30 17:13 ` Рысь 0 siblings, 2 replies; 29+ messages in thread From: Rich Felker @ 2015-08-30 5:30 UTC (permalink / raw) To: musl On Sun, Aug 30, 2015 at 12:13:53PM +0700, Рысь wrote: > The snowball effect I referred to is that your silent decision will > amplify in future by allowing third parties to use symvers more and > more in their code, This is what I don't buy. You're seriously overestimating the degree to which people doing this care about musl. Most likely they are not even aware of musl; much less do they care if I tell them they can't use symbol versioning. > then it will be so widespread it will be impossible > to run anything without them. That hasn't happened yet despite the people who use it only caring about "GNU/Linux" (glibc). > Instead, you and community as an > alternative to glibc should teach them not to do. This is not a purely > technical talk, sorry. You can't refere to pure technical reasons here. > Why you want to be clean and straight in the area where hacks are in > general use? I think there's a good opportunity for education not to use symbol versioning just when people ask why musl libc.so itself doesn't have versions or why they can't use versioning to achieve something musl-internal. There's certainly also _some_ weight to be had in filing bug reports that say "this is breaking when you run it on musl libc because it uses symbol versioning", but it seems a lot more likely to me that this just becomes a disincentive for them to address any musl-related bug reports. They can come back and say "symbol versions are a documented ELF feature in the toolchain, and if musl's ldso doesn't support them it's broken". I disagree with this view, but it's a plausible position they can take. So from my standpoint, "we support this misfeature despite it being a misfeature, but you really should make things work without it because static linking is broken, and our users need static linking" is a much more diplomatic argument. > For myself I already did everything in my systems to be sure that there > is no such treat as "symbol versioning": patching binutils ld not to > accept them, then patching every thing which silently requires them > (alarmed by build failures in this way of course). That's a perfectly fine approach, and if that's your setup, no harm can come from patching hypothetical symver support out of musl since there are no symbol versions present to begin with. > I also do not use > recent gcc, so I don't know which problems have arised again and why > now again "libgcc_s breaks" (and why I don't even have it in my > systems, both "desktop" and "server" configurations which include C++ > complex code like qt4). This is not an option for everyone. For example your version of gcc does not have aarch64 support, nor does it have risc-v support which is not even in upstream gcc yet afaik, and which will be very important for future open hardware that free and auditable all the way down to the metal. Backporting new archs or features from new language standards to old gcc is A LOT of work. I would love to see somebody do it, but I'm not going to be the one who does. > And how other non-glibc build environments solve libgcc_s breakage? > Or they dead already? This use of versions is Linux/ELF-specific, and the only other player, uClibc, supports symbol versioning AFAIK. Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-30 5:30 ` Rich Felker @ 2015-08-30 9:00 ` Szabolcs Nagy 2015-08-30 17:09 ` Rich Felker 2015-08-30 17:13 ` Рысь 1 sibling, 1 reply; 29+ messages in thread From: Szabolcs Nagy @ 2015-08-30 9:00 UTC (permalink / raw) To: musl * Rich Felker <dalias@libc.org> [2015-08-30 01:30:37 -0400]: > On Sun, Aug 30, 2015 at 12:13:53PM +0700, ???????? wrote: > > I also do not use > > recent gcc, so I don't know which problems have arised again and why > > now again "libgcc_s breaks" (and why I don't even have it in my > > systems, both "desktop" and "server" configurations which include C++ > > complex code like qt4). > > This is not an option for everyone. For example your version of gcc > does not have aarch64 support, nor does it have risc-v support which > is not even in upstream gcc yet afaik, and which will be very > important for future open hardware that free and auditable all the way the libgcc_s issue is x86 specific (because only x86 had the broken ifunc based multiversioning support) it is possible to patch out, but would make it impossible to use musl-gcc wrapper with existing gcc-6.* on x86 there are other reasons for symvers: debian is willing to accept patches for a musl based debian, however they use symbol versioning a lot to avoid frequently changing the .so name. i.e. if we support symvers we may be able to create a musl based debian package repo without much effort or maintainance work. (it is probably possible to rebuild all dependent packages on every minor abi change in a library, but then somebody would need to do that work and users will need to download more packages on an update). ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-30 9:00 ` Szabolcs Nagy @ 2015-08-30 17:09 ` Rich Felker 2015-08-30 17:45 ` u-wsnj 0 siblings, 1 reply; 29+ messages in thread From: Rich Felker @ 2015-08-30 17:09 UTC (permalink / raw) To: musl On Sun, Aug 30, 2015 at 11:00:13AM +0200, Szabolcs Nagy wrote: > * Rich Felker <dalias@libc.org> [2015-08-30 01:30:37 -0400]: > > On Sun, Aug 30, 2015 at 12:13:53PM +0700, ???????? wrote: > > > I also do not use > > > recent gcc, so I don't know which problems have arised again and why > > > now again "libgcc_s breaks" (and why I don't even have it in my > > > systems, both "desktop" and "server" configurations which include C++ > > > complex code like qt4). > > > > This is not an option for everyone. For example your version of gcc > > does not have aarch64 support, nor does it have risc-v support which > > is not even in upstream gcc yet afaik, and which will be very > > important for future open hardware that free and auditable all the way > > the libgcc_s issue is x86 specific (because only x86 had the > broken ifunc based multiversioning support) > > it is possible to patch out, but would make it impossible to > use musl-gcc wrapper with existing gcc-6.* on x86 I don't think this is an actual concern. musl-gcc should insist on using static libgcc. If it doesn't, that's a bug: libgcc_s.so.1 isn't even going to be present in the musl library path unless the user manually copies/links it there. > there are other reasons for symvers: debian is willing to > accept patches for a musl based debian, however they use > symbol versioning a lot to avoid frequently changing the .so > name. i.e. if we support symvers we may be able to create > a musl based debian package repo without much effort or > maintainance work. (it is probably possible to rebuild all > dependent packages on every minor abi change in a library, > but then somebody would need to do that work and users will > need to download more packages on an update). This seems like an area where there would be concrete benefit. Any word from ppl interested in Debian/musl? Being that the only significant feedback on symbol versioning so far has been negative, I lean towards holding off on prioritizing this issue while we wait for more opinions or compelling reasons to support it. It would be nice if the libgcc issue could be fixed independently of version support so that it's not compelling a particular course of action on our side; I think the alternatives you proposed on the gcc side were cleaner than what they're doing now anyway. But I'm concerned we may not be able to get upstream to accept them. Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-30 17:09 ` Rich Felker @ 2015-08-30 17:45 ` u-wsnj 0 siblings, 0 replies; 29+ messages in thread From: u-wsnj @ 2015-08-30 17:45 UTC (permalink / raw) To: musl On Sun, Aug 30, 2015 at 01:09:32PM -0400, Rich Felker wrote: > On Sun, Aug 30, 2015 at 11:00:13AM +0200, Szabolcs Nagy wrote: > > there are other reasons for symvers: debian is willing to > > accept patches for a musl based debian, however they use > > symbol versioning a lot to avoid frequently changing the .so > > name. i.e. if we support symvers we may be able to create > > a musl based debian package repo without much effort or > > maintainance work. (it is probably possible to rebuild all > > dependent packages on every minor abi change in a library, > > but then somebody would need to do that work and users will > > need to download more packages on an update). It would be interesting to have some idea about the frequency of the cases when libraries break the ABIs and the number/percentage of the packages which would need rebuilding because of this. > This seems like an area where there would be concrete benefit. Any > word from ppl interested in Debian/musl? Indeed it would be interesting to hear from Debian. Such a port would be of course a nice testbed and showcase for musl. If despite the expectations this could be done without symbol versioning, then it would also serve as an indication of the importance of the versioning or lack thereof. > Being that the only significant feedback on symbol versioning so far > has been negative, I lean towards holding off on prioritizing this > issue while we wait for more opinions or compelling reasons to support +1 (the less developer time is consumed before the need is proven, the better) Rune ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-30 5:30 ` Rich Felker 2015-08-30 9:00 ` Szabolcs Nagy @ 2015-08-30 17:13 ` Рысь 2015-08-30 17:38 ` Rich Felker 1 sibling, 1 reply; 29+ messages in thread From: Рысь @ 2015-08-30 17:13 UTC (permalink / raw) To: musl On Sun, 30 Aug 2015 01:30:37 -0400 Rich Felker <dalias@libc.org> wrote: > On Sun, Aug 30, 2015 at 12:13:53PM +0700, Рысь wrote: > > The snowball effect I referred to is that your silent decision will > > amplify in future by allowing third parties to use symvers more and > > more in their code, > > This is what I don't buy. You're seriously overestimating the degree > to which people doing this care about musl. Most likely they are not > even aware of musl; much less do they care if I tell them they can't > use symbol versioning. Do they listen to reasons or just reject claims in "drepper" style? That's sad, I understand it's hard to change anything, especially in gcc/gnu camp. But musl already made it's way into OpenWRT trunk and some news shouted already. Even if they're not aware, some will face it in niche products. > > > then it will be so widespread it will be impossible > > to run anything without them. > > That hasn't happened yet despite the people who use it only caring > about "GNU/Linux" (glibc). The "GNU/Linux" already crossed some points of no return, some fundamental even a decade ago. And you can name some of them. And even because of them musl is gaining attention as alternative libc and forming a crowd of those who prefer "green organic Linux product" :-) > > > Instead, you and community as an > > alternative to glibc should teach them not to do. This is not a > > purely technical talk, sorry. You can't refere to pure technical > > reasons here. Why you want to be clean and straight in the area > > where hacks are in general use? > > I think there's a good opportunity for education not to use symbol > versioning just when people ask why musl libc.so itself doesn't have > versions or why they can't use versioning to achieve something > musl-internal. > > There's certainly also _some_ weight to be had in filing bug reports > that say "this is breaking when you run it on musl libc because it > uses symbol versioning", but it seems a lot more likely to me that > this just becomes a disincentive for them to address any musl-related > bug reports. They can come back and say "symbol versions are a > documented ELF feature in the toolchain, and if musl's ldso doesn't > support them it's broken". I disagree with this view, but it's a > plausible position they can take. So from my standpoint, "we support > this misfeature despite it being a misfeature, but you really should > make things work without it because static linking is broken, and our > users need static linking" is a much more diplomatic argument. The support for static linking indeed must be always available with musl. However some programs are designed with dynamic linking in mind, and some even require dangerous mechanisms like dlopen. And symvers is a deeper level of hell. I myself consider such programs or libraries (more common) broken anyway (they usually break the build after for silly reasons like binding hard to glibc internals), and if I can't proceed without them (for example, alsa) a huge local patchwork takes place. Btw your wiki page about alternatives is a good collection here. > > > For myself I already did everything in my systems to be sure that > > there is no such treat as "symbol versioning": patching binutils ld > > not to accept them, then patching every thing which silently > > requires them (alarmed by build failures in this way of course). > > That's a perfectly fine approach, and if that's your setup, no harm > can come from patching hypothetical symver support out of musl since > there are no symbol versions present to begin with. OK. If symvers hack goes only into dynamic linker, that's no more than a commit that I (if I want to cutoff such a code like binutils ld) can revert. I hope you will continue to consider symvers a misfeature then. > > > I also do not use > > recent gcc, so I don't know which problems have arised again and why > > now again "libgcc_s breaks" (and why I don't even have it in my > > systems, both "desktop" and "server" configurations which include > > C++ complex code like qt4). > > This is not an option for everyone. For example your version of gcc > does not have aarch64 support, nor does it have risc-v support which > is not even in upstream gcc yet afaik, and which will be very > important for future open hardware that free and auditable all the way > down to the metal. Backporting new archs or features from new language > standards to old gcc is A LOT of work. I would love to see somebody do > it, but I'm not going to be the one who does. (unneeded text with blaming gcc so fat/slow/hard to configure) Btw who decided arm64 to be such a stupid name? > > > And how other non-glibc build environments solve libgcc_s breakage? > > Or they dead already? > > This use of versions is Linux/ELF-specific, and the only other player, > uClibc, supports symbol versioning AFAIK. OK. > > Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-30 17:13 ` Рысь @ 2015-08-30 17:38 ` Rich Felker 0 siblings, 0 replies; 29+ messages in thread From: Rich Felker @ 2015-08-30 17:38 UTC (permalink / raw) To: musl On Mon, Aug 31, 2015 at 12:13:29AM +0700, Рысь wrote: > On Sun, 30 Aug 2015 01:30:37 -0400 > Rich Felker <dalias@libc.org> wrote: > > > On Sun, Aug 30, 2015 at 12:13:53PM +0700, Рысь wrote: > > > The snowball effect I referred to is that your silent decision will > > > amplify in future by allowing third parties to use symvers more and > > > more in their code, > > > > This is what I don't buy. You're seriously overestimating the degree > > to which people doing this care about musl. Most likely they are not > > even aware of musl; much less do they care if I tell them they can't > > use symbol versioning. > > Do they listen to reasons or just reject claims in "drepper" style? > That's sad, I understand it's hard to change anything, especially in > gcc/gnu camp. The GCC/GNU side is actually a lot easier to work with these days. Anecdotally I've had the most bad responses ("it works for me with glibc", "static linking is deprecated", etc.) from the "desktop Linux" sector. > But musl already made it's way into OpenWRT trunk and some news shouted > already. Even if they're not aware, some will face it in niche products. Generally the projects actively trying to support musl are those that are either already following best practices on things like this, or at least aware of where they're not and conscious of not actively breaking things. > > > For myself I already did everything in my systems to be sure that > > > there is no such treat as "symbol versioning": patching binutils ld > > > not to accept them, then patching every thing which silently > > > requires them (alarmed by build failures in this way of course). > > > > That's a perfectly fine approach, and if that's your setup, no harm > > can come from patching hypothetical symver support out of musl since > > there are no symbol versions present to begin with. > > OK. If symvers hack goes only into dynamic linker, that's no more than > a commit that I (if I want to cutoff such a code like binutils ld) can > revert. It may not be quite that easy since future commits will build on code with some extra arguments, local vars, etc. and not cleanly apply with that reverted. You'd probably have an easier time finding a clever way to make the code optimize-out with dead code elimination by adding a single line somewhere to null-out the version table pointer or similar. > I hope you will continue to consider symvers a misfeature then. Of course. > > > I also do not use > > > recent gcc, so I don't know which problems have arised again and why > > > now again "libgcc_s breaks" (and why I don't even have it in my > > > systems, both "desktop" and "server" configurations which include > > > C++ complex code like qt4). > > > > This is not an option for everyone. For example your version of gcc > > does not have aarch64 support, nor does it have risc-v support which > > is not even in upstream gcc yet afaik, and which will be very > > important for future open hardware that free and auditable all the way > > down to the metal. Backporting new archs or features from new language > > standards to old gcc is A LOT of work. I would love to see somebody do > > it, but I'm not going to be the one who does. > > (unneeded text with blaming gcc so fat/slow/hard to configure) Bloat, C++, lib dependencies, etc. are certainly valid complaints, but from my perspective the build process has gotten a lot better. Cross compilers and sysroot used to be very fragile and had some outright breakage in the build system that required arcane workarounds; now they just work fine out of the box. Aside from the excessive build time and disk space, I'd much rather be working with gcc 5.2 than 4.2. The time cost to me fixing all the stupid stuff that breaks on old gcc is greater than the cpu time cost compiling new gcc... Of course I understand and respect that the situation is the opposite for many users. I just don't have the time/resources to clean up and revamp old gcc for them, tho. :( > Btw who decided arm64 to be such a stupid name? arm64 was not usable because so much stuff matches arm* to catch armv4t, armv5, armv6, etc. and it would wrongly identify arm64 as being the existing arm arch rather than a new one. Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 2:43 Adjustments to roadmap Rich Felker 2015-08-28 3:38 ` Khem Raj 2015-08-28 7:24 ` u-wsnj @ 2015-08-30 5:18 ` Рысь 2015-08-30 5:31 ` Rich Felker 2015-08-31 7:09 ` Adjustments to roadmap Rich Felker 3 siblings, 1 reply; 29+ messages in thread From: Рысь @ 2015-08-30 5:18 UTC (permalink / raw) To: musl On Thu, 27 Aug 2015 22:43:48 -0400 Rich Felker <dalias@libc.org> wrote: > In addition to the above, there are all of the existing roadmap items > on the wiki which are open for discussion of how they should be > prioritized. The big projects are roughly: > > - Atomics refactorization/deduplication > - Bits refactorization/deduplication > - Out-of-tree builds > - LC_COLLATE > - IDN > - Advanced glibc ABI-compat features in dynamic linker > - Documentation > > Apologies for the slow progress lately. Don't worry though, there's > still lots more good stuff to come for musl. > > Rich How about libc message localization, or MUSL_LOCPATH and msgfmt stuff? You wanted to translate musl messages some time ago and I even did sent a draft before question was abandoned. I also can't get it to work and I don't quite understand the requirements for it. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-30 5:18 ` Рысь @ 2015-08-30 5:31 ` Rich Felker 2015-08-30 17:21 ` Рысь 0 siblings, 1 reply; 29+ messages in thread From: Rich Felker @ 2015-08-30 5:31 UTC (permalink / raw) To: musl On Sun, Aug 30, 2015 at 12:18:47PM +0700, Рысь wrote: > On Thu, 27 Aug 2015 22:43:48 -0400 > Rich Felker <dalias@libc.org> wrote: > > > In addition to the above, there are all of the existing roadmap items > > on the wiki which are open for discussion of how they should be > > prioritized. The big projects are roughly: > > > > - Atomics refactorization/deduplication > > - Bits refactorization/deduplication > > - Out-of-tree builds > > - LC_COLLATE > > - IDN > > - Advanced glibc ABI-compat features in dynamic linker > > - Documentation > > > > Apologies for the slow progress lately. Don't worry though, there's > > still lots more good stuff to come for musl. > > How about libc message localization, or MUSL_LOCPATH and msgfmt stuff? > You wanted to translate musl messages some time ago and I even did > sent a draft before question was abandoned. I also can't get it to > work and I don't quite understand the requirements for it. Yes, that would be great. Can you point me back at the message with the draft? I'll take a look again and see if I can get it to work. Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-30 5:31 ` Rich Felker @ 2015-08-30 17:21 ` Рысь 2015-08-30 19:29 ` Message localization [Was: Re: [musl] Adjustments to roadmap] Rich Felker 0 siblings, 1 reply; 29+ messages in thread From: Рысь @ 2015-08-30 17:21 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 1832 bytes --] On Sun, 30 Aug 2015 01:31:47 -0400 Rich Felker <dalias@libc.org> wrote: > On Sun, Aug 30, 2015 at 12:18:47PM +0700, Рысь wrote: > > On Thu, 27 Aug 2015 22:43:48 -0400 > > Rich Felker <dalias@libc.org> wrote: > > > > > In addition to the above, there are all of the existing roadmap > > > items on the wiki which are open for discussion of how they > > > should be prioritized. The big projects are roughly: > > > > > > - Atomics refactorization/deduplication > > > - Bits refactorization/deduplication > > > - Out-of-tree builds > > > - LC_COLLATE > > > - IDN > > > - Advanced glibc ABI-compat features in dynamic linker > > > - Documentation > > > > > > Apologies for the slow progress lately. Don't worry though, > > > there's still lots more good stuff to come for musl. > > > > How about libc message localization, or MUSL_LOCPATH and msgfmt > > stuff? You wanted to translate musl messages some time ago and I > > even did sent a draft before question was abandoned. I also can't > > get it to work and I don't quite understand the requirements for it. > > Yes, that would be great. Can you point me back at the message with > the draft? I'll take a look again and see if I can get it to work. > > Rich Here it is: http://www.openwall.com/lists/musl/2015/03/17/3 (or "[musl] libintl: stubs or working functions?" Tue, 17 Mar 2015 13:59:16 +0700) I don't remember what did not work, I just get back to it again two days ago - recovered a file attached, compiled with msgfmt, set environment: MUSL_LOCPATH=/tmp LC_ALL=ru_RU ./date and it still prints in English. I had at the time file ru_RU in /tmp and strace confirmed musl mmaped that file. I attach "date.c" test program I expected to work. I probably messed up with msgfmt or something I do not remember now. [-- Attachment #2: date.c --] [-- Type: application/octet-stream, Size: 250 bytes --] #include <stdio.h> #include <time.h> #include <locale.h> int main(void) { time_t t; struct tm tmx; char B[1024]; setlocale(LC_ALL, ""); time(&t); localtime_r(&t, &tmx); strftime(B, sizeof(B), "%c", &tmx); printf("%s\n", B); return 0; } ^ permalink raw reply [flat|nested] 29+ messages in thread
* Message localization [Was: Re: [musl] Adjustments to roadmap] 2015-08-30 17:21 ` Рысь @ 2015-08-30 19:29 ` Rich Felker 2015-09-01 4:26 ` Рысь 0 siblings, 1 reply; 29+ messages in thread From: Rich Felker @ 2015-08-30 19:29 UTC (permalink / raw) To: musl On Mon, Aug 31, 2015 at 12:21:08AM +0700, Рысь wrote: > > > How about libc message localization, or MUSL_LOCPATH and msgfmt > > > stuff? You wanted to translate musl messages some time ago and I > > > even did sent a draft before question was abandoned. I also can't > > > get it to work and I don't quite understand the requirements for it. > > > > Yes, that would be great. Can you point me back at the message with > > the draft? I'll take a look again and see if I can get it to work. > > > > Rich > > Here it is: http://www.openwall.com/lists/musl/2015/03/17/3 (or > "[musl] libintl: stubs or working functions?" Tue, 17 Mar 2015 13:59:16 > +0700) > > I don't remember what did not work, I just get back to it again two > days ago - recovered a file attached, compiled with msgfmt, set > environment: > > MUSL_LOCPATH=/tmp LC_ALL=ru_RU ./date > > and it still prints in English. I had at the time file ru_RU in /tmp > and strace confirmed musl mmaped that file. > > I attach "date.c" test program I expected to work. > > I probably messed up with msgfmt or something I do not remember now. It works fine for me. I did: $ msgfmt -o ru_RU ru_RU.po $ gcc date.c $ LC_ALL=ru_RU MUSL_LOCPATH=. ./a.out Вск Авг 30 15:24:44 2015 Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Message localization [Was: Re: [musl] Adjustments to roadmap] 2015-08-30 19:29 ` Message localization [Was: Re: [musl] Adjustments to roadmap] Rich Felker @ 2015-09-01 4:26 ` Рысь 2015-09-01 4:47 ` Rich Felker 0 siblings, 1 reply; 29+ messages in thread From: Рысь @ 2015-09-01 4:26 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 1675 bytes --] On Sun, 30 Aug 2015 15:29:53 -0400 Rich Felker <dalias@libc.org> wrote: > On Mon, Aug 31, 2015 at 12:21:08AM +0700, Рысь wrote: > > > > How about libc message localization, or MUSL_LOCPATH and msgfmt > > > > stuff? You wanted to translate musl messages some time ago and I > > > > even did sent a draft before question was abandoned. I also > > > > can't get it to work and I don't quite understand the > > > > requirements for it. > > > > > > Yes, that would be great. Can you point me back at the message > > > with the draft? I'll take a look again and see if I can get it to > > > work. > > > > > > Rich > > > > Here it is: http://www.openwall.com/lists/musl/2015/03/17/3 (or > > "[musl] libintl: stubs or working functions?" Tue, 17 Mar 2015 > > 13:59:16 +0700) > > > > I don't remember what did not work, I just get back to it again two > > days ago - recovered a file attached, compiled with msgfmt, set > > environment: > > > > MUSL_LOCPATH=/tmp LC_ALL=ru_RU ./date > > > > and it still prints in English. I had at the time file ru_RU in /tmp > > and strace confirmed musl mmaped that file. > > > > I attach "date.c" test program I expected to work. > > > > I probably messed up with msgfmt or something I do not remember now. > > It works fine for me. I did: > > $ msgfmt -o ru_RU ru_RU.po > $ gcc date.c > $ LC_ALL=ru_RU MUSL_LOCPATH=. ./a.out > Вск Авг 30 15:24:44 2015 > > Rich This is what I get when running musl 1.1.11 without patches (untouched): % MUSL_LOCPATH=/tmp/l LC_ALL=ru_RU ./lib/libc.so ../date Tue Sep 1 11:22:31 2015 I attached strace output as well as compiled ru_RU data. [-- Attachment #2: strace_locale.log --] [-- Type: application/octet-stream, Size: 2399 bytes --] 551 execve("./lib/libc.so", ["./lib/libc.so", "../date"], ["MUSL_LOCPATH=/tmp/l", "LC_ALL=ru_RU", "SHELL=/bin/bash", "TERM=rxvt-256color", "HISTSIZE=10000", "WINDOWID=106954758", "OLDPWD=/tmp/l", "USER=lynx", "PATH=/local/mybin:/local/mybin/g:/bin:/local/bin:/local/X11/bin:/local/qt4/bin:/local/qemu/bin", "INPUTRC=/etc/inputrc", "PWD=/tmp/l/musl-1.1.11", "PS1=% ", "HISTIGNORE=&:[bf]g:exit", "SHLVL=1", "COLORFGBG=default;default;0", "HOME=/u/lynx", "LOGNAME=lynx", "UID=1000", "DISPLAY=:0.0", "HISTTIMEFORMAT=%h/%d - %H:%M:%S ", "COLORTERM=rxvt-xpm", "_=/bin/strace"]) = 0 551 set_thread_area({entry_number:-1 -> 12, base_addr:0xf772b924, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 551 set_tid_address(0xf772b940) = 551 551 open("../date", O_RDONLY|O_LARGEFILE) = 3 551 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\2\0\3\0\1\0\0\0\334\202\4\0104\0\0\0\274\4\0\0\0\0\0\0004\0 \0\6\0(\0\20\0\r\0\6\0\0\0004\0\0\0004\200\4\0104\200\4\10\300\0\0\0\300\0\0\0\5\0\0\0\4\0\0\0\3\0\0\0\364\0\0\0\364\200\4\10\364\200\4\10\n\0\0\0\n\0\0\0\4\0\0\0\1\0\0\0\1\0\0\0\0\0\0\0\0\200\4\10\0\200\4\10\217\3\0\0\217\3\0\0\5\0\0\0\0\20\0\0\1\0"..., 936) = 936 551 mmap2() = 0x8048000 551 mmap2() = 0x8049000 551 close(3) = 0 551 open("/tmp/l/ru_RU", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC) = 3 551 fstat64(3, {st_dev=makedev(0, 17), st_ino=0, st_mode=036475, st_nlink=15677, st_uid=37258, st_gid=807, st_blksize=1000, st_blocks=1000, st_size=1, st_atime=0, st_mtime=1970/01/01-07:00:10, st_ctime=0}) = 0 551 mmap2() = 0xf7692000 551 close(3) = 0 551 clock_gettime(CLOCK_REALTIME, {1441081380, 104814240}) = 0 551 open("/etc/localtime", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_CLOEXEC) = 3 551 fstat64(3, {st_dev=makedev(8, 2), st_ino=0, st_mode=0134100, st_nlink=63346, st_uid=18148, st_gid=10, st_blksize=0, st_blocks=0, st_size=1, st_atime=0, st_mtime=1965/06/10-03:51:00, st_ctime=0}) = 0 551 mmap2() = 0xf7691000 551 close(3) = 0 551 ioctl(1, TIOCGWINSZ, {ws_row=40, ws_col=100, ws_xpixel=800, ws_ypixel=560}) = 0 551 writev(1, [{"Tue Sep 1 11:23:00 2015", 24}, {"\n", 1}], 2) = 25 551 exit_group(0) = ? [-- Attachment #3: ru_RU --] [-- Type: application/octet-stream, Size: 11118 bytes --] ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Message localization [Was: Re: [musl] Adjustments to roadmap] 2015-09-01 4:26 ` Рысь @ 2015-09-01 4:47 ` Rich Felker 2015-09-02 2:26 ` Рысь 0 siblings, 1 reply; 29+ messages in thread From: Rich Felker @ 2015-09-01 4:47 UTC (permalink / raw) To: musl On Tue, Sep 01, 2015 at 11:26:55AM +0700, Рысь wrote: > This is what I get when running musl 1.1.11 without patches (untouched): > > % MUSL_LOCPATH=/tmp/l LC_ALL=ru_RU ./lib/libc.so ../date > Tue Sep 1 11:22:31 2015 > > I attached strace output as well as compiled ru_RU data. The mo file is broken; it looks like it was produced with the buggy msgfmt from the sabotage repo that outputs the strings in the order they were in the source, rather than sorting them. If you write a perl/awk/whatever script to pre-sort them you should be able to get by with this msgfmt; otherwise, you could use the GNU version or try the other branch of gettext-tiny. Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Message localization [Was: Re: [musl] Adjustments to roadmap] 2015-09-01 4:47 ` Rich Felker @ 2015-09-02 2:26 ` Рысь 2015-09-02 2:28 ` Rich Felker 0 siblings, 1 reply; 29+ messages in thread From: Рысь @ 2015-09-02 2:26 UTC (permalink / raw) To: musl On Tue, 1 Sep 2015 00:47:30 -0400 Rich Felker <dalias@libc.org> wrote: > On Tue, Sep 01, 2015 at 11:26:55AM +0700, Рысь wrote: > > This is what I get when running musl 1.1.11 without patches > > (untouched): > > > > % MUSL_LOCPATH=/tmp/l LC_ALL=ru_RU ./lib/libc.so ../date > > Tue Sep 1 11:22:31 2015 > > > > I attached strace output as well as compiled ru_RU data. > > The mo file is broken; it looks like it was produced with the buggy > msgfmt from the sabotage repo that outputs the strings in the order > they were in the source, rather than sorting them. If you write a > perl/awk/whatever script to pre-sort them you should be able to get by > with this msgfmt; otherwise, you could use the GNU version or try the > other branch of gettext-tiny. > > Rich How to sort strings (or there is a script I don't know about)? Because I can't use gnu version, and using gettext-tiny which was not updated for 3 years. Or there is another version with this feature implemented? ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Message localization [Was: Re: [musl] Adjustments to roadmap] 2015-09-02 2:26 ` Рысь @ 2015-09-02 2:28 ` Rich Felker 2015-09-07 13:51 ` Рысь 0 siblings, 1 reply; 29+ messages in thread From: Rich Felker @ 2015-09-02 2:28 UTC (permalink / raw) To: musl On Wed, Sep 02, 2015 at 09:26:57AM +0700, Рысь wrote: > On Tue, 1 Sep 2015 00:47:30 -0400 > Rich Felker <dalias@libc.org> wrote: > > > On Tue, Sep 01, 2015 at 11:26:55AM +0700, Рысь wrote: > > > This is what I get when running musl 1.1.11 without patches > > > (untouched): > > > > > > % MUSL_LOCPATH=/tmp/l LC_ALL=ru_RU ./lib/libc.so ../date > > > Tue Sep 1 11:22:31 2015 > > > > > > I attached strace output as well as compiled ru_RU data. > > > > The mo file is broken; it looks like it was produced with the buggy > > msgfmt from the sabotage repo that outputs the strings in the order > > they were in the source, rather than sorting them. If you write a > > perl/awk/whatever script to pre-sort them you should be able to get by > > with this msgfmt; otherwise, you could use the GNU version or try the > > other branch of gettext-tiny. > > How to sort strings (or there is a script I don't know about)? Because I > can't use gnu version, and using gettext-tiny which was not updated for > 3 years. Or there is another version with this feature implemented? It was in gettext-tiny but then reverted because making msgfmt work exposed other limitations in it. You can check out the second-to-newest version or the 'full' branch that still has a working version. Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Message localization [Was: Re: [musl] Adjustments to roadmap] 2015-09-02 2:28 ` Rich Felker @ 2015-09-07 13:51 ` Рысь 2015-09-08 15:18 ` Rich Felker 0 siblings, 1 reply; 29+ messages in thread From: Рысь @ 2015-09-07 13:51 UTC (permalink / raw) To: musl On Tue, 1 Sep 2015 22:28:56 -0400 Rich Felker <dalias@libc.org> wrote: > On Wed, Sep 02, 2015 at 09:26:57AM +0700, Рысь wrote: > > On Tue, 1 Sep 2015 00:47:30 -0400 > > Rich Felker <dalias@libc.org> wrote: > > > > > On Tue, Sep 01, 2015 at 11:26:55AM +0700, Рысь wrote: > > > > This is what I get when running musl 1.1.11 without patches > > > > (untouched): > > > > > > > > % MUSL_LOCPATH=/tmp/l LC_ALL=ru_RU ./lib/libc.so ../date > > > > Tue Sep 1 11:22:31 2015 > > > > > > > > I attached strace output as well as compiled ru_RU data. > > > > > > The mo file is broken; it looks like it was produced with the > > > buggy msgfmt from the sabotage repo that outputs the strings in > > > the order they were in the source, rather than sorting them. If > > > you write a perl/awk/whatever script to pre-sort them you should > > > be able to get by with this msgfmt; otherwise, you could use the > > > GNU version or try the other branch of gettext-tiny. > > > > How to sort strings (or there is a script I don't know about)? > > Because I can't use gnu version, and using gettext-tiny which was > > not updated for 3 years. Or there is another version with this > > feature implemented? > > It was in gettext-tiny but then reverted because making msgfmt work > exposed other limitations in it. You can check out the > second-to-newest version or the 'full' branch that still has a working > version. > > Rich Many Thanks! Sorry for late feedback, now only had time to test it. It works! % MUSL_LOCPATH=/tmp/l LC_ALL=ru_RU ./date Пн Сен 7 20:47:05 2015 Meow!! I guess there is a root of lack of translation in my case: many programs use msgfmt and friends to make .mo from .po. So if I want to solve the problem, I must "update" gettext-tiny then rebuild all affected software. Am I right? ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Message localization [Was: Re: [musl] Adjustments to roadmap] 2015-09-07 13:51 ` Рысь @ 2015-09-08 15:18 ` Rich Felker 0 siblings, 0 replies; 29+ messages in thread From: Rich Felker @ 2015-09-08 15:18 UTC (permalink / raw) To: musl On Mon, Sep 07, 2015 at 08:51:19PM +0700, Рысь wrote: > On Tue, 1 Sep 2015 22:28:56 -0400 > Rich Felker <dalias@libc.org> wrote: > > > On Wed, Sep 02, 2015 at 09:26:57AM +0700, Рысь wrote: > > > On Tue, 1 Sep 2015 00:47:30 -0400 > > > Rich Felker <dalias@libc.org> wrote: > > > > > > > On Tue, Sep 01, 2015 at 11:26:55AM +0700, Рысь wrote: > > > > > This is what I get when running musl 1.1.11 without patches > > > > > (untouched): > > > > > > > > > > % MUSL_LOCPATH=/tmp/l LC_ALL=ru_RU ./lib/libc.so ../date > > > > > Tue Sep 1 11:22:31 2015 > > > > > > > > > > I attached strace output as well as compiled ru_RU data. > > > > > > > > The mo file is broken; it looks like it was produced with the > > > > buggy msgfmt from the sabotage repo that outputs the strings in > > > > the order they were in the source, rather than sorting them. If > > > > you write a perl/awk/whatever script to pre-sort them you should > > > > be able to get by with this msgfmt; otherwise, you could use the > > > > GNU version or try the other branch of gettext-tiny. > > > > > > How to sort strings (or there is a script I don't know about)? > > > Because I can't use gnu version, and using gettext-tiny which was > > > not updated for 3 years. Or there is another version with this > > > feature implemented? > > > > It was in gettext-tiny but then reverted because making msgfmt work > > exposed other limitations in it. You can check out the > > second-to-newest version or the 'full' branch that still has a working > > version. > > > > Rich > > Many Thanks! > > Sorry for late feedback, now only had time to test it. It works! > > % MUSL_LOCPATH=/tmp/l LC_ALL=ru_RU ./date > Пн Сен 7 20:47:05 2015 > > Meow!! > > I guess there is a root of lack of translation in my case: many > programs use msgfmt and friends to make .mo from .po. So if I > want to solve the problem, I must "update" gettext-tiny then rebuild > all affected software. Am I right? Yes, but the reason this code is not in 'master' of gettext-tiny yet (actually it was committed there then reverted) is that gettext-tiny's msgfmt is not really working; it's missing some features/behavior needed to correctly handle real-world .po files from many apps. If you really can't use the GNU version of msgfmt you might need to see if there's some other version (maybe a BSD one?) you can use or else find someone who can fix/finish gettext-tiny's msgfmt for you. Rich ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Adjustments to roadmap 2015-08-28 2:43 Adjustments to roadmap Rich Felker ` (2 preceding siblings ...) 2015-08-30 5:18 ` Рысь @ 2015-08-31 7:09 ` Rich Felker 3 siblings, 0 replies; 29+ messages in thread From: Rich Felker @ 2015-08-31 7:09 UTC (permalink / raw) To: musl On Thu, Aug 27, 2015 at 10:43:48PM -0400, Rich Felker wrote: > Apologies for the slow progress lately. Don't worry though, there's > still lots more good stuff to come for musl. Based on this thread and consideration of what's in demand, I've updated the roadmap on the wiki: http://wiki.musl-libc.org/wiki/Roadmap I've copied the text (which as always is subject to change) below for reference. Malloc did not make the list yet since there's not much that can be done without some serious design work and research; it's still something on my mind but I don't know when progress is practical to expect, and it very well could be post-1.2. I'm also holding off on making any decisions on symbol versioning, though having resolved the libgcc_s issue in some way is a goal for 1.1.13. I do want to go ahead with the static/shared object files unification since it's important to static-PIE deployment and fits in with the build system overhaul that's been on the agenda for a long time now. By virtue of static-PIE, it's also linked to the NOMMU support I'm working on enhancing, which is intended to go into actual J2 deployments, but we might also get NOMMU ARM support as part of the process since it has better toolchain support and will likely be easier to develop and test the FDPIC functionality on. Rich ---------------------------------------------------------------------- = musl 1.1.12 = Estimated release: September Primary targets: * NOMMU enhancements ** FDPIC entry point and dynamic linker code ** Making all ARM assembly thumb2-compatible for Cortex-M support ** Support for FDPIC on at least one arch * Build system overhaul ** Unifying static/shared libc object files ** Bits headers deduplication * Merging atomics refactorization/deduplication Secondary targets: * Out-of-tree builds * Removing crt/*/*crt1.s in favor of using crt_arch.h everywhere * Cleaning up mcontext_t access in cancellation signal handler (member names vs offset hacks) * Enhanced LSB/glibc ABI-compat, especially fortify __*_chk symbols = musl 1.1.13 = Estimated release: Late October Primary targets: * LC_COLLATE implementation * IDN support in DNS resolver * Message translation support for dynamic linker Secondary targets: * Further dynamic linker performance improvements and clean-up * Resolving GCC symbol-versioning incompatibility issue - see http://www.openwall.com/lists/musl/2015/05/10/1 * Remapping of glibc-ABI-incompatible symbols (regexec, etc.) by dynamic linker * New getlogin[_r] with lookup via controlling tty ^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2015-09-08 15:18 UTC | newest] Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-08-28 2:43 Adjustments to roadmap Rich Felker 2015-08-28 3:38 ` Khem Raj 2015-08-28 4:21 ` Rich Felker 2015-08-28 6:57 ` Вася Бойцов 2015-08-28 12:47 ` Rich Felker 2015-08-28 12:16 ` Justin Cormack 2015-08-28 7:24 ` u-wsnj 2015-08-28 11:39 ` Рысь 2015-08-28 17:18 ` Rich Felker 2015-08-30 4:21 ` Рысь 2015-08-30 4:46 ` Rich Felker 2015-08-30 5:13 ` Рысь 2015-08-30 5:30 ` Rich Felker 2015-08-30 9:00 ` Szabolcs Nagy 2015-08-30 17:09 ` Rich Felker 2015-08-30 17:45 ` u-wsnj 2015-08-30 17:13 ` Рысь 2015-08-30 17:38 ` Rich Felker 2015-08-30 5:18 ` Рысь 2015-08-30 5:31 ` Rich Felker 2015-08-30 17:21 ` Рысь 2015-08-30 19:29 ` Message localization [Was: Re: [musl] Adjustments to roadmap] Rich Felker 2015-09-01 4:26 ` Рысь 2015-09-01 4:47 ` Rich Felker 2015-09-02 2:26 ` Рысь 2015-09-02 2:28 ` Rich Felker 2015-09-07 13:51 ` Рысь 2015-09-08 15:18 ` Rich Felker 2015-08-31 7:09 ` Adjustments to roadmap Rich Felker
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).