* AM_GNU_GETTEXT without referring internal symbols? @ 2016-04-01 12:56 Masanori Ogino [not found] ` <CAA-4+jeUF8c+Kmv0UQi0akMAtc2hmi3pC_8=nBNcsfuRcjSgNA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 0 siblings, 1 reply; 14+ messages in thread From: Masanori Ogino @ 2016-04-01 12:56 UTC (permalink / raw) To: bug-gnu-gettext; +Cc: musl Hello, Now AM_GNU_GETTEXT uses _nl_msg_cat_cntr and _nl_expand_alias to check whether the implementation is compatible with GNU gettext. However, the symbols don't appear in libintl.h so it seems that they are not part of the public API. Actually, musl libc implements libintl features and the score of gettext-tools' testsuite is equal to that with the internal libintl, using a modified AM_GNU_GETTEXT. The musl's libintl.h even defines __USE_GNU_GETTEXT and __GNU_GETTEXT_SUPPORTED_REVISION, but it does not imitate private symbols. I had checked the archive and I've found some discussions: https://lists.gnu.org/archive/html/bug-gnu-utils/2006-03/msg00011.html http://lists.gnu.org/archive/html/bug-gettext/2015-11/msg00015.html So, if the goal of the macro is check if the implementation is compatible with GNU gettext, why don't we check the public API rather than using internal symbols? Is it possible to check if the implementation is not one of known "broken" implementations and/or it is really compatible? Regards, -- Masanori Ogino ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <CAA-4+jeUF8c+Kmv0UQi0akMAtc2hmi3pC_8=nBNcsfuRcjSgNA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: AM_GNU_GETTEXT without referring internal symbols? [not found] ` <CAA-4+jeUF8c+Kmv0UQi0akMAtc2hmi3pC_8=nBNcsfuRcjSgNA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2016-04-04 2:23 ` Daiki Ueno [not found] ` <m3h9fit8lg.fsf-ueno-mXXj517/zsQ@public.gmane.org> 0 siblings, 1 reply; 14+ messages in thread From: Daiki Ueno @ 2016-04-04 2:23 UTC (permalink / raw) To: Masanori Ogino Cc: musl-ZwoEplunGu1jrUoiu81ncdBPR1lH4CV8, bug-gnu-gettext-mXXj517/zsQ Hello, Masanori Ogino <masanori.ogino-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes: > Now AM_GNU_GETTEXT uses _nl_msg_cat_cntr and _nl_expand_alias to check > whether the implementation is compatible with GNU gettext. However, > the symbols don't appear in libintl.h so it seems that they are not > part of the public API. > > Actually, musl libc implements libintl features and the score of > gettext-tools' testsuite is equal to that with the internal libintl, > using a modified AM_GNU_GETTEXT. > > The musl's libintl.h even defines __USE_GNU_GETTEXT and > __GNU_GETTEXT_SUPPORTED_REVISION, but it does not imitate private > symbols. > > I had checked the archive and I've found some discussions: > https://lists.gnu.org/archive/html/bug-gnu-utils/2006-03/msg00011.html > http://lists.gnu.org/archive/html/bug-gettext/2015-11/msg00015.html > > So, if the goal of the macro is check if the implementation is > compatible with GNU gettext, why don't we check the public API rather > than using internal symbols? Is it possible to check if the > implementation is not one of known "broken" implementations and/or it > is really compatible? I agree that it would be desirable, but doubt that it is possible (at least reliably), because: - For some reason, there is no public API to directly load arbitrary MO files and we need to mimic the behavior of translated applications: prepare a directory structure (e.g. DIR/fr/domain.mo), call bindtextdomain() for the directory, and finally call gettext(). - That requires that at least one non-POSIX locale is available on the system, to pick the translation. However, even if the system is glibc based, not all locales might be available thanks to sub-packaging (Fedora) or user configuration (Debian). So I suppose the only feasible option here is to somehow whitelist the implementations by checking macros or symbols. Does musl provides anything like that[1]? Regards, Footnotes: [1] https://sourceforge.net/p/predef/wiki/Libraries/ -- Daiki Ueno ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <m3h9fit8lg.fsf-ueno-mXXj517/zsQ@public.gmane.org>]
* Re: AM_GNU_GETTEXT without referring internal symbols? [not found] ` <m3h9fit8lg.fsf-ueno-mXXj517/zsQ@public.gmane.org> @ 2016-04-04 4:23 ` Masanori Ogino [not found] ` <CAA-4+jcpfOXoYx1g_48M0B7punK7r2AOU7YU-V_pdQNGYcFmng-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 0 siblings, 1 reply; 14+ messages in thread From: Masanori Ogino @ 2016-04-04 4:23 UTC (permalink / raw) To: Daiki Ueno Cc: musl-ZwoEplunGu1jrUoiu81ncdBPR1lH4CV8, bug-gnu-gettext-mXXj517/zsQ Thank you for your reply, Daiki. 2016-04-04 11:23 GMT+09:00 Daiki Ueno <ueno-mXXj517/zsQ@public.gmane.org>: > Hello, > > Masanori Ogino <masanori.ogino-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes: > >> Now AM_GNU_GETTEXT uses _nl_msg_cat_cntr and _nl_expand_alias to check >> whether the implementation is compatible with GNU gettext. However, >> the symbols don't appear in libintl.h so it seems that they are not >> part of the public API. >> >> Actually, musl libc implements libintl features and the score of >> gettext-tools' testsuite is equal to that with the internal libintl, >> using a modified AM_GNU_GETTEXT. >> >> The musl's libintl.h even defines __USE_GNU_GETTEXT and >> __GNU_GETTEXT_SUPPORTED_REVISION, but it does not imitate private >> symbols. >> >> I had checked the archive and I've found some discussions: >> https://lists.gnu.org/archive/html/bug-gnu-utils/2006-03/msg00011.html >> http://lists.gnu.org/archive/html/bug-gettext/2015-11/msg00015.html >> >> So, if the goal of the macro is check if the implementation is >> compatible with GNU gettext, why don't we check the public API rather >> than using internal symbols? Is it possible to check if the >> implementation is not one of known "broken" implementations and/or it >> is really compatible? > > I agree that it would be desirable, but doubt that it is possible (at > least reliably), because: > > - For some reason, there is no public API to directly load arbitrary MO > files and we need to mimic the behavior of translated applications: > prepare a directory structure (e.g. DIR/fr/domain.mo), call > bindtextdomain() for the directory, and finally call gettext(). > > - That requires that at least one non-POSIX locale is available on the > system, to pick the translation. However, even if the system is glibc > based, not all locales might be available thanks to sub-packaging > (Fedora) or user configuration (Debian). OK, I understood now. > So I suppose the only feasible option here is to somehow whitelist the > implementations by checking macros or symbols. Does musl provides > anything like that[1]? No, it doesn't on purpose. Here is the entry on this topic in the FAQ: http://wiki.musl-libc.org/wiki/FAQ#Q:_why_is_there_no_MUSL_macro_.3F Also, I'd like to point out some detailed explanations in a recent discussion: http://www.openwall.com/lists/musl/2016/03/23/6 http://www.openwall.com/lists/musl/2016/03/23/7 http://www.openwall.com/lists/musl/2016/03/23/9 That is why I proposed to have a blacklist of "broken" implementations as an option. AFAIK there have already been some blacklisting in autotools e.g. checking the version of glibc to reject specific broken implementation of a function. Thus, I think it's acceptable to use a blacklist. What do you think about it? -- Masanori Ogino ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <CAA-4+jcpfOXoYx1g_48M0B7punK7r2AOU7YU-V_pdQNGYcFmng-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: [musl] Re: AM_GNU_GETTEXT without referring internal symbols? [not found] ` <CAA-4+jcpfOXoYx1g_48M0B7punK7r2AOU7YU-V_pdQNGYcFmng-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2016-04-07 2:26 ` Daiki Ueno 2016-04-07 5:34 ` Re: [bug-gettext] " Masanori Ogino 0 siblings, 1 reply; 14+ messages in thread From: Daiki Ueno @ 2016-04-07 2:26 UTC (permalink / raw) To: Masanori Ogino Cc: bug-gnu-gettext-mXXj517/zsQ, musl-ZwoEplunGu1jrUoiu81ncdBPR1lH4CV8 Masanori Ogino <masanori.ogino-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes: >> So I suppose the only feasible option here is to somehow whitelist the >> implementations by checking macros or symbols. Does musl provides >> anything like that[1]? > > No, it doesn't on purpose. Here is the entry on this topic in the FAQ: > http://wiki.musl-libc.org/wiki/FAQ#Q:_why_is_there_no_MUSL_macro_.3F > > Also, I'd like to point out some detailed explanations in a recent discussion: > http://www.openwall.com/lists/musl/2016/03/23/6 > http://www.openwall.com/lists/musl/2016/03/23/7 > http://www.openwall.com/lists/musl/2016/03/23/9 Thanks for the information. > That is why I proposed to have a blacklist of "broken" implementations > as an option. > > AFAIK there have already been some blacklisting in autotools e.g. > checking the version of glibc to reject specific broken implementation > of a function. Thus, I think it's acceptable to use a blacklist. What > do you think about it? Yes, that sounds like a good idea. But I guess we then need to collect information about incompatible implementations. In this regard I'm actually not sure if the gettext-tools test coverage can be used as an indicator of compatibility. By the way, musl defines __GNU_GETTEXT_SUPPORTED_REVISION in the same way as glibc: #define __GNU_GETTEXT_SUPPORTED_REVISION(major) ((major) == 0 ? 1 : -1) Is major = 1 + minor = 1 actually supported in musl? After briefly checking Solaris 11 variants have: #define __GNU_GETTEXT_SUPPORTED_REVISION(m) \ ((((m) == 0) || ((m) == 1)) ? 1 : -1) Regards, -- Daiki Ueno ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Re: [bug-gettext] AM_GNU_GETTEXT without referring internal symbols? 2016-04-07 2:26 ` [musl] " Daiki Ueno @ 2016-04-07 5:34 ` Masanori Ogino [not found] ` <CAA-4+jfbUEi1bBBP7iT2_9zaxyR+XvRb00yrWsoFj7iWiN3uUw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2016-04-07 6:26 ` Re: [bug-gettext] " Rich Felker 0 siblings, 2 replies; 14+ messages in thread From: Masanori Ogino @ 2016-04-07 5:34 UTC (permalink / raw) To: musl; +Cc: bug-gnu-gettext 2016-04-07 11:26 GMT+09:00 Daiki Ueno <ueno@gnu.org>: > Masanori Ogino <masanori.ogino@gmail.com> writes: >> That is why I proposed to have a blacklist of "broken" implementations >> as an option. >> >> AFAIK there have already been some blacklisting in autotools e.g. >> checking the version of glibc to reject specific broken implementation >> of a function. Thus, I think it's acceptable to use a blacklist. What >> do you think about it? > > Yes, that sounds like a good idea. But I guess we then need to collect > information about incompatible implementations. In this regard I'm > actually not sure if the gettext-tools test coverage can be used as an > indicator of compatibility. Indeed. > By the way, musl defines __GNU_GETTEXT_SUPPORTED_REVISION in the same > way as glibc: > > #define __GNU_GETTEXT_SUPPORTED_REVISION(major) ((major) == 0 ? 1 : -1) > > Is major = 1 + minor = 1 actually supported in musl? musl doesn't support "%Id" (major 1) IIRC. I suspect that musl actually supports "system dependent segment" (minor 1) as the GNU implementation does. On the other hand, glibc's definition is questionable too since it seems that glibc's gettext implements major 1. > After briefly checking Solaris 11 variants have: > > #define __GNU_GETTEXT_SUPPORTED_REVISION(m) \ > ((((m) == 0) || ((m) == 1)) ? 1 : -1) -- Masanori Ogino ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <CAA-4+jfbUEi1bBBP7iT2_9zaxyR+XvRb00yrWsoFj7iWiN3uUw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: [musl] Re: AM_GNU_GETTEXT without referring internal symbols? [not found] ` <CAA-4+jfbUEi1bBBP7iT2_9zaxyR+XvRb00yrWsoFj7iWiN3uUw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2016-04-07 5:36 ` Masanori Ogino 2016-05-26 6:09 ` Daiki Ueno 1 sibling, 0 replies; 14+ messages in thread From: Masanori Ogino @ 2016-04-07 5:36 UTC (permalink / raw) To: musl-ZwoEplunGu1jrUoiu81ncdBPR1lH4CV8; +Cc: bug-gnu-gettext-mXXj517/zsQ 2016-04-07 14:34 GMT+09:00 Masanori Ogino <masanori.ogino-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>: > musl doesn't support "%Id" (major 1) IIRC. I suspect that musl > actually supports "system dependent segment" (minor 1) as the GNU > implementation does. I mean, I suspect that musl *doesn't* actually supports "system dependent segment." -- Masanori Ogino <masanori.ogino-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> http://twitter.com/omasanori http://gplus.to/omasanori ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [musl] Re: AM_GNU_GETTEXT without referring internal symbols? [not found] ` <CAA-4+jfbUEi1bBBP7iT2_9zaxyR+XvRb00yrWsoFj7iWiN3uUw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2016-04-07 5:36 ` [musl] " Masanori Ogino @ 2016-05-26 6:09 ` Daiki Ueno 2016-05-26 12:53 ` Re: [bug-gettext] " Masanori Ogino 1 sibling, 1 reply; 14+ messages in thread From: Daiki Ueno @ 2016-05-26 6:09 UTC (permalink / raw) To: Masanori Ogino Cc: bug-gnu-gettext-mXXj517/zsQ, musl-ZwoEplunGu1jrUoiu81ncdBPR1lH4CV8 [-- Attachment #1: Type: text/plain, Size: 1494 bytes --] Hello, Masanori Ogino <masanori.ogino-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes: > 2016-04-07 11:26 GMT+09:00 Daiki Ueno <ueno-mXXj517/zsQ@public.gmane.org>: >> Masanori Ogino <masanori.ogino-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes: >>> That is why I proposed to have a blacklist of "broken" implementations >>> as an option. >>> >>> AFAIK there have already been some blacklisting in autotools e.g. >>> checking the version of glibc to reject specific broken implementation >>> of a function. Thus, I think it's acceptable to use a blacklist. What >>> do you think about it? >> >> Yes, that sounds like a good idea. But I guess we then need to collect >> information about incompatible implementations. In this regard I'm >> actually not sure if the gettext-tools test coverage can be used as an >> indicator of compatibility. > > Indeed. I was wondering if there is anything could be done in the upcoming gettext release. Let's go back to the original explanation by Bruno: https://lists.gnu.org/archive/html/bug-gnu-utils/2006-03/msg00011.html where he states two things: 1. The purpose of the checks are excluding incompatible implementations, e.g., NetBSD (around 1.5?) and Solaris 7 2. The __GNU_GETTEXT_SUPPORTED_REVISION macro is a recent addition In that case, I guess we could bypass the symbol checks if __GNU_GETTEXT_SUPPORTED_REVISION is defined, as long as broken implementations do not define it. How about the attached patch? Regards, -- Daiki Ueno [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-m4-Rely-less-on-internal-symbols.patch --] [-- Type: text/x-patch, Size: 3400 bytes --] From 77a71921d4925db2abc53a4610e89799e19fb113 Mon Sep 17 00:00:00 2001 From: Daiki Ueno <ueno-mXXj517/zsQ@public.gmane.org> Date: Thu, 26 May 2016 13:38:57 +0900 Subject: [PATCH] m4: Rely less on internal symbols * gettext-runtime/m4/gettext.m4 (AM_GNU_GETTEXT): Skip checks for the internal symbols _nl_msg_cat_cntr, _nl_domain_bindings, and _nl_expand_alias, if __GNU_GETTEXT_SUPPORTED_REVISION is defined. --- gettext-runtime/m4/gettext.m4 | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/gettext-runtime/m4/gettext.m4 b/gettext-runtime/m4/gettext.m4 index 1f6979a..32ff263 100644 --- a/gettext-runtime/m4/gettext.m4 +++ b/gettext-runtime/m4/gettext.m4 @@ -1,4 +1,4 @@ -# gettext.m4 serial 67 (gettext-0.19.6) +# gettext.m4 serial 68 (gettext-0.19.6) dnl Copyright (C) 1995-2014, 2016 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -161,13 +161,18 @@ changequote([,])dnl [AC_LANG_PROGRAM( [[ #include <libintl.h> -$gt_revision_test_code +#ifndef __GNU_GETTEXT_SUPPORTED_REVISION extern int _nl_msg_cat_cntr; extern int *_nl_domain_bindings; +#define _GT_COMPAT (_nl_msg_cat_cntr + *_nl_domain_bindings) +#else +#define _GT_COMPAT 0 +#endif +$gt_revision_test_code ]], [[ bindtextdomain ("", ""); -return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_domain_bindings +return * gettext ("")$gt_expression_test_code + _GT_COMPAT ]])], [eval "$gt_func_gnugettext_libc=yes"], [eval "$gt_func_gnugettext_libc=no"])]) @@ -193,17 +198,22 @@ return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_domain_b [AC_LANG_PROGRAM( [[ #include <libintl.h> -$gt_revision_test_code +#ifndef __GNU_GETTEXT_SUPPORTED_REVISION extern int _nl_msg_cat_cntr; extern #ifdef __cplusplus "C" #endif const char *_nl_expand_alias (const char *); +#define _GT_COMPAT (_nl_msg_cat_cntr + *_nl_expand_alias ("")) +#else +#define _GT_COMPAT 0 +#endif +$gt_revision_test_code ]], [[ bindtextdomain ("", ""); -return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_alias ("") +return * gettext ("")$gt_expression_test_code + _GT_COMPAT ]])], [eval "$gt_func_gnugettext_libintl=yes"], [eval "$gt_func_gnugettext_libintl=no"]) @@ -214,17 +224,22 @@ return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_a [AC_LANG_PROGRAM( [[ #include <libintl.h> -$gt_revision_test_code +#ifndef __GNU_GETTEXT_SUPPORTED_REVISION extern int _nl_msg_cat_cntr; extern #ifdef __cplusplus "C" #endif const char *_nl_expand_alias (const char *); +#define _GT_COMPAT (_nl_msg_cat_cntr + *_nl_expand_alias ("")) +#else +#define _GT_COMPAT 0 +#endif +$gt_revision_test_code ]], [[ bindtextdomain ("", ""); -return * gettext ("")$gt_expression_test_code + _nl_msg_cat_cntr + *_nl_expand_alias ("") +return * gettext ("")$gt_expression_test_code + _GT_COMPAT ]])], [LIBINTL="$LIBINTL $LIBICONV" LTLIBINTL="$LTLIBINTL $LTLIBICONV" -- 2.5.5 ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Re: [bug-gettext] AM_GNU_GETTEXT without referring internal symbols? 2016-05-26 6:09 ` Daiki Ueno @ 2016-05-26 12:53 ` Masanori Ogino 2016-05-26 19:36 ` Rich Felker 0 siblings, 1 reply; 14+ messages in thread From: Masanori Ogino @ 2016-05-26 12:53 UTC (permalink / raw) To: musl; +Cc: bug-gnu-gettext Hello Daiki, 2016-05-26 15:09 GMT+09:00 Daiki Ueno <ueno@gnu.org>: > Hello, > > Masanori Ogino <masanori.ogino@gmail.com> writes: > >> 2016-04-07 11:26 GMT+09:00 Daiki Ueno <ueno@gnu.org>: >>> Masanori Ogino <masanori.ogino@gmail.com> writes: >>>> That is why I proposed to have a blacklist of "broken" implementations >>>> as an option. >>>> >>>> AFAIK there have already been some blacklisting in autotools e.g. >>>> checking the version of glibc to reject specific broken implementation >>>> of a function. Thus, I think it's acceptable to use a blacklist. What >>>> do you think about it? >>> >>> Yes, that sounds like a good idea. But I guess we then need to collect >>> information about incompatible implementations. In this regard I'm >>> actually not sure if the gettext-tools test coverage can be used as an >>> indicator of compatibility. >> >> Indeed. > > I was wondering if there is anything could be done in the upcoming > gettext release. Let's go back to the original explanation by Bruno: > https://lists.gnu.org/archive/html/bug-gnu-utils/2006-03/msg00011.html > where he states two things: > > 1. The purpose of the checks are excluding incompatible implementations, > e.g., NetBSD (around 1.5?) and Solaris 7 > > 2. The __GNU_GETTEXT_SUPPORTED_REVISION macro is a recent addition > > In that case, I guess we could bypass the symbol checks if > __GNU_GETTEXT_SUPPORTED_REVISION is defined, as long as broken > implementations do not define it. > > How about the attached patch? It looks essentially good to me. You can remove the "if test $gt_api_version -ge 3; then ... fi" part before where you modified too if it is not used anywhere else, I guess. Thank you for working on! -- Masanori Ogino ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Re: [bug-gettext] AM_GNU_GETTEXT without referring internal symbols? 2016-05-26 12:53 ` Re: [bug-gettext] " Masanori Ogino @ 2016-05-26 19:36 ` Rich Felker [not found] ` <20160526193658.GA21636-C3MtFaGISjmo6RMmaWD+6Sb1p8zYI1N1@public.gmane.org> 0 siblings, 1 reply; 14+ messages in thread From: Rich Felker @ 2016-05-26 19:36 UTC (permalink / raw) To: Masanori Ogino; +Cc: musl, bug-gnu-gettext On Thu, May 26, 2016 at 09:53:49PM +0900, Masanori Ogino wrote: > Hello Daiki, > > 2016-05-26 15:09 GMT+09:00 Daiki Ueno <ueno@gnu.org>: > > Hello, > > > > Masanori Ogino <masanori.ogino@gmail.com> writes: > > > >> 2016-04-07 11:26 GMT+09:00 Daiki Ueno <ueno@gnu.org>: > >>> Masanori Ogino <masanori.ogino@gmail.com> writes: > >>>> That is why I proposed to have a blacklist of "broken" implementations > >>>> as an option. > >>>> > >>>> AFAIK there have already been some blacklisting in autotools e.g. > >>>> checking the version of glibc to reject specific broken implementation > >>>> of a function. Thus, I think it's acceptable to use a blacklist. What > >>>> do you think about it? > >>> > >>> Yes, that sounds like a good idea. But I guess we then need to collect > >>> information about incompatible implementations. In this regard I'm > >>> actually not sure if the gettext-tools test coverage can be used as an > >>> indicator of compatibility. > >> > >> Indeed. > > > > I was wondering if there is anything could be done in the upcoming > > gettext release. Let's go back to the original explanation by Bruno: > > https://lists.gnu.org/archive/html/bug-gnu-utils/2006-03/msg00011.html > > where he states two things: > > > > 1. The purpose of the checks are excluding incompatible implementations, > > e.g., NetBSD (around 1.5?) and Solaris 7 > > > > 2. The __GNU_GETTEXT_SUPPORTED_REVISION macro is a recent addition > > > > In that case, I guess we could bypass the symbol checks if > > __GNU_GETTEXT_SUPPORTED_REVISION is defined, as long as broken > > implementations do not define it. > > > > How about the attached patch? > > It looks essentially good to me. You can remove the "if test > $gt_api_version -ge 3; then ... fi" part before where you modified too > if it is not used anywhere else, I guess. > > Thank you for working on! I haven't tested it but the concept looks good to me too. Rich ^ permalink raw reply [flat|nested] 14+ messages in thread
[parent not found: <20160526193658.GA21636-C3MtFaGISjmo6RMmaWD+6Sb1p8zYI1N1@public.gmane.org>]
* Re: [musl] Re: AM_GNU_GETTEXT without referring internal symbols? [not found] ` <20160526193658.GA21636-C3MtFaGISjmo6RMmaWD+6Sb1p8zYI1N1@public.gmane.org> @ 2016-05-27 8:48 ` Daiki Ueno 0 siblings, 0 replies; 14+ messages in thread From: Daiki Ueno @ 2016-05-27 8:48 UTC (permalink / raw) To: Rich Felker Cc: musl-ZwoEplunGu1jrUoiu81ncdBPR1lH4CV8, bug-gnu-gettext-mXXj517/zsQ Rich Felker <dalias-8zAoT0mYgF4@public.gmane.org> writes: > On Thu, May 26, 2016 at 09:53:49PM +0900, Masanori Ogino wrote: >> >> It looks essentially good to me. You can remove the "if test >> $gt_api_version -ge 3; then ... fi" part before where you modified too >> if it is not used anywhere else, I guess. I think it is still needed when AM_GNU_GETTEXT is invoked as: AM_GNU_GETTEXT([external], [need-formatstring-macros]) > I haven't tested it but the concept looks good to me too. Thanks both for looking at it. I have pushed it as: http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=b67399b40bc5bf3165b09e6a095ec941d4b30a97 For the record, I have tested it on the Alpine Linux image obtained from http://www.nongnu.org/pretest/ using a test package created with: $ cp .../gettext.m4 gnulib/m4 $ ./gnulib/gnulib-tool --create-testdir --dir=t gettext (I had to copy libintl.h to the VM as Alpine doesn't install it) Regards, -- Daiki Ueno ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Re: [bug-gettext] AM_GNU_GETTEXT without referring internal symbols? 2016-04-07 5:34 ` Re: [bug-gettext] " Masanori Ogino [not found] ` <CAA-4+jfbUEi1bBBP7iT2_9zaxyR+XvRb00yrWsoFj7iWiN3uUw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2016-04-07 6:26 ` Rich Felker 2016-04-07 7:12 ` Masanori Ogino 1 sibling, 1 reply; 14+ messages in thread From: Rich Felker @ 2016-04-07 6:26 UTC (permalink / raw) To: Masanori Ogino; +Cc: musl, bug-gnu-gettext On Thu, Apr 07, 2016 at 02:34:01PM +0900, Masanori Ogino wrote: > 2016-04-07 11:26 GMT+09:00 Daiki Ueno <ueno@gnu.org>: > > Masanori Ogino <masanori.ogino@gmail.com> writes: > >> That is why I proposed to have a blacklist of "broken" implementations > >> as an option. > >> > >> AFAIK there have already been some blacklisting in autotools e.g. > >> checking the version of glibc to reject specific broken implementation > >> of a function. Thus, I think it's acceptable to use a blacklist. What > >> do you think about it? > > > > Yes, that sounds like a good idea. But I guess we then need to collect > > information about incompatible implementations. In this regard I'm > > actually not sure if the gettext-tools test coverage can be used as an > > indicator of compatibility. > > Indeed. > > > By the way, musl defines __GNU_GETTEXT_SUPPORTED_REVISION in the same > > way as glibc: > > > > #define __GNU_GETTEXT_SUPPORTED_REVISION(major) ((major) == 0 ? 1 : -1) > > > > Is major = 1 + minor = 1 actually supported in musl? > > musl doesn't support "%Id" (major 1) IIRC. I suspect that musl > actually supports "system dependent segment" (minor 1) as the GNU > implementation does. > > On the other hand, glibc's definition is questionable too since it > seems that glibc's gettext implements major 1. The intent is that musl supports the _API_ fully, not the (IMO awful, and against the whole spirit of gettext) GNU implementation of sysdep strings as a segment of the mo file that's patched at runtime and wastes core in every process. Instead, msgfmt (there's a prototype version of the utility that does this, but it needs work) should generate all possible combinations of the expansion of the sysdep macros at mo generation time, and "sysdep" translations magically work with no runtime cost. At some point I want to prepare a patch for upstream msgfmt to do this but I haven't gotten around to it yet. I'm not sure what the %Id thing you're referring to is; can you point me to a description of it? Rich ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Re: [bug-gettext] AM_GNU_GETTEXT without referring internal symbols? 2016-04-07 6:26 ` Re: [bug-gettext] " Rich Felker @ 2016-04-07 7:12 ` Masanori Ogino 2016-04-07 10:57 ` Szabolcs Nagy 0 siblings, 1 reply; 14+ messages in thread From: Masanori Ogino @ 2016-04-07 7:12 UTC (permalink / raw) To: Rich Felker; +Cc: musl, bug-gnu-gettext 2016-04-07 15:26 GMT+09:00 Rich Felker <dalias@libc.org>: > On Thu, Apr 07, 2016 at 02:34:01PM +0900, Masanori Ogino wrote: >> 2016-04-07 11:26 GMT+09:00 Daiki Ueno <ueno@gnu.org>: >> > Masanori Ogino <masanori.ogino@gmail.com> writes: >> >> That is why I proposed to have a blacklist of "broken" implementations >> >> as an option. >> >> >> >> AFAIK there have already been some blacklisting in autotools e.g. >> >> checking the version of glibc to reject specific broken implementation >> >> of a function. Thus, I think it's acceptable to use a blacklist. What >> >> do you think about it? >> > >> > Yes, that sounds like a good idea. But I guess we then need to collect >> > information about incompatible implementations. In this regard I'm >> > actually not sure if the gettext-tools test coverage can be used as an >> > indicator of compatibility. >> >> Indeed. >> >> > By the way, musl defines __GNU_GETTEXT_SUPPORTED_REVISION in the same >> > way as glibc: >> > >> > #define __GNU_GETTEXT_SUPPORTED_REVISION(major) ((major) == 0 ? 1 : -1) >> > >> > Is major = 1 + minor = 1 actually supported in musl? >> >> musl doesn't support "%Id" (major 1) IIRC. I suspect that musl >> actually supports "system dependent segment" (minor 1) as the GNU >> implementation does. >> >> On the other hand, glibc's definition is questionable too since it >> seems that glibc's gettext implements major 1. > > The intent is that musl supports the _API_ fully, not the (IMO awful, > and against the whole spirit of gettext) GNU implementation of sysdep > strings as a segment of the mo file that's patched at runtime and > wastes core in every process. Instead, msgfmt (there's a prototype > version of the utility that does this, but it needs work) should > generate all possible combinations of the expansion of the sysdep > macros at mo generation time, and "sysdep" translations magically work > with no runtime cost. At some point I want to prepare a patch for > upstream msgfmt to do this but I haven't gotten around to it yet. > > I'm not sure what the %Id thing you're referring to is; can you point > me to a description of it? %Id is essentially an extension of printf(3) in glibc 2.2 and later. It can be implemented by the same way as how sysdep is implemented; it just depends on the current locale, not the ABI. Search http://man7.org/linux/man-pages/man3/sprintf.3.html by "glibc 2.2 adds" for details. Fortunately, https://www.gnu.org/software/gettext/manual/html_node/c_002dformat.html describes that, if I understand this correctly, gettext can ignore the I flag if printf does not recognize it. I don't know whether such implementation matches the requirement of major 1 GNU mo format. -- Masanori Ogino ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Re: [bug-gettext] AM_GNU_GETTEXT without referring internal symbols? 2016-04-07 7:12 ` Masanori Ogino @ 2016-04-07 10:57 ` Szabolcs Nagy 2016-04-07 13:53 ` Rich Felker 0 siblings, 1 reply; 14+ messages in thread From: Szabolcs Nagy @ 2016-04-07 10:57 UTC (permalink / raw) To: Masanori Ogino; +Cc: Rich Felker, musl, bug-gnu-gettext * Masanori Ogino <masanori.ogino@gmail.com> [2016-04-07 16:12:39 +0900]: > 2016-04-07 15:26 GMT+09:00 Rich Felker <dalias@libc.org>: > > I'm not sure what the %Id thing you're referring to is; can you point > > me to a description of it? > > %Id is essentially an extension of printf(3) in glibc 2.2 and later. > It can be implemented by the same way as how sysdep is implemented; it > just depends on the current locale, not the ABI. > > Search http://man7.org/linux/man-pages/man3/sprintf.3.html by "glibc > 2.2 adds" for details. i think it is not acceptable that glibc introduces extensions that can collide with future versions of the c and posix standards. but it is even worse if a translation file format depends on such extensions which makes it hard to fix the problem once the collision happens (like it happened with scanf %a). there are other problems with this flag, so gettext manual should recommend some other solution. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Re: [bug-gettext] AM_GNU_GETTEXT without referring internal symbols? 2016-04-07 10:57 ` Szabolcs Nagy @ 2016-04-07 13:53 ` Rich Felker 0 siblings, 0 replies; 14+ messages in thread From: Rich Felker @ 2016-04-07 13:53 UTC (permalink / raw) To: Masanori Ogino, musl, bug-gnu-gettext On Thu, Apr 07, 2016 at 12:57:47PM +0200, Szabolcs Nagy wrote: > * Masanori Ogino <masanori.ogino@gmail.com> [2016-04-07 16:12:39 +0900]: > > 2016-04-07 15:26 GMT+09:00 Rich Felker <dalias@libc.org>: > > > I'm not sure what the %Id thing you're referring to is; can you point > > > me to a description of it? > > > > %Id is essentially an extension of printf(3) in glibc 2.2 and later. > > It can be implemented by the same way as how sysdep is implemented; it > > just depends on the current locale, not the ABI. > > > > Search http://man7.org/linux/man-pages/man3/sprintf.3.html by "glibc > > 2.2 adds" for details. > > i think it is not acceptable that glibc introduces > extensions that can collide with future versions of > the c and posix standards. > > but it is even worse if a translation file format > depends on such extensions which makes it hard to > fix the problem once the collision happens > (like it happened with scanf %a). > > there are other problems with this flag, so gettext > manual should recommend some other solution. In that case this has nothing to do with gettext capabilities; gettext does not process format strings but simply translates one format strign to another, and if a program is using non-portable format strings, it's going to break whether it's using gettext or not. So I don't think it's relevant to this discussion of whether _gettext_ is compatible. Rich ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2016-05-27 8:48 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-04-01 12:56 AM_GNU_GETTEXT without referring internal symbols? Masanori Ogino [not found] ` <CAA-4+jeUF8c+Kmv0UQi0akMAtc2hmi3pC_8=nBNcsfuRcjSgNA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2016-04-04 2:23 ` Daiki Ueno [not found] ` <m3h9fit8lg.fsf-ueno-mXXj517/zsQ@public.gmane.org> 2016-04-04 4:23 ` Masanori Ogino [not found] ` <CAA-4+jcpfOXoYx1g_48M0B7punK7r2AOU7YU-V_pdQNGYcFmng-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2016-04-07 2:26 ` [musl] " Daiki Ueno 2016-04-07 5:34 ` Re: [bug-gettext] " Masanori Ogino [not found] ` <CAA-4+jfbUEi1bBBP7iT2_9zaxyR+XvRb00yrWsoFj7iWiN3uUw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2016-04-07 5:36 ` [musl] " Masanori Ogino 2016-05-26 6:09 ` Daiki Ueno 2016-05-26 12:53 ` Re: [bug-gettext] " Masanori Ogino 2016-05-26 19:36 ` Rich Felker [not found] ` <20160526193658.GA21636-C3MtFaGISjmo6RMmaWD+6Sb1p8zYI1N1@public.gmane.org> 2016-05-27 8:48 ` [musl] " Daiki Ueno 2016-04-07 6:26 ` Re: [bug-gettext] " Rich Felker 2016-04-07 7:12 ` Masanori Ogino 2016-04-07 10:57 ` Szabolcs Nagy 2016-04-07 13:53 ` Rich Felker
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).