From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/11039 Path: news.gmane.org!.POSTED!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: Re: a bug in bindtextdomain() and strip '.UTF-8' Date: Mon, 13 Feb 2017 12:12:36 -0500 Message-ID: <20170213171236.GI1520@brightrain.aerifal.cx> References: <20170129163329.GL1533@brightrain.aerifal.cx> <20170208143147.GY1533@brightrain.aerifal.cx> <20170211023610.GA1520@brightrain.aerifal.cx> <20170212023422.GE1520@brightrain.aerifal.cx> <20170213132816.GG1520@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1487005971 9139 195.159.176.226 (13 Feb 2017 17:12:51 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 13 Feb 2017 17:12:51 +0000 (UTC) User-Agent: Mutt/1.5.21 (2010-09-15) To: musl@lists.openwall.com Original-X-From: musl-return-11054-gllmg-musl=m.gmane.org@lists.openwall.com Mon Feb 13 18:12:48 2017 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1cdKBC-00022d-4b for gllmg-musl@m.gmane.org; Mon, 13 Feb 2017 18:12:46 +0100 Original-Received: (qmail 17433 invoked by uid 550); 13 Feb 2017 17:12:50 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 17415 invoked from network); 13 Feb 2017 17:12:49 -0000 Content-Disposition: inline In-Reply-To: Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:11039 Archived-At: On Mon, Feb 13, 2017 at 10:06:49PM +0800, He X wrote: > no, it's on musl, i just tested it with my patches, with vim, stripping > will lead to unknown characters. That's not a matter of the locale being non-UTF-8 (it's UTF-8) but of the application doing something broken. The locale is UTF-8 because nl_langinfo(CODESET) says it is and because mb/wc conversion functions process UTF-8. That's what it means for the locale to be UTF-8. > I mean, .mo files under zh_CN/ of vim is GBK set, while zh_CN/ of other > apps is UTF-8 set, that meas there may be other apps like vim, we should be > more cautious, add a check before map the .mo files, and fail non-UTF8 set > in setlocale. All musl locale files are required to be UTF-8. If an application has translation files that are not UTF-8, they're not usable. This could be fixed in the application or by using a fixed version of msgfmt that converts to UTF-8 before producing the .mo file. > Btw, _nl_msg_cat_cntr & _nl_domain_bindings will block apps compiling with > the native intl of musl, and after i added a dump for these two symbols, The autoconf text for gettext is supposed to be getting fixed not to do that anymore, but I'm not sure what the progress on upstreaming it is. > gnu tar showed me segfaults, because he passed a zero msgid1 causing > __mo_lookup segfault, we should add a check in dcngettext to avoid it(if > (!msgid1) goto notrans;): > > #2 0x00007ffff7d82a6f in dcngettext (domainname=0x6737a0 "tar", > msgid1=0x0, msgid2=0x0, n=1, > category=5) at src/locale/dcngettext.c:211 Is it expecting gettext to return a null pointer in this case, or to return something else (like the "header", i.e. the translation of "")? I think it's acceptable to change this behavior as long as we do it right, but it should be a separate patch since it's an independent change. Rich