From: Rich Felker <dalias@libc.org>
To: musl@lists.openwall.com
Subject: Re: printf doesn't respect locale
Date: Wed, 11 Sep 2019 09:47:27 -0400 [thread overview]
Message-ID: <20190911134727.GU9017@brightrain.aerifal.cx> (raw)
In-Reply-To: <20190911145336.08554c4e@inria.fr>
On Wed, Sep 11, 2019 at 02:53:36PM +0200, Jens Gustedt wrote:
> Hello Rich,
>
> On Wed, 11 Sep 2019 07:44:37 -0400 Rich Felker <dalias@libc.org> wrote:
>
> > On Wed, Sep 11, 2019 at 12:07:22PM +0200, Jens Gustedt wrote:
>
> > > I think that WG14 would be happy to hear any suggestions how we
> > > could get out of this trap, a proposal for C2x would even be
> > > better.
> >
> > The obvious solution is a modifier character to printf/scanf format
> > strings that applies to numeric conversions and means "always
> > format/interpret this as if in the C locale". However this is hard to
> > test for at build time unless there's a macro declaring its
> > availability, so ideally WG14 would also adopt the sort of
> > fine-grained feature availability macros some of us have been
> > proposing for extensions.
>
> If such a proposal would be made, it would have to be based on a
> reference implementation in the field. Would musl be willing to be
> such a reference implementation?
Possibly, contingent on some willingness of other parties to be on
board with it (even if not implementing it at first). I don't want
musl to be in the position of implementing something new that's not
standardized and likely to *conflict* with future standards, which
custom format flags could do.
> In addition, I would think that it should not switch off all locale
> feature but should leave the encoding properties such as UTF-8
> functional.
Absolutely, but encoding is not relevant to numeric fields. Everything
else is strictly specified, at least for formatting (printf). For
conversion (scanf) implementation-defined locale-specific forms are
also allowed, but this is probably not wanted when you're processing
data from a serialized form that's intended to be universal.
> > An alternative/additional solution, which I actually might like
> > better, is having a function which sets a thread-local flag to treat
> > certain locale properties (at least the problematic LC_NUMERIC ones)
> > as if the current locale were "C". This is weaker than the uselocale
> > API from POSIX, but doesn't have the problems with the possibility of
> > failure (likely with no way to make forward progress) like it does,
> > and more importantly, would avoid *breaking* m17n/i18n functionality
> > by turning off other unrelated, non-problematic locale features.
> > Application or library code could then just set/restore this flag
> > around *printf/*scanf/strto*/etc calls, or could set it and leave it
> > if they never want to see ',' again.
>
> Interesting.
>
> Would this be difficult to implement in musl? (I guess not)
I would think not, but I'd have to look at the details a little more.
One other advantage of this approach is that it has a more graceful
fallback. If an application needs portable LC_NUMERIC behavior, it can
check at build time for the presence of the new interface. If present,
LC_NUMERIC can be set to "" (user's preference) and the new interface
can be used to get the needed behavior. If absent, the application can
refrain from setting LC_NUMERIC, only setting the other categories and
leaving it as "C" (default).
Note that having it be thread-locally stateful is, in my opinion, much
better than having new variants of the affected functions or new
formats, since a caller using LC_NUMERIC can set/restore the state to
safely call library code that's completely unaware of the new
interfaces.
Of course there may be complications I haven't thought of. One that
comes to mind right away is what localeconv() should return under such
conditions.
> Would you be willing to write this up?
What form would it need to be in?
> Once we'd have that in musl (even before having it in C2x) it could be
> easier for ourselves to convice us to have full locale support.
By "full" you mean variable radix point? I'm not sure it makes a big
difference in that it won't help code that's not prepared for radix
point to vary. What it does help is making it so code that is being
careful to avoid the breakage can still use LC_NUMERIC when it wants
to, without depending on POSIX.
Rich
next prev parent reply other threads:[~2019-09-11 13:47 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-09 16:31 Daniel Schoepe
2019-09-09 16:39 ` Daniel Schoepe
2019-09-09 16:51 ` Szabolcs Nagy
2019-09-09 17:55 ` Rich Felker
2019-09-09 17:54 ` Rich Felker
2019-09-10 16:00 ` Daniel Schoepe
2019-09-10 16:31 ` Szabolcs Nagy
2019-09-10 16:44 ` Tim Tassonis
2019-09-10 17:30 ` Rich Felker
2019-09-10 17:10 ` Daniel Schoepe
2019-09-10 17:33 ` Rich Felker
2019-09-10 18:43 ` Szabolcs Nagy
2019-09-10 21:55 ` A. Wilcox
2019-09-11 10:01 ` Szabolcs Nagy
2019-09-11 10:07 ` Jens Gustedt
2019-09-11 11:44 ` Rich Felker
2019-09-11 12:53 ` Jens Gustedt
2019-09-11 13:47 ` Rich Felker [this message]
2019-09-11 15:15 ` Jens Gustedt
2019-09-11 15:38 ` Rich Felker
2019-09-11 18:08 ` Jens Gustedt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190911134727.GU9017@brightrain.aerifal.cx \
--to=dalias@libc.org \
--cc=musl@lists.openwall.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/musl/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).