[Caml-list] string_of_float -> float_of_string locale dependency bug

[Caml-list] string_of_float -> float_of_string locale dependency bug

[Evgeny Chukreev]

Hello!

/tmp% echo $LANG
ru_RU.KOI8-R

/tmp% ocaml -I /usr/local/lib/ocaml/3.07/camomile/ bigarray.cma camomile.cma
        Objective Caml version 3.07+2

# float_of_string "0,";;
- : float = 0.
# string_of_float 0,;;
Syntax error
# string_of_float 0.;;
Fatal error: exception Failure("float_of_string")

Looks like string_of_float returns "0." (not depends on locale)
but float_of_string can't parse such string due to locale dependency.

-- 
... WBR, Evgeny ...

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] string_of_float -> float_of_string locale dependency bug

[Yamagata Yoriyuki]

I think you should post a bug report to OCaml, because it is clearly a bug.
  http://caml.inria.fr/bin/caml-bugs

--
Yamagata Yoriyuki

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] string_of_float -> float_of_string locale dependency bug

[Xavier Leroy]

> Hello!
> 
> /tmp% echo $LANG
> ru_RU.KOI8-R
> 
> /tmp% ocaml -I /usr/local/lib/ocaml/3.07/camomile/ bigarray.cma camomile.cma
>         Objective Caml version 3.07+2
> 
> # float_of_string "0,";;
> - : float = 0.
> # string_of_float 0,;;
> Syntax error
> # string_of_float 0.;;
> Fatal error: exception Failure("float_of_string")

Do you have the same error if you don't load camomile.cma?  From a
quick test here, I believe not.

The Caml runtime system does depend on the LC_NUMERIC locale begin set
to its default value "C", but it ensures that this is the case by never
calling setlocale(LC_ALL, "") nor setlocale(LC_NUMERIC, "").

Third-party libraries can invalidate this invariant by calling e.g.
setlocale(LC_ALL, "").  Two possibilities:

- The library doesn't really need LC_ALL, e.g. it would be enough
  to set LC_CTYPE or LC_COLLATE and leave LC_NUMERIC unchanged.
  In this case, the library should be fixed.

- The library really needs to set LC_NUMERIC, in which case it's
  impossible to use that library with the Caml toplevel.

The C library API for internationalization is largely broken, and as
you can see there is nothing we can do to work around the fact that
the current locale is a global variable for the whole program.

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] string_of_float -> float_of_string locale dependency bug

[Evgeny Chukreev]

On Mon, 14 Jun 2004 11:54:44 +0200
    /Xavier/ /Leroy/ <xavier.leroy@inria.fr> wrote me:

XL> The C library API for internationalization is largely broken, and as
XL> you can see there is nothing we can do to work around the fact that
XL> the current locale is a global variable for the whole program.
Thanks for answer. I think it is good idea to include this topic into
O'Caml documentation (Interfacing C with Objective Caml).

-- 
... WBR, Evgeny ...

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] string_of_float -> float_of_string locale dependency bug

[Yamagata Yoriyuki]

From: Xavier Leroy <xavier.leroy@inria.fr>
Subject: Re: [Caml-list] string_of_float -> float_of_string locale dependency bug
Date: Mon, 14 Jun 2004 11:54:44 +0200

> The Caml runtime system does depend on the LC_NUMERIC locale begin set
> to its default value "C", but it ensures that this is the case by never
> calling setlocale(LC_ALL, "") nor setlocale(LC_NUMERIC, "").
> 
> Third-party libraries can invalidate this invariant by calling e.g.
> setlocale(LC_ALL, "").  Two possibilities:

Or applications which run ocaml script internally, which become
common recently.

Assuming a particular locale is quite problematic.

> 
> - The library doesn't really need LC_ALL, e.g. it would be enough
>   to set LC_CTYPE or LC_COLLATE and leave LC_NUMERIC unchanged.
>   In this case, the library should be fixed.
> 
> - The library really needs to set LC_NUMERIC, in which case it's
>   impossible to use that library with the Caml toplevel.

Camomile needs to set LC_ALL to acquire LC_NUMERIC value.  Camomile
does not use LC_NUMERIC value currently, but it provides API to
getting all LC_* values for the user.

> The C library API for internationalization is largely broken, and as
> you can see there is nothing we can do to work around the fact that
> the current locale is a global variable for the whole program.

You can temporally save the current LC_NUMERIC value, change its value
to C, and restore the value after the conversion.  What is a problem?

--
Yamagata Yoriyuki

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] string_of_float -> float_of_string locale dependency bug

[Evgeny Chukreev]

On Tue, 15 Jun 2004 00:23:47 +0900 (JST)
    /Yamagata/ /Yoriyuki/ <yoriyuki@mbg.ocn.ne.jp> wrote:

XL>> The C library API for internationalization is largely broken, and as
XL>> you can see there is nothing we can do to work around the fact that
XL>> the current locale is a global variable for the whole program.

YY> You can temporally save the current LC_NUMERIC value, change its value
YY> to C, and restore the value after the conversion.  What is a problem?

Performance?

Another approach: write own strtod function which doesn't depend on
the LC_NUMERIC locale.

Or take a look at Python's: _localemodule.c and locale.py.

-- 
... WBR, Evgeny ...

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] string_of_float -> float_of_string locale dependency bug

[Xavier Leroy]

> Camomile needs to set LC_ALL to acquire LC_NUMERIC value.  Camomile
> does not use LC_NUMERIC value currently, but it provides API to
> getting all LC_* values for the user.

Fine.  So what about restoring LC_NUMERIC after getting its value, like you
helpfully suggest that the Caml run-time system does?

> > The C library API for internationalization is largely broken, and as
> > you can see there is nothing we can do to work around the fact that
> > the current locale is a global variable for the whole program.
> 
> You can temporally save the current LC_NUMERIC value, change its value
> to C, and restore the value after the conversion.  What is a problem?

Speed.  setlocale() is quite expensive, and string <-> float
conversions can be extremely frequent.

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] string_of_float -> float_of_string locale dependency bug

[Xavier Leroy]

> Another approach: write own strtod function which doesn't depend on
> the LC_NUMERIC locale.

Of course, having our own string <-> float conversions would work
around this issue and other issues as well, such as the inability of
the C library functions to print a float *exactly* (i.e. with just
enough decimals that it reads back to the very same float).

This is no small undertaking, however, and one has a feeling of
reinventing the wheel.

The ANSI C standard library is quite small to begin with, but the
addition of locales managed to render it even less useful...

- Xavier Leroy

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] string_of_float -> float_of_string locale dependency bug

[Shawn Wagner]

On Mon, Jun 14, 2004 at 06:04:20PM +0200, Xavier Leroy wrote:
> > Another approach: write own strtod function which doesn't depend on
> > the LC_NUMERIC locale.
> 
> Of course, having our own string <-> float conversions would work
> around this issue and other issues as well, such as the inability of
> the C library functions to print a float *exactly* (i.e. with just
> enough decimals that it reads back to the very same float).
> 
> This is no small undertaking, however, and one has a feeling of
> reinventing the wheel.
> 
> The ANSI C standard library is quite small to begin with, but the
> addition of locales managed to render it even less useful...
> 

While I agree that LC_NUMERIC makes converting floats into strings that are
intended to be converted back into floats by other programs a major pain,
locales aren't all bad. Adding locale support and ctype.h routines was one
of the first things I did for annexlib. Being able to do things like case
conversion according to the user's character set is very nice.

*braces for people griping about how that's fundamentally broken too*

-- 
Shawn Wagner
shawnw@speakeasy.org

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] string_of_float -> float_of_string locale dependency bug

[Yamagata Yoriyuki]

From: Xavier Leroy <xavier.leroy@inria.fr>
Subject: Re: [Caml-list] string_of_float -> float_of_string locale dependency bug
Date: Mon, 14 Jun 2004 17:58:22 +0200

> > Camomile needs to set LC_ALL to acquire LC_NUMERIC value.  Camomile
> > does not use LC_NUMERIC value currently, but it provides API to
> > getting all LC_* values for the user.
> 
> Fine.  So what about restoring LC_NUMERIC after getting its value, like you
> helpfully suggest that the Caml run-time system does?

Because I did not aware that this causes a problem.  I will do
something to fix this problem.

> > > The C library API for internationalization is largely broken, and as
> > > you can see there is nothing we can do to work around the fact that
> > > the current locale is a global variable for the whole program.
> > 
> > You can temporally save the current LC_NUMERIC value, change its value
> > to C, and restore the value after the conversion.  What is a problem?
> 
> Speed.  setlocale() is quite expensive, and string <-> float
> conversions can be extremely frequent.

Then, how about wrapping the entire ocaml-runtime by this "locale-free
zone"?  C primitives would be executed in "C" locale, but let them
have an access to the saved locale and temporarily revert back to the
original locale.

--
Yamagata Yoriyuki

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners