Re: localization, internationalization and Caml

[skaller <skaller@maxtal.com.au>]

Francis Dupont wrote:
> 
>  In your previous mail you wrote:
> 
>         The current 'support' for 8 bit characters in ocaml should be
>    deprecated immediately. It is an extremely bad thing to have, since
>    Latin-1 et al are archaic 8 bit standards incompatible with the
>    international standard for ISO10646 communication, namely
>    the UTF-8 encoding.
> 
> => there is a rather strong opposition against UTF-8 in France
> because it is not a natural encoding (ie. if ASCII maps to ASCII
> it is not the case for ISO 8859-* characters, imagine a new UTF-X
> encoding maps ASCII to strange things and you'd be able to understand
> our concern).

	I do understand the concern, but the decision on
the International Standards has been made. The transition
for ISO 8859-x clients will involve some pain. Better to
start going through the pain now :-(

>    Yes, I know Latin-1 is useful now for French.
> 
> => it is more than useful, Latin-1 (soon ISO IS 8859-15) is necessary
> if you need really readable texts in French.

	No, what you mean is that with _current technology_
there is plenty of support for 8 bit characters, using code pages,
so that Latin-1 is well supported.

	For example, there are a lot of text editors that
accept 8 bit characters, and even permit switching code pages.
There are almost none that work with ISO10646 or unicode,
let alone accept UTF-8 encoding. (Yudit is the only one I know of).

>    The way forward may well be to provide an input filter to convert
>    Latin-1 (or any other encoding) to UTF8, and have ocaml process that.
> 
> => my problem is the output of the filter will be no more readable when
> I've put too much French in the program (in comments for instance).

	You will have no problem with the right tools: the difference
will be transparent. Of course, you will need the right tools.
For example, you will need a browser like Internet Explorer 5,
which processes UTF-8 encoding correctly.

	I agree that this is a problem, but supporting
Latin-1, or any other archaic standard, is not going
to help move forward. It is bad enough that most vendors
only support Unicode, which is a small, almost filled,
16 bit subset of the full 31 bit ISO-10646 Standard.
 
>    This requires almost no changes to the compiler: the design should
>    open the set of characters acceptable in identifiers, probably
>    to some subset of the set recommended in one of the ISO10646 related
>    documents; the other change required is to accept \uXXXX and \UXXXXXXXX
>    escapes in strings. String processing functions should generally
>    continue to be 8 bit [per octet]: full internationalisation of client
>    string handling functions is a very complex, non-trivial, task]
> 
> => I believe internationalization should not be done by countries
> where English is the only used language: this is at least awkward...

	I believe people with international concerns can work
together no matter what their native language. Some English
speakers may be concerned, some, like me, are somewhat
embarrased to be non-fluent in _any_ other language.
[I speak a smattering of high school German]

	However, Australia, where I live, has migrants
from all over the world and support for many languages
is an important issue here. Particularly Asian languages.
And ISO-8859-x is not much help there :-)

-- 
John Skaller, mailto:skaller@maxtal.com.au
1/10 Toxteth Rd Glebe NSW 2037 Australia
homepage: http://www.maxtal.com.au/~skaller
downloads: http://www.triode.net.au/~skaller