Re: Full mandoc locale support committed.

[Yuri Pankov <yuri.pankov@gmail.com>]

On Wed, May 18, 2011 at 01:07:28AM +0200, Kristaps Dzonsons wrote:
> Hi,
> 
> With this last commit, initial [full] locale support has been fitted 
> into mandoc!  Attached is eye-candy: a manual full of random Unicode 
> input (\[uNNNN]) first with -Tascii, then with -Tlocale.

Unicode escapes work for me, but does that mean that existing localized
manpages can't be used? Tried several ja and ru from debian 6, all of
them make mandoc die with "FATAL: line scope broken, syntax violated"
(-Tlint talks a lot about "ERROR: skipping bad character: ignoring
byte").

>  From the manual:
> 
>     Locale Output
>       Locale-depending output encoding is triggered with -Tlocale.
>       This option is not available on all systems: systems without
>       locale support, or those whose internal representation is not
>       natively UCS-4, will fall back to -Tascii.  See ASCII Output
>       for font style specification and available command-line
>       arguments.
> 
> The check-ins:
> 
>    (1) http://mdocml.bsd.lv/archives/source/0920.html

A small typo here in Makefile's comment - USE_CHAR.

>    (2) http://mdocml.bsd.lv/archives/source/0919.html
> 
> This support is /very/ fast, and any overhead occurs if and only if 
> -Tlocale is selected AND supported.  If this doesn't hold, then mandoc 
> runs at -Tascii "native" speed.
> 
> The bad news: it comes at a price.  -Tlocale only works if Unicode 
> code-point values (defined in the UCS-4 (-2?) standards) can be 
> transformed directly into wide-character values usable by the system.  I 
> check this with an optional C99 feature, __STDC_ISO_10646__.  See
> 
>   http://www.cl.cam.ac.uk/~mgk25/ucs/iso2022-wc.html
> 
> for details.  Unfortunately, this seems only to be exported on glibc. 
> I'm told the conditions hold on OpenBSD and FreeBSD.  NetBSD?
> 
> If your system abides by these rules and doesn't export this symbol, 
> please let me know and we can special-case the macro test for this 
> feature.  There is a way to convert from Unicode to a system's 
> wide-character support without this feature, but it isn't pretty.  I'll 
> probably have to implement this anyway for portability.  For now, if a 
> system doesn't do __STDC_ISO_10646__, -Tlocale is a synonym for -Tascii.

Checked on Solaris 11 and Illumos - both do not export the symbol,
though commenting out #undef USE_WCHAR in term_ascii.c makes -Tlocale
work.

> Note that if this method is unilaterally hated, it's easy to switch to 
> another method.  This was simply the fastest, simplest, and most 
> transparent to implement.  All of the logic either way is in one file, 
> and easy to manipulate:
> 
>   http://mdocml.bsd.lv/cgi-bin/cvsweb/term_ascii.c?cvsroot=mdocml
> 
> Thoughts?


Yuri
--
 To unsubscribe send an email to discuss+unsubscribe@mdocml.bsd.lv