[TUHS] Character sets

[cowan@mercury.ccil.org (John Cowan)]

Johnny Billquist scripsit:

> >>Haha. Yes... Except that you now have multiple representations of each
> >>character within one character set. So what has improved???

Mojibake, though not unknown, is now much less common, and the number
of documents on the web that are in UTF-8 (including its ASCII subset)
is at 85% and rising.

> >In the Good Old Days, characters were all the same size, and you could
> >do nice, simple things like
> >
> >   while (*c && *c++ != " ");

That particular piece of code still works if the encoding is UTF-8.
Fundamentally, Unicode is complicated because human writing systems
are complicated.

> Another one I noted a while ago was that functions and command in
> Unix, such as lpq, which try to print things in nice columns now
> fail, because the code don't actually know how many characters have
> been output.

Well, if the font isn't fixed-width, you're screwed anyway.  But if
it is, there is information in the Unicode tables that tells you which
characters have widths of 0, 1, or 2.  Print programs can be modified
to use that information.

> And let's not even talk about such wonderful concepts as colors in
> the character set definition... Unicode seems to have it all... 

Colors are optional.

> I wonder how many code points exist for 'A'. It's definitely more than
> one...

Other than Greek and Cyrillic A letters, there are the math letters, which
are used *in plain text* to designate semantic differences: plain A,
italic A, and bold A mean different things mathematically.  Using the
math italics for emphasis or book titles is a Bad Thing.

-- 
John Cowan          http://www.ccil.org/~cowan        cowan at ccil.org
   There was an old man                Said with a laugh, "I
     From Peru, whose lim'ricks all      Cut them in half, the pay is
       Look'd like haiku.  He              Much better for two."
                                             --Emmet O'Brien