Re: (Con)TeX(t), Unicode and accented characters

["Adam Lindsay" <atl@comp.lancs.ac.uk>]

r.ermers@hccnet.nl said this at Tue, 21 Dec 2004 08:56:40 +0100:

>In Latex the combination \"{a} can mean two things:
>1. in most fonts: show the charachter on the a given numerical position,
>which means that there is one character ä.
>
>2. in some other fonts \"{a} means: combine " with a and make an ä. This
>means that " is combined with the character on the numerical position of
>a. TeX does this very well and thus construes very acceptable diacritical
>signs like \"{q}, \d{o}, \v{o}, which do not exist in regular fonts.

Robert,

That's a helpful explanation. I'll try to expand on that in the ConTeXt
case, just in case people are curious or are led into thinking it's just
the same:

In ConTeXt, the combination \"{a} means one thing: \adiaeresis (see enco-
acc). This \adiaeresis can mean one of two things, depending on the encoding:
1. Numerical position, or
2. The fallback case (defined in enco-def), where a diaeresis/umlaut is
placed atop an 'a' glyph. Hyphenation implications as Hans described.

The interesting/helpful thing about ConTeXt is that internally, that
glyph is given a consistent name, no matter how it is input or output.
So, if you type ä in your given input regime, and that encoding is
properly set, that numerical ä (e.g., character #228 in the windows
regime) is mapped to \adiaeresis.

Wanna know what happens in UTF-8? Here's my 'simplified' explanation:
In a UTF-8 bytestream, that character "ä" is signified by two bytes:
0xC3, 0xA4. That first byte triggers a conversion of both bytes into two
different bytes, the actual Unicode number, 0x00 0xE4 (or: 0, 228).
ConTeXt then looks into internal hashes set up (in this case, the unic-
000 vector), looks at the 228th element, and sees that it's \adiaeresis.
Things then proceed as normal. :)

(It's also interesting to note that for PostScript and TrueType fonts,
that number > name > number (glyph) mapping happens yet again in the
driver. But all that is outside of TeX proper, so to say any more would
be confusing.)
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Adam T. Lindsay, Computing Dept.     atl@comp.lancs.ac.uk
 Lancaster University, InfoLab21        +44(0)1524/510.514
 Lancaster, LA1 4WA, UK             Fax:+44(0)1524/510.492
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-