Math encoding in XeTeX

Math encoding in XeTeX

[Ricard Roca]

Hi,

With ConTeXt and pdfTeX you can use \enableregime[utf] (utf8 encoding) with 
normal mode and math mode (no need to write \pm, \alpha or \times). But with 
ConTeXt and XeTeX you don't have to specify utf8 regime, because XeTeX can 
read utf8 directly. However, math mode in XeTeX is a different story ( 256 
character tfm's are needed and so, and the encoding has to be ascii, so you 
have to write \pm, and ± doesn't work). 

It would be nice if utf8 regime (using uni-xxx files) could be enabled only 
for math mode when using XeTeX, and so you would be able to input unicode 
characters in all modes (normal and math).

Thanks in advance,

Ricard Roca

Re: Math encoding in XeTeX

[Mojca Miklavec]

On 5/21/06, Ricard Roca wrote:
> Hi,
>
> With ConTeXt and pdfTeX you can use \enableregime[utf] (utf8 encoding) with
> normal mode and math mode (no need to write \pm, \alpha or \times). But with
> ConTeXt and XeTeX you don't have to specify utf8 regime, because XeTeX can
> read utf8 directly. However, math mode in XeTeX is a different story ( 256
> character tfm's are needed and so, and the encoding has to be ascii, so you
> have to write \pm, and ± doesn't work).
>
> It would be nice if utf8 regime (using uni-xxx files) could be enabled only
> for math mode when using XeTeX, and so you would be able to input unicode
> characters in all modes (normal and math).

According to one of the recent threads you can most probably do
something like this (I have to reboot first to try it out):
    \catcode±=\active
    \def±{\pm}

There are actually two possibilities:

1. define a whole lot of active characters in the spirit of the above
definition, then ConTeXt will render them OK (a matter of a slight
redefinition of \enableregime macro or perhaps an additional macro to
switch this option on or off)

2. better support for non-OpenType fonts in XeTeX.
XeTeX currently does a very good job with OpenType. I guess that it
would be doable that someone would tell to XeTeX which glyphs are
present in a certain font/encoding (partially it could guess that
alone) and then things could happen automatically even without
defining active characters.

(or a mixture of both)

Also, ConTeXt can fake many characters if they are not present in the
font. Without defining active characters this isn't possible since
ConTeXt never sees them.

Mojca

Re: Math encoding in XeTeX

[Mojca Miklavec]

On 5/21/06, Mojca Miklavec wrote:
> According to one of the recent threads you can most probably do

I'm sorry for not being precise enough, the thread was on the XeTeX
mailing list, not here. (How to use EC font encoding in XeTeX?)

Mojca

PS: I would vote for better support of encodings (Type1 fonts) in
XeTeX, but I doubt that doing things without active characters can be
as good as it is now in ConTeXt+pdfTeX.

Re: Math encoding in XeTeX

[Ricard Roca]

> I'm sorry for not being precise enough, the thread was on the XeTeX
> mailing list, not here. (How to use EC font encoding in XeTeX?)

Don't worry. I am subscribed to that list too :)

Thanks a lot

Ricard

Re: Math encoding in XeTeX

[Hans Hagen]

Mojca Miklavec wrote:
> On 5/21/06, Ricard Roca wrote:
>   
>> Hi,
>>
>> With ConTeXt and pdfTeX you can use \enableregime[utf] (utf8 encoding) with
>> normal mode and math mode (no need to write \pm, \alpha or \times). But with
>> ConTeXt and XeTeX you don't have to specify utf8 regime, because XeTeX can
>> read utf8 directly. However, math mode in XeTeX is a different story ( 256
>> character tfm's are needed and so, and the encoding has to be ascii, so you
>> have to write \pm, and � doesn't work).
>>
>> It would be nice if utf8 regime (using uni-xxx files) could be enabled only
>> for math mode when using XeTeX, and so you would be able to input unicode
>> characters in all modes (normal and math).
>>     
>
> According to one of the recent threads you can most probably do
> something like this (I have to reboot first to try it out):
>     \catcode�=\active
>     \def�{\pm}
>
> There are actually two possibilities:
>
> 1. define a whole lot of active characters in the spirit of the above
> definition, then ConTeXt will render them OK (a matter of a slight
> redefinition of \enableregime macro or perhaps an additional macro to
> switch this option on or off)
>
> 2. better support for non-OpenType fonts in XeTeX.
> XeTeX currently does a very good job with OpenType. I guess that it
> would be doable that someone would tell to XeTeX which glyphs are
> present in a certain font/encoding (partially it could guess that
> alone) and then things could happen automatically even without
> defining active characters.
>
> (or a mixture of both)
>
> Also, ConTeXt can fake many characters if they are not present in the
> font. Without defining active characters this isn't possible since
> ConTeXt never sees them.
>   
i think that using \enableregime[utf] should work ok xetex (but in loading patterns) 

You can play with 

\startregime[none]
  \dostepwiserecurse{128}{255}{1}
    {\expanded{\defineactivecharacter 
       {\recurselevel} 
       {\rawcharacter{\recurselevel}}}}
\stopregime

\enableregime[none]

\appendtoks
  \enableregime[utf]%
%   \everyhbox\expandafter{\the\everyhbox\enableregime[none]}% fails 
\to \everymathematics

\starttext

text : ç ß

math : $ç ß [\text{\bf\enableregime[none]ç ß}] ç ß$

\stoptext

Taco may know why active chars behave a bit strange in math (may hav eto do with the multiple passes in math and info getting lost 

Hans 



-----------------------------------------------------------------
                                          Hans Hagen | PRAGMA ADE
              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                             | www.pragma-pod.nl
-----------------------------------------------------------------

_______________________________________________
ntg-context mailing list
ntg-context@ntg.nl
http://www.ntg.nl/mailman/listinfo/ntg-context

Re: Math encoding in XeTeX

[Mojca Miklavec]

On 5/21/06, Hans Hagen wrote:
> Mojca Miklavec wrote:

> i think that using \enableregime[utf] should work ok xetex (but in loading patterns)
>
> You can play with
>
> \startregime[none]
>   \dostepwiserecurse{128}{255}{1}
>     {\expanded{\defineactivecharacter
>        {\recurselevel}
>        {\rawcharacter{\recurselevel}}}}
> \stopregime
>
> \enableregime[none]
>
> \appendtoks
>   \enableregime[utf]%
> %   \everyhbox\expandafter{\the\everyhbox\enableregime[none]}% fails
> \to \everymathematics
>
> \starttext
>
> text : ç ß
>
> math : $ç ß [\text{\bf\enableregime[none]ç ß}] ç ß$
>
> \stoptext
>
> Taco may know why active chars behave a bit strange in math (may hav eto do with the multiple passes in math and info getting lost

I'm only speculating here because I don't have my computer & linux
here, but may it be that it's because XeTeX handles characters in a
completely different way than pdfTeX? If you define 128 active
characters, the thing will behave pseudo-randomly.

Active characters have to be defined in the "unicode-way", not by
reimplementing Unicode inside ConTeXt macros (which is what ConTeXt
currently does for pdfTeX). I guess that the unicode vectors should be
redefined (or at least generalized) for the purposes of XeTeX.

I would suggest to create active characters for math and the "extended
latin" sections of Unicode, so that "faking" characters and using
ec-encoded fonts would still work (although XeTeX should also support
EC encoding in my opinion, which it currently doesn't). You can then
leave the rest of Unicode to be handled by XeTeX.

Explanation: if I use ec-encoded Type1 fonts, \ccaron will work, but
typing "č" directly won't. ConTeXt knows where \ccaron is located in
an EC font and is able to use it properly, but XeTeX doesn't know
where to look for the glyph "whatever the unicode number of ccaron
is". The only way out of it is to say
    \catcode`č=\active \defč{\ccaron}
So my suggestion would be to define active characters for those
Unicode letters where it makes sense to do it (accented latin
characters, math) and where they can be faked.

There was already a discussion about a similar topic some time ago
(but it might take too much processing time):
   if glyph exists in the font ( \ifnum\XeTeXcharglyph"xxxx>0 )
      use it
   else
      fake it

\enableregime should also be modified in order to handle regimes other
that UTF properly in XeTeX. There are two possibilities: either to let
XeTeX do it his own way or to read everything as "byte" and let
ConTeXt handle regimes in its own way.
If XeTeX does the job, EC-encoded fonts won't work. If you let ConTeXt
process it, you might loose support for some more obscure regimes that
XeTeX can already handle.

Mojca
_______________________________________________
ntg-context mailing list
ntg-context@ntg.nl
http://www.ntg.nl/mailman/listinfo/ntg-context