[NTG-context] Mathdigits fallback does not work

[NTG-context] Mathdigits fallback does not work

[Mohammad Hossein Bateni]

[-- Attachment #1.1: Type: text/plain, Size: 6315 bytes --]

Hello,

I use fallbacks to substitute "Arabic" digits in math to Indic ones. The
mechanism doesn't work in the newest context. I tried it with mkxl version
in 2025.02.04 17:52 in a clean install. The email is a bit long, but please
bear with me, so I can learn a few other things along the way.

Compiling type-imp-mathdigits.mkiv (provided in the distribution) does not
change the digits to Indic form.

This is the part after the typescript definition:

\usetypescript [mathdigits] [xits-dejavu] [extendedarabicindic]
> \setupbodyfont[dejavu]
> \starttext
>     $3+2=5 \quad \bf 3+2=5$
> \stoptext


The logs contain the following, which shows dejavusansmono is not loaded.

mkiv lua stats  > loaded fonts: 3 files: texgyredejavu-math.otf,
> dejavuserif-bold.ttf, dejavuserif.ttf


IIUC one issue is that setupbodyfont kind of undoes the usetypescript
command because dejavu typescripts already define MathRoman.

Could you also explain the difference between setupbodyfont and
switchtobodyfont? Do I need the latter?

If I use the following instead, the Arabic digits do turn into Indic ones.

\usetypescript [mathdigits] [xits-dejavu] [extendedarabicindic]
> \setupbodyfont[dejavu]
> \definefontsynonym[MathRoman][MathRomanBold][fallbacks=mathdigits]
> \switchtobodyfont[sans]


With some caveats:
1) This still ignores the MathRoman definition in the above typescript (if
I change the above to use euler, e.g., I see no change).
2) This only changes MathRoman and not MathRomanBold. So if I have
something like {\bf $2+3$}, then the fallback does not kick in.
3) I can't use
\definefontsynonym[MathRoman][MathRoman][fallbacks=mathdigits] because it
creates a loop. Is there a way to "deep copy" font definitions (rather than
the "copy by reference") we have here? That way, I could run the fallback
for both MathRoman and MathRomanBold.

Speaking of which, what's the difference between \definefont and
\definefontsynonym?

What happens when either of these appear within a typescript definition? Is
the only difference, that within typescripts we have a separate namespace
to which the Sans, Serif, MathRoman, etc. are attached? Can I access the
MathRoman defined within typescript MyTypescript as "MyTypescriptMathRoman"
or something like that? What about if I define BaseMath (a non-standard
name, other than Sans, SansBold, MathRoman, etc.)? Are these also attached
to that namespace?

What is typescriptcollection BTW? Is it only cosmetic? Or can we pull in
all definitions from a collection? Does it have to be the same as the file
name?

What's the difference between MathRoman and MathRomanBold? I thought when
we would automatically get the Bold, Italic, BoldItalic variants when we
define something like Sans. Or do we need to pull in some setups for that?
For math, it seems that MathRoman has all the bold glyphs inside (at the
digitsbold offset). Does it also have the bold version of other symbols?
Then, what's the use of MathRomanBold?

In the math typescripts, I see the feature "math\mathsizesuffix"? Is it for
normal, script, scriptscript size? Who sets this macro? Are the math
typescripts loaded multiple times with different \mathsizesuffix values? At
what time?

Back to the original question, if I switch usetypescript and setupbodyfont
(see below), I see the math font changes taking effect (e.g., by picking up
euler-math.otf instead of xits-math.otf). However, the fallback still
doesn't work.

\setupbodyfont[dejavu]
> \usetypescript [mathdigits] [xits-dejavu] [extendedarabicindic]
> %\definefontsynonym[MathRoman][MathRomanBold][fallbacks=mathdigits]
> \switchtobodyfont[sans,20pt]


I need to uncomment the third line above to get the fallbacks to work. I
feel that the digitsnormal in MathRomanBold (used in the fallback) is
slightly bolder than the normal one, but my eyes may be failing me :)

Now it gets more mysterious. See this code snippet:

% no change from mathdigits file
> \starttypescript [mathdigits] [dejavu] [arabicindic,extendedarabicindic]
> \resetfontfallback  [mathdigits]
> \definefontfallback [mathdigits] [dejavusansmono]
> [digits\typescriptthree] [check=yes,force=yes,offset=digitsnormal]
> \definefontfallback [mathdigits] [dejavusansmonobold]
> [digits\typescriptthree] [check=yes,force=yes,offset=digitsbold]
> \stoptypescript



\starttypescript [mathdigits] [xits-dejavu]
> [arabicindic,extendedarabicindic]
> \usetypescript [mathdigits] [dejavu] [\typescriptthree]
> \definefontsynonym[MathRoman]
> [file:xits-math.otf][features=math\mathsizesuffix,goodies=xits-math,fallbacks=mathdigits]
> % copied from comments of the same file
> \definefontsynonym[MathRomanDigitsXitsDejavu]
> [file:xits-math.otf][features=math\mathsizesuffix,goodies=xits-math,fallbacks=mathdigits]
> \stoptypescript



\setupbodyfont[dejavu]
> \usetypescript [mathdigits] [xits-dejavu] [extendedarabicindic]
> % copied essentially from comments of the same file
> \definefontsynonym [MathRoman] [MathRomanDigitsXitsDejavu]
> \switchtobodyfont[sans,20pt]



\starttext
>     $3+2=5 \quad \bf 3+2=5$ math with fallback\\
>     {\bf$3+2=5 \quad \bf 3+2=5$ bold math}\\
>     {۳ + ۲ = ۵ \bf ۳ + ۲ = ۵ text Indic digits}\\
>     {3 + 2 = 5 \bf 3 + 2 = 5 text Arabic diigts}\\
>     %$$\sum_{i=1}^{8} x^2 = \pi\alpha + e^{-\beta}$$
> \stoptext


If the last line is uncommented, I ran into the following error.

Sorry, but I can't typeset math unless various parameters have been set.
This is
normally done by loading special math fonts into the math family slots.
Your font
set is lacking at least the parameter mentioned earlier.

When it's commented, the fallbacks still don't happen, even if I rewrite
the last definition as \definefontsynonym [MathRoman]
[MathRomanDigitsXitsDejavu][fallbacks=mathdigits].

In some other tests (not with math), I noticed that in fallback definition,
I also need to set the target key to be the same as the second parameter of
the definition; something like:

\definefontfallback [mathdigits] [dejavusansmono] [digitsarabicindic]
> [check=yes,force=yes,offset=digitsnormal,target=digitsarabicindic]


But I don't have a MWE for it right now.

Thanks a lot for your help,
Mohammad

[-- Attachment #1.2: Type: text/html, Size: 8742 bytes --]

[-- Attachment #2: Type: text/plain, Size: 511 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Mathdigits fallback does not work

[Hans Hagen via ntg-context]

On 2/8/2025 3:05 PM, Mohammad Hossein Bateni wrote:
> Hello,
> 
> I use fallbacks to substitute "Arabic" digits in math to Indic ones. The 
> mechanism doesn't work in the newest context. I tried it with mkxl 
> version in 2025.02.04 17:52 in a clean install. The email is a bit long, 
> but please bear with me, so I can learn a few other things along the way.

\definefontfallback [Math] [dejavusansmono] [0x0660-0x0669] 
[check=yes,force=yes,offset=digitsnormal]

\setupbodyfont[dejavu,sans]

\starttext
\startlines
     1 : \im{3+2=5 \quad \bf 3+2=5} math with fallback
     2 : {\bf \im{3+2=5 \quad \bf 3+2=5} bold math}
     3 : {۳ + ۲ = ۵ \bf ۳ + ۲ = ۵ text Indic digits}
     4 : {3 + 2 = 5 \bf 3 + 2 = 5 text Arabic diigts}
     5 : \dm{\sum_{i=1}^{8} x^2 = \pi\alpha + e^{-\beta}}
\stoplines
\stoptext

>          %$$\sum_{i=1}^{8} x^2 = \pi\alpha + e^{-\beta}$$
 >
> If the last line is uncommented, I ran into the following error.

In ConTeXt the $$ is just seen as $ (inline math).

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Mathdigits fallback does not work

[Mohammad Hossein Bateni]

[-- Attachment #1.1: Type: text/plain, Size: 5674 bytes --]

This is great. Thanks a lot, Hans!
Documented on the wiki:
https://wiki.contextgarden.net/Input_and_compilation/Languages/Right-to-left_RTL

I tested the idea with all context math fonts. TeX Gyre, DejaVu, Modern,
STIX Two, Libertinus, XCharter, Kp Fonts, EB Garamond, Concrete, and
Erewhon work with different fallback fonts I tried.
On the other hand, I couldn't get Kurier and Plex to work (not important,
just reporting). See note about Pennstander below.

My typescript looks something like this:

%%% beginning of demo
%%%
%%% some definitions for vazir, sahel and vazircode to set up rm, ss, mm
%%% followed by the following for math

% SEC A: fallback
\starttypescript [math] [fall:vazir] [name]
\definefontfallback [mathdig] [vazirregular]
[digitsextendedarabicindic] [check=yes,force=yes,offset=digitsnormal]
\definefontfallback [mathdig] [vazirbold] [digitsextendedarabicindic]
[check=yes,force=yes,offset=digitsbold]
% I use the following to remap some other characters like Arabic decimal
separator, etc.
\definefontfallback [mathdig] [vazirregular]     [arabic]
 [check=yes,force=yes]
\stoptypescript

% SEC B: base
\starttypescript [math] [main:modern] [name]
  \definefontsynonym[MathRoman]
 [file:latinmodern-math][features={math\mathsizesuffix,lm:mathextra,lm-math},goodies=modern-math,fallbacks=mathdig]

\definefontsynonym[MathRomanBold][file:latinmodern-math][features={math\mathsizesuffix,lm:mathextra,lm-math-bold,lm-math},goodies=modern-math,fallbacks=mathdig]
\stoptypescript

% SEC C: combination
\starttypescript  [myfont]
  \definetypeface [\typescriptone][rm] [serif][sahel]    [default]
  \definetypeface [\typescriptone][ss] [sans] [vazir]    [default]
  \definetypeface [\typescriptone][tt] [mono] [vazircode][default]
  %\definetypeface [\typescriptone][mm] [math] [modern]    [default]
  \definetypeface [\typescriptone][mm] [math] [fall:vazir,main:modern]
 [default]
\stoptypescript

%%% end of demo

Still unresolved (less important mostly out of curiosity):

1. I'd have preferred something like "[math] [fall] [vazir] [default]" and
"[math] [base] [modern] [default]" but couldn't get it to work. Is there
any other way to make the above more readable? In my setup, I have copies
of SEC A for each fallback font and a copy of SEC B for each base math font.
2. I don't understand how \definefontsynonym works. Does it have to be
within a typescript? When (and how many times) are the definitions inside
"executed"?
3. I couldn't get synonyms to work in the second argument
of \definefontfallback.
4. Can we pass multiple fallbacks to \definefontsynonym definition like
fallbacks={a,b}. I couldn't get this to work.
5. Can we do transitive definefontsynonyms and add features and fallback
iteratively? I mean, define B from A with some features and fallbacks, and
then C from B with some additional features and fallbacks.
6. I see that we have digitsnormal and digitsbold in the normal (non-bold)
math font. Do we have the bold version of every character in there?
7. When I tried to apply fallback on Pennstander (
https://github.com/juliusross1/Pennstander) I discovered that the fallback
does not take effect unless I add "presets.fallbacks {}," instead of the
tweaks of the goodies file.
8. What is a "font class". I couldn't make much sense of the magical code
for \definefontsynonym, etc. though something I found there was that there
exists a class fallbacks and a (non-class?) fallback for each font. In the
logs, I always saw the class one output as empty.

--Mohammad

On Wed, Feb 12, 2025 at 7:37 AM Hans Hagen via ntg-context <
ntg-context@ntg.nl> wrote:

> On 2/8/2025 3:05 PM, Mohammad Hossein Bateni wrote:
> > Hello,
> >
> > I use fallbacks to substitute "Arabic" digits in math to Indic ones. The
> > mechanism doesn't work in the newest context. I tried it with mkxl
> > version in 2025.02.04 17:52 in a clean install. The email is a bit long,
> > but please bear with me, so I can learn a few other things along the way.
>
> \definefontfallback [Math] [dejavusansmono] [0x0660-0x0669]
> [check=yes,force=yes,offset=digitsnormal]
>
> \setupbodyfont[dejavu,sans]
>
> \starttext
> \startlines
>      1 : \im{3+2=5 \quad \bf 3+2=5} math with fallback
>      2 : {\bf \im{3+2=5 \quad \bf 3+2=5} bold math}
>      3 : {۳ + ۲ = ۵ \bf ۳ + ۲ = ۵ text Indic digits}
>      4 : {3 + 2 = 5 \bf 3 + 2 = 5 text Arabic diigts}
>      5 : \dm{\sum_{i=1}^{8} x^2 = \pi\alpha + e^{-\beta}}
> \stoplines
> \stoptext
>
> >          %$$\sum_{i=1}^{8} x^2 = \pi\alpha + e^{-\beta}$$
>  >
> > If the last line is uncommented, I ran into the following error.
>
> In ConTeXt the $$ is just seen as $ (inline math).
>
> Hans
>
> -----------------------------------------------------------------
>                                            Hans Hagen | PRAGMA ADE
>                Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
>         tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
> -----------------------------------------------------------------
>
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to
> the Wiki!
>
> maillist : ntg-context@ntg.nl /
> https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
> webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
> archive  : https://github.com/contextgarden/context
> wiki     : https://wiki.contextgarden.net
>
> ___________________________________________________________________________________
>

[-- Attachment #1.2: Type: text/html, Size: 7461 bytes --]

[-- Attachment #2: Type: text/plain, Size: 511 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Mathdigits fallback does not work

[Hans Hagen]

On 2/23/2025 8:46 PM, Mohammad Hossein Bateni wrote:
> This is great. Thanks a lot, Hans!

i had made a TABLE test but your gmail bounces

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________