update / punctuation / math

[Hans Hagen via ntg-context <ntg-context@ntg.nl>]

[-- Attachment #1: Type: text/plain, Size: 9411 bytes --]

Hi,

There have been some mails about punctuation spacing and a fix was added 
to the engine that related to that. As tests showed it to be okay so we 
made an update. It took a bit longer than normal because we were in the 
middle of some other math stuff: additional fonts and extensibles.

Daniel Flipo maintains a few math fonts (like concrete, xcharter, 
erewhon, kp, euler) and the last few weeks more extensive support for 
extensibles was added and concrete became quite nice too, so these fonts 
make a nice benchmark. As they are part of the lmtx install and we made 
sure to support them.

In the process we adapted our 2023 roadmap of which part is attached (we 
included an example end then decided to show of concrete).

When we go through the process of 'upgrading' we noticed some 
interesting names for symbols and 'constructs'. Quite some come from 
plain and/or amsmath (in the past taco and aditya did some porting to 
context) and we're not always sure if something is really used (or even 
what it was intended for) so if you notice something weird or missing, 
let us know. Examples are welcome too. It might also be that something 
can go away because it's obsolete or never needed (so far we could 
resist te kick-out-symbole-name temptation when it comes to symbol names 
that we think no sane user can remember or imagine to be there).

When often add extra tests to the test suite (math subsection).

Hans & Mikael

ps. Alan and I are still messing around with some cross referencing. 
That code is still experimental and can have issues that we're looking 
at but hard to nail down (huge complex cross-referencing documents). 
More about that later.

==================================

We added the tex of the pdf below

====== extract from roadmap ======

\usemodule[article-basic,abbreviations-logos]

\setupbodyfont[concrete]

\starttext

\startsubject[title=Math in \CONTEXT\ roadmap]

\startitemize[n]

\startitem
     After playing with math support for more than a year, we have come 
to the
     conclusion that it is time to move on. We have already discarded italic
     correction and now are replacing rules with extensibles. Much was 
already in
     place (and applied) but experiences with type one antykwas made us 
review
     some \OPENTYPE\ fonts. Not using rules makes some of them look 
better. The
     effect is subtle and probably not \AMS\ compliant, but we think 
that it will
     work out well for simple math like fractions of decimal numbers.
     Consequently, we have added to our shrinking to-do list the burden to
     investigate whether we can remove those obsolete code paths from 
the engine.
     After all, who needs italic correction, who prefers ugly rules to 
beautiful
     glyphs, and who understands all these font parameters? Furthermore, 
after all
     these years, we don't expect \OPENTYPE\ font and \UNICODE\ math 
technologies
     to improve much; we don't know if \MICROSOFT\ is developing their 
technology
     further at all. Therefore, we are confident that what we are doing 
is the way
     it should have been done when math was upgraded. Hopefully users 
will notice
     the improvements.
\stopitem

\startitem
     Math also means physics and units (that topic was brought up 
recently on the
     list by Gavin). Therefore, because we're in cleanup mode, we decided to
     eliminate some more. With \ISO\ now in place for a long time, we 
are going to
     ignore the existence of the inch as unit from now on. The unit will 
probably
     remain in the engine for nostalgic reasons, but it will no be 
accepted in
     MWE. Instead, we will provide some more modern, culturally correct,
     kid-friendly units that we will use in examples, manuals and such. 
Because
     the four-person strong team dealing with this wants to avoid making 
mistakes,
     we will go through a careful and scientifically sound process of 
calibration
     first, using a selected tex savvy audience. We expect these new 
units to be
     stable a month from now. Believe it or not, in the process of 
documenting all
     this, we found a buglet in the new math dimension spacing, so it 
has already
     paid off. Expect to hear more in a month or so, and enjoy your 
inches as long
     as you still can. In case you wonder how this relates to math other 
than
     mentioned: the math subsystem has 'mu' as adaptive unit, and that 
inspired is
     to come up with one for text (in addition to two new more or less fixed
     units).
\stopitem

\startitem
     The math family model is a fundamental concept in \TEX\ but we 
think we can
     do without. First of all, \OPENTYPE\ math fonts have (design) 
script and
     scriptscript sizes built in, so for that we have one family. 
Second, only
     full bold (heavy) makes sense as companion for regular math which is
     something that in practice we can support otherwise. So, this makes us
     consider dropping families altogether which then provides (mem) 
space for
     even more classes or dictionaries. If we nevertheless decide to keep
     families, we can certainly go with less than we have now, maybe two 
(or four
     if we want to be generous and also resemble original tex) of them 
is enough.
     We cannot imagine users wanting more. As a side note: completely 
divorcing
     families could make the math engine a bit leaner. It is hard to 
explain and
     users only care about the outcome. So more on this later.
\stopitem

\startitem
     Another path to explore is to identify the few building blocks that are
     needed for typesetting math, and then doing a bit more at the tex 
end. Of
     course that would nil quite some earlier effort, which is a bit 
frustrating,
     but still \unknown\ maybe the math engine can be reduced to a 
fraction of
     what is is now.
\stopitem

\startitem
     When we look at the math fonts and some characters in there, we 
sometimes
     wonder what makes sense. For some, searching in e.g. arXiv brings 
no hit.
     Basically we have obsolete math symbols and currently used one. 
That made us
     think about ancient math versus modern math, just like there is 
ancient greek
     and modern greek. Because math is a script one can wonder about 
obsolete math
     dialects with symbols just like there are plenty deal scripts in 
\UNICODE. We
     already are working on dictionaries but another axis is useability.
\stopitem

\startitem
     We no longer have the small / large extensible family model so we can
     simplify delimiters in the engine. Not something users should worry 
about.
\stopitem

\startitem
     We're not sure why math is considered stable because everything moved
     forward. Therefore we're preparing a bid for extra math symbols as 
needed in
     modern explorative and daring math thesis. When symbols are really 
used, and
     we have proof of that, it should be possible to get them un 
\UNICODE, just
     like all these emoji. We welcome input and as an example of 
currently faked
     symbols we added some to the distribution as easter eggs. One example:

     Mikael got contacted by a stressed student working on a thesis on
     probability. This student needed to typeset the characteristic 
function of a
     random variable \im {X} with density function \im {f_{X}}, and it was
     insisted to use another notation than the (wide) hat, that was 
already used
     for something else. For this reason the \tex {widerandomhat} was 
introduced,

     \useMPlibrary[newmath]

     \startformula
         E[\ee^{\ii tX}] = \widerandomhat{f_{X}}(t)\mtp{,}
         E[\ee^{\ii t(X_1+X_2)}] = \widerandomhat{f_{X_1} \ast 
f_{X_2}}(t)\mtp{.}
     \stopformula

     Naturally, it is automatically scaled, just like the ordinary wide hat

     \startformula
         \widehat{a+b+c+d+e+f} \neq \widerandomhat{a+b+c+d+e+f}
     \stopformula

     Once the thesis is printed, we will contact the \UNICODE\ Consortium to
     suggest that it gets a slot.
\stopitem

\startitem
     Our most ambitious project is a reverse engineering one, which is 
why it is
     conducted at the engineering faculty of the Dnul university (we cannot
     reveal the real name yet). In math articles one can find 
visualizing like
     $x\leftarrow x$ and there are plenty of \TEX\ commands that have 
arrow or
     hook in their names. If you look at the names of math symbols plenty
     are kind of weird. We think it is not natural and are considering a 
\quote
     {natural language math input} project, where you tell what it is 
and get the
     symbols you expect. For that we need to analyze typeset math and 
from the
     context as well as visualization derive a dataset that we can feed 
into a
     machine learning subsystem that then can be used to turn input into 
type. We
     have several stages in mind spanning years but it can be fun. Think 
of it
     like \quote {untagged math} which then of course results in \quote 
{untagged
     pdf}, but better!
\stopitem

\stopitemize

Mikael & Hans

\stopsubject

\stoptext


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------

[-- Attachment #2: 230401-0.pdf --]
[-- Type: application/pdf, Size: 67658 bytes --]

[-- Attachment #3: Type: text/plain, Size: 496 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : https://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : https://contextgarden.net
___________________________________________________________________________________