[luatex-plain] disappearing math-on node

[luatex-plain] disappearing math-on node

[Dohyun Kim]

Hi,

This is a bug report based on the issue discussed at
https://github.com/lualatex/luaotfload/issues/212

\font\tenrm{file:lmroman10-regular.otf:mode=node;script=latn}\tenrm
\setbox0\vbox{%
  x\penalty-10000
  $a$x$a$
}
\unvbox0
\setbox0\lastbox
\unhbox0
\end

The plain tex code shown above fails with a lua error:
luatex-fonts-merged.lua:9616: attempt to index local 'current' (a nil value)

My guess is: as math-on (math-off too) is a "discardable" item after a
linebreak, the first math-on node has gone away. So the
math-on/math-off pair has become broken, by which the behavior of
"node.end_of_math" is confused.

Regards,
-- 
Dohyun Kim
Seoul, Republic of Korea
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [luatex-plain] disappearing math-on node

[Hans Hagen]

On 4/3/2014 6:29 PM, Dohyun Kim wrote:
> Hi,
>
> This is a bug report based on the issue discussed at
> https://github.com/lualatex/luaotfload/issues/212
>
> \font\tenrm{file:lmroman10-regular.otf:mode=node;script=latn}\tenrm
> \setbox0\vbox{%
>    x\penalty-10000
>    $a$x$a$
> }
> \unvbox0
> \setbox0\lastbox
> \unhbox0
> \end
>
> The plain tex code shown above fails with a lua error:
> luatex-fonts-merged.lua:9616: attempt to index local 'current' (a nil value)
>
> My guess is: as math-on (math-off too) is a "discardable" item after a
> linebreak, the first math-on node has gone away. So the
> math-on/math-off pair has become broken, by which the behavior of
> "node.end_of_math" is confused.

mtxrun --script plain test.tex

runs ok here so i need an example that fails with context or luatex-plain

normally the font callback is called on a hlist and there are always 
paired math nodes

(what is line 9616 in the current beta?)

If indeed the end_of_math is a problem i can remove that optimiziation 
in the generic code

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [luatex-plain] disappearing math-on node

[Philipp Gesang]

[-- Attachment #1.1.1: Type: text/plain, Size: 1634 bytes --]

···<date: 2014-04-03, Thursday>···<from: Hans Hagen>···

> On 4/3/2014 6:29 PM, Dohyun Kim wrote:
> > This is a bug report based on the issue discussed at
> > https://github.com/lualatex/luaotfload/issues/212
> >
> > \font\tenrm{file:lmroman10-regular.otf:mode=node;script=latn}\tenrm
> > \setbox0\vbox{%
> >    x\penalty-10000
> >    $a$x$a$
> > }
> > \unvbox0
> > \setbox0\lastbox
> > \unhbox0
> > \end
> >
> > The plain tex code shown above fails with a lua error:
> > luatex-fonts-merged.lua:9616: attempt to index local 'current' (a nil value)
> >
> > My guess is: as math-on (math-off too) is a "discardable" item after a
> > linebreak, the first math-on node has gone away. So the
> > math-on/math-off pair has become broken, by which the behavior of
> > "node.end_of_math" is confused.
> 
> mtxrun --script plain test.tex
> 
> runs ok here so i need an example that fails with context or luatex-plain

It fails with the luatex-plain format from the minimals as of
yesterday and Luatex 4972:

    error: .../context/tex/texmf-context/tex/context/base/font-ota.lua:144: attempt to index local 'current' (a nil value)
    .
    \bye ^^@-\par 
                \vfill \supereject \end 
    l.17 \bye

Though the issue arises only in node mode.

> normally the font callback is called on a hlist and there are always 
> paired math nodes
> 
> (what is line 9616 in the current beta?)

Line 9626 with yesterday’s code. Attached are patches for
font-ota.lua and luatex-fonts-otn.lua that guard all invocations
of end_of_math() with checks for the subtype.

Best,
Philipp


[-- Attachment #1.1.2: luatex-fonts-otn.lua_end_of_math.patch --]
[-- Type: text/x-diff, Size: 911 bytes --]

--- luatex-fonts-otn.lua.orig	2014-04-05 15:02:34.711828352 +0200
+++ luatex-fonts-otn.lua	2014-04-05 15:02:36.955171990 +0200
@@ -2375,7 +2375,7 @@
                                 end
                             end
                             start = start.next
-                        elseif id == math_code then
+                        elseif id == math_code and start.subtype == 0 then
                             start = end_of_math(start).next
                         else
                             start = start.next
@@ -2567,7 +2567,7 @@
                             end
                         end
                         start = start.next
-                    elseif id == math_code then
+                    elseif id == math_code and start.subtype == 0 then
                         start = end_of_math(start).next
                     else
                         start = start.next

[-- Attachment #1.1.3: font-ota.lua_end_of_math.patch --]
[-- Type: text/x-diff, Size: 706 bytes --]

--- font-ota.lua.orig	2014-04-05 14:55:53.789969704 +0200
+++ font-ota.lua	2014-04-05 14:59:32.487654509 +0200
@@ -137,7 +137,7 @@
                 last[a_state] = s_fina
             end
             first, last, n = nil, nil, 0
-            if id == math_code then
+            if id == math_code and current.subtype == 0 then
                 current = end_of_math(current)
             end
         end
@@ -423,7 +423,7 @@
             if first or last then
                 first, last = finish(first,last)
             end
-            if id == math_code then
+            if id == math_code and current.subtype == 0 then
                 current = end_of_math(current)
             end
         end

[-- Attachment #1.2: Type: application/pgp-signature, Size: 490 bytes --]

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [luatex-plain] disappearing math-on node

[Hans Hagen]

On 4/5/2014 3:12 PM, Philipp Gesang wrote:
> ···<date: 2014-04-03, Thursday>···<from: Hans Hagen>···
>
>> On 4/3/2014 6:29 PM, Dohyun Kim wrote:
>>> This is a bug report based on the issue discussed at
>>> https://github.com/lualatex/luaotfload/issues/212
>>>
>>> \font\tenrm{file:lmroman10-regular.otf:mode=node;script=latn}\tenrm
>>> \setbox0\vbox{%
>>>     x\penalty-10000
>>>     $a$x$a$
>>> }
>>> \unvbox0
>>> \setbox0\lastbox
>>> \unhbox0
>>> \end
>>>
>>> The plain tex code shown above fails with a lua error:
>>> luatex-fonts-merged.lua:9616: attempt to index local 'current' (a nil value)
>>>
>>> My guess is: as math-on (math-off too) is a "discardable" item after a
>>> linebreak, the first math-on node has gone away. So the
>>> math-on/math-off pair has become broken, by which the behavior of
>>> "node.end_of_math" is confused.
>>
>> mtxrun --script plain test.tex
>>
>> runs ok here so i need an example that fails with context or luatex-plain
>
> It fails with the luatex-plain format from the minimals as of
> yesterday and Luatex 4972:
>
>      error: .../context/tex/texmf-context/tex/context/base/font-ota.lua:144: attempt to index local 'current' (a nil value)
>      .
>      \bye ^^@-\par
>                  \vfill \supereject \end
>      l.17 \bye
>
> Though the issue arises only in node mode.
>
>> normally the font callback is called on a hlist and there are always
>> paired math nodes
>>
>> (what is line 9616 in the current beta?)
>
> Line 9626 with yesterday’s code. Attached are patches for
> font-ota.lua and luatex-fonts-otn.lua that guard all invocations
> of end_of_math() with checks for the subtype.

I think that the removed math node is a bad thing so that will change in 
a next luatex release (i tested a patch but it will not go in the tl 
release). (Just like the math node at the end of a line it will have 
zero surround in order to make the backend happy.)

In order to avoid the 'current' issue I'll use an overload of 
end_of_math in the generic code.

The real solution is to make the built-in end_of_math more strict 
(trivial patch ready and tested so that can end up in the next luatex).

In the context code i won't intercept it because end_of_math will do 
that at some point.

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [luatex-plain] disappearing math-on node

[Philipp Gesang]

[-- Attachment #1.1: Type: text/plain, Size: 2711 bytes --]

···<date: 2014-04-06, Sunday>···<from: Hans Hagen>···

> On 4/5/2014 3:12 PM, Philipp Gesang wrote:
> > ···<date: 2014-04-03, Thursday>···<from: Hans Hagen>···
> >
> >> On 4/3/2014 6:29 PM, Dohyun Kim wrote:
> >>> This is a bug report based on the issue discussed at
> >>> https://github.com/lualatex/luaotfload/issues/212
> >>>
> >>> \font\tenrm{file:lmroman10-regular.otf:mode=node;script=latn}\tenrm
> >>> \setbox0\vbox{%
> >>>     x\penalty-10000
> >>>     $a$x$a$
> >>> }
> >>> \unvbox0
> >>> \setbox0\lastbox
> >>> \unhbox0
> >>> \end
> >>>
> >>> The plain tex code shown above fails with a lua error:
> >>> luatex-fonts-merged.lua:9616: attempt to index local 'current' (a nil value)
> >>>
> >>> My guess is: as math-on (math-off too) is a "discardable" item after a
> >>> linebreak, the first math-on node has gone away. So the
> >>> math-on/math-off pair has become broken, by which the behavior of
> >>> "node.end_of_math" is confused.
> >>
> >> mtxrun --script plain test.tex
> >>
> >> runs ok here so i need an example that fails with context or luatex-plain
> >
> > It fails with the luatex-plain format from the minimals as of
> > yesterday and Luatex 4972:
> >
> >      error: .../context/tex/texmf-context/tex/context/base/font-ota.lua:144: attempt to index local 'current' (a nil value)
> >      .
> >      \bye ^^@-\par
> >                  \vfill \supereject \end
> >      l.17 \bye
> >
> > Though the issue arises only in node mode.
> >
> >> normally the font callback is called on a hlist and there are always
> >> paired math nodes
> >>
> >> (what is line 9616 in the current beta?)
> >
> > Line 9626 with yesterday’s code. Attached are patches for
> > font-ota.lua and luatex-fonts-otn.lua that guard all invocations
> > of end_of_math() with checks for the subtype.
> 
> I think that the removed math node is a bad thing so that will change in 
> a next luatex release (i tested a patch but it will not go in the tl 
> release). (Just like the math node at the end of a line it will have 
> zero surround in order to make the backend happy.)

I’d be interested in testing the patch as well.

> In order to avoid the 'current' issue I'll use an overload of 
> end_of_math in the generic code.
> 
> The real solution is to make the built-in end_of_math more strict 
> (trivial patch ready and tested so that can end up in the next luatex).
> 
> In the context code i won't intercept it because end_of_math will do 
> that at some point.

Good news. Though I’m a little worried about the fontloader in
Context and the generic format diverging too much.

Thanks for looking into this,
Philipp


[-- Attachment #1.2: Type: application/pgp-signature, Size: 490 bytes --]

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: [luatex-plain] disappearing math-on node

[Philipp Gesang]

[-- Attachment #1.1: Type: text/plain, Size: 2467 bytes --]

···<date: 2014-04-06, Sunday>···<from: Hans Hagen>···

> On 4/5/2014 3:12 PM, Philipp Gesang wrote:
> > ···<date: 2014-04-03, Thursday>···<from: Hans Hagen>···
> >
> >> On 4/3/2014 6:29 PM, Dohyun Kim wrote:
> >>> This is a bug report based on the issue discussed at
> >>> https://github.com/lualatex/luaotfload/issues/212
> >>>
> >>> \font\tenrm{file:lmroman10-regular.otf:mode=node;script=latn}\tenrm
> >>> \setbox0\vbox{%
> >>>     x\penalty-10000
> >>>     $a$x$a$
> >>> }
> >>> \unvbox0
> >>> \setbox0\lastbox
> >>> \unhbox0
> >>> \end
> >>>
> >>> The plain tex code shown above fails with a lua error:
> >>> luatex-fonts-merged.lua:9616: attempt to index local 'current' (a nil value)
> >>>
> >>> My guess is: as math-on (math-off too) is a "discardable" item after a
> >>> linebreak, the first math-on node has gone away. So the
> >>> math-on/math-off pair has become broken, by which the behavior of
> >>> "node.end_of_math" is confused.
> >>
> >> mtxrun --script plain test.tex
> >>
> >> runs ok here so i need an example that fails with context or luatex-plain
> >
> > It fails with the luatex-plain format from the minimals as of
> > yesterday and Luatex 4972:
> >
> >      error: .../context/tex/texmf-context/tex/context/base/font-ota.lua:144: attempt to index local 'current' (a nil value)
> >      .
> >      \bye ^^@-\par
> >                  \vfill \supereject \end
> >      l.17 \bye
> >
> > Though the issue arises only in node mode.
> >
> >> normally the font callback is called on a hlist and there are always
> >> paired math nodes
> >>
> >> (what is line 9616 in the current beta?)
> >
> > Line 9626 with yesterday’s code. Attached are patches for
> > font-ota.lua and luatex-fonts-otn.lua that guard all invocations
> > of end_of_math() with checks for the subtype.
> 
> I think that the removed math node is a bad thing so that will change in 
> a next luatex release (i tested a patch but it will not go in the tl 
> release). (Just like the math node at the end of a line it will have 
> zero surround in order to make the backend happy.)
> 
> In order to avoid the 'current' issue I'll use an overload of 
> end_of_math in the generic code.
> 
> The real solution is to make the built-in end_of_math more strict 
> (trivial patch ready and tested so that can end up in the next luatex).

With experimental revision 4974 the bug is gone. Thanks!
Philipp


[-- Attachment #1.2: Type: application/pgp-signature, Size: 490 bytes --]

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________