* underbraces get progressively more misaligned (w/ minimal example)
@ 2014-06-29 10:13 Sanjoy Mahajan
2014-06-29 10:24 ` Sanjoy Mahajan
0 siblings, 1 reply; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-06-29 10:13 UTC (permalink / raw)
To: ntg-context
Dear list,
As I copyedited my book, I noticed examples where the underbraces in
math formulas were not aligned vertically, especially toward the end of
the book. Eventually I realized it was not due to my growing
copyediting skill, and I have made the following minimal example showing
the problem (I am using 2014.05.17 MkIV):
\setupformulas[indentnext=auto, spacebefore=none, spaceafter=none]
\starttext
\dorecurse{3000}{
\placeformula\startformula
\underbrace{\hbox{yyyy}}_{\rho} =
\underbrace{\hbox{yyyy yyyy yyyy}}_{\rho}
\stopformula
}
\stoptext
It'll produce a 162-page file. On the formulas on page 1, the two
underbraces in each equation are aligned. But there seems to be some
kind of slow leak, because the underbraces become ever more misaligned.
By page 162, they are visibly misaligned by maybe 3pt. If I increase
the recursion count to 30000, on page 2129 the misalignment is about 30pt.
Taking out the \placeformula makes the misalignment go away.
I tried to reproduce the problem with plain LuaTeX, using the tex file
below, but the underbraces came out aligned.
\newcount\foo
\foo=3000
\loop
$$
\underbrace{\hbox{yyyy}}_{\rho} =
\underbrace{\hbox{yyyy yyyy yyyy}}_{\rho}
$$
\advance \foo -1
\ifnum \foo>0
\repeat
\end
Any thoughts on this very strange issue?
-Sanjoy
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-06-29 10:13 underbraces get progressively more misaligned (w/ minimal example) Sanjoy Mahajan
@ 2014-06-29 10:24 ` Sanjoy Mahajan
2014-07-03 10:49 ` Sanjoy Mahajan
0 siblings, 1 reply; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-06-29 10:24 UTC (permalink / raw)
To: ntg-context
[-- Attachment #1: Type: text/plain, Size: 83 bytes --]
Here is a 72dpi rendering of the last page (with recursion count=30000).
-Sanjoy
[-- Attachment #2: page with large misalignments --]
[-- Type: image/png, Size: 19406 bytes --]
[-- Attachment #3: Type: text/plain, Size: 485 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-06-29 10:24 ` Sanjoy Mahajan
@ 2014-07-03 10:49 ` Sanjoy Mahajan
2014-07-03 19:21 ` Wolfgang Schuster
0 siblings, 1 reply; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-07-03 10:49 UTC (permalink / raw)
To: ntg-context
I thought the problem might lie with the general
\definemathunderextensible (math-stc.mkvi), because underbrace is
defined as
\definemathunderextensible [vfenced] [underbrace] ["FE3DF] % ["023DF]
Other underextensibles include \underbar:
\definemathunderextensible [vfenced] [underbar] ["FE33F]
% ["203E]
So I tried my example replacing \underbrace with \underbar but the
alignment comes out fine (and \overbrace works fine too). So that
debugging idea didn't work, except to indicate that something special
about underbrace isn't working right.
Does anyone have suggestions for further debugging ideas?
-Sanjoy
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-03 10:49 ` Sanjoy Mahajan
@ 2014-07-03 19:21 ` Wolfgang Schuster
2014-07-03 21:03 ` Sanjoy Mahajan
2014-07-07 13:53 ` Sanjoy Mahajan
0 siblings, 2 replies; 20+ messages in thread
From: Wolfgang Schuster @ 2014-07-03 19:21 UTC (permalink / raw)
To: mailing list for ConTeXt users; +Cc: Hans Hagen, sanjoy
Am 03.07.2014 um 12:49 schrieb Sanjoy Mahajan <sanjoy@MIT.EDU>:
> I thought the problem might lie with the general
> \definemathunderextensible (math-stc.mkvi), because underbrace is
> defined as
>
> \definemathunderextensible [vfenced] [underbrace] ["FE3DF] % ["023DF]
>
> Other underextensibles include \underbar:
>
> \definemathunderextensible [vfenced] [underbar] ["FE33F]
> % ["203E]
>
> So I tried my example replacing \underbrace with \underbar but the
> alignment comes out fine (and \overbrace works fine too). So that
> debugging idea didn't work, except to indicate that something special
> about underbrace isn't working right.
>
> Does anyone have suggestions for further debugging ideas?
math-stc.mkvi:
\unexpanded\def\math_stackers_make_double#top#bottom#category#codepoint#codeextra#text%
{\begingroup
\edef\currentmathstackers{#category}%
\mathstackersparameter\c!left\relax
\ifmmode\math_class_by_parameter\mathstackersparameter\else\dontleavehmode\fi
{\edef\currentmathstackers{#category}%
\edef\m_math_stackers_text_middle {#text}%
%
...
%
- \ifx\currentmathtext\empty
+ \ifx\m_math_stackers_text_middle\empty
\setbox\scratchboxthree\emptyhbox
\else
\setmathtextbox\scratchboxthree\hbox{\math_stackers_middletext}%
\fi
...
\fi}%
\mathstackersparameter\c!right\relax
\edef\p_limits{\mathstackersparameter\c!mathlimits}%
\ifx\p_limits\v!yes
\expandafter\endgroup\expandafter\limits
\else
\expandafter\endgroup
\fi}
Wolfgang
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-03 19:21 ` Wolfgang Schuster
@ 2014-07-03 21:03 ` Sanjoy Mahajan
2014-07-03 22:09 ` Hans Hagen
2014-07-07 13:53 ` Sanjoy Mahajan
1 sibling, 1 reply; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-07-03 21:03 UTC (permalink / raw)
To: Wolfgang Schuster, mailing list for ConTeXt users; +Cc: Hans Hagen
[-- Attachment #1: Type: text/plain, Size: 1939 bytes --]
> math-stc.mkvi:
>
> \unexpanded\def\math_stackers_make_double#top#bottom#category#codepoint#codeextra#text%
> ...
> - \ifx\currentmathtext\empty
> + \ifx\m_math_stackers_text_middle\empty
> \setbox\scratchboxthree\emptyhbox
> \else
Thanks for looking into this, Wolfgang.
I made this change to math-stc.mkvi in the 2014.05.17 beta, then remade
the MkIV format with
mtxrun --selfupdate
mtxrun --generate
luatools --generate
context --make
Then I ran context on my test file (with 30000 as the \dorecurse count).
However, the offset underbraces remain. I'm attaching a image from
p. 2128, by which point the offset is 20 or 30pt. I am fairly sure I am
running the newly created version, because remaking the format included
these lines:
open source > 1 > 256 > /home/sanjoy/context/2014.05.17-beta/tex/texmf-context/tex/context/base/math-stc.mkvi
loading > ConTeXt Math Macros / Stackers
close source > 1 > 256 > /home/sanjoy/context/2014.05.17-beta/tex/texmf-context/tex/context/base/math-stc.mkvi
and the log output from running the minimal.tex shows that the the fmt
is indeed dated 2014.7.3:
mtx-context | run 1: luatex --fmt="/home/sanjoy/context/2014.05.17-beta/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e/formats/luatex/cont-en" --jobname="minimal" --lua="/home/sanjoy/context/2014.05.17-beta/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e/formats/luatex/cont-en.lui" --no-parse-first-line --c:currentrun=1 --c:fulljobname="./minimal.tex" --c:input="./minimal.tex" --c:kindofrun=1 --c:maxnofruns=8 "cont-yes.mkiv"
This is LuaTeX, Version beta-0.79.1 (TeX Live 2014/dev) (rev 4972)
\write18 enabled.
open source > 1 > 1 > /home/sanjoy/context/2014.05.17-beta/tex/texmf-context/tex/context/base/cont-yes.mkiv
ConTeXt ver: 2014.05.17 23:46 MKIV beta fmt: 2014.7.3 int: english/english
-Sanjoy
[-- Attachment #2: page 2128 (PNG) --]
[-- Type: image/png, Size: 13724 bytes --]
[-- Attachment #3: Type: text/plain, Size: 485 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-03 21:03 ` Sanjoy Mahajan
@ 2014-07-03 22:09 ` Hans Hagen
2014-07-03 22:37 ` Sanjoy Mahajan
0 siblings, 1 reply; 20+ messages in thread
From: Hans Hagen @ 2014-07-03 22:09 UTC (permalink / raw)
To: Sanjoy Mahajan, Wolfgang Schuster, mailing list for ConTeXt users
On 7/3/2014 11:03 PM, Sanjoy Mahajan wrote:
>> math-stc.mkvi:
>>
>> \unexpanded\def\math_stackers_make_double#top#bottom#category#codepoint#codeextra#text%
>> ...
>> - \ifx\currentmathtext\empty
>> + \ifx\m_math_stackers_text_middle\empty
>> \setbox\scratchboxthree\emptyhbox
>> \else
>
> Thanks for looking into this, Wolfgang.
>
> I made this change to math-stc.mkvi in the 2014.05.17 beta, then remade
> the MkIV format with
>
> mtxrun --selfupdate
> mtxrun --generate
> luatools --generate
> context --make
context --make does a --generate so you can skip that and the luatools
run is not needed either (selfupdate is seldom needed too)
> Then I ran context on my test file (with 30000 as the \dorecurse count).
>
> However, the offset underbraces remain. I'm attaching a image from
> p. 2128, by which point the offset is 20 or 30pt. I am fairly sure I am
> running the newly created version, because remaking the format included
> these lines:
>
> open source > 1 > 256 > /home/sanjoy/context/2014.05.17-beta/tex/texmf-context/tex/context/base/math-stc.mkvi
> loading > ConTeXt Math Macros / Stackers
> close source > 1 > 256 > /home/sanjoy/context/2014.05.17-beta/tex/texmf-context/tex/context/base/math-stc.mkvi
>
> and the log output from running the minimal.tex shows that the the fmt
> is indeed dated 2014.7.3:
>
> mtx-context | run 1: luatex --fmt="/home/sanjoy/context/2014.05.17-beta/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e/formats/luatex/cont-en" --jobname="minimal" --lua="/home/sanjoy/context/2014.05.17-beta/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e/formats/luatex/cont-en.lui" --no-parse-first-line --c:currentrun=1 --c:fulljobname="./minimal.tex" --c:input="./minimal.tex" --c:kindofrun=1 --c:maxnofruns=8 "cont-yes.mkiv"
> This is LuaTeX, Version beta-0.79.1 (TeX Live 2014/dev) (rev 4972)
> \write18 enabled.
> open source > 1 > 1 > /home/sanjoy/context/2014.05.17-beta/tex/texmf-context/tex/context/base/cont-yes.mkiv
>
> ConTeXt ver: 2014.05.17 23:46 MKIV beta fmt: 2014.7.3 int: english/english
we get a .5pt height difference somewhere (nothing else different in a
trace) ... smells like a luatex bug (testing that now)
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-03 22:09 ` Hans Hagen
@ 2014-07-03 22:37 ` Sanjoy Mahajan
2014-07-03 22:47 ` luigi scarso
` (2 more replies)
0 siblings, 3 replies; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-07-03 22:37 UTC (permalink / raw)
To: Hans Hagen, Wolfgang Schuster, mailing list for ConTeXt users
[-- Attachment #1: Type: text/plain, Size: 2480 bytes --]
> context --make does a --generate so you can skip that and the luatools
> run is not needed either (selfupdate is seldom needed too)
So 'context --make' is all one needs? I'll update the wiki.
> we get a .5pt height difference somewhere (nothing else different in a
> trace) ... smells like a luatex bug (testing that now)
That is my suspicion as well. But I couldn't reproduce it with plain
luatex. Though maybe I needed to use equation numbers, however that is
done in plain tex (the context run doesn't show the problem without the
\placeformula).
I suspect some internal state is getting slightly corrupted and getting
progressively worse with the 30,000 repetitions.
I've been finding many unexplained 0.5pt or so differences, which may be
related to the underbrace problem. The context, so to speak, is that I
am putting in the last edits to the book, after having gone through and
checked every page many times. So I want to ensure that none of the
small changes has a big effect, but I don't want to check every page
again (or I'll vomit).
Thus, I run my compare-pdfs.sh script on the previous and the latest
pdf. It renders each page to a B/W png (using mupdf) and then compares
corresponding pages using 'compare' (one of the ImageMagick tools). Any
differing pixels are in red, and the shared pixels are in very light
gray. Then all the pages with enough differences are viewed with 'feh',
in order of most-to-least differences. And I check that nothing major
has changed.
What I keep finding is that pages long after where I make a small change
somehow get changed, with roughly one-pixel shifts to parts of the page.
I'm attaching an example pixel diff. It shows changes on p.72 of the
mss. However, the only change to the source code was on p.58 (and was
designed not to change any page breaks, which it didn't). That same
page (p.72) often gets similar changes with all kinds of small changes.
[Note to me: In case I need to recreate the source for debugging
purposes, it's the change from rev 41ff54 to 2467d5]
In case the script is useful to others (earlier versions are in the
contexttest repository somewhere), I am attaching it. I use it as
follows with a bash command line. (The awk in the pipeline makes sure
that only pages with some difference make it to the viewer.)
DPI=72 compare-pdfs.sh old.pdf new.pdf | tee book-print.compare72dpi | awk '{if (($3 > 0.02)) print;}' | sort -nr -k3 | awk '{print $1}' | xargs -r feh
-Sanjoy
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: compare script --]
[-- Type: text/x-sh, Size: 1210 bytes --]
#! /bin/bash
# Usage: $0 file1.pdf file2.pdf
#
# compares file1.pdf and file2.pdf by rendering each page and using
# the 'compare' ImageMagick utility
#
# Copyright 2007-2014 Sanjoy Mahajan. Licensed under the GNU GPL version 3
# or (at your option) any later version.
#
# HISTORY
# 2014-06-22: Use mudraw instead of pdftoppm. GPL v3+
# 2009-09-30: Fix capture of dB output; don't use a viewer; use pdftoppm
# 2007-01-15: First version
#
if [ -z "$DPI" ]; then
DPI=72
fi
ext=png
if [ -z "$1" -o -z "$2" ]; then
echo "Usage: $0 file1.pdf file2.pdf"
exit 3
fi
# generate the many page images in a temporary directory
d=`mktemp -d`
mkdir -p $d/a $d/b
mudraw -r $DPI -g -o $d/a/%03d.$ext $1
mudraw -r $DPI -g -o $d/b/%03d.$ext $2
wait
# find the union of the page numbers (in case one pdf has more pages)
pages=`ls $d/{a,b}/*.$ext | sed "s%.*/\([0-9][0-9]*\).$ext%\1%" | sort -un`
# compare each page
for p in $pages ; do
if ! [ -e "$d/a/$p.$ext" ] ; then
echo "$p: missing from $1"
continue
fi
if ! [ -e "$d/b/$p.$ext" ] ; then
echo "$p: missing from $2"
continue
fi
echo -n "$d/diff-$p.$ext $p "
compare -metric mae $d/{a,b}/$p.$ext $d/diff-$p.$ext 2>&1
done
[-- Attachment #3: pixel diff --]
[-- Type: image/png, Size: 35769 bytes --]
[-- Attachment #4: Type: text/plain, Size: 485 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-03 22:37 ` Sanjoy Mahajan
@ 2014-07-03 22:47 ` luigi scarso
2014-07-07 13:59 ` Sanjoy Mahajan
2014-07-03 22:51 ` Hans Hagen
2014-07-07 13:56 ` Sanjoy Mahajan
2 siblings, 1 reply; 20+ messages in thread
From: luigi scarso @ 2014-07-03 22:47 UTC (permalink / raw)
To: mailing list for ConTeXt users
[-- Attachment #1.1: Type: text/plain, Size: 2968 bytes --]
On Fri, Jul 4, 2014 at 12:37 AM, Sanjoy Mahajan <sanjoy@mit.edu> wrote:
> > context --make does a --generate so you can skip that and the luatools
> > run is not needed either (selfupdate is seldom needed too)
>
> So 'context --make' is all one needs? I'll update the wiki.
>
> > we get a .5pt height difference somewhere (nothing else different in a
> > trace) ... smells like a luatex bug (testing that now)
>
> That is my suspicion as well. But I couldn't reproduce it with plain
> luatex. Though maybe I needed to use equation numbers, however that is
> done in plain tex (the context run doesn't show the problem without the
> \placeformula).
>
> I suspect some internal state is getting slightly corrupted and getting
> progressively worse with the 30,000 repetitions.
>
> I've been finding many unexplained 0.5pt or so differences, which may be
> related to the underbrace problem. The context, so to speak, is that I
> am putting in the last edits to the book, after having gone through and
> checked every page many times. So I want to ensure that none of the
> small changes has a big effect, but I don't want to check every page
> again (or I'll vomit).
>
> Thus, I run my compare-pdfs.sh script on the previous and the latest
> pdf. It renders each page to a B/W png (using mupdf) and then compares
> corresponding pages using 'compare' (one of the ImageMagick tools). Any
> differing pixels are in red, and the shared pixels are in very light
> gray. Then all the pages with enough differences are viewed with 'feh',
> in order of most-to-least differences. And I check that nothing major
> has changed.
>
> What I keep finding is that pages long after where I make a small change
> somehow get changed, with roughly one-pixel shifts to parts of the page.
> I'm attaching an example pixel diff. It shows changes on p.72 of the
> mss. However, the only change to the source code was on p.58 (and was
> designed not to change any page breaks, which it didn't). That same
> page (p.72) often gets similar changes with all kinds of small changes.
>
> [Note to me: In case I need to recreate the source for debugging
> purposes, it's the change from rev 41ff54 to 2467d5]
>
> In case the script is useful to others (earlier versions are in the
> contexttest repository somewhere), I am attaching it. I use it as
> follows with a bash command line. (The awk in the pipeline makes sure
> that only pages with some difference make it to the viewer.)
>
> DPI=72 compare-pdfs.sh old.pdf new.pdf | tee book-print.compare72dpi | awk
> '{if (($3 > 0.02)) print;}' | sort -nr -k3 | awk '{print $1}' | xargs -r feh
>
> Thank you for the script.
Under linux you can also try diffpdf
$> aptitude search diffpdf
i diffpdf
- compare two PDF
files textually or visually
p diffpdf:i386
- compare two PDF
files textually or visually
--
luigi
[-- Attachment #1.2: Type: text/html, Size: 4161 bytes --]
[-- Attachment #2: Type: text/plain, Size: 485 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-03 22:37 ` Sanjoy Mahajan
2014-07-03 22:47 ` luigi scarso
@ 2014-07-03 22:51 ` Hans Hagen
2014-07-04 4:58 ` Wolfgang Schuster
2014-07-07 13:56 ` Sanjoy Mahajan
2 siblings, 1 reply; 20+ messages in thread
From: Hans Hagen @ 2014-07-03 22:51 UTC (permalink / raw)
To: Sanjoy Mahajan, Wolfgang Schuster, mailing list for ConTeXt users
On 7/4/2014 12:37 AM, Sanjoy Mahajan wrote:
>> context --make does a --generate so you can skip that and the luatools
>> run is not needed either (selfupdate is seldom needed too)
>
> So 'context --make' is all one needs? I'll update the wiki.
>
>> we get a .5pt height difference somewhere (nothing else different in a
>> trace) ... smells like a luatex bug (testing that now)
>
> That is my suspicion as well. But I couldn't reproduce it with plain
> luatex. Though maybe I needed to use equation numbers, however that is
> done in plain tex (the context run doesn't show the problem without the
> \placeformula).
>
> I suspect some internal state is getting slightly corrupted and getting
> progressively worse with the 30,000 repetitions.
here it happens after a pagebreak (a few lines overflow are ok)
> I've been finding many unexplained 0.5pt or so differences, which may be
> related to the underbrace problem. The context, so to speak, is that I
> am putting in the last edits to the book, after having gone through and
> checked every page many times. So I want to ensure that none of the
> small changes has a big effect, but I don't want to check every page
> again (or I'll vomit).
>
> Thus, I run my compare-pdfs.sh script on the previous and the latest
> pdf. It renders each page to a B/W png (using mupdf) and then compares
> corresponding pages using 'compare' (one of the ImageMagick tools). Any
> differing pixels are in red, and the shared pixels are in very light
> gray. Then all the pages with enough differences are viewed with 'feh',
> in order of most-to-least differences. And I check that nothing major
> has changed.
>
> What I keep finding is that pages long after where I make a small change
> somehow get changed, with roughly one-pixel shifts to parts of the page.
> I'm attaching an example pixel diff. It shows changes on p.72 of the
> mss. However, the only change to the source code was on p.58 (and was
> designed not to change any page breaks, which it didn't). That same
> page (p.72) often gets similar changes with all kinds of small changes.
>
> [Note to me: In case I need to recreate the source for debugging
> purposes, it's the change from rev 41ff54 to 2467d5]
>
> In case the script is useful to others (earlier versions are in the
> contexttest repository somewhere), I am attaching it. I use it as
> follows with a bash command line. (The awk in the pipeline makes sure
> that only pages with some difference make it to the viewer.)
>
> DPI=72 compare-pdfs.sh old.pdf new.pdf | tee book-print.compare72dpi | awk '{if (($3 > 0.02)) print;}' | sort -nr -k3 | awk '{print $1}' | xargs -r feh
>
> -Sanjoy
>
--
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-03 22:51 ` Hans Hagen
@ 2014-07-04 4:58 ` Wolfgang Schuster
2014-07-04 8:22 ` Hans Hagen
2014-07-04 9:34 ` Sanjoy Mahajan
0 siblings, 2 replies; 20+ messages in thread
From: Wolfgang Schuster @ 2014-07-04 4:58 UTC (permalink / raw)
To: Hans Hagen; +Cc: mailing list for ConTeXt users, Sanjoy Mahajan
[-- Attachment #1.1: Type: text/plain, Size: 1438 bytes --]
Am 04.07.2014 um 00:51 schrieb Hans Hagen <pragma@wxs.nl>:
> On 7/4/2014 12:37 AM, Sanjoy Mahajan wrote:
>>> context --make does a --generate so you can skip that and the luatools
>>> run is not needed either (selfupdate is seldom needed too)
>>
>> So 'context --make' is all one needs? I'll update the wiki.
>>
>>> we get a .5pt height difference somewhere (nothing else different in a
>>> trace) ... smells like a luatex bug (testing that now)
>>
>> That is my suspicion as well. But I couldn't reproduce it with plain
>> luatex. Though maybe I needed to use equation numbers, however that is
>> done in plain tex (the context run doesn't show the problem without the
>> \placeformula).
>>
>> I suspect some internal state is getting slightly corrupted and getting
>> progressively worse with the 30,000 repetitions.
>
> here it happens after a pagebreak (a few lines overflow are ok)
It does also depend on the width of the text because when I use 48pt in the example below
the height of the brace is always the same but with 49pt the height increases after the first page.
\starttext
\dorecurse{100}
{\startformula
\mathord
{%\setbox\scratchbox\mathstylehbox{\Umathaccent\fam\zerocount"FE3DF{\hskip48pt}}%
\setbox\scratchbox\mathstylehbox{\Umathaccent\fam\zerocount"FE3DF{\hskip49pt}}%
\the\userpageno:\the\ht\scratchbox}%
\stopformula}
\stoptext
Wolfgang
[-- Attachment #1.2: Type: text/html, Size: 2454 bytes --]
[-- Attachment #2: Type: text/plain, Size: 485 bytes --]
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-04 4:58 ` Wolfgang Schuster
@ 2014-07-04 8:22 ` Hans Hagen
2014-07-04 9:34 ` Sanjoy Mahajan
1 sibling, 0 replies; 20+ messages in thread
From: Hans Hagen @ 2014-07-04 8:22 UTC (permalink / raw)
To: Wolfgang Schuster; +Cc: mailing list for ConTeXt users, Sanjoy Mahajan
On 7/4/2014 6:58 AM, Wolfgang Schuster wrote:
>
> Am 04.07.2014 um 00:51 schrieb Hans Hagen <pragma@wxs.nl
> <mailto:pragma@wxs.nl>>:
>
>> On 7/4/2014 12:37 AM, Sanjoy Mahajan wrote:
>>>> context --make does a --generate so you can skip that and the luatools
>>>> run is not needed either (selfupdate is seldom needed too)
>>>
>>> So 'context --make' is all one needs? I'll update the wiki.
>>>
>>>> we get a ..5pt height difference somewhere (nothing else different in a
>>>> trace) .... smells like a luatex bug (testing that now)
>>>
>>> That is my suspicion as well. But I couldn't reproduce it with plain
>>> luatex. Though maybe I needed to use equation numbers, however that is
>>> done in plain tex (the context run doesn't show the problem without the
>>> \placeformula).
>>>
>>> I suspect some internal state is getting slightly corrupted and getting
>>> progressively worse with the 30,000 repetitions.
>>
>> here it happens after a pagebreak (a few lines overflow are ok)
>
> It does also depend on the width of the text because when I use 48pt in
> the example below
> the height of the brace is always the same but with 49pt the height
> increases after the first page.
>
> \starttext
>
> \dorecurse{100}
> {\startformula
> \mathord
>
> {%\setbox\scratchbox\mathstylehbox{\Umathaccent\fam\zerocount"FE3DF{\hskip48pt}}%
>
> \setbox\scratchbox\mathstylehbox{\Umathaccent\fam\zerocount"FE3DF{\hskip49pt}}%
> \the\userpageno:\the\ht\scratchbox}%
> \stopformula}
>
> \stoptext
\starttext
\dorecurse{1000}
{\setbox\scratchbox\hbox{$\Umathaccent\fam\zerocount"FE3DF{\hskip49pt}$}%
\the\ht\scratchbox\par}
\stoptext
that was the one i used when seeing some trigger from the page break
(the two at the top of page 2 are the overshoot before tex triggers the
page)
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-04 4:58 ` Wolfgang Schuster
2014-07-04 8:22 ` Hans Hagen
@ 2014-07-04 9:34 ` Sanjoy Mahajan
2014-07-05 0:30 ` Hans Hagen
1 sibling, 1 reply; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-07-04 9:34 UTC (permalink / raw)
To: Wolfgang Schuster, Hans Hagen; +Cc: mailing list for ConTeXt users
Wolfgang Schuster <schuster.wolfgang@gmail.com> writes:
> It does also depend on the width of the text because when I use 48pt
> in the example below the height of the brace is always the same but
> with 49pt the height increases after the first page.
An interesting bug. That's also consistent with my experiments, in which
\underbrace{\hbox{yyyy yyyy}}_{\rho}
triggers the problem, but
\underbrace{\hbox{yyyy}}_{\rho}
does not.
-Sanjoy
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-04 9:34 ` Sanjoy Mahajan
@ 2014-07-05 0:30 ` Hans Hagen
2014-07-07 2:16 ` Sanjoy Mahajan
0 siblings, 1 reply; 20+ messages in thread
From: Hans Hagen @ 2014-07-05 0:30 UTC (permalink / raw)
To: Sanjoy Mahajan, mailing list for ConTeXt users
On 7/4/2014 11:34 AM, Sanjoy Mahajan wrote:
> Wolfgang Schuster <schuster.wolfgang@gmail.com> writes:
>
>> It does also depend on the width of the text because when I use 48pt
>> in the example below the height of the brace is always the same but
>> with 49pt the height increases after the first page.
>
> An interesting bug. That's also consistent with my experiments, in which
>
> \underbrace{\hbox{yyyy yyyy}}_{\rho}
>
> triggers the problem, but
>
> \underbrace{\hbox{yyyy}}_{\rho}
>
> does not.
Hi Sanjoy,
After a day of testing, tracing, wondering etc Luigi and I figured out
what was the problem it tooks us lots of debug prints -). Indeed there
is a bug in luatex. An extensible is made up of a sequence
char
skip 0pt- xpt
char
skip 0pt- xpt
char
skip 0pt- xpt
char
And the code that determines the height/depth took the dimensions of
skips into the equation while in fact glue has no such dimensions
(leaders that are a special kind of glue have dimensions). Not the first
place one looks for such an issue. Probably a side effect of copying a
function and patching part.
So, some random value (not entirely random as it depends on what mem
area the glue node is allocated from and that is kind of stable in a
run) was added which results in the effects you noticed.
Because it is stable within a run (same successive errors) it doesn't
result in oscillating (too many runs due to differences in results) and
this is probably why it went unnoticed so long. Interesting. (Normally
the pseudo random ht/dp were small.)
So, once this fix moves upstream you can check your suspicious cases.
Luigi will push it in our experimental branch and I suppose that Taco
will then push it into trunk and after that you can compile or wait till
the garden carries it.
Thanks for noticing!
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-05 0:30 ` Hans Hagen
@ 2014-07-07 2:16 ` Sanjoy Mahajan
2014-07-07 3:48 ` Sanjoy Mahajan
0 siblings, 1 reply; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-07-07 2:16 UTC (permalink / raw)
To: Hans Hagen, mailing list for ConTeXt users
Hans Hagen <pragma@wxs.nl> writes:
> After a day of testing, tracing, wondering etc Luigi and I figured out
> what was the problem it tooks us lots of debug prints -). Indeed there
> is a bug in luatex.
Great detective work!
> So, some random value (not entirely random as it depends on what mem
> area the glue node is allocated from and that is kind of stable in a
> run) was added which results in the effects you noticed.
Ah, so running it 30000 times would allocate the glue node from higher
and higher mem, and thereby making the not-so-random value larger and
larger.
> So, once this fix moves upstream you can check your suspicious cases.
> Luigi will push it in our experimental branch and I suppose that Taco
> will then push it into trunk and after that you can compile
Once it in the luatex trunk, I will check it right away. I am also
happy to test the experimental branch if you give me a pointer to it.
> or wait till the garden carries it.
Do you mean in the minimals?
> Thanks for noticing!
For sure. I am very happy that it is fixed, and fixed before I have to
give in my absolute final PDF file.
My only worry is that the carefully tuned page breaks, some of which are
sensitive to 1pt insertions of vertical space, might change once the
underbraces return to their homes. But I'll tweak it with a bit of
vertical space.
-Sanjoy
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-07 2:16 ` Sanjoy Mahajan
@ 2014-07-07 3:48 ` Sanjoy Mahajan
0 siblings, 0 replies; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-07-07 3:48 UTC (permalink / raw)
To: Hans Hagen, mailing list for ConTeXt users
> Once it in the luatex trunk, I will check it right away.
I just compiled and installed luatex 0.79.1 r5033, remade the formats,
and the result passes my underbrace tests (the 30,000 iterations and the
examples in my book mss).
Thank you for the great debugging.
-Sanjoy
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-03 19:21 ` Wolfgang Schuster
2014-07-03 21:03 ` Sanjoy Mahajan
@ 2014-07-07 13:53 ` Sanjoy Mahajan
1 sibling, 0 replies; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-07-07 13:53 UTC (permalink / raw)
To: Wolfgang Schuster, mailing list for ConTeXt users; +Cc: Hans Hagen
Wolfgang Schuster <schuster.wolfgang@gmail.com> writes:
> math-stc.mkvi:
>
> %
> - \ifx\currentmathtext\empty
> + \ifx\m_math_stackers_text_middle\empty
> \setbox\scratchboxthree\emptyhbox
Is this change still also needed (in addition to the luatex fix)?
-Sanjoy
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-03 22:37 ` Sanjoy Mahajan
2014-07-03 22:47 ` luigi scarso
2014-07-03 22:51 ` Hans Hagen
@ 2014-07-07 13:56 ` Sanjoy Mahajan
2 siblings, 0 replies; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-07-07 13:56 UTC (permalink / raw)
To: Hans Hagen, Wolfgang Schuster, mailing list for ConTeXt users
Sanjoy Mahajan <sanjoy@mit.edu> writes:
> I've been finding many unexplained 0.5pt or so differences, which may
> be related to the underbrace problem. ... What I keep finding is that
> pages long after where I make a small change somehow get changed, with
> roughly one-pixel shifts to parts of the page.
These small shifts were also caused by the underbrace luatex bug (these
pages all had underbraces on them)---many birds with one stone.
-Sanjoy
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-03 22:47 ` luigi scarso
@ 2014-07-07 13:59 ` Sanjoy Mahajan
0 siblings, 0 replies; 20+ messages in thread
From: Sanjoy Mahajan @ 2014-07-07 13:59 UTC (permalink / raw)
To: luigi scarso, mailing list for ConTeXt users
luigi scarso <luigi.scarso@gmail.com> writes:
> Under linux you can also try diffpdf
Thanks -- I'd forgotten about that and just tried it. If you are
looking for pixel-level changes, my script makes it easier to see
exactly what the changes are. On the other hand, diffpdf is more
polished than my script.
-Sanjoy
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: underbraces get progressively more misaligned (w/ minimal example)
2014-07-04 7:12 Akira Kakuto
@ 2014-07-04 8:17 ` Hans Hagen
0 siblings, 0 replies; 20+ messages in thread
From: Hans Hagen @ 2014-07-04 8:17 UTC (permalink / raw)
To: mailing list for ConTeXt users
On 7/4/2014 9:12 AM, Akira Kakuto wrote:
>> It does also depend on the width of the text because when I use 48pt
>> in the example below
>> the height of the brace is always the same but with 49pt the height
>> increases after the first page.
>
> An experiment:
>
> width = 48.0719985pt
> height = 4.968pt (constant)
>
> width = 48.0719986pt
> height = unpredictable various values
So now we need to figure out where it comes from; maybe some math
parameter get messed up ... it has to do with the mechanism that builds
extensibles from snippets (begin - connector - middle - connector - end
glyphs). Depending on the width this goes in steps: first fixed prebuild
glyphs, later extensibles.
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | voip: 087 875 68 74 | www.pragma-ade.com
| www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
* underbraces get progressively more misaligned (w/ minimal example)
@ 2014-07-04 7:12 Akira Kakuto
2014-07-04 8:17 ` Hans Hagen
0 siblings, 1 reply; 20+ messages in thread
From: Akira Kakuto @ 2014-07-04 7:12 UTC (permalink / raw)
To: ntg-context
> It does also depend on the width of the text because when I use 48pt in the example below
> the height of the brace is always the same but with 49pt the height increases after the first page.
An experiment:
width = 48.0719985pt
height = 4.968pt (constant)
width = 48.0719986pt
height = unpredictable various values
Best regards,
Akira
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://tex.aanhet.net
archive : http://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___________________________________________________________________________________
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2014-07-07 13:59 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-06-29 10:13 underbraces get progressively more misaligned (w/ minimal example) Sanjoy Mahajan
2014-06-29 10:24 ` Sanjoy Mahajan
2014-07-03 10:49 ` Sanjoy Mahajan
2014-07-03 19:21 ` Wolfgang Schuster
2014-07-03 21:03 ` Sanjoy Mahajan
2014-07-03 22:09 ` Hans Hagen
2014-07-03 22:37 ` Sanjoy Mahajan
2014-07-03 22:47 ` luigi scarso
2014-07-07 13:59 ` Sanjoy Mahajan
2014-07-03 22:51 ` Hans Hagen
2014-07-04 4:58 ` Wolfgang Schuster
2014-07-04 8:22 ` Hans Hagen
2014-07-04 9:34 ` Sanjoy Mahajan
2014-07-05 0:30 ` Hans Hagen
2014-07-07 2:16 ` Sanjoy Mahajan
2014-07-07 3:48 ` Sanjoy Mahajan
2014-07-07 13:56 ` Sanjoy Mahajan
2014-07-07 13:53 ` Sanjoy Mahajan
2014-07-04 7:12 Akira Kakuto
2014-07-04 8:17 ` Hans Hagen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).