staticMPfigures can overrun output streams

staticMPfigures can overrun output streams

[Sanjoy Mahajan]

The following minimal file overruns the available output streams
(version 2006.08.08 and context live):

\starttext
\dorecurse{16}{\startstaticMPfigure{\recurselevel}
  fill fullcircle scaled 1cm withcolor red;
\stopstaticMPfigure}
\stoptext

At the end are the last few lines of the log.  Here are the last few
lines of the last few lines:

! Bad number (16).
\openMPgraphicfile ...immediate \openout \MPwrite 
                                                  \MPgraphicfile .mp\relax \...


After experiments and digging in the TeXbook I'm pretty sure the cause
is as follows:

  The macro \openMPgraphicfile (supp-mps.tex) constructs the name of a new
  output stream and allocates a new write to it:

       \donetrue \@EA\@EA\csname newwrite\endcsname\csname\@@MPG\@@MPG\MPgraphicfile\endcsname

  Then \MPwrite is set to this stream number:

       \@EA\let\@EA\MPwrite\csname\@@MPG\@@MPG\MPgraphicfile\endcsname

  And then it's opened:

       \immediate\openout\MPwrite\MPgraphicfile.mp\relax

  which is where TeX complains about the bad number (16).  

The number gets that high because each static figure goes into a
separate file, and each file requires a \newwrite.  I thought it would
not run out of streams because the allocation is all enclosed in
\bgroup..\egroup (in \startstaticMPgraphic) and the \egroup would pop
the write-stream allocation stack.  But no such luck.  The workhorse
\alloc (p.347 of the TeXBook) uses a \global assignment to the control
sequence.

Here's a possible solution that works here but is a hack and may make
unwarranted assumptions about control-sequence names.  It require a new
if to say whether we are processing a static MP figure, and then using a
dedicated output stream for those writes.

======================== cut here ========================
\unprotect

\newif\ifinstaticMPfigure
\newwrite\MPstaticwrite

\long\def\startstaticMPgraphic#1#2\stopstaticMPgraphic
  {\bgroup
   \instaticMPfiguretrue  % value is restored by the \egroup
   \setMPrandomseedfalse
   \def\MPgraphicfile{#1}% no \jobname here
   \let\allocateMPslot  \gobbleoneargument
   \let\deallocateMPslot\gobbleoneargument
   \let\runMPgraphic    \gobbleoneargument
   \runMPgraphicstrue
   \enableincludeMPgraphics
   \currentMPgraphic\plusone  % hack, else no close
   \startMPgraphic#2\stopMPgraphic
   \executeMPgraphicfile
   \egroup}

\def\openMPgraphicfile#1#2% #1=alwaysopen #2=message
  {%\doifundefinedelse{\@@MPG\@@MPG\MPgraphicfile}
   %  {\donetrue
   %   \@EA\newwrite\csname\@@MPG\@@MPG\MPgraphicfile\endcsname}%
   %  {\ifcase#1\donefalse\else\donetrue\fi}%
   \@EA\ifx\csname\@@MPG\@@MPG\MPgraphicfile\endcsname\relax
    %\donetrue \@EA\newwrite\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
    % for the sake of plain usage
     \donetrue
     \ifinstaticMPfigure
       \let\MPwrite\MPstaticwrite
     \else
       \@EA\@EA\csname newwrite\endcsname\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
     \fi
     \doglobal\addtocommalist\MPgraphicfile\MPgraphicfiles
   \else
     \ifcase#1\relax\donefalse\else\donetrue\fi
   \fi
   \ifinstaticMPfigure
   \else
     \@EA\let\@EA\MPwrite\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
   \fi
   \ifdone
     \immediate\openout\MPwrite\MPgraphicfile.mp\relax
     \ifx\MPinputtranslation\empty\else
       \immediate\write\MPwrite{\MPinputtranslation}%
       \immediate\write\MPwrite{verbatimtex \MPinputtranslation etex ;}%
     \fi
     \immediate\write\MPwrite{\letterpercent\space #2 graphics of job "\jobname"}%
     \writeMPgraph
     \setMPrandomseed
   \fi}

\protect

\starttext
\dorecurse{16}{\startstaticMPfigure{\recurselevel}
  fill fullcircle scaled 1cm withcolor red;
\stopstaticMPfigure}
\stoptext
======================== cut here ========================

As a diff:

======================== cut here ========================
--- a/supp-mps.tex	2006-07-22 11:26:47.000000000 -0400
+++ b/supp-mps.tex	2006-09-24 15:32:19.000000000 -0400
@@ -315,7 +315,9 @@
 %D setting the next boolean to true.
 
 \newwrite\MPwrite
+\newwrite\MPstaticwrite
 
+\newif\ifinstaticMPfigure
 \newif\iflongMPlines \longMPlinestrue % we now have \obeyMPlines
 
 \ifx \overlaywidth     \undefined \def \overlaywidth     {4cm} \fi
@@ -720,12 +722,20 @@
    \@EA\ifx\csname\@@MPG\@@MPG\MPgraphicfile\endcsname\relax
     %\donetrue \@EA\newwrite\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
     % for the sake of plain usage
-     \donetrue \@EA\@EA\csname newwrite\endcsname\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
+     \donetrue
+     \ifinstaticMPfigure
+       \let\MPwrite\MPstaticwrite
+     \else
+       \@EA\@EA\csname newwrite\endcsname\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
+     \fi
      \doglobal\addtocommalist\MPgraphicfile\MPgraphicfiles
    \else
      \ifcase#1\relax\donefalse\else\donetrue\fi
    \fi
-   \@EA\let\@EA\MPwrite\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
+   \ifinstaticMPfigure
+   \else
+     \@EA\let\@EA\MPwrite\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
+   \fi
    \ifdone
      \immediate\openout\MPwrite\MPgraphicfile.mp\relax
      \ifx\MPinputtranslation\empty\else
@@ -1983,6 +1993,7 @@
 
 \long\def\startstaticMPgraphic#1#2\stopstaticMPgraphic
   {\bgroup
+   \instaticMPfiguretrue  % value is restored by the \egroup
    \setMPrandomseedfalse
    \def\MPgraphicfile{#1}% no \jobname here
    \let\allocateMPslot  \gobbleoneargument
======================== cut here ========================

Which has left a minor puzzle.  Without the patch, where did the warning
from plain TeX go?  When you overrun the number of writes, it should
say:

! No room for (\write) \H.

e.g. try this file:

\starttext
\dorecurse{17}{\newwrite\H}
\stoptext

But with the staticMPfigure test file, the "No room for (\write)"
message didn't appear, and instead the TeX complained only when \openout
was tried on the resulting stream.

Meanwhile here's more of the original log:

======================== cut here ========================
\openout14 = `toomany-9.mp'.

\openout5 = `toomany-mpgraph.mpb'.

system(texmfstart --ifchanged=toomany-9.mp texexec --mpgraphic toomany-9.mp)...
executed.

color           : currentcolor (def) is not defined 
\openout15 = `toomany-10.mp'.

\openout5 = `toomany-mpgraph.mpb'.

system(texmfstart --ifchanged=toomany-10.mp texexec --mpgraphic toomany-10.mp).
..executed.

color           : currentcolor (def) is not defined 
! Bad number (16).
\openMPgraphicfile ...immediate \openout \MPwrite 
                                                  \MPgraphicfile .mp\relax \...

\startwritingMPgraphic ...MPgraphicfile 1{runtime}
                                                  \theMPextensions \theMPinc...

\startMPgraphic ...aphic ->\startwritingMPgraphic 
                                                  \writecheckedMPgraphic {#1...

\startstaticMPgraphic ...graphic #2\stopMPgraphic 
                                                  \executeMPgraphicfile \egr...

\dostartstaticMPfigure ...}#2\stopstaticMPgraphic 
                                                  \endgroup 
\redorecurse ->\expandrecursecontent 
                                     \dodorecurse 
...
l.20 \stopstaticMPfigure}
                         
? 
======================== cut here ========================

-Sanjoy

`A society of sheep must in time beget a government of wolves.'
   -- Bertrand de Jouvenal

Re: staticMPfigures can overrun output streams

[Hans Hagen]

Sanjoy Mahajan wrote:
> Here's a possible solution that works here but is a hack and may make
> unwarranted assumptions about control-sequence names.  It require a new
> if to say whether we are processing a static MP figure, and then using a
> dedicated output stream for those writes.
>
> ======================== cut here ========================
> \unprotect
>
> \newif\ifinstaticMPfigure
>   
interesting is that we already had a boolean but somehow it was not used 
(only at one place)

\newwrite\MPstaticwrite

\long\def\startstaticMPgraphic#1#2\stopstaticMPgraphic
  {\bgroup
   \MPstaticgraphictrue\MPstaticgraphictrue
   \setMPrandomseedfalse
   \def\MPgraphicfile{#1}% no \jobname here
   \let\allocateMPslot  \gobbleoneargument
   \let\deallocateMPslot\gobbleoneargument
   \let\runMPgraphic    \gobbleoneargument
   \runMPgraphicstrue
   \enableincludeMPgraphics
   \currentMPgraphic\plusone  % hack, else no close
   \startMPgraphic#2\stopMPgraphic
   \executeMPgraphicfile
   \egroup}

\def\openMPgraphicfile#1#2% #1=alwaysopen #2=message
  {\ifMPstaticgraphic
     \donetrue
     \let\MPwrite\MPstaticwrite
   \else
     \@EA\ifx\csname\@@MPG\@@MPG\MPgraphicfile\endcsname\relax
       \donetrue
       % \@EA\newwrite\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
       % for the sake of plain usage
       \@EA\@EA\csname 
newwrite\endcsname\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
       \doglobal\addtocommalist\MPgraphicfile\MPgraphicfiles
     \else
       \ifcase#1\relax\donefalse\else\donetrue\fi
     \fi
     \@EA\let\@EA\MPwrite\csname\@@MPG\@@MPG\MPgraphicfile\endcsname
   \fi
   \ifdone
     \immediate\openout\MPwrite\MPgraphicfile.mp\relax
     \ifx\MPinputtranslation\empty\else
       \immediate\write\MPwrite{\MPinputtranslation}%
       \immediate\write\MPwrite{verbatimtex \MPinputtranslation etex ;}%
     \fi
     \immediate\write\MPwrite{\letterpercent\space #2 graphics of job 
"\jobname"}%
     \writeMPgraph
     \setMPrandomseed
   \fi}

i'll add this


> ! No room for (\write) \H.
>
> e.g. try this file:
>
> \starttext
> \dorecurse{17}{\newwrite\H}
> \stoptext
>
> But with the staticMPfigure test file, the "No room for (\write)"
> message didn't appear, and instead the TeX complained only when \openout
> was tried on the resulting stream.
>   
it is because we open files without closing them; somehow i think that this is a bug (taco may know)

Hans 


-----------------------------------------------------------------
                                          Hans Hagen | PRAGMA ADE
              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                             | www.pragma-pod.nl
-----------------------------------------------------------------

Re: staticMPfigures can overrun output streams

[Sanjoy Mahajan]

>   \MPstaticgraphictrue\MPstaticgraphictrue

Just one setting needed?

> > But with the staticMPfigure test file, the "No room for (\write)"
> > message didn't appear, and instead the TeX complained only when
> > \openout was tried on the resulting stream.
> it is because we open files without closing them; somehow i think
> that this is a bug (taco may know)

It may be two separate issues: (1) When the \newwrite is executed,
plain TeX should complain before anyone even tries to open the stream;
and (2) not closing files.  I checked quickly for the second one, but
couldn't be sure that each open matched a close.

It'll be nice when the awful string, filename, and file manipulations
can be done in lua...

-Sanjoy

`Never underestimate the evil of which men of power are capable.'
         --Bertrand Russell, _War Crimes in Vietnam_, chapter 1.

Re: staticMPfigures can overrun output streams

[Taco Hoekwater]

Hi,

> it is because we open files without closing them; somehow i 
> think that this is a bug (taco may know)

No, that is not a bug. The error is equivalent to

    \setbox234567=\hbox{}

It is the macro program's duty to keep the numbers in check.

Taco

Re: staticMPfigures can overrun output streams

[Sanjoy Mahajan]

In another thread, Hans mentioned the 2006.09.25 beta.  I tested it
against this test file and it works great:

\starttext
\dorecurse{20}{\startreusableMPgraphic{\recurselevel}
  draw fullcircle scaled 3mm;
\stopreusableMPgraphic
\placefigure[here,none]{}{\reuseMPgraphic{\recurselevel}}}
\stoptext

I just added this test to the contexttest svn repo.

-Sanjoy

`Never underestimate the evil of which men of power are capable.'
         --Bertrand Russell, _War Crimes in Vietnam_, chapter 1.