Buffers (gone) in MK IV (was: MK IV & Lilypond)

Buffers (gone) in MK IV (was: MK IV & Lilypond)

[Mojca Miklavec]

On 8/21/07, David Wooten wrote:
> Greetings all,
>
> Has anyone had any luck getting the lilypond module to work with mkiv/
> luatex engine? It fails for me (works fine with mkii/pdftex). I can
> send along some log info if others have found it to work.

Hans has mentioned several times that buffers are not going to be
written to files with luaTeX any more. This has a weird consequence on
many modules which relied exactly on that trick.

Short example of what goes wrong:

\starttext

\startbuffer[a]
Hello world!
\stopbuffer
\typebuffer[a]

%\executesystemcommand{dosomethingwith \jobname-a.tmp}
\stoptext

The problem is that MK II created a file \jobname-a.tmp and lilypond
module processed exactly that file further. In MK IV, that file is not
created any more (you can check that by running the above example with
--lua first and then with pdfTeX again. Only in the second case a file
[whateverthefilename]-a.tmp is created.)

Lilypond module relied on that "feature" (I would rather call it
dirty-but-very-useful-trick).

Hans, what's the current general recipe for this kind of [mis]use of buffers?

Mojca
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Buffers (gone) in MK IV (was: MK IV & Lilypond)

[Hans Hagen]

Mojca Miklavec wrote:
> On 8/21/07, David Wooten wrote:
>> Greetings all,
>>
>> Has anyone had any luck getting the lilypond module to work with mkiv/
>> luatex engine? It fails for me (works fine with mkii/pdftex). I can
>> send along some log info if others have found it to work.
> 
> Hans has mentioned several times that buffers are not going to be
> written to files with luaTeX any more. This has a weird consequence on
> many modules which relied exactly on that trick.
> 
> Short example of what goes wrong:
> 
> \starttext
> 
> \startbuffer[a]
> Hello world!
> \stopbuffer
> \typebuffer[a]
> 
> %\executesystemcommand{dosomethingwith \jobname-a.tmp}
> \stoptext
> 
> The problem is that MK II created a file \jobname-a.tmp and lilypond
> module processed exactly that file further. In MK IV, that file is not
> created any more (you can check that by running the above example with
> --lua first and then with pdfTeX again. Only in the second case a file
> [whateverthefilename]-a.tmp is created.)
> 
> Lilypond module relied on that "feature" (I would rather call it
> dirty-but-very-useful-trick).
> 
> Hans, what's the current general recipe for this kind of [mis]use of buffers?

I'll add savebuffer to core-buf.*

% engine=luatex

\startluacode
     if not buffers.save then
         function buffers.save(name)
             if not name or name == "" then
                 name = tex.jobname
             end
             local b, f = buffers.data[name], 
string.format("%s-%s.tmp",tex.jobname,name)
             b = (b and type(b) == "table" and table.join(b)) or b or ""
             io.savedata(f,b)
         end
     end
\stopluacode

\def\savebuffer{\dosingleempty\dosavebuffer}

\def\dosavebuffer[#1]{\ctxlua{buffers.save("#1")}}

\starttext

\startbuffer[oeps]
oeps
\stopbuffer

\startbuffer
oepsoeps
\stopbuffer

\savebuffer[oeps] \savebuffer

\stoptext



-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Buffers (gone) in MK IV (was: MK IV & Lilypond)

[Mojca Miklavec]

On 8/21/07, Hans Hagen wrote:
> Mojca Miklavec wrote:
> > On 8/21/07, David Wooten wrote:
> >> Greetings all,
> >>
> >> Has anyone had any luck getting the lilypond module to work with mkiv/
> >> luatex engine? It fails for me (works fine with mkii/pdftex). I can
> >> send along some log info if others have found it to work.
> >
> > Hans has mentioned several times that buffers are not going to be
> > written to files with luaTeX any more. This has a weird consequence on
> > many modules which relied exactly on that trick.
> >
> > Short example of what goes wrong:
> >
> > \starttext
> >
> > \startbuffer[a]
> > Hello world!
> > \stopbuffer
> > \typebuffer[a]
> >
> > %\executesystemcommand{dosomethingwith \jobname-a.tmp}
> > \stoptext
> >
> > The problem is that MK II created a file \jobname-a.tmp and lilypond
> > module processed exactly that file further. In MK IV, that file is not
> > created any more (you can check that by running the above example with
> > --lua first and then with pdfTeX again. Only in the second case a file
> > [whateverthefilename]-a.tmp is created.)
> >
> > Lilypond module relied on that "feature" (I would rather call it
> > dirty-but-very-useful-trick).
> >
> > Hans, what's the current general recipe for this kind of [mis]use of buffers?
>
> I'll add savebuffer to core-buf.*

Thanks a lot for that one!

> % engine=luatex
>
> \startluacode
>      if not buffers.save then
>          function buffers.save(name)
>              if not name or name == "" then
>                  name = tex.jobname
>              end
>              local b, f = buffers.data[name],
> string.format("%s-%s.tmp",tex.jobname,name)
>              b = (b and type(b) == "table" and table.join(b)) or b or ""
>              io.savedata(f,b)
>          end
>      end
> \stopluacode
>
> \def\savebuffer{\dosingleempty\dosavebuffer}
>
> \def\dosavebuffer[#1]{\ctxlua{buffers.save("#1")}}
>
> \starttext
>
> \startbuffer[oeps]
> oeps
> \stopbuffer
>
> \startbuffer
> oepsoeps
> \stopbuffer
>
> \savebuffer[oeps] \savebuffer
>
> \stoptext

I now tried to add the following three lines to t-lilypond.tex:

	\def\PDF{texmfstart --ifchanged=\lily!filename.eps pstopdf \lily!filename.eps}
	\beginLUATEX
	\savebuffer[lilypond-\the\lily!figures]%
	\endLUATEX
	\ifeof18
		\installprogram{\LP}%
etc.

But luaTeX now reports the following to console:

system (LUATEX) : [line 32] \savebuffer [lilypond-\the \lily!figures
]\endLUATEX \ifeof 18 \installprogram {\LP }\doif \jobsuffix
{pdf}{\installprogram {\PDF }}\else \executesystemcommand {\LP }\doif
\jobsuffix {pdf}{\executesystemcommand {\PDF }}\fi \doifelse
\jobsuffix {pdf} {\edef \lily!img {\lily!filename .pdf}}{\edef
\lily!img {\lily!filename .eps}}\ifvmode \getfiguredimensions
[\lily!filename .pdf]\leavevmode \newdimen \FigWidth \FigWidth
=\figurewidth \ifdim \FigWidth >\localhsize \!!dimena =\localhsize
\advance \!!dimena by-\FigWidth \noindent \hskip \!!dimena \fi \fi
\externalfigure [\lily!img ]\egroup

and then

systems         : end file li at line 34
 )
(\end occurred inside a group at level 1)

### simple group (level 1) entered at line 32 ({)
### bottom level
...
No pages of output.
Transcript written on li.log.


It seems like a problem with grouping, but I have no idea how to
prevent luaTeX from parsing the input further from what it's supposed
to read.

Mojca
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Buffers (gone) in MK IV

[Hans Hagen]

Mojca Miklavec wrote:

> 	\def\PDF{texmfstart --ifchanged=\lily!filename.eps pstopdf \lily!filename.eps}
> 	\beginLUATEX
> 	\savebuffer[lilypond-\the\lily!figures]%
> 	\endLUATEX

\beginLUATEX is not to be used in a macro

\beginLUATEX message to screen
	bla bla
\endLUATEX

then becomes

\beginLUATEX message to screen bla bla \endLUATEX

and all goes to the message

just do

\ifx\savebuffer\undefined \else \savebuffer[] \fi

or so

since when don't you read the source code any more -)

Hans
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Buffers (gone) in MK IV (was: MK IV & Lilypond)

[David Wooten]

Greetings,

Did we end up with a working solution for t-lilypond? I've tinkered a  
bit Hans' response, too, but without luck.

Dave

On Aug. 21, 2007, at Aug 21, 3:53 PM, Mojca Miklavec wrote:

> On 8/21/07, Hans Hagen wrote:
>> Mojca Miklavec wrote:
>>> On 8/21/07, David Wooten wrote:
>>>> Greetings all,
>>>>
>>>> Has anyone had any luck getting the lilypond module to work with  
>>>> mkiv/
>>>> luatex engine? It fails for me (works fine with mkii/pdftex). I can
>>>> send along some log info if others have found it to work.
>>>
>>> Hans has mentioned several times that buffers are not going to be
>>> written to files with luaTeX any more. This has a weird  
>>> consequence on
>>> many modules which relied exactly on that trick.
>>>
>>> Short example of what goes wrong:
>>>
>>> \starttext
>>>
>>> \startbuffer[a]
>>> Hello world!
>>> \stopbuffer
>>> \typebuffer[a]
>>>
>>> %\executesystemcommand{dosomethingwith \jobname-a.tmp}
>>> \stoptext
>>>
>>> The problem is that MK II created a file \jobname-a.tmp and lilypond
>>> module processed exactly that file further. In MK IV, that file  
>>> is not
>>> created any more (you can check that by running the above example  
>>> with
>>> --lua first and then with pdfTeX again. Only in the second case a  
>>> file
>>> [whateverthefilename]-a.tmp is created.)
>>>
>>> Lilypond module relied on that "feature" (I would rather call it
>>> dirty-but-very-useful-trick).
>>>
>>> Hans, what's the current general recipe for this kind of [mis]use  
>>> of buffers?
>>
>> I'll add savebuffer to core-buf.*
>
> Thanks a lot for that one!
>
>> % engine=luatex
>>
>> \startluacode
>>      if not buffers.save then
>>          function buffers.save(name)
>>              if not name or name == "" then
>>                  name = tex.jobname
>>              end
>>              local b, f = buffers.data[name],
>> string.format("%s-%s.tmp",tex.jobname,name)
>>              b = (b and type(b) == "table" and table.join(b)) or b  
>> or ""
>>              io.savedata(f,b)
>>          end
>>      end
>> \stopluacode
>>
>> \def\savebuffer{\dosingleempty\dosavebuffer}
>>
>> \def\dosavebuffer[#1]{\ctxlua{buffers.save("#1")}}
>>
>> \starttext
>>
>> \startbuffer[oeps]
>> oeps
>> \stopbuffer
>>
>> \startbuffer
>> oepsoeps
>> \stopbuffer
>>
>> \savebuffer[oeps] \savebuffer
>>
>> \stoptext
>
> I now tried to add the following three lines to t-lilypond.tex:
>
> 	\def\PDF{texmfstart --ifchanged=\lily!filename.eps pstopdf \lily! 
> filename.eps}
> 	\beginLUATEX
> 	\savebuffer[lilypond-\the\lily!figures]%
> 	\endLUATEX
> 	\ifeof18
> 		\installprogram{\LP}%
> etc.
>
> But luaTeX now reports the following to console:
>
> system (LUATEX) : [line 32] \savebuffer [lilypond-\the \lily!figures
> ]\endLUATEX \ifeof 18 \installprogram {\LP }\doif \jobsuffix
> {pdf}{\installprogram {\PDF }}\else \executesystemcommand {\LP }\doif
> \jobsuffix {pdf}{\executesystemcommand {\PDF }}\fi \doifelse
> \jobsuffix {pdf} {\edef \lily!img {\lily!filename .pdf}}{\edef
> \lily!img {\lily!filename .eps}}\ifvmode \getfiguredimensions
> [\lily!filename .pdf]\leavevmode \newdimen \FigWidth \FigWidth
> =\figurewidth \ifdim \FigWidth >\localhsize \!!dimena =\localhsize
> \advance \!!dimena by-\FigWidth \noindent \hskip \!!dimena \fi \fi
> \externalfigure [\lily!img ]\egroup
>
> and then
>
> systems         : end file li at line 34
>  )
> (\end occurred inside a group at level 1)
>
> ### simple group (level 1) entered at line 32 ({)
> ### bottom level
> ...
> No pages of output.
> Transcript written on li.log.
>
>
> It seems like a problem with grouping, but I have no idea how to
> prevent luaTeX from parsing the input further from what it's supposed
> to read.
>
> Mojca
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Buffers (gone) in MK IV (was: MK IV & Lilypond)

[Mojca Miklavec]

[-- Attachment #1: Type: text/plain, Size: 747 bytes --]

On 8/25/07, David Wooten wrote:
> Greetings,
>
> Did we end up with a working solution for t-lilypond?

I'm sorry. The file is attached. The only line added is
    \ifx\savebuffer\undefined \else \savebuffer[lilypond-\the\lily!figures]\fi
and one probably needs the latest ConTeXt.

Mojca

------------

        \def\PDF{texmfstart --ifchanged=\lily!filename.eps pstopdf
\lily!filename.eps}
+%D The following line is needed for LuaTeX, where buffers are kept in memory
+%D in contrast to other (older) flavours of TeX where they're written to a
+%D file automatically. We have to require writing to a file explicitely
+%D with \type{\savebuffer}.
+       \ifx\savebuffer\undefined \else
\savebuffer[lilypond-\the\lily!figures]\fi
        \ifeof18

[-- Attachment #2: t-lilypond.tex --]
[-- Type: application/x-tex, Size: 10340 bytes --]

[-- Attachment #3: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________