How to preserve a TeX token list across passes (the ConTeXt way)?

How to preserve a TeX token list across passes (the ConTeXt way)?

[Jyrki Ruuskanen]

I made a plain TeX module for simple abbreviation and bibliography  
handling. The problem is it currently uses a temporary file to keep  
track of used bibs / abbreviations. Essentially it writes \addToList 
[\usedAbbreviations]{<abbr>} and \addToList[\usedReferences]{<cite>}  
commands to it, and this file is input on the second pass.

After finding out that ConTeXt *really is* well thought of and robust  
I changed the functionality to use TeXutil with \writeutilitycommand.  
Everything seems to work, the commands appear in .tui/.tuo files and  
are executed, but the token lists are empty when execution reaches  
the document. I couldn't figure out how to fix it as I don't yet know  
enough about the anatomy of a ConTeXt run and the system.

Jyrki Ruuskanen

Re: How to preserve a TeX token list across passes (the ConTeXt way)?

[Taco Hoekwater]

Jyrki Ruuskanen wrote:
> I made a plain TeX module for simple abbreviation and bibliography  
> handling. The problem is it currently uses a temporary file to keep  
> track of used bibs / abbreviations. Essentially it writes \addToList 
> [\usedAbbreviations]{<abbr>} and \addToList[\usedReferences]{<cite>}  
> commands to it, and this file is input on the second pass.
> 
> After finding out that ConTeXt *really is* well thought of and robust  
> I changed the functionality to use TeXutil with \writeutilitycommand.  
> Everything seems to work, the commands appear in .tui/.tuo files and  
> are executed, but the token lists are empty when execution reaches  
> the document. I couldn't figure out how to fix it as I don't yet know  
> enough about the anatomy of a ConTeXt run and the system.

Some small example files would help a lot. We have to know what
*precisely* is in the tui file.

Cheers ,Taco

Re: How to preserve a TeX token list across passes (the ConTeXt way)?

[Hans Hagen]

Jyrki Ruuskanen wrote:
> I made a plain TeX module for simple abbreviation and bibliography  
> handling. The problem is it currently uses a temporary file to keep  
> track of used bibs / abbreviations. Essentially it writes \addToList 
> [\usedAbbreviations]{<abbr>} and \addToList[\usedReferences]{<cite>}  
> commands to it, and this file is input on the second pass.
>
> After finding out that ConTeXt *really is* well thought of and robust  
> I changed the functionality to use TeXutil with \writeutilitycommand.  
> Everything seems to work, the commands appear in .tui/.tuo files and  
> are executed, but the token lists are empty when execution reaches  
> the document. I couldn't figure out how to fix it as I don't yet know  
> enough about the anatomy of a ConTeXt run and the system.
>   
can you make a small example? maybe a grouping issue 

Hans 

-----------------------------------------------------------------
                                          Hans Hagen | PRAGMA ADE
              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                             | www.pragma-pod.nl
-----------------------------------------------------------------

Re: How to preserve a TeX token list across passes (the ConTeXt way)?

[Hans Hagen]

Jyrki Ruuskanen wrote:
> Ok, the module is read in cont-sys.rme with \usemodule[jtex]. It 
> initializes the lists inside protect / unprotect pair with 
> \newtoks\usedAbbreviations and \newtoks\usedReferences along with 
> macro definitions.
first of all, here is an alternative to setting your texts; instead of doing the check-next-char stuff yourself ...

%%% handle multilingual texts (and uppercase)
% usage: \setText[lang]{key}{value}
\def\setText{\futurelet\testchar\maybeLangSetText}
\def\maybeLangSetText%
	{\ifx[\testchar \let\next\langSetText
	\else \let\next\defaultSetText \fi \next}
\def\langSetText[#1]#2#3{\def\name{#2@#1}\def\bigname{#2@#1@uppercase}
	\expandafter\def\csname \name \endcsname{#3}
	\expandafter\def\csname \bigname \endcsname{\uppercase{#3}}}
\def\defaultSetText#1#2{\langSetText[\currentmainlanguage]{#1}{#2}}

% usage: \getText{key}
\def\langGetText[#1]#2{\def\name{#2@#1}%
	\expandafter\ifx\csname \name \endcsname\relax
	\name missing!\else\csname \name \endcsname\fi}
\def\getText#1{\def\name{#1@\currentlanguage}%
	\expandafter\ifx\csname \name \endcsname\relax
	\langGetText[\currentmainlanguage]{#1}
	\else\langGetText[\currentlanguage]{#1}\fi}


... you can use the built in functionality: 

\let\dowithText\firstofoneargument

\def\setText
  {\dosingleempty\dosetText}

\def\dosetText[#1]#2#3%
  {\setvalue{\iffirstargument#1\else\currentmainlanguage\fi @TXT@#2}{\dowithText{#3}}}

\def\getText
  {\let\dowithText\firstofoneargument
   \dosingleempty\dogetText}

\def\dogetText[#1]#2%
  {\executeifdefined{\iffirstargument#1\else\currentlanguage\fi @TXT@#2}%
     {\executeifdefined{\iffirstargument#1\else\currentmainlanguage\fi @TXT@#2}%
        {missing #2}}}

\def\getUppercaseText
  {\let\dowithText\WORD
   \dosingleempty\dogetText}


\starttext

\setText{oeps}{whatever}
\setText[de]{oeps}{de-whatever}

\getText{oeps}
\getText[de]{oeps}
\getUppercaseText{oeps}
\getUppercaseText[de]{oeps}

\stoptext

but, when possible, use: 

\setuplabeltext[oeps=whatever]
\setuplabeltext[de][oeps=de-whatever]

{\labeltext{oeps} \mainlanguage[de] \labeltext{oeps} \LABELTEXT{oeps}}

because that is kind of integrated 

Hans 


-----------------------------------------------------------------
                                          Hans Hagen | PRAGMA ADE
              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                             | www.pragma-pod.nl
-----------------------------------------------------------------

Re: How to preserve a TeX token list across passes (the ConTeXt way)?

[Hans Hagen]

Jyrki Ruuskanen wrote:
> Ok, the module is read in cont-sys.rme with \usemodule[jtex]. It 
> initializes the lists inside protect / unprotect pair with 
> \newtoks\usedAbbreviations and \newtoks\usedReferences along with 
> macro definitions.
>
> Abbreviations are introduced with 
> \setAbbreviation[<key>]{<abbr>}{<meaning>}{<description>} which 
> defines macros \<key> \<key>@meaning and \<key>@description. Each call 
> to \<key> calls 
> \writeutilitycommand{\addToList[\string\usedAbbreviations]{<key>}, so 
> after the first run the .tui file has a number of "c 
> \addToList[\usedAbbreviations]{something}" lines in it.
>
> Functionality is similar for the bibliography. The \cite command 
> writes \addToList[\usedReferences]{<cite>} into .tui.
>
> So, the .tuo file should populate used* lists and I could typeset the 
> list of used abbreviations in initialization order with 
> \executeListIntersection[\abbreviations][\usedAbbreviations]{\printAbbreviation}. 
> Same for used bib references. But well, lists remain empty.
>
> I'll attach the module and the .tui file.
is there also a test file? 

Hans 

-----------------------------------------------------------------
                                          Hans Hagen | PRAGMA ADE
              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                             | www.pragma-pod.nl
-----------------------------------------------------------------

Re: How to preserve a TeX token list across passes (the ConTeXt way)?

[Hans Hagen]

Jyrki Ruuskanen wrote:
> Ok, the module is read in cont-sys.rme with \usemodule[jtex]. It 
> initializes the lists inside protect / unprotect pair with 
> \newtoks\usedAbbreviations and \newtoks\usedReferences along with 
> macro definitions.
>
> Abbreviations are introduced with 
> \setAbbreviation[<key>]{<abbr>}{<meaning>}{<description>} which 
> defines macros \<key> \<key>@meaning and \<key>@description. Each call 
> to \<key> calls 
> \writeutilitycommand{\addToList[\string\usedAbbreviations]{<key>}, so 
> after the first run the .tui file has a number of "c 
> \addToList[\usedAbbreviations]{something}" lines in it.
>
> Functionality is similar for the bibliography. The \cite command 
> writes \addToList[\usedReferences]{<cite>} into .tui.
>
> So, the .tuo file should populate used* lists and I could typeset the 
> list of used abbreviations in initialization order with 
> \executeListIntersection[\abbreviations][\usedAbbreviations]{\printAbbreviation}. 
> Same for used bib references. But well, lists remain empty.
>
> I'll attach the module and the .tui file.
>
> And excuse me, I have never ever used a mailing list. How am I 
> supposed to answer to a certain message in the chain? Thanks guys.
i'm away two days (ntg tex days) and will oook into it after that 

Hans 

-----------------------------------------------------------------
                                          Hans Hagen | PRAGMA ADE
              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                             | www.pragma-pod.nl
-----------------------------------------------------------------

How to preserve a TeX token list across passes (the ConTeXt way)?

[Jyrki Ruuskanen]

[-- Attachment #1.1: Type: text/plain, Size: 800 bytes --]

Thanks for the cont-sys tip, good to know.

Yes, I have used \definesynonyms, but I need abbreviations to have  
three levels: the abbreviation itself, the meaning of abbreviation  
and the description of the abbreviation (for example, "WWW" is short  
for "World Wide Web" and is described as a "Worldwide interconnected  
mess of multimedia, which took off in early 90's."). ConTeXt synonyms  
seem to have two.

I'm sure there are a million ways of doing this. I decided to  
implement it from the ground up to learn some plain TeX. But now I  
need to integrate my macros with ConTeXt to make passing the lists  
graceful without extra auxiliary files and forced double runs.

And as for the reply button, I guess I have to subscribe to get the  
mails in my mailbox. I'll do that now ;)

Jyrki


[-- Attachment #1.2: Type: text/html, Size: 3701 bytes --]

[-- Attachment #2: Type: text/plain, Size: 139 bytes --]

_______________________________________________
ntg-context mailing list
ntg-context@ntg.nl
http://www.ntg.nl/mailman/listinfo/ntg-context

Re: How to preserve a TeX token list across passes (the ConTeXt way)?

[Hans Hagen]

Jyrki Ruuskanen wrote:
> Ok, the module is read in cont-sys.rme with \usemodule[jtex]. It 
> initializes the lists inside protect / unprotect pair with 
> \newtoks\usedAbbreviations and \newtoks\usedReferences along with 
> macro definitions.
copy cont-sys.rme -> cont.sys.tex

and add the module loading there (cont-rme will be overwritten when you 
update)
>
> Abbreviations are introduced with 
> \setAbbreviation[<key>]{<abbr>}{<meaning>}{<description>} which 
> defines macros \<key> \<key>@meaning and \<key>@description. Each call 
> to \<key> calls 
> \writeutilitycommand{\addToList[\string\usedAbbreviations]{<key>}, so 
> after the first run the .tui file has a number of "c 
> \addToList[\usedAbbreviations]{something}" lines in it.
>
> Functionality is similar for the bibliography. The \cite command 
> writes \addToList[\usedReferences]{<cite>} into .tui.
>
> So, the .tuo file should populate used* lists and I could typeset the 
> list of used abbreviations in initialization order with 
> \executeListIntersection[\abbreviations][\usedAbbreviations]{\printAbbreviation}. 
> Same for used bib references. But well, lists remain empty.
so how about using \definesynonyms (see end of core-syn.tex)
>
>
> I'll attach the module and the .tui file.
>
> And excuse me, I have never ever used a mailing list. How am I 
> supposed to answer to a certain message in the chain? Thanks guys.
just hit the reply button in your mail agent 

Hans 

-----------------------------------------------------------------
                                          Hans Hagen | PRAGMA ADE
              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                             | www.pragma-pod.nl
-----------------------------------------------------------------

How to preserve a TeX token list across passes (the ConTeXt way)?

[Jyrki Ruuskanen]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

Ok, the module is read in cont-sys.rme with \usemodule[jtex]. It  
initializes the lists inside protect / unprotect pair with \newtoks 
\usedAbbreviations and \newtoks\usedReferences along with macro  
definitions.

Abbreviations are introduced with \setAbbreviation[<key>]{<abbr>} 
{<meaning>}{<description>} which defines macros \<key> \<key>@meaning  
and \<key>@description. Each call to \<key> calls \writeutilitycommand 
{\addToList[\string\usedAbbreviations]{<key>}, so after the first run  
the .tui file has a number of "c \addToList[\usedAbbreviations] 
{something}" lines in it.

Functionality is similar for the bibliography. The \cite command  
writes \addToList[\usedReferences]{<cite>} into .tui.

So, the .tuo file should populate used* lists and I could typeset the  
list of used abbreviations in initialization order with  
\executeListIntersection[\abbreviations][\usedAbbreviations] 
{\printAbbreviation}. Same for used bib references. But well, lists  
remain empty.

I'll attach the module and the .tui file.

And excuse me, I have never ever used a mailing list. How am I  
supposed to answer to a certain message in the chain? Thanks guys.

Jyrki


[-- Attachment #2: final.tui --]
[-- Type: application/octet-stream, Size: 35260 bytes --]

[-- Attachment #3: t-jtex.tex --]
[-- Type: application/octet-stream, Size: 5192 bytes --]

%D \enableregime[il1]
%D \module
%D	[file=t-jtex,
%D	version=2006.0,
%D	title=\CONTEXT\ User Module,
%D	subtitle=Miscellaneous Macros,
%D	author=Jyrki Ruuskanen,
%D	date=\currentdate,
%D	copyright=Jyrki Ruuskanen]

\writestatus{loading}{J's TeX Extensions}

\unprotect

% add \currentsecondlanguage variable
\def\secondlanguage[#1]{\def\currentsecondlanguage{#1}}

%%% handle multilingual texts (and uppercase)
% usage: \setText[lang]{key}{value}
\def\setText{\futurelet\testchar\maybeLangSetText}
\def\maybeLangSetText%
	{\ifx[\testchar \let\next\langSetText
	\else \let\next\defaultSetText \fi \next}
\def\langSetText[#1]#2#3{\def\name{#2@#1}\def\bigname{#2@#1@uppercase}
	\expandafter\def\csname \name \endcsname{#3}
	\expandafter\def\csname \bigname \endcsname{\uppercase{#3}}}
\def\defaultSetText#1#2{\langSetText[\currentmainlanguage]{#1}{#2}}

% usage: \getText{key}
\def\langGetText[#1]#2{\def\name{#2@#1}%
	\expandafter\ifx\csname \name \endcsname\relax
	\name missing!\else\csname \name \endcsname\fi}
\def\getText#1{\def\name{#1@\currentlanguage}%
	\expandafter\ifx\csname \name \endcsname\relax
	\langGetText[\currentmainlanguage]{#1}
	\else\langGetText[\currentlanguage]{#1}\fi}

% usage: \getUppercaseText{key}
\def\langGetUppercaseText[#1]#2{\def\name{#2@#1@uppercase}%
	\expandafter\ifx\csname \name \endcsname\relax
	\name missing!\else\csname \name \endcsname\fi}
\def\getUppercaseText#1{\def\name{#1@\currentlanguage @uppercase}%
	\expandafter\ifx\csname \name \endcsname\relax
	\langGetUppercaseText[\currentmainlanguage]{#1}
	\else\langGetUppercaseText[\currentlanguage]{#1}\fi}

% usage: \startTextBuffer[lang]{key}...
\def\startTextBuffer{\futurelet\testchar\maybeLangStartTextBuffer}
\def\maybeLangStartTextBuffer%
	{\ifx[\testchar \let\next\langStartTextBuffer
	\else \let\next\defaultStartTextBuffer \fi \next}
\def\langStartTextBuffer[#1]#2{\startbuffer[#2@#1]}
\def\defaultStartTextBuffer#1{\langStartTextBuffer[\currentmainlanguage]{#1}}

% NOTE doesn't work, use \stopbuffer for now
% usage: ...\stopTextBuffer
\let\stopTextBuffer=\stopbuffer

% usage: \getTextBuffer{key}
\def\getTextBuffer#1{\getbuffer[#1@\currentlanguage]}

%%% handle abbreviations
% usage: \setAbbreviation[key]{abbr}{meaning}{description}
\newtoks\usedAbbreviations
\newtoks\abbreviations
\def\setAbbreviation{\futurelet\testchar\maybeOptSetAbbreviation}
\def\maybeOptSetAbbreviation%
	{\ifx[\testchar \let\next\optSetAbbreviation
	\else \let\next\defaultSetAbbreviation \fi \next}
\def\optSetAbbreviation[#1]#2#3#4%
	{\addToList[\abbreviations]{#1}
	\expandafter\def\csname#1\endcsname%
		{#2\writeutilitycommand{\addToList[\string\usedAbbreviations]{#1}}}
	\setAsKeyAndValue[meaning]{#3}{#1}
	\setAsKeyAndValue[description]{#4}{#1}}
\def\defaultSetAbbreviation#1#2#3{\optSetAbbreviation[#1]{#1}{#2}{#3}}

% usage: \getMeaning{key}
\def\getMeaning#1{\def\meaning{#1@meaning}%
	\csname \meaning \endcsname}
\let\infull=\getMeaning

% usage: \getDescription{key}
\def\getDescription#1{\def\description{#1@description}%
	\csname \description \endcsname}

% usage: \printAbbreviation{key}
\def\printAbbreviation#1%
	{{\bf\csname#1\endcsname} (\getMeaning{#1}). \getDescription{#1}\par}

%%% handle references
% usage: \setReference[key]{cite}{description}
\newtoks\usedReferences
\newtoks\references
\def\setReference[#1]#2#3{\def\citename{#1@cite}%
	\addToList[\references]{#1}
	\expandafter\def\csname\citename\endcsname%
		{#2\writeutilitycommand{\addToList[\string\usedReferences]{#1}}}
	\setAsKeyAndValue[description]{#3}{#1}}
	
% usage: \cite[key]
\def\cite[#1]{\def\citename{#1@cite}%
	\csname \citename \endcsname}

% usage: \printReference{key}
\def\printReference#1%
	{\getDescription{#1}\par}

%%% token list operations
% usage: \addToList[\list]{value}
\def\addToList[#1]#2{\let\\=\relax
	\edef\act{\noexpand#1={\the#1 \\{#2}}}%
	\act}

% usage: \executeList[\list]{\command#1}
\def\executeList[#1]#2{\def\\##1{#2{##1}}%
	\the#1}

% usage: \executeListIntersection[\listone][\listtwo]{\command#1}
\def\executeOnMatch#1#2{\def\comparison{#2@comparison}%
	\expandafter\ifx\csname \comparison \endcsname\relax
	\else#1{#2} \fi}
\def\executeListIntersection[#1][#2]#3%
	{\executeList[#2]{\setAsKeyAndValue[comparison]{true}}
	\executeList[#1]{\executeOnMatch{#3}}}

% usage: \addToAssociative[\list]{value}{key}
\def\addToAssociative[#1]#2#3{\let\\=\relax
	\edef\act{\noexpand#1={\the#1 \\{#2}{#3}}}%
	\act}

% usage: \executeAssociative[\list]{\command#1#2}
\def\executeAssociative[#1]#2{\def\\##1##2{#2{##1}{##2}}%
	\the#1}

% usage: \setAsKeyAndValue[suffix]{value}{key}
\def\setAsKeyAndValue{\futurelet\testchar\maybeSetAsKeyAndValueWithSuffix}
\def\maybeSetAsKeyAndValueWithSuffix%
	{\ifx[\testchar \let\next\setAsKeyAndValueWithSuffix
	\else \let\next\setAsKeyAndValueWithoutSuffix \fi \next}
\def\setAsKeyAndValueWithSuffix[#1]#2#3{\def\name{#3@#1}%
	\expandafter\def\csname \name \endcsname{#2}}
\def\setAsKeyAndValueWithoutSuffix#1#2{\def\name{#2}%
	\expandafter\def\csname \name \endcsname{#1}}

% usage: \getFromAssociative[\list]{key}
\def\getFromAssociative[#1]#2{\def\name{#2@temp}%
	\executeAssociative[#1]{\setAsKeyAndValue[temp]}
	\csname \name \endcsname}

\protect

[-- Attachment #4: Type: text/plain, Size: 139 bytes --]

_______________________________________________
ntg-context mailing list
ntg-context@ntg.nl
http://www.ntg.nl/mailman/listinfo/ntg-context