Recommendations for Speed?

Recommendations for Speed?

[Duane Johnson]

Hello,

I've been perusing the list archives for a couple of days now, and  
reading up on articles (such as Hans' regarding optimizing in TeX and  
eTeX) to find guidance on a handful of decisions.  Our company makes  
a web-based publishing solution called MemoryPress.com that uses  
conTeXt for on-demand creation of PDFs which are then converted to  
jpeg format for display in a web browser.

We are currently looking into ways to speed things up and since many  
of you are far more qualified than I am in this area, I wondered if  
you might advise me on the following:

1. Is there a way to keep TeX in-memory (i.e. as a server or daemon  
process) so that it doesn't have to load and reload fonts and the  
environment?  Our system makes repeated requests for typeset  
documents and we are wondering if there's a way to remove the  
overhead of re-running pdftex.

2. Among the many different command-line TeX options, what might  
optimize for speed?  I've done some rudimentary benchmarking and  
found that executing etex (without conTeXt) is much faster than  
texexec, and wondered if there is any advice in this area.  Is there  
any way to shave off some of the overhead of using conTeXt?

3. I've noticed that there is some kind of caching going on (tui/tuo  
files?) that helps speed things up after the first run.  How can I  
best take advantage of this facility?  What kinds of things will  
require TeX to start from scratch, vs. use some or all of this cached  
information?

4. Are there any other areas I should consider when looking for ways  
to use TeX as an on-demand typesetting engine?

Thank you,

Duane Johnson
Programming Manager
FamilyLearn
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Recommendations for Speed?

[Taco Hoekwater]

Hi Duane,

Duane Johnson wrote:
> 
> 1. Is there a way to keep TeX in-memory (i.e. as a server or daemon  
> process) so that it doesn't have to load and reload fonts and the  
> environment?  Our system makes repeated requests for typeset  
> documents and we are wondering if there's a way to remove the  
> overhead of re-running pdftex.

No, there is not (at least not using any of the 'stock' tex
distributions). The one thing you could do is preload the fonts
and modules you need into the format file, but setting this up
is not completely trivial either.

> 2. Among the many different command-line TeX options, what might  
> optimize for speed?  I've done some rudimentary benchmarking and  
> found that executing etex (without conTeXt) is much faster than  
> texexec, and wondered if there is any advice in this area.  Is there  
> any way to shave off some of the overhead of using conTeXt?

You can only get rid of texexec and the .tuo|.tui handling if you
do not need any kind of referencing at all in your document. If
that is the case, you can run texexec with the --once switch, and
that will help speed up run time quite a bit.

texexec (without --once) will typically run the tex engine one time
too many to make sure that all references are correctly resolved,
so if you know behorehand how many runs are needed, you can speed
up by eliminating that last run.

> 3. I've noticed that there is some kind of caching going on (tui/tuo  
> files?) that helps speed things up after the first run.  How can I  
> best take advantage of this facility?  What kinds of things will  
> require TeX to start from scratch, vs. use some or all of this cached  
> information?

The information in the tui file is not really cached, it is needed
for cross references.

> 4. Are there any other areas I should consider when looking for ways  
> to use TeX as an on-demand typesetting engine?

The best way right now to do general typesetting on demand using TeX
is by creating asynchronous jobs. If that is not an option, then for
the general case, you could consider paying (or hiring) a TeX developer
to implement a daemon feature. Or you could out-source the whole affair:
I know Pragma-ADE (Hans' company) has plans to set up a typesetting-
on-demand service.

I have found that in most specific cases, you can simplify by using
either a dedicated format that runs on a bare-boned tex engine, or by 
just simply having people wait a while.

Of course, it all depends on your actual situation.

Best wishes,
Taco



___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Recommendations for Speed?

[Duncan Hothersall]

Taco said:
> Duane Johnson wrote:
>   
>> 1. Is there a way to keep TeX in-memory (i.e. as a server or daemon  
>> process) so that it doesn't have to load and reload fonts and the  
>> environment?  Our system makes repeated requests for typeset  
>> documents and we are wondering if there's a way to remove the  
>> overhead of re-running pdftex.
>>     
>
> No, there is not (at least not using any of the 'stock' tex
> distributions). The one thing you could do is preload the fonts
> and modules you need into the format file, but setting this up
> is not completely trivial either.
>   
Indeed not trivial. But something I played with once upon a time for 
much the same purpose. You should take a look at Jonathan Fine's TeX 
daemon project texd (http://sourceforge.net/projects/texd) if you 
haven't already - it runs as a pipe (unix) and only loads formats once; 
it would take a little bit work to get it going reliably with ConTeXt, 
especially to do so in a way that is easily updated when a new ConTeXt 
is released, and I'm not sure if Jonathan is still actively working on 
the project. But it demonstrates an effective approach, which works with 
Plain (and LaTeX I think).

Duncan
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________