Re: distributed / parallel TeX?

[Lars Huttar <lars_huttar@sil.org>]

Thanks, everybody, for the discussion on running TeX distributed / in
parallel.
I am much educated about the state of the art. :-)

Summary ...

- There is plenty of optimization that normally can be done. If a
ConTeXt run is taking a really long time, chances are that something is
not being done according to the design.

- For most (current) purposes, documents are small enough and ConTeXt is
fast enough that the effort to automate distribution of typesetting runs
may not be worthwhile. On the other hand, the usage of TeX might expand
if greater throughput were available.

- However, as things stand now, one can always divide documents up by
hand, typeset the parts independently, and stitch them back together
using tools such as divert/undivert. One can even design a document with
the spec that the canonical typesetting process is to typeset the
sections independently; then the sections can never affect each other,
except for explicitly added inter-section effects like page reference
updates.

If you're not aware of MarkMail, it's a handy place to browse / search
archives of mailing lists. This thread can be found at
http://markmail.org/search/?q=ntg+context+distributed+parallel

On 12/17/2008 2:47 AM, Taco Hoekwater wrote:
> There are some interesting ideas in this discussion, but with
> the current state of the code base all of this will be exceedingly
> difficult (especially because of all the synchronisation issues).
>
> Unless someone wants to work on this idea him/herself (and that
> would be great, there are not nearly enough people working on TeX
> development!), you could remind me, say, two years from now?

Sure. Thank you for your interest.

I wasn't asking for someone to implement new features for this, though I
would be happy to see it happen if it is worthwhile for the community.

As Dr Dobb's says, "Single core systems are history"
(http://www.ddj.com/hpc-high-performance-computing/207100560). Software
that can take advantage of multiple cores (or threads, or distributed
nodes) will continue to scale. Of course some effort, and often some
adjustment, is necessary to enable programs to effectively use parallelism.

I'll create a page at http://wiki.contextgarden.net/Parallel summarizing
this discussion if that's OK.

Regards,
Lars

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________