From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.comp.tex.context/46332 Path: news.gmane.org!not-for-mail From: Lars Huttar Newsgroups: gmane.comp.tex.context Subject: distributed / parallel TeX? Date: Mon, 15 Dec 2008 17:06:10 -0600 Message-ID: <4946E2E2.1050108@sil.org> Reply-To: mailing list for ConTeXt users NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1229382519 25548 80.91.229.12 (15 Dec 2008 23:08:39 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 15 Dec 2008 23:08:39 +0000 (UTC) To: mailing list for ConTeXt users Original-X-From: ntg-context-bounces@ntg.nl Tue Dec 16 00:09:42 2008 Return-path: Envelope-to: gctc-ntg-context-518@m.gmane.org Original-Received: from ronja.vet.uu.nl ([131.211.172.88] helo=ronja.ntg.nl) by lo.gmane.org with esmtp (Exim 4.50) id 1LCMZI-00016Q-Ip for gctc-ntg-context-518@m.gmane.org; Tue, 16 Dec 2008 00:09:40 +0100 Original-Received: from localhost (localhost [127.0.0.1]) by ronja.ntg.nl (Postfix) with ESMTP id 811011FC27; Tue, 16 Dec 2008 00:08:26 +0100 (CET) Original-Received: from ronja.ntg.nl ([127.0.0.1]) by localhost (smtp.ntg.nl [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 06374-01-8; Tue, 16 Dec 2008 00:07:16 +0100 (CET) Original-Received: from ronja.vet.uu.nl (localhost [127.0.0.1]) by ronja.ntg.nl (Postfix) with ESMTP id 4DBE51FB8F; Tue, 16 Dec 2008 00:07:16 +0100 (CET) Original-Received: from localhost (localhost [127.0.0.1]) by ronja.ntg.nl (Postfix) with ESMTP id 19EE91FB8F for ; Tue, 16 Dec 2008 00:07:14 +0100 (CET) Original-Received: from ronja.ntg.nl ([127.0.0.1]) by localhost (smtp.ntg.nl [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 06323-01-8 for ; Tue, 16 Dec 2008 00:06:14 +0100 (CET) Original-Received: from smtp1.wsfo.org (smtp1.wsfo.org [208.145.81.51]) by ronja.ntg.nl (Postfix) with ESMTP id 158561FB6B for ; Tue, 16 Dec 2008 00:06:13 +0100 (CET) Original-Received: from mail.link77.net (mail.link77.net [172.22.0.125]) by smtp1.wsfo.org (8.13.1/8.13.1) with ESMTP id mBFN6CYs016834 (version=TLSv1/SSLv3 cipher=DES-CBC3-SHA bits=168 verify=NO) for ; Mon, 15 Dec 2008 18:06:12 -0500 X-CGP-ClamAV-Result: CLEAN X-VirusScanner: Niversoft's CGPClamav Helper v1.8.2 (ClamAV engine v0.94.1) Original-Received: from [172.20.4.229] (account lars_huttar@sil.org [172.20.4.229] verified) by mail.link77.net (CommuniGate Pro SMTP 5.2.10) with ESMTPSA id 203459984 for ntg-context@ntg.nl; Mon, 15 Dec 2008 18:06:12 -0500 User-Agent: Thunderbird 2.0.0.18 (Windows/20081105) X-Enigmail-Version: 0.95.7 X-Scanned-By: MIMEDefang 2.62 on 172.22.0.51 X-Virus-Scanned: amavisd-new at ntg.nl X-BeenThere: ntg-context@ntg.nl X-Mailman-Version: 2.1.11 Precedence: list List-Id: mailing list for ConTeXt users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: ntg-context-bounces@ntg.nl Errors-To: ntg-context-bounces@ntg.nl X-Virus-Scanned: amavisd-new at ntg.nl Xref: news.gmane.org gmane.comp.tex.context:46332 Archived-At: Hello, We've been using TeX to typeset a 1200-page book, and at that size, the time it takes to run becomes a big issue (especially with multiple passes... about 8 on average). It takes us anywhere from 80 minutes on our fastest machine, to 9 hours on our slowest laptop. So the question comes up, can TeX runs take advantage of parallelized or distributed processing? As many in the computer industries are aware, processor speed (clock rate) has plateaued; it is not going to continue rising at the rate it had been. Hence the common move to dual-core, quad-core, etc. machines. But applications in general cannot take advantage of multiple cores to speed their work unless they are architected to take advantage of them. We googled around a bit but were surprised not to find any real references to efforts at running TeX in parallel or on distributed networks or clusters. Wouldn't this be something that a lot of people would find useful? Or does everyone only use TeX for typesetting short papers? Sure, you can use manual tricks to speed up TeX processing. You can "comment out" sections of a document, or select them via modes. But then you have to remember where you did the commenting out, so you can reverse it. And you have no guarantees as to whether the inclusion/exclusion of section B will affect the layout of section C or not. Wouldn't it be nice if TeX (or a TeX wrapper, or macro package, or "typesetting system") could take care of this for you? What if you had a language -- or a few extensions to existing languages -- to give your typesetting engine hints or commands about where to split up your long document into fairly-independent chunks? What if you designed your document specifically to be typeset in independent, parallel pieces so that you could guarantee that you would get the same result for section B whether or not you were typesetting the whole book at the same time? What if the typesetting system automated the stitching-back-together process of the chunks, gathering page reference info from each chunk to inform the next iteration of typesetting the other chunks? Has anyone been working on this already? It seems like it must have been discussed, but I don't know where to go to look for that discussion. Thanks, Lars ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________