From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.comp.tex.context/36856 Path: news.gmane.org!not-for-mail From: Norbert Preining Newsgroups: gmane.comp.tex.context,gmane.comp.tex.context.devel Subject: Re: [dev-context] integrating context mkiv, luatex, and fmtutil, mktexlsr, etc Date: Wed, 3 Oct 2007 15:18:56 +0200 Message-ID: <20071003131856.GE3597@gamma.logic.tuwien.ac.at> References: <20071002135500.GH32335@gamma.logic.tuwien.ac.at> <47025A4D.6070404@wxs.nl> <20071003111513.GA31958@gamma.logic.tuwien.ac.at> <47037B3C.5030002@wxs.nl> <20071003074824.GW9324@gamma.logic.tuwien.ac.at> <47037739.208@wxs.nl> <20071003111038.GB29863@gamma.logic.tuwien.ac.at> <47037AF1.4020104@wxs.nl> <20071003122106.GA3597@gamma.logic.tuwien.ac.at> <470393E3.1010803@wxs.nl> Reply-To: mailing list for ConTeXt users NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: sea.gmane.org 1191417594 29573 80.91.229.12 (3 Oct 2007 13:19:54 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Wed, 3 Oct 2007 13:19:54 +0000 (UTC) Cc: dev-context@ntg.nl, ntg-context@ntg.nl To: Hans Hagen Original-X-From: ntg-context-bounces@ntg.nl Wed Oct 03 15:19:50 2007 Return-path: Envelope-to: gctc-ntg-context-518@m.gmane.org Original-Received: from ronja.vet.uu.nl ([131.211.172.88] helo=ronja.ntg.nl) by lo.gmane.org with esmtp (Exim 4.50) id 1Id48e-0004og-7a for gctc-ntg-context-518@m.gmane.org; Wed, 03 Oct 2007 15:19:44 +0200 Original-Received: from localhost (localhost [127.0.0.1]) by ronja.ntg.nl (Postfix) with ESMTP id 871A41FD6D; Wed, 3 Oct 2007 15:19:39 +0200 (CEST) Original-Received: from ronja.ntg.nl ([127.0.0.1]) by localhost (smtp.ntg.nl [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 30415-05-4; Wed, 3 Oct 2007 15:19:15 +0200 (CEST) Original-Received: from ronja.vet.uu.nl (localhost [127.0.0.1]) by ronja.ntg.nl (Postfix) with ESMTP id A7ADD1FD19; Wed, 3 Oct 2007 15:19:14 +0200 (CEST) Original-Received: from localhost (localhost [127.0.0.1]) by ronja.ntg.nl (Postfix) with ESMTP id 846CC1FC27; Wed, 3 Oct 2007 15:19:08 +0200 (CEST) Original-Received: from ronja.ntg.nl ([127.0.0.1]) by localhost (smtp.ntg.nl [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 30415-05-3; Wed, 3 Oct 2007 15:18:58 +0200 (CEST) Original-Received: from mx.logic.tuwien.ac.at (mx.logic.tuwien.ac.at [128.130.175.19]) by ronja.ntg.nl (Postfix) with ESMTP id 9EEE81FD19; Wed, 3 Oct 2007 15:18:56 +0200 (CEST) Original-Received: from gamma.logic.tuwien.ac.at ([128.130.175.3] ident=Debian-exim) by mx.logic.tuwien.ac.at with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.63) (envelope-from ) id 1Id47s-0004I1-E4; Wed, 03 Oct 2007 15:18:56 +0200 Original-Received: from preining by gamma.logic.tuwien.ac.at with local (Exim 4.63) (envelope-from ) id 1Id47s-0002P5-BM; Wed, 03 Oct 2007 15:18:56 +0200 Content-Disposition: inline In-Reply-To: <470393E3.1010803@wxs.nl> User-Agent: Mutt/1.5.13 (2006-08-11) X-Virus-Scanned: amavisd-new at ntg.nl X-BeenThere: ntg-context@ntg.nl X-Mailman-Version: 2.1.9 Precedence: list List-Id: mailing list for ConTeXt users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: ntg-context-bounces@ntg.nl Errors-To: ntg-context-bounces@ntg.nl X-Virus-Scanned: amavisd-new at ntg.nl Xref: news.gmane.org gmane.comp.tex.context:36856 gmane.comp.tex.context.devel:683 Archived-At: HI Hans! On Mi, 03 Okt 2007, Hans Hagen wrote: > >database for luatex. But it is placed NOT alongside the ls-R files, > >right? > > well, "TEXMFSHARECACHE=yes" makes sure that it is put in the same path > as the ls-r file Ah, ok, that is with the new context you created today. No, didn't test this till now ;-) First I guess I need an updated luatex you said. > no, not the trees, but if one uses a font, say lmroman10-regular.otf, > then the 'converterd/prepared/whatever' datatable is cached so only that > file ends up in the cache (flat structure, so, when one needs > lmroman10-regular.otf, i first look in the cache (no database needed) > for a tmc/a file, if not present i generate one, using the otf file from > the normal tree, looked up in a kpse like way) [...] > >- preprocessed font data cache so that loading the stuff is done faster > > indeed, only at runtime, so normally in the users path someplace Hmm, I still don't get it. In all this processing, is there a job depending thing contained? I thought extracting all those tables etc can be done independently of all jobs (and will help all jobs). Anyway, I got it. So this is not a bad thing at all to have this stuff generated at job running time, so we (integrators) don't have to worry about it. Only finding all the files is important for a start, so ls-R replacement. > >$ ls -l ~/luatex-cache/context/f7d1b3c25487ab1e1035aff1c53b90da/formats/ > >-rw-r--r-- 1 norbert norbert 5669003 2007-10-03 14:07 cont-en.fmt > >-rw-r--r-- 1 norbert norbert 38786 2007-10-03 14:07 cont-en.log > >-rw-r--r-- 1 norbert norbert 159484 2007-10-03 14:07 cont-en.lua > >-rw-r--r-- 1 norbert norbert 112438 2007-10-03 14:07 cont-en.luc > >$ > > > >So the format is placed in some strange ;-) place under luatex-cache. > > hm, also with the new version and the env var set? here it nicely goes > to texmf-linux/web2c/luatex Probably an instance of "get the new beta", btw, where do I get it, from the usual download place? Or is there something else for betas? > >- formats??? what is saved for this > > a cont-en.fmt as well as a cont-en.luc file, the format and it sstub Yes of course, but what needs to be cached here but the sole information that the file exist? (ie updating ls-R/cache) > >But all of this is somehow static. On a normal system a normal user > >shouldn't have the necessity to change anything of the above. That > >should be done at install time of the respective stuff. > > no, the font cache is not pregenerated at all, think of it like this: > > - luatex loads otf font (takes time) > - provides it as table to mkiv (takes more time) > - mkiv adds a few things, preprocesses the font a bit > - then saves it as table (compiled) so that loading goes in an eyeblink Ok, so after the first processing later jobs will have access to these tables in a blink. Ok, don't mind. Good for me ;-) No need to worry about this. > >Furthermore, the cache could be used (no idea whether this is true) for: > >- single job caching of data > > wouldn't it be better to keep generated files like this in the > > cwd, like .aux, etc files > > we're talking of megabytes here Are these megabytes kept and are they useful for later runs? If yes, then it is ok to put them in a separate cache (I assume the stuff is useful, it is the font stuff). > >What was the reason to do the hashing somewhere else but add the md5sums > >etc for these trees? > > lengths of paths and such; i run luatex on the unix web/etc servers and > have multiple trees in parallel, so now i can with one variable changed > access all data; > > /temp/luatex/context//whatever > > > /temp/luatex/context/c:/blabla/tex/blabla/whatever > > > does not work that well, so i hash the treeroot Hmm, but if the caches would be in TEXMFTREE/lcache/ then the only thing necessary would be a change of TEXMFCNF referencing other trees. (Sorry, only trying to understand the details ...) Best wishes Norbert ------------------------------------------------------------------------------- Dr. Norbert Preining Vienna University of Technology Debian Developer Debian TeX Group gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094 ------------------------------------------------------------------------------- He stood up straight and looked the world squarely in the fields and hills. To add weight to his words he stuck the rabbit bone in his hair. He spread his arms out wide. `I will go mad!' he announced. --- Arthur discovering a way of coping with life on --- Prehistoric Earth. --- Douglas Adams, The Hitchhikers Guide to the Galaxy ___________________________________________________________________________________ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://tex.aanhet.net archive : https://foundry.supelec.fr/projects/contextrev/ wiki : http://contextgarden.net ___________________________________________________________________________________