Re: Best way to create a large number of documents from database

From: Hans Hagen <j.hagen@xs4all.nl>
To: mailing list for ConTeXt users <ntg-context@ntg.nl>,
	Mojca Miklavec <mojca.miklavec.lists@gmail.com>
Subject: Re: Best way to create a large number of documents from database
Date: Thu, 16 Apr 2020 16:52:12 +0200	[thread overview]
Message-ID: <780bff75-0ea2-f150-5b6e-5aed60085c49@xs4all.nl> (raw)
In-Reply-To: <CALBOmsa8zz=++caJUv+cToXevXbs-g=-b8BJHqF8A1=NseGTXg@mail.gmail.com>

On 4/16/2020 4:38 PM, Mojca Miklavec wrote:
> On Thu, 16 Apr 2020 at 11:29, Taco Hoekwater wrote:
>>> On 16 Apr 2020, at 11:12, Mojca Miklavec wrote:
>>>
>>> I have been asked to create a few thousand PDF documents from a CSV
>>> "database" today
>>
>> In CPU cycles, the fastest way is to do a single context —once
>> run generating all the pages as a single document, then using
>> mutool merge to split it into separate documents using a (shell)
>> loop.
> 
> Just to make it clear: I don't really need to optimize on the CPU end,
> as the bottleneck is on the other side of the keyboard, so as long as
> the CPU can process 5k pages today, I'm fine with it :) :) :)

5K is nothing ... so that will work

>>> One option is that I quickly draft a python script that creates a few
>>> thousand TeX documents and compiles them individually, but it might be
>>> easier if there was a way to just create a single template document
>>> and then run something like
>>>     context --some-params --N=42 --output=document-0042.pdf template.tex
>>> or something along those lines.
>>
>> If you want to go this route (and you may have to if not each record
>> fits exactly within a single page),
> 
> I do have one page per document. The more annoying part is having
> strange document names that need more attention when mapping page
> number -> name (I'm not saying this is not doable).

so, don't make files:

- write a tex file foo.tex
- process it: context --batch --result=1 --once foo

etc ... so, use --result for the target name and use the same input name

(I won't bother you with the template system in context that no one 
knows of.)

  Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________