[NTG-context] new upload

[NTG-context] new upload

[Hans Hagen]

Hi,

In todays upload there is a follow up on a new feature that is already 
present in the previous upload (but that was not yet advertised):

   context --parallel test1.tex test2.tex test3.tex

   context --parallel --pattern=test*.tex

   context --parallellist myfile.cmd  (filters context processing lines)

   context --parallel test1.tex test2.tex test3.tex --terminal

The --parellel feature will run in parallel, By default upto 8 jobs but 
you can specify less or more depending on your hardware but on my 
machine this is reasonable.

See workflows-parrallel at the end for more info. That one also mentions 
the new --forceinput flag which makes that when these

   context foo-1.tex --mode=m-1 --forceinput=whatever.tex
   context foo-3.tex --mode=m-3 --forceinput=whatever.tex

are run in parallel there is no clash in tuc/pdf. Of course not all 
scenarios might work out well. Commandline arguments given are passed on 
to the different jobs but it's good enough for the purpose I have in mind.

fwiw: a single pass teststuite run (some 1950 files) goes down from 1350 
to 700 seconds but another run with 1200 complex xml documents goes 4 
times faster. In the end this is way cheaper than updating a 5 year old 
dev machine (a new one would run about twice as fast which would bring 
about the same gain).

Hans

ps. Because tex is a memory intense application it doesn't scale with 
the number of cores unless one has the latest greatest fast memory as 
well as generous cpu caches. There's only so much one can do in the code 
base about it.

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : https://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : https://contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: new upload

[Rik Kabel]

On 7/18/2023 4:12 PM, Hans Hagen wrote:
> Hi,
>
> In todays upload there is a follow up on a new feature that is already 
> present in the previous upload (but that was not yet advertised):
>
>   context --parallel test1.tex test2.tex test3.tex
>
>   context --parallel --pattern=test*.tex
>
>   context --parallellist myfile.cmd  (filters context processing lines)
>
>   context --parallel test1.tex test2.tex test3.tex --terminal
>
> The --parellel feature will run in parallel, By default upto 8 jobs 
> but you can specify less or more depending on your hardware but on my 
> machine this is reasonable.
>
> See workflows-parrallel at the end for more info. That one also 
> mentions the new --forceinput flag which makes that when these
>
>   context foo-1.tex --mode=m-1 --forceinput=whatever.tex
>   context foo-3.tex --mode=m-3 --forceinput=whatever.tex
>
> are run in parallel there is no clash in tuc/pdf. Of course not all 
> scenarios might work out well. Commandline arguments given are passed 
> on to the different jobs but it's good enough for the purpose I have 
> in mind.
>
> fwiw: a single pass teststuite run (some 1950 files) goes down from 
> 1350 to 700 seconds but another run with 1200 complex xml documents 
> goes 4 times faster. In the end this is way cheaper than updating a 5 
> year old dev machine (a new one would run about twice as fast which 
> would bring about the same gain).
>
> Hans
>
> ps. Because tex is a memory intense application it doesn't scale with 
> the number of cores unless one has the latest greatest fast memory as 
> well as generous cpu caches. There's only so much one can do in the 
> code base about it.
>
>
Nice. Some of my make files use concurrence (MAKEFLAGS += --jobs=5). 
That seems to bring similar benefits. Is this different beyond the syntax?

-- 
Rik

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : https://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : https://contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: new upload

[Hans Hagen via ntg-context]

On 7/18/2023 11:53 PM, Rik Kabel wrote:

> Nice. Some of my make files use concurrence (MAKEFLAGS += --jobs=5). 
> That seems to bring similar benefits. Is this different beyond the syntax?
they are different processes indeed, but the implementation is somewhat 
curious as it needs to be portable across operating systems and i don't 
want to add dependencies to the engine, so we go for a little less 
performance but not that much in practice

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : https://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : https://contextgarden.net
___________________________________________________________________________________

[NTG-context] new upload

[Hans Hagen]

Hi,

new upload with packed itemize fix .. hopefyully no side effects of some 
work in progress (but no crashes in test suite so likely ok)

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : https://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : https://contextgarden.net
___________________________________________________________________________________

[NTG-context] new upload

[Hans Hagen]

Hi,

I uploaded a new lmtx (some new - experimental - things will be 
explained after the ctx meeting), but there's one thing that is somewhat 
important.

In addition to the built in units we can now have so called user units. 
There is some more info about this in the lowlevel-registers manual. One 
of the predefined units is 'uu' with an associated \ununit (dimension) 
variable and it's is also interfaced to metafun. That means that when 
you use uu in metapost graphics you have to make sure that you save the 
meaning and/or numeric uu now.

The idea is that just as in mp one can say 20xx with xx being a (sort 
of) unit, one can do the same in tex. Think of

\newdimension \MyUnitA  \MyUnitA 1.23pt

\pushoverloadmode % just in case
     \newuserunit\MyUnitA ua % binds the unit to the variable
\popoverloadmode

x\hskip 10ua\relax x

The units starting with 'u' are considered user units but be carefulk 
with other combinations as we have (and will have more) built into 
context ones, like

pi : pi for Mikael
ft : foot for Alan
fs : (global body) font size
tw : (layout) text width
th : (layout) text height
hs : (current) hsize
vs : (current) vsize
cd : (when set) column distance
cw : (when set) column width
cx : combination cell width
uu : user unit (\METAFUN)

So Mikael can now have

   test \hskip 3pi\relax test

and Alan can have

   \setuplayout
     [width=1ft]

to be consistent with inches.

The reason why we have this mechanism is that it provides a level of 
abstraction. It might evolve over time but don't expect the 600+ units 
possible to show up because one needs sensible shortcuts. Already built 
in unit always win and you cna only bind to already defined quantities.

Here's one for those who want to go beyond the normal:

\newdimension \FooA   \FooA 1.23pt
\newdimen     \FooB   \FooB 12.3pt
\protected\def\FooC   {\the\dimexpr\FooA +\FooB\relax}

So you can now figure how 'cx' is defined in a dynamic way.


Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : https://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : https://contextgarden.net
___________________________________________________________________________________