texexec with 2 processors?

texexec with 2 processors?

[Xan]

Hi,

I have Intel Core Duo and when I run texexec the CPU resource in gnome 
is marked about 50%. Is it mean that texexec only use one processor? If 
it's, can you think to improve it for scalability?

Just a curious question.

Xan.

PS: Please, CCme.
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: texexec with 2 processors?

[Taco Hoekwater]

Xan wrote:
> Hi,
> 
> I have Intel Core Duo and when I run texexec the CPU resource in gnome 
> is marked about 50%. Is it mean that texexec only use one processor? If 
> it's, can you think to improve it for scalability?
> 
> Just a curious question.

Run two jobs at once ;-)

Seriously: no, typesetting (currently) is a single-process streaming
task, and that is not easily fixed at all.

Best wishes,
Taco
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: texexec with 2 processors?

[Xan]

En/na Taco Hoekwater ha escrit:
> Xan wrote:
>> Hi,
>>
>> I have Intel Core Duo and when I run texexec the CPU resource in 
>> gnome is marked about 50%. Is it mean that texexec only use one 
>> processor? If it's, can you think to improve it for scalability?
>>
>> Just a curious question.
>
> Run two jobs at once ;-)
>
> Seriously: no, typesetting (currently) is a single-process streaming
> task, and that is not easily fixed at all.
>
> Best wishes,
> Taco
Confirmed so :-)
Yes, I understand it's not easy to "fix" it. I think it's a huge code task.

It could be useful for offering typesetting services. I'm thinking about 
having a computation service like alpha.wolfram but with ConTeXt. Like 
ConTeXt online but benefit from multiple processors....

Joking, perhaps you could fix it in Mark VII ;-)

Regards,
Xan.


___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: texexec with 2 processors?

[Mojca Miklavec]

On Fri, Aug 28, 2009 at 00:34, Xan wrote:
> En/na Taco Hoekwater ha escrit:
>>
>> Xan wrote:
>>>
>>> Hi,
>>>
>>> I have Intel Core Duo and when I run texexec the CPU resource in gnome is
>>> marked about 50%. Is it mean that texexec only use one processor? If it's,
>>> can you think to improve it for scalability?
>>>
>>> Just a curious question.
>>
>> Run two jobs at once ;-)
>>
>> Seriously: no, typesetting (currently) is a single-process streaming
>> task, and that is not easily fixed at all.
>>
>> Best wishes,
>> Taco
>
> Confirmed so :-)
> Yes, I understand it's not easy to "fix" it. I think it's a huge code task.

You cannot start processing page 500 before you know where page 499
wil end. TeX is not really a kind of application where you would gain
a lot by parallelization. And honestly, I don't remember seing many
applications using both cores. (Plus: I'm happy if other applications
continue to run smoothly instead of being blocked by TeX using 95% of
both processors.)

> It could be useful for offering typesetting services. I'm thinking about
> having a computation service like alpha.wolfram but with ConTeXt. Like
> ConTeXt online but benefit from multiple processors....

If you would offer a service, you would get multiple requests at the
same time anyway, so there's no real need for that in this case
(you'll face many more serious problems when offering typesetting
service).

> Joking, perhaps you could fix it in Mark VII ;-)

You can try XeTeX if you want to put load on both processors. It does
offer a parallel process as far as I heard, but then you'll probably
want support for quad-core once you get a better computer :)

Mojca
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: texexec with 2 processors?

[luigi scarso]

> You cannot start processing page 500 before you know where page 499
> wil end. TeX is not really a kind of application where you would gain
> a lot by parallelization. And honestly, I don't remember seing many
> applications using both cores. (Plus: I'm happy if other applications
> continue to run smoothly instead of being blocked by TeX using 95% of
> both processors.)
Hm, not so sure. Think for example to a book with 3 chapters, and you
know at priori that there are not relations
among them (references etc).
You can
1) typeset each of them in a concurrent way,
2) recompose the final document to correct pagenumbers and build indeces

(this is a TeX concurrency )

>
>> It could be useful for offering typesetting services. I'm thinking about
>> having a computation service like alpha.wolfram but with ConTeXt. Like
>> ConTeXt online but benefit from multiple processors....
>
> If you would offer a service, you would get multiple requests at the
> same time anyway, so there's no real need for that in this case
It's another kind of concurrency, it's about servers and OS (apache -
Linux vs IIE - Windows server etc)

> You can try XeTeX if you want to put load on both processors. It does
> offer a parallel process as far as I heard, but then you'll probably
> want support for quad-core once you get a better computer :)
Again, another kind of concurrency.
it's about luatex or pdftext or xetex C programs


-- 
luigi
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: texexec with 2 processors?

[Taco Hoekwater]

luigi scarso wrote:
>> You cannot start processing page 500 before you know where page 499
>> wil end.  []

> Hm, not so sure. Think for example to a book with 3 chapters, and you
> know at priori that there are not relations

The root of the problem is that because TeX is a programming language
as well as a typesetting engine, there is no way of knowing what
the state of the engine will be after the next token has been read,
never mind the next chapter.

Mojca wrote:
>> You can try XeTeX if you want to put load on both processors. It does
>> offer a parallel process as far as I heard

Xetex runs the typesetting engine and the PDF generation backend in
separate processes. Luatex may eventually start doing that also.

Best wishes,
Taco


___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: texexec with 2 processors?

[luigi scarso]

On Fri, Aug 28, 2009 at 9:15 AM, Taco Hoekwater<taco@elvenkind.com> wrote:
> luigi scarso wrote:
>>>
>>> You cannot start processing page 500 before you know where page 499
>>> wil end.  []
>
>> Hm, not so sure. Think for example to a book with 3 chapters, and you
>> know at priori that there are not relations
>
> The root of the problem is that because TeX is a programming language
> as well as a typesetting engine, there is no way of knowing what
> the state of the engine will be after the next token has been read,
> never mind the next chapter.

I'm think at something like this
%%
here Global macros and
data
(better if readonly)
%%
\StartTask
\chaper{..}
:
\StopTask

\StartTask
\chaper{..}
:
\StopTask

\StartTask
\chaper{..}
:
\StopTask

ie **you** marked that contents between \StartTask and \StopTask are
suitable for concurrency


-- 
luigi
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: texexec with 2 processors?

[Xan]

En/na Mojca Miklavec ha escrit:
>   
>
> You cannot start processing page 500 before you know where page 499
> wil end. TeX is not really a kind of application where you would gain
> a lot by parallelization. And honestly, I don't remember seing many
> applications using both cores. (Plus: I'm happy if other applications
> continue to run smoothly instead of being blocked by TeX using 95% of
> both processors.)
>   
I did not say to process page 500 before the page 499. I just say use 
all power of the multiple processors.
>   
>> It could be useful for offering typesetting services. I'm thinking about
>> having a computation service like alpha.wolfram but with ConTeXt. Like
>> ConTeXt online but benefit from multiple processors....
>>     
>
> If you would offer a service, you would get multiple requests at the
> same time anyway, so there's no real need for that in this case
> (you'll face many more serious problems when offering typesetting
> service).
>
>   
You are right here.
>> Joking, perhaps you could fix it in Mark VII ;-)
>>     
>
> You can try XeTeX if you want to put load on both processors. It does
> offer a parallel process as far as I heard,
Is it true?
>  but then you'll probably
> want support for quad-core once you get a better computer :)
>
> Mojca
>   
Xan.
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: texexec with 2 processors?

[Mojca Miklavec]

On Fri, Aug 28, 2009 at 10:26, Xan wrote:
> En/na Mojca Miklavec ha escrit:
>>
>>
>> You cannot start processing page 500 before you know where page 499
>> wil end. TeX is not really a kind of application where you would gain
>> a lot by parallelization. And honestly, I don't remember seing many
>> applications using both cores. (Plus: I'm happy if other applications
>> continue to run smoothly instead of being blocked by TeX using 95% of
>> both processors.)
>>
>
> I did not say to process page 500 before the page 499. I just say use all
> power of the multiple processors.

Well, yes. But you need to delegate a time consuming task A to
processor 1 and another time consuming task B to processor 2 where
both tasks need to be independent from each other and then you may
join the results at the end, else you spend more resources for
communication between processors than for actual work.

>> You can try XeTeX if you want to put load on both processors. It does
>> offer a parallel process as far as I heard,
>
> Is it true?

As Taco confirmed, it seem to run xetex and xdipdfmx in separate processes.

Mojca
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: texexec with 2 processors?

[Xan]

En/na Mojca Miklavec ha escrit:
> Well, yes. But you need to delegate a time consuming task A to
> processor 1 and another time consuming task B to processor 2 where
> both tasks need to be independent from each other and then you may
> join the results at the end, else you spend more resources for
> communication between processors than for actual work.
>
>   

Mmm..... it should be analyzed ;-)

Well, thanks a lot for the discussion,
Xan.
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: texexec with 2 processors?

[Hans Hagen]

Mojca Miklavec wrote:

> As Taco confirmed, it seem to run xetex and xdipdfmx in separate processes.

another example is running luatex in a virtual machine using one cpu in 
which case some file/disk io is done by an other (due the layered disk 
handling)

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________