[NTG-context] Why LuaMetaTex is so slow?

[NTG-context] Why LuaMetaTex is so slow?

[Shiv Shankar Dayal]

[-- Attachment #1.1: Type: text/plain, Size: 234 bytes --]

I can process TeXBook in almost an instant using PDFTeX which is about 450+
pages but my math book takes around 30 seconds averaging ~40 pages/second.

Why LuaMetaTeX is so slow? How can I speed it up?
-- 
Respect,
Shiv Shankar Dayal

[-- Attachment #1.2: Type: text/html, Size: 470 bytes --]

[-- Attachment #2: Type: text/plain, Size: 511 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Pablo Rodriguez via ntg-context]

On 1/18/24 18:24, Shiv Shankar Dayal wrote:
> I can process TeXBook in almost an instant using PDFTeX which is about
> 450+ pages but my math book takes around 30 seconds averaging ~40
> pages/second.

Hi Shiv Shankar Dayal,

I guess that “The TeXbook” (having been written by Knuth himself) may be
optimized for speed.

Your math book may use features (I’m guessing again) that require more
resources than the features used by “The TeXbook”.

> Why LuaMetaTeX is so slow? How can I speed it up?

https://www.pragma-ade.com/general/manuals/musings.pdf#page=96 might
explain why Lua(Meta)TeX is slower.

Other chapters in that document from Hans might give you some insight on
the reasons why LMTX has to pay a price for being more feature-rich. I
got such impression when I read it.

Just in case it might help,

Pablo
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Henning Hraban Ramm]

Am 18.01.24 um 18:47 schrieb Pablo Rodriguez via ntg-context:
> On 1/18/24 18:24, Shiv Shankar Dayal wrote:
>> I can process TeXBook in almost an instant using PDFTeX which is about
>> 450+ pages but my math book takes around 30 seconds averaging ~40
>> pages/second.
> 
> Hi Shiv Shankar Dayal,
> 
> I guess that “The TeXbook” (having been written by Knuth himself) may be
> optimized for speed.
> 
> Your math book may use features (I’m guessing again) that require more
> resources than the features used by “The TeXbook”.
> 
>> Why LuaMetaTeX is so slow? How can I speed it up?
> 
> https://www.pragma-ade.com/general/manuals/musings.pdf#page=96 might
> explain why Lua(Meta)TeX is slower.
> 
> Other chapters in that document from Hans might give you some insight on
> the reasons why LMTX has to pay a price for being more feature-rich. I
> got such impression when I read it.
> 
> Just in case it might help,

In comparison to LaTeX with LuaHBTeX, ConTeXt with LuaMetaTeX is 
lightning fast.

pdfTeX is 8bit, and Knuth’s plain TeX is very low level – for a fairer 
comparison, try the TeXbook on LuaMetaTeX (I don’t know if “plain” works 
though).

Hraban
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Shiv Shankar Dayal]

[-- Attachment #1.1: Type: text/plain, Size: 448 bytes --]

In comparison to LaTeX with LuaHBTeX, ConTeXt with LuaMetaTeX is
> lightning fast.
>
> pdfTeX is 8bit, and Knuth’s plain TeX is very low level – for a fairer
> comparison, try the TeXbook on LuaMetaTeX (I don’t know if “plain” works
> though).
>

I understand that why it is slow, but the problem is that the difference is
too high.
My book has lots of math, so perhaps that is the reason.

-- 
Respect,
Shiv Shankar Dayal

[-- Attachment #1.2: Type: text/html, Size: 871 bytes --]

[-- Attachment #2: Type: text/plain, Size: 511 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Mikael Sundqvist]

Hi,

On Thu, Jan 18, 2024 at 7:58 PM Shiv Shankar Dayal
<shivshankar.dayal@gmail.com> wrote:
>
>
>
>
>> In comparison to LaTeX with LuaHBTeX, ConTeXt with LuaMetaTeX is
>> lightning fast.
>>
>> pdfTeX is 8bit, and Knuth’s plain TeX is very low level – for a fairer
>> comparison, try the TeXbook on LuaMetaTeX (I don’t know if “plain” works
>> though).
>
>
> I understand that why it is slow, but the problem is that the difference is too high.
> My book has lots of math, so perhaps that is the reason.
>

For what it is worth, I have a math book (lots of math and references
back and forth and some figures) and the 300 pages compiles in around
10s, so 30 pages/s. I don't have too much to compare with, and that is
a bit slower than your 40 pages/s (but 40*30 is 1200 and not 450), but
I consider that pretty fast.

One thing you can try is \enableexperiments[fonts.compact] (as it
sounds, it is still experimental).

Also, if you do not provide any example code, it will be difficult for
people to guess how _you_ can speed up your compilation (if at all).

/Mikael
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Hans Hagen]

On 1/18/2024 7:56 PM, Shiv Shankar Dayal wrote:
> 
> 
> 
>     In comparison to LaTeX with LuaHBTeX, ConTeXt with LuaMetaTeX is
>     lightning fast.
> 
>     pdfTeX is 8bit, and Knuth’s plain TeX is very low level – for a fairer
>     comparison, try the TeXbook on LuaMetaTeX (I don’t know if “plain”
>     works
>     though).
> 
> I understand that why it is slow, but the problem is that the difference 
> is too high.
> My book has lots of math, so perhaps that is the reason.
You can run the texbook with \tracingall and see what actually happens.

The texbook uses plain tex (very lightweight) with a dedicated style 
(also lightweight). On the contrary context uses layers of abstraction, 
a more complex font, math, structure, page etc etc etc sytstem.

In fact, plain tex had to be as it is because otherwise it would take 
ages to process teh texbook. On a pc one could see the page numbers 
crawl so in the end wr're quite okay nowadays.

If you run context with \tracingall you'll see that more goes on.

For me 40 pps sounds not that bad. If you use only text you can get to 
500 pps but that's not realistic. Maybe your style is suboptimal.

Also, open type fonts, opentype math, unicode, color, etc etc comes at a 
price. And, compared to pdftex, in most cases luametatex is faster on 
documents of average complexity. If you also use xml and/or metapost you 
migh tbe dragged down to a few pages per second while lmtx doesn't 
suffer much.

Anyway, if you think it can be made faster you can always post code to 
this list that identifies bottlenecks.

Of course we'll never be as fast as (pure) plain tex (no layers added) 
or latex (which often is advocated as faster than context).

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Gavin via ntg-context]

Hi Shiv,

ConTeXt’s project structure has many benefits. One is speed. See https://wiki.contextgarden.net/Project_structure.

My books and long papers are products, each containing many components (chapters or sections). When I’m working, I have both the product and a component open. I frequently typeset the component as I’m working. This is very quick. I only typeset the entire product if I intend print or send it, or when I need to check some product-wide element like references. Components can be nested, so take this approach to whatever extreme you desire.

You can also use buffers if there are specific parts of your book (like diagrams) that demand significant processing. Buffers can save the typeset version of the contents. These will only be typeset again if you change the buffer contents (or if you are typesetting with a different version of ConTeXt.) See https://wiki.contextgarden.net/Command/typesetbuffer

My biggest book is about 200 pages. It takes 4 minutes to fully typeset on an M1 Mac. (I saw this time after downloading a new ConTeXt version.) This is mostly due to the 100+ complex TikZ diagrams. However, all of those diagrams are buffered, and when I’m working I only typeset the component. A typical chapter typesets in 1.6 seconds if I haven’t changed the diagrams. (I just did one to check.) That’s fast enough for me.

Hope that helps,
Gavin


> On Jan 18, 2024, at 10:24 AM, Shiv Shankar Dayal <shivshankar.dayal@gmail.com> wrote:
> 
> I can process TeXBook in almost an instant using PDFTeX which is about 450+ pages but my math book takes around 30 seconds averaging ~40 pages/second.
> 
> Why LuaMetaTeX is so slow? How can I speed it up?
> -- 
> Respect,
> Shiv Shankar Dayal
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
> webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
> archive  : https://github.com/contextgarden/context
> wiki     : https://wiki.contextgarden.net
> ___________________________________________________________________________________

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Joseph Wright]

On 18/01/2024 20:55, Gavin via ntg-context wrote:
> That’s fast enough for me.

That's surely the key statement - 'fast' here is a personal thing, and 
40 pages a second is fast enough for many people. It's certainly fast 
when you look at what LuaMetaTeX is doing, as Hans has said.

Joseph

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Bruce Horrocks]

On 18 Jan 2024, at 18:56, Shiv Shankar Dayal <shivshankar.dayal@gmail.com> wrote:
> 
> I understand that why it is slow, but the problem is that the difference is too high.
> My book has lots of math, so perhaps that is the reason.


Option 1: Buy or borrow a faster computer. You'll get your desired speed-up far faster than waiting for Hans to make changes, even assuming there is something that can be changed. I'd recommend a Mac M3 :-)

Option 2: Split your book into a project with each chapter, or even part of a chapter as a separate component and compile only the chapter you are working on. Set up a nighttime (or lunchtime) run to regenerate the entire book while you're away from the computer and speed doesn't matter.

—
Bruce Horrocks
Hampshire, UK

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Aditya Mahajan]

On Thu, 18 Jan 2024, Hans Hagen wrote:

> Of course we'll never be as fast as (pure) plain tex (no layers added) or
> latex (which often is advocated as faster than context).

I was curious, so did a quick experiment: typeset \input knuth 500 times. Here are the results:

pdflatex: 547 pages/sec
lualatex: 308 pages/sec

MKIV: 130 pages/sec
LMTX: 147 pages/sec

Interestingly, pdflatex, lualatex, and MkIV all generated pdf with 1112 pages, while LMTX generated a pdf with 1250 pages. Both MkIV and LMTX take the same time to run (8.5 sec) but LMTX generates more pages so it is faster. 

Of course, this is a meaningless experiment from a practical point of view as a 1000 page document will be more complex than just text.

Aditya
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Hans Hagen via ntg-context]

On 1/19/2024 12:34 AM, Aditya Mahajan wrote:
> On Thu, 18 Jan 2024, Hans Hagen wrote:
> 
>> Of course we'll never be as fast as (pure) plain tex (no layers added) or
>> latex (which often is advocated as faster than context).
> 
> I was curious, so did a quick experiment: typeset \input knuth 500 times. Here are the results:
> 
> pdflatex: 547 pages/sec
> lualatex: 308 pages/sec

opentype or type 1 ?

> MKIV: 130 pages/sec
> LMTX: 147 pages/sec

that is weird as lmtx is always faster here (we loose some 15% on the 
backend compared to luatex, maybe even more, but we gain back quite a 
bit elsewhere for various reasons)

> Interestingly, pdflatex, lualatex, and MkIV all generated pdf with 1112 pages, while LMTX generated a pdf with 1250 pages. Both MkIV and LMTX take the same time to run (8.5 sec) but LMTX generates more pages so it is faster.

puzzling

> Of course, this is a meaningless experiment from a practical point of view as a 1000 page document will be more complex than just text.
% engine=pdftex      1.2
% engine=luatmetatex 1.6
% engine=luatex      1.8
% engine=xetex       2.4

\starttext

\dorecurse{500} {
     \input knuth \par
}

\stoptext

125 pages, so how does your test file look

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Aditya Mahajan]

On Fri, 19 Jan 2024, Hans Hagen via ntg-context wrote:

> On 1/19/2024 12:34 AM, Aditya Mahajan wrote:
> > On Thu, 18 Jan 2024, Hans Hagen wrote:
> > 
> >> Of course we'll never be as fast as (pure) plain tex (no layers added) or
> >> latex (which often is advocated as faster than context).
> > 
> > I was curious, so did a quick experiment: typeset \input knuth 500 times.
> > Here are the results:
> > 
> > pdflatex: 547 pages/sec
> > lualatex: 308 pages/sec
> 
> opentype or type 1 ?

type 1. I did not change anything in the defaults. I'll try to figure out how to load opentype fonts in latex, and test lualatex. 

> > MKIV: 130 pages/sec
> > LMTX: 147 pages/sec
> 
> that is weird as lmtx is always faster here (we loose some 15% on the backend
> compared to luatex, maybe even more, but we gain back quite a bit elsewhere
> for various reasons)

LMTX is faster for me as well.

> > Interestingly, pdflatex, lualatex, and MkIV all generated pdf with 1112
> > pages, while LMTX generated a pdf with 1250 pages. Both MkIV and LMTX take
> > the same time to run (8.5 sec) but LMTX generates more pages so it is
> > faster.
> 
> puzzling
> 
> > Of course, this is a meaningless experiment from a practical point of view
> > as a 1000 page document will be more complex than just text.
> % engine=pdftex      1.2
> % engine=luatmetatex 1.6
> % engine=luatex      1.8
> % engine=xetex       2.4
> 
> \starttext
> 
> \dorecurse{500} {
>     \input knuth \par
> }
> 
> \stoptext
> 
> 125 pages, so how does your test file look

5000 recursions (not 500, sorry about that). 


\starttext

\dorecurse{5000}{\ReadFile{knuth}}

\stoptext

I'll send you the PDFs offlist. 

Aditya
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Hans Hagen]

On 1/19/2024 1:33 AM, Aditya Mahajan wrote:

> \starttext
> 
> \dorecurse{5000}{\ReadFile{knuth}}
> 
> \stoptext
> 
> I'll send you the PDFs offlist.
lmtx defaults to 'modern' so you need to

\setupbodyfont[modern]

for mkiv

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Hans Hagen]

On 1/18/2024 11:15 PM, Bruce Horrocks wrote:
> On 18 Jan 2024, at 18:56, Shiv Shankar Dayal <shivshankar.dayal@gmail.com> wrote:
>>
>> I understand that why it is slow, but the problem is that the difference is too high.
>> My book has lots of math, so perhaps that is the reason.
> 
> 
> Option 1: Buy or borrow a faster computer. You'll get your desired speed-up far faster than waiting for Hans to make changes, even assuming there is something that can be changed. I'd recommend a Mac M3 :-)

As tex is a single core process (ok, the os might delegate some file 
handling to other cores) it benefits from  jump in core speed. I use a 
2017 laptop with a reasonable fast intel xeon.  I have a few docks 
spread round with different resolution monitors (1920x1200, 2560×1440, 
4096×2160) and performance also depends on the connected monitor.

I'd be curious to know how much these new processors impact performance 
over time because the ages of dohbling performance every year are past.

There are several factors that impact performance:

tex engine : quite a bit faster in luametatex
mp  engine : quite a bit faster in luametatex
lua engine : not much we can do about

but inefficient macros and usage can offset any gain in engine.

One thing to keep in mind is that tex is an interpreter and all gets 
tokenized which is more costly in a 32 bit engine than in an 8 bit one 
(like pdftex). And although luametatex is more efficient than luatex 
that gets offset by more features that come at a price, but we still 
win. It helps a bit to know what tex does and what macros add to that. 
Simple timing operations is useless as in practice we have less cache 
hits than with a simple loop test: tex is jumping all over memory. 
Bottlenecks are often noe where one expects them.

Anyway, if one can choose: go for a few high perfotmance cores over 
dozens of so called efficient cores.

(that said: luametatex runs fine on my mobile phone)

> Option 2: Split your book into a project with each chapter, or even part of a chapter as a separate component and compile only the chapter you are working on. Set up a nighttime (or lunchtime) run to regenerate the entire book while you're away from the computer and speed doesn't matter.
Indeed. That's the natural way to deal with large documents.

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Henning Hraban Ramm]

Am 19.01.24 um 10:31 schrieb Hans Hagen:
> (that said: luametatex runs fine on my mobile phone)

What’s the smallest/oldest device someone tried LMTX on?
(We were joking about “Hans’ thermostate and Alan’s fridge“…)
Would it run on some 8 bit homecomputer? (I’d guess the RAM is too 
limited, and loading from datasette would need hours…)

Just curious.

Hraban
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Taco Hoekwater]

> On 19 Jan 2024, at 11:00, Henning Hraban Ramm <texml@fiee.net> wrote:
> 
> Am 19.01.24 um 10:31 schrieb Hans Hagen:
>> (that said: luametatex runs fine on my mobile phone)
> 
> What’s the smallest/oldest device someone tried LMTX on?
> (We were joking about “Hans’ thermostate and Alan’s fridge“…)
> Would it run on some 8 bit homecomputer? (I’d guess the RAM is too limited, and loading from datasette would need hours…)

No. You need integers (and pointers) to be at least 32 bits wide. And you also need about 90Mb of RAM for the initial malloc()s.

Historical 8-bit and 16-bit CPUs will not work. A modern fridge might, though. :)

— 
Taco Hoekwater              E: taco@bittext.nl
genderfluid (all pronouns)


___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Hans Hagen]

On 1/19/2024 11:00 AM, Henning Hraban Ramm wrote:
> Am 19.01.24 um 10:31 schrieb Hans Hagen:
>> (that said: luametatex runs fine on my mobile phone)
> 
> What’s the smallest/oldest device someone tried LMTX on?
> (We were joking about “Hans’ thermostate and Alan’s fridge“…)
> Would it run on some 8 bit homecomputer? (I’d guess the RAM is too 
> limited, and loading from datasette would need hours…)
> 
> Just curious.
The smallest i have here is a https://en.wikipedia.org/wiki/RCA_1802 
which actually was a very nicely designed thing (also powers the 
hubble); processors never became cleaner. Now, as the kit I used only 
had 512 bytes ram (later 2K or so) it really learns one how to program 
efficiently (thihk of writing a simple game, save and load from cassette 
player, etc in that space; dynamic adaptive functions and such) and you 
know where I'm coming from'(later pascal and modula).

That said: luametatex has a smaller memory footprint than luatex 
although nodes are way bigger but even that won't make it run well on an 
8 bit processor, assuming it can address 32 bit mem.

Context mkii started on 4.7 Mhz intel (386) so now we're 1000 times 
faster but performance gain is not linear. After all, tex has to compete 
with other applications running at the same time (browsers, remote 
clients, media apps, web polling apps, zoom, etc.) so it's kin dof 
amazing that it actually performs okay.

What boosted performance is for instance ssd's. What might boost is ddr5 
memory but that has to catch up with ddr4 first. What also boosted 
luametatex is cpu cache. I remember when taco and I moved to the math 
part we noticed a drop in performance, until machines got updated with 
larger cache. We'd just crossed some magic threshold.

Actually, I kind of assume that we're stuck with cpu performance but 
nevertheless luametatex became faster. It helps to develop on an older 
machine and not on a yearly updated one as that obscures matters. (Not 
that I'd not like a faster machine, but that comes at a steep price.)

Hans

ps. In the tranmsition from pdftex to luatex to luametatex one should 
not underestimate Taco's effort of going from basically unreadable and 
useless C output by web2c (pascal -> c) to what became the luatex C code 
base (with embedded original documentation). From that we added more 
functionality gradually over a decade till luametatex spun off, which 
then lead to quite a bit of refactoring again (but trying to maintain 
the spirit of how tex is programmed).


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Bruce Horrocks]

> On 19 Jan 2024, at 09:31, Hans Hagen <j.hagen@xs4all.nl> wrote:
> 
> On 1/18/2024 11:15 PM, Bruce Horrocks wrote:
>> On 18 Jan 2024, at 18:56, Shiv Shankar Dayal <shivshankar.dayal@gmail.com> wrote:
>>> 
>>> I understand that why it is slow, but the problem is that the difference is too high.
>>> My book has lots of math, so perhaps that is the reason.
>> Option 1: Buy or borrow a faster computer. You'll get your desired speed-up far faster than waiting for Hans to make changes, even assuming there is something that can be changed. I'd recommend a Mac M3 :-)
> 
> As tex is a single core process (ok, the os might delegate some file handling to other cores) it benefits from  jump in core speed. I use a 2017 laptop with a reasonable fast intel xeon.  I have a few docks spread round with different resolution monitors (1920x1200, 2560×1440, 4096×2160) and performance also depends on the connected monitor.
> 
> I'd be curious to know how much these new processors impact performance over time because the ages of dohbling performance every year are past.

I mentioned the Mac M3 because it's not the cores that make the difference but the speed of the SSD. So reading the source and writing out the result, plus writing/reading any intermediate files is astonishingly fast on the Mac. Also the RAM is on the same die as the chip so RAM access speeds are high. Even if the app is single-threaded, the speed increases are significant - in fact especially when the app is single is threaded. :-)

—
Bruce Horrocks
Hampshire, UK

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________

[NTG-context] Re: Why LuaMetaTex is so slow?

[Hans Hagen]

On 1/19/2024 7:13 PM, Bruce Horrocks wrote:
> 
> 
>> On 19 Jan 2024, at 09:31, Hans Hagen <j.hagen@xs4all.nl> wrote:
>>
>> On 1/18/2024 11:15 PM, Bruce Horrocks wrote:
>>> On 18 Jan 2024, at 18:56, Shiv Shankar Dayal <shivshankar.dayal@gmail.com> wrote:
>>>>
>>>> I understand that why it is slow, but the problem is that the difference is too high.
>>>> My book has lots of math, so perhaps that is the reason.
>>> Option 1: Buy or borrow a faster computer. You'll get your desired speed-up far faster than waiting for Hans to make changes, even assuming there is something that can be changed. I'd recommend a Mac M3 :-)
>>
>> As tex is a single core process (ok, the os might delegate some file handling to other cores) it benefits from  jump in core speed. I use a 2017 laptop with a reasonable fast intel xeon.  I have a few docks spread round with different resolution monitors (1920x1200, 2560×1440, 4096×2160) and performance also depends on the connected monitor.
>>
>> I'd be curious to know how much these new processors impact performance over time because the ages of dohbling performance every year are past.
> 
> I mentioned the Mac M3 because it's not the cores that make the difference but the speed of the SSD. So reading the source and writing out the result, plus writing/reading any intermediate files is astonishingly fast on the Mac. Also the RAM is on the same die as the chip so RAM access speeds are high. Even if the app is single-threaded, the speed increases are significant - in fact especially when the app is single is threaded. :-)

File caching helps a lot indeed. Just compare

   context --make

after a reboot with the second or third time after that.

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage  : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive  : https://github.com/contextgarden/context
wiki     : https://wiki.contextgarden.net
___________________________________________________________________________________