Re: TeX capacity exceeded

[Hans Hagen <j.hagen@xs4all.nl>]

On 5/17/2021 11:43 PM, Thomas A. Schmitz wrote:
> 
> Hi all,
> 
> I have a rather large (almost 120,000 lines) xml file, processing pulls 
> in another large xml file. The project compiles under mkiv; with lmtx, I 
> get
> 
> tex error       > tex error on line 8 in file : TeX capacity exceeded, 
> sorry [reference count=32767]

This is not the normal memory consumption that bothers you

\enabletrackers[system.usage]

will show you more detail. Normally the default context configuration 
will provide enough. In lmts memory is abit more dynamic so we start otu 
smaller and then stepwise grow. And one can configure even more 
(actually i just updated some configuration options for that).

Anyway: this more smells like some nesting going wrong and therefore 
excessive token sharing

> <macro> \reallanguagetag
> 
> just twenty pages shy of the end of the document.
> 
> There is an error log with this content:
> 
> return {
>   ["errortype"]=3,
>   ["filename"]="",
>   ["lastcontext"]="\n\n\n<macro> \\reallanguagetag \n    \n<macro> 
> \\reallanguagetag #1->\\ifcsname \\??languagelinked #1\n    \\endcsname 
> \\lastnamedcs \\else #1\\fi \n<macro> \\setusedlanguage #1->\\edef 
> \\currentusedlanguage {\\reallanguagetag {#1}\n    }\\ifempty 
> \\currentusedlanguage \\let \\currentusedlanguage \\currentlanguage 
> \\always\\let \\usedlanguageparameter \\languageparameter \\orelse \\ifx 
> \\currentusedlanguage \\v!global \\let \\currentusedlanguage\n\n<macro> 
> \\typo_delimited_push ...\\currentdelimitedtext {#1}\\edef 
> \\m_delimited_argument {#2}\\ifempty \\m_delimited_argument 
> \\typo_delimited_set_language_nop \n    \\else 
> \\typo_delimited_set_language_yes \\fi \\let 
> \\currentparentdelimitedtext \\currentdelimitedtext \\global \\advance 
> \\c_typo...\n\n<macro> \\delimitedtext [#1]#*[#2]->\\dontleavehmode 
> \\typo_delimited_push {#1}{#2}\n    \\edef \\p_delimited_method 
> {\\delimitedtextparameter \\c!method }\\ifx \\p_delimited_method 
> \\v!font \\expandafter \\typo_delimited_fontdriven \\else \\expandafter 
> \\typo_delimited_other \\fi \n\n<to be read again> \n 
> {\n...\n\n<initial> \n    ",
>   ["lastluaerror"]="",
>   ["lasttexerror"]="TeX capacity exceeded, sorry [reference count=32767]",
>   ["lasttexhelp"]="If you really absolutely need more capacity, you can 
> ask a wizard to enlarge me.",
>   ["linenumber"]=8,
>   ["offset"]=10,
>   ["skiplinenumber"]=8,
> }
> 
>   In mkiv, these are the stats at the end of the run:

mwe ...

> mkiv lua stats  > used config file: 
> selfautoparent:/texmf/web2c/texmfcnf.lua
> mkiv lua stats  > used cache path: 
> /mnt/shared/lmtx/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e 
> 
> mkiv lua stats  > resource resolver: loadtime 0.009 seconds, 1 scans 
> with scantime 0.005 seconds, 0 shared scans, 27 found files, scanned 
> paths: /home/tas/texmf
> mkiv lua stats  > stored bytecode data: 455 modules (0.122 sec), 96 
> tables (0.007 sec), 551 chunks (0.130 sec)
> mkiv lua stats  > traced context: maxstack: 1422, freed: 2, unreachable: 
> 1420
> mkiv lua stats  > cleaned up reserved nodes: 63 nodes, 9 lists of 442
> mkiv lua stats  > node memory usage: 17 attribute, 4 attribute_list, 1 
> glue, 51 glue_spec, 3 kern, 2 penalty, 2 temp, 2 user_defined
> mkiv lua stats  > node list callback tasks: 8 unique task lists, 9336 
> instances (re)created, 177413 calls
> mkiv lua stats  > h-node processing time: 4.800 seconds including kernel
> mkiv lua stats  > attribute processing time: 0.617 seconds front- and 
> backend
> mkiv lua stats  > driver time: 0.886 seconds
> mkiv lua stats  > used backend: pdf (backend for directly generating pdf 
> output)
> mkiv lua stats  > jobdata time: 0.161 seconds saving, 0.052 seconds loading
> mkiv lua stats  > callbacks: internal: 261417, file: 319, direct: 3, 
> late: 31831, function 200128, total: 493698 (464 per page)
> mkiv lua stats  > randomizer: resumed with value 0.47143577262518
> mkiv lua stats  > positions: 9535 collected, 9535 regular, 0 special
> mkiv lua stats  > v-node processing time: 0.401 seconds
> mkiv lua stats  > result saved in file: plutarchusvitae.pdf, 
> compresslevel 3, objectcompresslevel 1
> mkiv lua stats  > loaded patterns: de::3 en::1 la::65, load time: 0.000
> mkiv lua stats  > xml load time: 0.648 seconds, 4 files, 4 converted
> mkiv lua stats  > lxml preparation time: 0.626 seconds, 233814 nodes, 
> 578788 lpath calls, 578743 cached calls
> mkiv lua stats  > interactive elements: 0 references, 37 destinations
> mkiv lua stats  > margin data: 10798 entries, 0 pending
> mkiv lua stats  > loaded fonts: 9 files: garamondpremrpro.otf, 
> sabonltstd-bold.otf, sabonltstd-italic.otf, sabonltstd-roman.otf, 
> kadmosu.otf, oldstandard-bold.otf, oldstandard-italic.otf, 
> xitsmath-regular.otf, dejavusansmono.ttf
> mkiv lua stats  > font engine: otf 3.113, afm 1.513, tfm 1.000, 17 
> instances, 6 shared in backend, 0 common vectors, 6 common hashes, load 
> time 0.146 seconds
> mkiv lua stats  > used platform: linux-64, type: unix, binary subtree: 
> texmf-linux-64
> mkiv lua stats  > used engine: luatex version: 1.13, functionality 
> level: 7420, banner: this is luatex, version 1.13.0 (tex live 2021)
> mkiv lua stats  > used hash slots: 47131 of 65536 + 250000
> mkiv lua stats  > lua properties: engine: lua 5.3, used memory: 601 MB, 
> ctx: 579 MB, max: unknown MB, hash chars: min(64,40), symbol mask: utf 
> (τεχ)
> mkiv lua stats  > runtime: 22.310 seconds, 1068 processed pages, 1068 
> shipped pages, 47.871 pages/second
> system          | total runtime: 213.928 seconds

hardly demanding in terms of memory

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________