bib module in pdftex and luatex

bib module in pdftex and luatex

[Thomas A. Schmitz]

Hi all (esp. Taco ;-),

here's a very fundamental question about the bib-module: my main bib  
file now has ~ 1,700 entries. When I tried to compile a file which  
includes bibliographic references in mkii, I got this error:

! TeX capacity exceeded, sorry [hash size=60000].
\@@shortsectionnumber ...\c!conversion \endcsname
                                                   \relax \@EA \ifx  
\csname \...

\@@longsectionnumber ...\@@shortsectionnumber {#1}
                                                   \fi
\finalsectionnumber ...alue {\@@sectie \c!number }
                                                   \fi \fi
\dodododoconstructhead ...er {\someheadconversion
                                                   }\getvalue {\??ko  
#1\c!inb...
l.162 \section{Textausgaben}

!  ==> Fatal error occurred, no output PDF file produced!

The same file compiles without errors in mkiv. So I'm curious: Is it  
possible that the bib file is just too large for mkii? Or is it more  
probable that there's some error in it such as missing crossref etc?  
Does anyone have experience with large bib files and the bib module?

Thanks, and best wishes

Thomas
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bib module in pdftex and luatex

[Hans Hagen]

Thomas A. Schmitz wrote:
> Hi all (esp. Taco ;-),
> 
> here's a very fundamental question about the bib-module: my main bib  
> file now has ~ 1,700 entries. When I tried to compile a file which  
> includes bibliographic references in mkii, I got this error:
> 
> ! TeX capacity exceeded, sorry [hash size=60000].
> \@@shortsectionnumber ...\c!conversion \endcsname
>                                                    \relax \@EA \ifx  
> \csname \...
> 
> \@@longsectionnumber ...\@@shortsectionnumber {#1}
>                                                    \fi
> \finalsectionnumber ...alue {\@@sectie \c!number }
>                                                    \fi \fi
> \dodododoconstructhead ...er {\someheadconversion
>                                                    }\getvalue {\??ko  
> #1\c!inb...
> l.162 \section{Textausgaben}
> 
> !  ==> Fatal error occurred, no output PDF file produced!
> 
> The same file compiles without errors in mkiv. So I'm curious: Is it  
> possible that the bib file is just too large for mkii? Or is it more  
> probable that there's some error in it such as missing crossref etc?  
> Does anyone have experience with large bib files and the bib module?

you can increase the hash size in texmf.cnf ... if so, then also remake 
the format

in mkiv we need less hash space if only because it got rid of encodings 
and regimes

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bib module in pdftex and luatex

[Thomas A. Schmitz]

On Dec 21, 2007, at 1:47 PM, Hans Hagen wrote:

> you can increase the hash size in texmf.cnf ... if so, then also  
> remake
> the format
>
> in mkiv we need less hash space if only because it got rid of  
> encodings
> and regimes

Indeed, increasing the hash_extra size and rebuilding the formats  
worked. Remarkable that mkiv is already more efficient wrt hash space!

Thanks, and best

Thomas
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bib module in pdftex and luatex

[Taco Hoekwater]

Thomas A. Schmitz wrote:
> 
> The same file compiles without errors in mkiv. So I'm curious: Is it  
> possible that the bib file is just too large for mkii? 

Just to be complete: In general, the answer to that is "yes", even for
mkiv.

Best wishes,
Taco


___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bib module in pdftex and luatex

[Thomas A. Schmitz]

On Dec 21, 2007, at 2:37 PM, Taco Hoekwater wrote:

> Just to be complete: In general, the answer to that is "yes", even for
> mkiv.
>
> Best wishes,
> Taco
>

Hmm, okay, thanks, Taco. Any rule of thumb for just how big is too  
big? And what do people do in cases like this? One of the wonderful  
things about bibtex for me was that I could build up one big database  
which I could reuse for all my needs.

Best

Thomas
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bib module in pdftex and luatex

[Taco Hoekwater]

Thomas A. Schmitz wrote:
> On Dec 21, 2007, at 2:37 PM, Taco Hoekwater wrote:
> 
>> Just to be complete: In general, the answer to that is "yes", even for
>> mkiv.
>>
>> Best wishes,
>> Taco
>>
> 
> Hmm, okay, thanks, Taco. Any rule of thumb for just how big is too big? 

Depends on your texmf.cnf. IIRC, I have twenty or so csnames per
entry, and the baseline for mkii is a bit over 40.000 csnames,
so you need about

    40.000 + 20 * (numberofentries)

> And what do people do in cases like this? One of the wonderful  

Increase hash_extra, as Hans said. There is (virtuallly) no limit
to that.

Best wishes,
Taco

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bib module in pdftex and luatex

[Thomas A. Schmitz]

On Dec 21, 2007, at 4:56 PM, Taco Hoekwater wrote:

> Depends on your texmf.cnf. IIRC, I have twenty or so csnames per
> entry, and the baseline for mkii is a bit over 40.000 csnames,
> so you need about
>
>    40.000 + 20 * (numberofentries)
>
>> And what do people do in cases like this? One of the wonderful
>
> Increase hash_extra, as Hans said. There is (virtuallly) no limit
> to that.
>
> Best wishes,
> Taco

Ah, okay! Thanks a lot. So I will pit my ever larger and larger bib  
files against the ever more powerful computers that we will have in  
the future and let them fight it out!

Thomas
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: bib module in pdftex and luatex

[Hans Hagen]

Thomas A. Schmitz wrote:

> Ah, okay! Thanks a lot. So I will pit my ever larger and larger bib  
> files against the ever more powerful computers that we will have in  
> the future and let them fight it out!

we can manage the bib data in lua and do all bibtex actions there,

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
      tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                              | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________