Invalid UTF-8 sequence in bbl file

Invalid UTF-8 sequence in bbl file

[Oleg Kolosov]

[-- Attachment #1: Type: text/plain, Size: 659 bytes --]

Hi all, Taco!

I was experimenting with bib module and discovered something which looks like
a bug.

'texexec' ('texexec bibtest') runs fine but typesets references ('s' key)
incorrectly.

As MkIV does not run bibtex on input files automatically, I run 'bibtex'
manually once ('bibtex bibtest'), then 'context' ('context bibtest') gives
'Invalid UTF-8 sequence' error.

The resulting .bbl is the same in both cases.

It seems that 'author' field and 's' key in bbl file are generated
incorrectly, while others like 'title', 'city', etc. are fine.

See the attached files.

I'm using just updated minimals. t-bib version 2008.04.17.

-- 
Regards,
Oleg Kolosov

[-- Attachment #2: bibtest.bbl --]
[-- Type: text/plain, Size: 351 bytes --]

\setuppublicationlist[samplesize={АÐ02},totalnumber=1]

\startpublication[k=parintro,t=book,
a={{Антонов}},y=2002,
n=1,s=АÐ02]
\author[]{А.~С.}[Ð.~Ð.]{}{Антонов}
\pubyear{2002}
\title{Введение в параллельные вычисления}
\city{Москва}
\pubname{МГУ им. Ломоносова}
\stoppublication


[-- Attachment #3: bibtest.tex --]
[-- Type: text/x-tex, Size: 176 bytes --]

\enableregime[utf]
\useregime[utf]

\usemodule[bib]
\setupbibtex[database=bibtest.bib]

\starttext
\section Bibmodule test

Cite\cite[parintro].

\placepublications

\stoptext

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: bibtest.bib --]
[-- Type: text/x-bibtex; name="bibtest.bib", Size: 306 bytes --]

@book{parintro,
  author = "А. С. Антонов",
  title = "Введение в параллельные вычисления",
  type = "Методическое пособие",
  pages = "69",
  address = "Москва",
  publisher = "МГУ им. Ломоносова",
  year = "2002"
}

[-- Attachment #5: Type: text/plain, Size: 487 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Invalid UTF-8 sequence in bbl file

[Taco Hoekwater]

Oleg Kolosov wrote:

> 'texexec' ('texexec bibtest') runs fine but typesets references ('s' key)
> incorrectly.
> 
> As MkIV does not run bibtex on input files automatically, I run 'bibtex'
> manually once ('bibtex bibtest'), then 'context' ('context bibtest') gives
> 'Invalid UTF-8 sequence' error.

This is caused by the fact that bibtex doesn't understand UTF-8 at all
(it is only able to do something useful with byte encodings). I have
no immediate solution for you, the best advice I can offer right now is
for you to manually fix the bbl file after running bibtex.

The whole database is converted in one go, and there is never a need to
re-run bibtex unless when you delete the bbl by accident, so this is
a single (laborious) step (under the assumption that the bib file is
finished already). A long-term solution will depend on a more modern
bib database processor (like J.M. Hufflen's mlbibtex).

Best wishes,
Taco
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Invalid UTF-8 sequence in bbl file

[Oleg Kolosov]

Taco Hoekwater wrote:
>> As MkIV does not run bibtex on input files automatically, I run 'bibtex'
>> manually once ('bibtex bibtest'), then 'context' ('context bibtest') gives
>> 'Invalid UTF-8 sequence' error.
> 
> This is caused by the fact that bibtex doesn't understand UTF-8 at all
> (it is only able to do something useful with byte encodings). I have
> no immediate solution for you, the best advice I can offer right now is
> for you to manually fix the bbl file after running bibtex.
> 
> The whole database is converted in one go, and there is never a need to
> re-run bibtex unless when you delete the bbl by accident, so this is
> a single (laborious) step (under the assumption that the bib file is
> finished already). A long-term solution will depend on a more modern
> bib database processor (like J.M. Hufflen's mlbibtex).
> 

I didn't think about bibtex as a separate package from old times, was too
carried away by luatex fancies. Thanks for clarifying this out.

I've used the following workaround: write bib file in 8-bit encoding (koi8-r
for example), run bibtex on it once, then convert bbl to UTF-8. This works
perfectly, should have guessed it earlier. I don't change bib files
frequently, so this solution suits me well.

-- 
Regards,
Oleg Kolosov
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________