Problem processing a non-UTF8 Lua script

* Problem processing a non-UTF8 Lua script
@ 2010-10-21  8:00 Procházka Lukáš
  2010-10-21 15:49 ` Vedran Miletić
  0 siblings, 1 reply; 7+ messages in thread
From: Procházka Lukáš @ 2010-10-21  8:00 UTC (permalink / raw)
  To: ConTeXt

[-- Attachment #1: Type: text/plain, Size: 5957 bytes --]

Hello,

I have a problem with processing a Lua file similar to that with \typefile described several days (weeks?) ago.

Let's have the following files:

- Ctx:

---
\mainlanguage[cz]
\enableregime[cp1250]

\starttext
   AAA

   \startluacode
     local t = dofile("t.lua")

     context(t)
   \stopluacode
\stoptext
---

- And the t.lua:

---
-- return "111" -- This worked OK
return "žšč" -- This is problem for Ctx
---

When I try the t.lua from Win console by "Lua -e dofile('t.lua')", there is no problem.

When I run the Ctx file, it has problem processing characters with diacritics (like "čřž..."); the t.lua is not encoded in UTF-8 (which seems to be supposed by Ctx-Lua), but I'm using the cp1250 code page + each character = 1 B.

The error message is:

---
MTXrun | run 1: luatex --fmt="c:/ConTeXt/tex/texmf-cache/luatex-cache/context/f53042fa2e1c106bc7e3383ec8c3a00c/formats/cont-en" --lua="c:/ConTeXt/tex/texmf-cache/luatex-cache/context/f53042fa2e1c106bc7e3383ec8c3a00c/formats/cont-en.lui" --backend=pdf "D:/L
ukas/ConTeXt/Samples/U8/t-U8.mkiv"This is LuaTeX, Version beta-0.63.0-2010090921 (rev 3873)
  \write18 enabled.
(D:/Lukas/ConTeXt/Samples/U8/t-U8.mkiv
jobcontrol      > resuming randomizer with 0.50767540513321

ConTeXt  ver: 2010.10.20 21:33 MKIV  fmt: 2010.10.21  int: english/english

system          : cont-new loaded
(c:/ConTeXt/tex/texmf-context/tex/context/base/cont-new.tex
systems         : beware: some patches loaded from cont-new.tex
(c:/ConTeXt/tex/texmf-context/tex/context/base/cont-new.mkiv))
system          : cont-fil.mkiv loaded
(c:/ConTeXt/tex/texmf-context/tex/context/base/cont-fil.mkiv
loading         : ConTeXt File Synonyms
)
system          : cont-sys.rme loaded
(c:/ConTeXt/tex/texmf-context/tex/context/user/cont-sys.rme (c:/ConTeXt/tex/texmf-context/tex/context/base/type-def.mkiv) (c:/ConTeXt/tex/texmf-context/tex/context/base/type-lua.mkiv) (c:/ConTeXt/tex/texmf-context/tex/context/base/type-siz.mkiv) (c:/ConTeX
t/tex/texmf-context/tex/context/base/type-otf.mkiv))
system          : cont-err loaded
(c:/ConTeXt/tex/texmf-context/tex/context/base/cont-err.tex
systems         : no file 'cont-sys.tex', using 'cont-sys.rme' instead
)
system          : t-U8.top loaded
(t-U8.top
)
fonts           : preloading latin modern fonts
{c:/ConTeXt/tex/texmf/fonts/map/dvips/lm/lm-math.map}{c:/ConTeXt/tex/texmf/fonts/map/dvips/lm/lm-rm.map}{c:/ConTeXt/tex/texmf-context/fonts/map/pdftex/context/mkiv-base.map}
bodyfont        : 12pt rm is loaded
language        : language en is active
publications    : loading formatting style from bxml-apa
(c:/ConTeXt/tex/texmf-context/tex/context/base/bxml-apa.mkiv)
systems         : begin file D:/Lukas/ConTeXt/Samples/U8/t-U8.mkiv at line 4
! String contains an invalid utf-8 sequence.

system          > error on line 1 in file D:/Lukas/ConTeXt/Samples/U8/t-U8.mkiv: String contains an invalid utf-8 sequence ...

  1 >>  \mainlanguage[cz]
  2     \enableregime[cp1250]
  3
  4     \starttext
  5       AAA
  6
  7       \startluacode
  8         local t = dofile("t.lua")
  9
10         context(t)
11       \stopluacode

l.1
     ×ÜŔ
   } context(t)> ...le("t.lua")

\dodostartluacode ...d \directlua \zerocount {#1}}

l.11   \stopluacode

backends        > using xmp file 'c:/ConTeXt/tex/texmf-context/tex/context/base/lpdf-pdx.xml'
pages           > flushing realpage 1, userpage 1, subpage 1
systems         : end file D:/Lukas/ConTeXt/Samples/U8/t-U8.mkiv at line 12
  )<c:/ConTeXt/tex/texmf/fonts/opentype/public/lm/lmroman12-regular.otf>
mkiv lua stats : used config file          - c:/ConTeXt/tex/texmf/web2c/texmfcnf.lua
mkiv lua stats : used cache path           - c:/ConTeXt/tex/texmf-cache/luatex-cache/context/f53042fa2e1c106bc7e3383ec8c3a00c
mkiv lua stats : input load time           - 0.094 seconds
mkiv lua stats : stored bytecode data      - 241 modules, 56 tables, 297 chunks
mkiv lua stats : cleaned up reserved nodes - 33 nodes, 9 lists of 411
mkiv lua stats : node memory usage         - 19 glue_spec
mkiv lua stats : node list callback tasks  - 6 unique task lists, 5 instances (re)created, 328 calls
mkiv lua stats : used backend              - pdf (backend for directly generating pdf output)
mkiv lua stats : loaded patterns           - cs::23 en::2
mkiv lua stats : language load time        - 0.156 seconds , nofpatterns: 2
mkiv lua stats : callbacks                 - direct: 480, indirect: 4507, total: 4987
mkiv lua stats : lxml preparation time     - 0.000 seconds, 0 nodes, 15 lpath calls, 0 cached calls
mkiv lua stats : result saved in file      - t-U8.pdf
mkiv lua stats : loaded fonts              - 33 files: stmary10.afm lmmono12-regular.otf lmmono8-regular.otf lmmono9-regular.otf lmroman12-bold.otf lmroman12-regular.otf lmroman7-bold.otf lmroman7-regular.otf lmroman9-bold.otf lmroman9-regular.otf lmsans12
-regular.otf lmsans8-regular.otf lmsans9-regular.otf eufb10.tfm eufb7.tfm eufm10.tfm eufm7.tfm msam10.tfm msam7.tfm msbm10.tfm msbm7.tfm lmex10.tfm lmmi12.tfm lmmi7.tfm lmmi9.tfm lmmib10.tfm lmmib7.tfm lmsy10.tfm lmsy7.tfm lmsy9.tfm rm-lmr12.tfm rm-lmr7.tf
m rm-lmr9.tfm
mkiv lua stats : fonts load time           - 1.141 seconds
mkiv lua stats : luatex banner             - this is luatex, version beta-0.63.0-2010090921 (web2c 2010) (rev 3873)
mkiv lua stats : control sequences         - 30353 of 165536
mkiv lua stats : current memory usage      - 33 MB (ctx: 33 MB)
mkiv lua stats : runtime                   - 3.531 seconds, 1 processed pages, 1 shipped pages, 0.283 pages/second
---

So would it be possible to specify for Ctx which coding to use when using a Lua file by 'require' or 'dofile'?

- Or could this be depending on \enableregime specified in Ctx source file before \startluacode?

Or how to make Ctx work with non-UTF8 Lua files?

Lukas

NB: Latest minimals, WinXP.

[-- Attachment #2: t.lua --]
[-- Type: application/octet-stream, Size: 22 bytes --]

[-- Attachment #3: t-U8.mkiv --]
[-- Type: application/octet-stream, Size: 146 bytes --]

\mainlanguage[cz]
\enableregime[cp1250]

\starttext
  AAA

  \startluacode
    local t = dofile("t.lua")

    context(t)
  \stopluacode
\stoptext

[-- Attachment #4: Type: text/plain, Size: 486 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

^ permalink raw reply	[flat|nested] 7+ messages in thread