Re: Problem with Lua processing UTF8 substrings

[Philipp Gesang <gesang@stud.uni-heidelberg.de>]

[-- Attachment #1.1: Type: text/plain, Size: 2407 bytes --]

On 2012-02-01 20:26, Jaroslav Hajtmar wrote:
> I want to use Lua to write characters (substrings) from a string,
> but I get an error message:
> 
> ! String contains an invalid utf-8 sequence.
> 
> Can you please someone help?

Have you tried the unicode library? The standard string library
operates on bytes, therefore extracting a single byte yields an
incomplete multibyte char if the codepoint is beyond ascii.

·································································

\def\mymacro#1{%
  \startluacode
    local utf = unicode.utf8
    local target = [==[\detokenize{#1}]==]
    for i=1, utf.len(target) do
      context(utf.sub(target,i,i)..", ")
    end
  \stopluacode%
}

%% alternatively, use utfcharacters
\define[1]\myothermacro{%
  \startluacode
    local result = { }
    for i in string.utfcharacters[==[\detokenize{#1}]==] do
      result[\letterhash result+1] = i
    end
    context(table.concat(result, ", "))
  \stopluacode
}

\starttext

\mymacro{šěřěžřýčřčžáýčý}\par
\myothermacro{šěřěžřýčřčžáýčý}

\stoptext

·································································

(Lazy people would just do a “local string = unicode.utf8” at the
top of the file.)

Regards
Philipp




> 
> Thanks
> Jaroslav Hajtmar
> 
> Here is my minimal example:
> 
> \def\mymacro#1{\ctxlua{for i=1, string.len('#1') do
> context(string.sub('#1',i,i)..", ") end}}
> 
> \starttext
> 
> %\mymacro{šěřěžřýčřčžáýčý} % Here is a problem
> \mymacro{asdfghjklqwertt} % Here is all OK
> 
> \stoptext
> 
> 
> 
> 
> 
> 
> 
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : http://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________

-- 
()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments

[-- Attachment #1.2: Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________