Re: Yet another cornercase of ^^ handling in LuaMetaTeX

["Marcel Fabian Krüger" <tex@2krueger.de>]

On Wed, Jun 17, 2020 at 03:43:49PM +0200, Hans Hagen wrote:
> On 6/17/2020 12:10 PM, Marcel Fabian Krüger wrote:
> > On Mon, Jun 15, 2020 at 04:16:06PM +0200, Hans Hagen wrote:
> > > 
> > > ah, i removed some doee that i though was never seen; i'll add that bit
> > > again
> > > 
> > > > Beside that, I noticed a smaller weird thing when using LuaMetaTeX in ini
> > > > mode without Context: The default mathcodes for digits and letters have
> > > > their class and family values switched. So e.g. "A" has a default
> > > > mathcode of 119537729=0x7200041, indicating class 1 and family 7,
> > > > instead of the expected 31457345=0x1E00041, indicating class 7 and
> > > > family 1. Of course this isn't really a problem because the format will
> > > > normally assign new mathcodes anyway, but especially class 1 seems like
> > > > a *very* unnatural default for normal letters.
> > > i'll check it (digits get class 0) ... indeed these defaults nowadays never
> > > kick in this way
> > 
> > Thanks. Is there any way to convert either a token number (as given e.g.
> > by t.tok) or a numeric cs identifier (as given e.g. by t.tok-cs_token_flag)
> > to a token uservalue?
> > In LuaTeX I used to use `token.new(tok, 0)` for this, but LuaMetaTeX
> > seems to do more checking here.
> 
> Indeed there is more checking. So, when you create a token you need to use a
> valid cmd code and a valid chr code.
> 
> \ctxlua{
>     local t = token.new(string.byte("X"),token.command_id("letter"))
>     tex.print(t)
> }
> 
> The interface might evolve a bit more. Valid commands (at this moment) are:
> 
> \ctxlua{
>     for k, v in ipairs(token.values("command")) do
>         print(k,v)
>     end
> }
> 
> So, things like this should work
> 
> \dimen3456=1234pt
> 
> \dimen3467=
> \ctxlua{
>     local t = token.new(3456,token.command_id("register_dimen"))
>   % local t = token.new("register_dimen",3456) % in upcoming version
>     tex.sprint(t)
> }
> 
> \the\dimen3467
> 
> As does:
> 
> \ctxlua{
>     local t1 = token.new(string.byte("{"),token.command_id("letter"))
>     local t2 = token.new(string.byte("}"),token.command_id("letter"))
>     tex.print(t1)
>     tex.print(t2)
> }
> 
> Properties like cs_token_flag are not user level and have no meaning in the
> interface. In fact, the 'token-as-number' is just some representation with
> no intended meaning. Also, tokens that are problematic or can interfere
> badly might become inaccessible in future versions.
> 
> (I don't expect many users to use the low level token interface anyway but
> it's there as part of the package.)
> 
> Hans
> 

How does this work with mark nodes? Their mark field has a table
containing triples of command, character and "cs" value. How can the
actual csname of a controlsequence be extracted from this in LuaMetaTeX?

Marcel
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________