Re: LMTX MkIV difference in expansion

[Rik Kabel <ConTeXt@rik.users.panix.com>]

On 11/19/2020 17:03, Hans Hagen wrote:
> On 11/19/2020 9:41 PM, Wolfgang Schuster wrote:
>> Rik Kabel schrieb am 19.11.2020 um 21:20:
>>> Another LMTX/MkIV difference, this time with expansion:
>>>
>>>     \define\Align{yes}
>>>     \starttext
>>>     \startalignment[\Align]
>>>          This works with MkIV but fails with LMTX, complaining: {\tt
>>>          tex error on line 3 in file G:/expand.mkvi: The file ended
>>>          when scanning an argument.}
>>>     \blank
>>>         It works in both when \tex{def} or \tex{defineexpandable} is
>>>         used instead of \tex{define}.
>>>     \blank
>>>         What changed?
>>>     \stopalignment
>>>     \stoptext
>>>
>>> It may well be that I have been abusing some laxity in MkIV and that 
>>> LMTX is a bit stricter in what it accepts, but I would like to know 
>>> if this is an expected difference.
>>
>> You have to wait for Hans to get an answer but here is a minimal 
>> example.
>>
>> \starttext
>>
>> \protected\def\testparameter{test}
>> %\def\testparameter{test}
>>
>> \def\test[#1]%
>> {\expandafter\let\expandafter\testargumentlist\csname#1\endcsname}
>>
>> \test[\testparameter]
>>
>> \stoptext
> Often arguments to commands like \startsomething[xx] let the xx end up 
> in some \(if)csname expansion. A protected (\unexpanded in context 
> speak) macro doesn't expand inside for instance an \edef (or 
> comparable expandable situation). Now, from that it makes perfect 
> sense to also not let it expand inside a \csname or \ifcsname. One 
> reason is that when it does expand, you can get a pretty wild (nested) 
> sequence of nested expansions and one can be pretty sure that we then 
> don't have a proper csname. This is why in luatex we have a catch for 
> running wild csname checking.
>
> The original \ifcsname test was inherited from etex. The \protected 
> feature also comes from etex. But \csname is a tex natural.
> In pdftex (and luatex) a protected macro inside an \(if)csname does 
> expand which to makes no sense and smells like a bug. Or maybe it was 
> tricky to catch (the implementation of protected a bit of a hack).
>
> In luametatex protected macros are native and in the process I also 
> decided to *not* expand them in a \(if)csname where I expect (as said) 
> protected macros to behave like in an edef. I nice side effect is that 
> running wild no longer happens (but we still catch it) which can save 
> quite some useless backup token list construction (needed because tex 
> has to push back stuff in order to be able to report an error).
>
> So, when you still don't understand it (which I can understand) I'm 
> sure Wolfgang can explain it better now.
>
> \starttext
>
>           \def\foo{foo}
> \protected\def\oof{oof}
>
> \csname foo\endcsname
> \csname oof\endcsname
> \csname \foo\endcsname
>
> % error in luametatex, ok in pdftex/luatex:
>
> % \csname \oof\endcsname
>
> \ifcsname  foo\endcsname yes\else nop\fi
> \ifcsname  oof\endcsname yes\else nop\fi
> \ifcsname \foo\endcsname yes\else nop\fi
>
> % nop in luametatex (error intercepted), yes in pdftex/luatex
>
> \ifcsname \oof\endcsname yes\else nop\fi
>
> \stoptext
>
> Now, one can argue that if I consider it a but in the other engines, 
> why I don't argue that it should be solved. Well, there is too much 
> legacy code already that might use it as feature so it will not 
> change. But in luametatex we can 'fix' these things. (We also use the 
> csname in a rather predictable way in context so i don't expect issues 
> in the core.)
>
> Hans
>
You are right about not quite understand.

Does this mean that I can have the same definitions in MkIV and LMTX 
(after some future update), or should I hunt down the \defines in both, 
or that I should fork (or mode test) my source environment files, one 
set for LMTX and one for MkIV?

-- 
Rik

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________