The odd semantics of \begincsname

The odd semantics of \begincsname

[Henri Menke]

Dear list,

According to the LuaTeX documentation:

    “The \begincsname primitive is like \csname but doesn’t create a
    relaxed equivalent when there is no such name.”

I thought it would be possible to use this fact to skip the \relax-ed
definition when \def-ining a new control sequence, but the following MWE
fails with \inaccessible:

    \expandafter\gdef\csname yes\endcsname{}
    \expandafter\gdef\begincsname no\endcsname{}
    \bye

Is this a bug or is this behaviour intended?  Could this be fixed by
making manufacture_csname aware whether it is in a def_cmd context or
not?

Cheers, Henri

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: The odd semantics of \begincsname

[Hans Hagen]

On 8/17/2019 9:19 AM, Henri Menke wrote:
> Dear list,
> 
> According to the LuaTeX documentation:
> 
>      “The \begincsname primitive is like \csname but doesn’t create a
>      relaxed equivalent when there is no such name.”
> 
> I thought it would be possible to use this fact to skip the \relax-ed
> definition when \def-ining a new control sequence, but the following MWE
> fails with \inaccessible:
> 
>      \expandafter\gdef\csname yes\endcsname{}
>      \expandafter\gdef\begincsname no\endcsname{}
>      \bye
> 
> Is this a bug or is this behaviour intended?  Could this be fixed by
> making manufacture_csname aware whether it is in a def_cmd context or
> not?
[sorry to those who are not interested in these low level issues, just skip]

intended ... it expands to basically nothing so you get no token 
representing a 'name' after the gdef .. the expansion is pushed in from 
of whatever comes next (which could be another \expandafter for instance)

you suggest that if \begincsname could behave differently when it's 
after a \def, \gdef, (and then quite some more definition related 
commands), it could behave differently but it not an option

for instance (as mentioned) there can be more than one expansion going 
on after these define commands, like expanding a macro that itself 
expands to \csname so one has several \expandafters before the gdef 
then); there is actually no looking back in scanning tokens unless a 
token has been scanned already and looking forward would involve 
expansion so a circular mess

an option could be not to push something on the save stack (a side 
effect of creating the csname, which has a little impact on performance 
and nesting) but removing that bit might give other side effects (e.g. 
for successive reassignments inside a group, maybe even mixed local and 
global); i did a quick test with that and it gives quite incompatible 
output in ConTeXt so that's definitely a no-go (adding all kind fo 
saveguards and checks in the engine doesn't pay off, especially not for 
something that never was a problem)

some time ago i considered a convenience command \[e]defcsname, as it 
saves a few tokens (no gain in performance as all the related things 
still need to happen); but even that one would probably create the name 
in the same way

so ... this is the way it is ... (i must admit that it never gave me any 
issues so whatever triggered the question, there's probbaly a way around 
it)

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: The odd semantics of \begincsname

[Henri Menke]

On 17/08/19 8:48 PM, Hans Hagen wrote:
> On 8/17/2019 9:19 AM, Henri Menke wrote:
>> Dear list,
>>
>> According to the LuaTeX documentation:
>>
>>      “The \begincsname primitive is like \csname but doesn’t create a
>>      relaxed equivalent when there is no such name.”
>>
>> I thought it would be possible to use this fact to skip the \relax-ed
>> definition when \def-ining a new control sequence, but the following MWE
>> fails with \inaccessible:
>>
>>      \expandafter\gdef\csname yes\endcsname{}
>>      \expandafter\gdef\begincsname no\endcsname{}
>>      \bye
>>
>> Is this a bug or is this behaviour intended?  Could this be fixed by
>> making manufacture_csname aware whether it is in a def_cmd context or
>> not?
> [sorry to those who are not interested in these low level issues, just skip]
> 
> intended ... it expands to basically nothing so you get no token 
> representing a 'name' after the gdef .. the expansion is pushed in from 
> of whatever comes next (which could be another \expandafter for instance)
> 
> you suggest that if \begincsname could behave differently when it's 
> after a \def, \gdef, (and then quite some more definition related 
> commands), it could behave differently but it not an option
> 
> for instance (as mentioned) there can be more than one expansion going 
> on after these define commands, like expanding a macro that itself 
> expands to \csname so one has several \expandafters before the gdef 
> then); there is actually no looking back in scanning tokens unless a 
> token has been scanned already and looking forward would involve 
> expansion so a circular mess
> 
> an option could be not to push something on the save stack (a side 
> effect of creating the csname, which has a little impact on performance 
> and nesting) but removing that bit might give other side effects (e.g. 
> for successive reassignments inside a group, maybe even mixed local and 
> global); i did a quick test with that and it gives quite incompatible 
> output in ConTeXt so that's definitely a no-go (adding all kind fo 
> saveguards and checks in the engine doesn't pay off, especially not for 
> something that never was a problem)
> 
> some time ago i considered a convenience command \[e]defcsname, as it 
> saves a few tokens (no gain in performance as all the related things 
> still need to happen); but even that one would probably create the name 
> in the same way
> 
> so ... this is the way it is ... (i must admit that it never gave me any 
> issues so whatever triggered the question, there's probbaly a way around 
> it)

I can accept this answer.  Just for a little context, the question was
triggered by this:

    https://tex.stackexchange.com/questions/504501/global-variant-of-csname-endcsname

In short: Having thousands of

    \expandafter\gdef\csname foo\endcsname{}

inside a group (as happens for xmltex), can lead to a save_stack
overflow.  One way around it is to do

    \begingroup\expandafter\endgroup\expandafter\gdef\csname foo\endcsname{}

The \expandafter inside the group will pull the evaluation of \csname
into the group which will discard the save_stack at the \endgroup, thus
avoiding the build-up.  However, this construction is a bit hard to
understand so I was wondering whether

    \expandafter\gdef\begincsname foo\endcsname{}

could be used instead to elide the save_stack (which doesn't work
because \begincsname does not actually build a \csname).

Cheers, Henri

> Hans
> 
> -----------------------------------------------------------------
>                                            Hans Hagen | PRAGMA ADE
>                Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
>         tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
> -----------------------------------------------------------------
> 

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: The odd semantics of \begincsname

[Hans Hagen]

On 8/17/2019 10:56 AM, Henri Menke wrote:
> On 17/08/19 8:48 PM, Hans Hagen wrote:
>> On 8/17/2019 9:19 AM, Henri Menke wrote:
>>> Dear list,
>>>
>>> According to the LuaTeX documentation:
>>>
>>>       “The \begincsname primitive is like \csname but doesn’t create a
>>>       relaxed equivalent when there is no such name.”
>>>
>>> I thought it would be possible to use this fact to skip the \relax-ed
>>> definition when \def-ining a new control sequence, but the following MWE
>>> fails with \inaccessible:
>>>
>>>       \expandafter\gdef\csname yes\endcsname{}
>>>       \expandafter\gdef\begincsname no\endcsname{}
>>>       \bye
>>>
>>> Is this a bug or is this behaviour intended?  Could this be fixed by
>>> making manufacture_csname aware whether it is in a def_cmd context or
>>> not?
>> [sorry to those who are not interested in these low level issues, just skip]
>>
>> intended ... it expands to basically nothing so you get no token
>> representing a 'name' after the gdef .. the expansion is pushed in from
>> of whatever comes next (which could be another \expandafter for instance)
>>
>> you suggest that if \begincsname could behave differently when it's
>> after a \def, \gdef, (and then quite some more definition related
>> commands), it could behave differently but it not an option
>>
>> for instance (as mentioned) there can be more than one expansion going
>> on after these define commands, like expanding a macro that itself
>> expands to \csname so one has several \expandafters before the gdef
>> then); there is actually no looking back in scanning tokens unless a
>> token has been scanned already and looking forward would involve
>> expansion so a circular mess
>>
>> an option could be not to push something on the save stack (a side
>> effect of creating the csname, which has a little impact on performance
>> and nesting) but removing that bit might give other side effects (e.g.
>> for successive reassignments inside a group, maybe even mixed local and
>> global); i did a quick test with that and it gives quite incompatible
>> output in ConTeXt so that's definitely a no-go (adding all kind fo
>> saveguards and checks in the engine doesn't pay off, especially not for
>> something that never was a problem)
>>
>> some time ago i considered a convenience command \[e]defcsname, as it
>> saves a few tokens (no gain in performance as all the related things
>> still need to happen); but even that one would probably create the name
>> in the same way
>>
>> so ... this is the way it is ... (i must admit that it never gave me any
>> issues so whatever triggered the question, there's probbaly a way around
>> it)
> 
> I can accept this answer.  Just for a little context, the question was
> triggered by this:
> 
>      https://tex.stackexchange.com/questions/504501/global-variant-of-csname-endcsname
> 
> In short: Having thousands of
> 
>      \expandafter\gdef\csname foo\endcsname{}
> 
> inside a group (as happens for xmltex), can lead to a save_stack
> overflow.  One way around it is to do
> 
>      \begingroup\expandafter\endgroup\expandafter\gdef\csname foo\endcsname{}

Sure, just group. But actually, if one needs that many csnames one can 
wonder about the approach. One can bump the save stack just like one 
also might have to bump the hash (extra) size (either of them can 
overflow).

Also, probably a bit of extra grouping can happen at a different level, 
not for each csname but for in this case an xml element, which is also 
more efficient

> The \expandafter inside the group will pull the evaluation of \csname
> into the group which will discard the save_stack at the \endgroup, thus
> avoiding the build-up.  However, this construction is a bit hard to
> understand so I was wondering whether

well, instead of this:

\begingroup\expandafter\endgroup\expandafter\gdef\csname foo\endcsname{}

one can just use this:

\begingroup\expandafter\gdef\csname foo\endcsname{}\endgroup

which is less tokens, less pushing/poping and therefore a litle faster 
(but often neglectable compared to other things that tex/macros do in 
most cases) but of course it looks less 'cool' and 'expert' and creates 
less 'awe' .. so let's add another one:

{\expandafter}\expandafter\gdef\csname foo\endcsname{}

this one is performance wise close to the second case (normal grouping)
but it might look more puzzling which is why i should wrap it:

\def\defcsname {{\expandafter}\expandafter\def \csname}
\def\gdefcsname{{\expandafter}\expandafter\gdef\csname}

which then is about as efficient as the first alternative with two 
\expandafter usage using \begingroup\endgroup (okay, efficiency depends 
of course on the engine too, and probably on the cpu as well)

(you can argue that \expandafter and \noexpand and \futurelet ... were 
added to tex so that one could boost his resume ... the more you use in 
sequence the more expert you are; but you can also argue that they add 
some charm to tex, a nice playground and such)

>      \expandafter\gdef\begincsname foo\endcsname{}
> 
> could be used instead to elide the save_stack (which doesn't work
> because \begincsname does not actually build a \csname).
it does when it's known and then it puts something in the input (a 
token), but when unknown it doesn't so you effectively get \def{} which 
is not what you want (ok, maybe some weird usage where { is defined as 
macro does, which actually can make sense when one handles xml with 
active characters).

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________