zle messes up 'words' variable?

zle messes up 'words' variable?

[Felipe Contreras]

Hi,

I'm trying the following:
---
set_vars ()
{
	cur="foo"
	words="foo"
	cwords="foo"
}

_foo ()
{
	local cur words cwords
	set_vars

	echo "cur=${cur} words=${words} cwords=${cwords}" >> /tmp/comp_test.txt
}

compdef _foo foo
---

Which results in:
cur=foo words= cwords=foo

Strangely enough, if I use the #compdef tag, the code works fine (words="foo").

Any ideas?

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Jérémie Roquet]

Hi,

2011/4/27 Felipe Contreras <felipe.contreras@gmail.com>:
> I'm trying the following:
> ---
> set_vars ()
> {
>        cur="foo"
>        words="foo"
>        cwords="foo"
> }
>
> _foo ()
> {
>        local cur words cwords
>        set_vars
>
>        echo "cur=${cur} words=${words} cwords=${cwords}" >> /tmp/comp_test.txt
> }
>
> compdef _foo foo
> ---
>
> Which results in:
> cur=foo words= cwords=foo
>
> Strangely enough, if I use the #compdef tag, the code works fine (words="foo").
>
> Any ideas?

$words is a special array used for completion. See `man zshcompsys'
for more about it.

Best regards,

-- 
Jérémie

Re: zle messes up 'words' variable?

[Felipe Contreras]

2011/4/27 Jérémie Roquet <arkanosis@gmail.com>:
> 2011/4/27 Felipe Contreras <felipe.contreras@gmail.com>:
>> I'm trying the following:
>> ---
>> set_vars ()
>> {
>>        cur="foo"
>>        words="foo"
>>        cwords="foo"
>> }
>>
>> _foo ()
>> {
>>        local cur words cwords
>>        set_vars
>>
>>        echo "cur=${cur} words=${words} cwords=${cwords}" >> /tmp/comp_test.txt
>> }
>>
>> compdef _foo foo
>> ---
>>
>> Which results in:
>> cur=foo words= cwords=foo
>>
>> Strangely enough, if I use the #compdef tag, the code works fine (words="foo").
>>
>> Any ideas?
>
> $words is a special array used for completion. See `man zshcompsys'
> for more about it.

Yeah, I know that, but I'm overriding it, which works fine with
#compdef, but not with compdef. How do you explain that?

Besides, the problem is with bash completion emulation where 'words'
is not supposed to be special, so zsh is messing things up.

--
#!bash

autoload -U +X bashcompinit && bashcompinit

set_vars ()
{
	cur="foo"
	words="foo"
	cwords="foo"
}

_foo ()
{
	local cur words cwords
	set_vars

	echo "cur=${cur} words=${words} cwords=${cwords}" >> /tmp/comp_test.txt
}

complete -F _foo foo
--

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Felipe Contreras]

2011/4/27 Felipe Contreras <felipe.contreras@gmail.com>:
> 2011/4/27 Jérémie Roquet <arkanosis@gmail.com>:
>> 2011/4/27 Felipe Contreras <felipe.contreras@gmail.com>:
>>> I'm trying the following:
>>> ---
>>> set_vars ()
>>> {
>>>        cur="foo"
>>>        words="foo"
>>>        cwords="foo"
>>> }
>>>
>>> _foo ()
>>> {
>>>        local cur words cwords

Also, Jonathan Nieder found that this works using 'typeset -h', but
that's not supported in bash. It would be nice to tell zsh to use
'typeset -h' whenever it finds 'local', or something like that.

>>>        set_vars
>>>
>>>        echo "cur=${cur} words=${words} cwords=${cwords}" >> /tmp/comp_test.txt
>>> }
>>>
>>> compdef _foo foo
>>> ---


-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Bart Schaefer]

On Apr 27, 11:26am, Felipe Contreras wrote:
}
} Strangely enough, if I use the #compdef tag, the code works fine
} (words="foo").
}
} Any ideas?

Hrm.  #compdef is equivalent to "compdef -na ..." and causes the
function to be autoloaded at the time it's referenced.  I don't know
of any other reason the -na options would change the behavior.  Does
the code work fine on repeated attempts or only the first time?

What may be happening is that "sticky emulation" is in effect, so that
when the function is autoloaded in bash compatibility mode, "words"
becomes non-special.  If so, I'm not familiar enough with bashcompinit
to say whether that's intentional or an accidental side-effect.

Re: zle messes up 'words' variable?

[Felipe Contreras]

On Wed, Apr 27, 2011 at 11:26 AM, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> I'm trying the following:
> ---
> set_vars ()
> {
>        cur="foo"
>        words="foo"
>        cwords="foo"
> }
>
> _foo ()
> {
>        local cur words cwords
>        set_vars
>
>        echo "cur=${cur} words=${words} cwords=${cwords}" >> /tmp/comp_test.txt
> }
>
> compdef _foo foo
> ---
>
> Which results in:
> cur=foo words= cwords=foo
>
> Strangely enough, if I use the #compdef tag, the code works fine (words="foo").
>
> Any ideas?

I saw the following reply online but I wasn't CC'ed:

> Hrm.  #compdef is equivalent to "compdef -na ..." and causes the function to
> be autoloaded at the time it's referenced.  I don't know of any other reason
> the -na options would change the behavior.  Does the code work fine on
> repeated attempts or only the first time?

Actually, I cannot get it to work any more with #compdef tag. I don't
know what I'm doing wrong. But compdef -na doesn't work either.

> What may be happening is that "sticky emulation" is in effect, so that when
> the function is autoloaded in bash compatibility mode, "words" becomes
> non-special.  If so, I'm not familiar enough with bashcompinit to say whether
> that's intentional or an accidental side-effect.

This has nothing to do with bashcompinit, the issue is with plain compinit.

BTW. This messes up with a very important function in bash completion:
_get_comp_words_by_ref()

Cheers.

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Bart Schaefer]

On Apr 28, 11:19pm, Felipe Contreras wrote:
}
} I saw the following reply online but I wasn't CC'ed:

I typically don't Cc on mail to the list unless the sender specifically
asks for that.  If you asked and I missed it, I apologize.

} > What may be happening is that "sticky emulation" is in effect, so
} > that when the function is autoloaded in bash compatibility mode,
} > "words" becomes non-special. If so, I'm not familiar enough with
} > bashcompinit to say whether that's intentional or an accidental
} > side-effect.
} 
} This has nothing to do with bashcompinit, the issue is with plain
} compinit.

Your example in workers/29086 explicitly loads and runs bashcompinit.
What am I missing?

} BTW. This messes up with a very important function in bash completion:
} _get_comp_words_by_ref()

I'm now entirely confused.

I no longer believe that I know what you mean by "in bash completion."

There's no shell function _get_comp_words_by_ref anywhere in the zsh
sources.  Where is that coming from?

If what you're doing is loading functions into zsh that were written to
be used with bash's compgen, then I don't follow how you can NOT be
using bashcompinit.

Either way I have no better explanation for why #compdef and directly
calling compdef would behave differently, but I don't see any uses of
"emulate sh -c" in bashcompinit so that's probably not right.

Re: zle messes up 'words' variable?

[Felipe Contreras]

On Fri, Apr 29, 2011 at 6:31 AM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On Apr 28, 11:19pm, Felipe Contreras wrote:
> }
> } I saw the following reply online but I wasn't CC'ed:
>
> I typically don't Cc on mail to the list unless the sender specifically
> asks for that.  If you asked and I missed it, I apologize.

Oh, I'm typically on mailing lists where Reply-To isn't munged so it's
not necessary to ask for that. But I'll do that.

> } > What may be happening is that "sticky emulation" is in effect, so
> } > that when the function is autoloaded in bash compatibility mode,
> } > "words" becomes non-special. If so, I'm not familiar enough with
> } > bashcompinit to say whether that's intentional or an accidental
> } > side-effect.
> }
> } This has nothing to do with bashcompinit, the issue is with plain
> } compinit.
>
> Your example in workers/29086 explicitly loads and runs bashcompinit.
> What am I missing?

What is 'workers/29086'?

This thread is titled "zle messes up 'words' variable?", and the test
has nothing to do with bashcompinit:
http://article.gmane.org/gmane.comp.shells.zsh.devel/22479

> } BTW. This messes up with a very important function in bash completion:
> } _get_comp_words_by_ref()
>
> I'm now entirely confused.
>
> I no longer believe that I know what you mean by "in bash completion."

By bash completion I mean:
http://bash-completion.alioth.debian.org/

> There's no shell function _get_comp_words_by_ref anywhere in the zsh
> sources.  Where is that coming from?

>From here:
http://git.debian.org/?p=bash-completion/bash-completion.git;a=blob;f=bash_completion;h=253d1fb99bcd253535b78c4cbf94e9322e60098c;hb=a43a68f2c60c9f29ab81986382b4063a3aa7bbbb#l422

But it seems it has changed.

> If what you're doing is loading functions into zsh that were written to
> be used with bash's compgen, then I don't follow how you can NOT be
> using bashcompinit.

I am using bashcompinit, but the issue appears without it.

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Bart Schaefer]

On May 2,  9:11pm, Felipe Contreras wrote:
}
} On Fri, Apr 29, 2011 at 6:31 AM, Bart Schaefer
} <schaefer@brasslantern.com> wrote:
} > Your example in workers/29086 explicitly loads and runs bashcompinit.
} > What am I missing?
} 
} What is 'workers/29086'?

Sorry, I'm writing in zsh-workers subscriber-speke.  Every message sent
to the zsh-workers (or zsh-users) list is tagged with an "X-Seq:" header
containing a number.  The archive server at zsh.org is able to look up
messages by sequence number.  So we shorthand "go to the zsh.org archive
and put 29086 into the workers lookup field" as "workers/29086".
 
} I am using bashcompinit, but the issue appears without it.

Without bashcompinit, there is by definition not an issue.  Or, if you
prefer, you should have expected this issue to appear, because words is
documented as a special variable in completion function context.

If what you want to discuss is ways to fix bashcompinit so that the
functions in bash completion work as expected, that's one thing.  If
what you're asking is for the special-ness of "words" to be undone
for completion in general, that's so unlikely to happen as to not be
worth continuing.

Re: zle messes up 'words' variable?

[Felipe Contreras]

On Tue, May 3, 2011 at 4:42 AM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On May 2,  9:11pm, Felipe Contreras wrote:
> } I am using bashcompinit, but the issue appears without it.
>
> Without bashcompinit, there is by definition not an issue.  Or, if you
> prefer, you should have expected this issue to appear, because words is
> documented as a special variable in completion function context.

The fact that it's documented as a special variable doesn't mean that
the behavior should be totally unexpected.

1) Without modifier

Then 'words' can be re-used like any other variable.

2) With 'local' modifier

Then modifying 'words' has an impact only on the current function, not
on the scope where the variable was defined as local.

2.1) When using #compdef tag

Then 'words' can be modified like any other variable.

3) With 'typedef -h'

Then 'words' can be modified like any other variable.

I was expecting an explanation of these discrepancies, and a way to
globally make 'words' work as expected without having to replace each
use of 'local' by 'typedef -h'.

I don't think anybody knows what's going on, so nobody knows how to do
that. In fact, this is probably a bug.

> If what you want to discuss is ways to fix bashcompinit so that the
> functions in bash completion work as expected, that's one thing.  If
> what you're asking is for the special-ness of "words" to be undone
> for completion in general, that's so unlikely to happen as to not be
> worth continuing.

You are not good at compartmentalizing aren't you? If you fix the
issue with my example, you fix the issue with bashcompinit. It's the
same issue. Or "riddle" if that word makes you more comfortable.

But if you want to mix the two things together, fine:

---
#!bash

autoload -U +X bashcompinit && bashcompinit

set_vars ()
{
	cur="foo"
	words="foo"
	cwords="foo"
}

_foo ()
{
	local cur words cwords
	set_vars

	echo "cur=${cur} words=${words} cwords=${cwords}" >> /tmp/comp_test.txt
}

complete -F _foo foo
---

cur=foo words= cwords=foo

Now that is clearly a bug.

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Bart Schaefer]

On May 3, 11:05am, Felipe Contreras wrote:
}
} The fact that it's documented as a special variable doesn't mean that
} the behavior should be totally unexpected.
} 
} 1) Without modifier
} 
} Then 'words' can be re-used like any other variable.
} 
} 2) With 'local' modifier
} 
} Then modifying 'words' has an impact only on the current function, not
} on the scope where the variable was defined as local.

Ahh, I see the problem.  You haven't read the entire documentation.
At the very top of (what in the info file is) section 19.2 is this
paragraph (caps mine for emphasis):

    Inside completion widgets, AND ANY FUNCTIONS CALLED FROM THEM, some
    parameters have special meaning; outside these functions they are
    not special to the shell in any way. These parameters are used to
    pass information between the completion code and the completion
    widget. Some of the builtin commands and the condition codes use or
    change the current values of these parameters. Any existing values
    will be hidden during execution of completion widgets; EXCEPT FOR
    COMPSTATE, THE PARAMETERS ARE RESET ON EACH FUNCTION EXIT (INCLUDING
    NESTED FUNCTION CALLS FROM WITHIN THE COMPLETION WIDGET) to the
    values they had when the function was entered.

In other words, these variable are always local in all scopes, unless
they are explicitly declared otherwise (which in this case requires
the use of typeset -h).  That's part of their special-ness, and is one
of the things I assert is extremely unlikely to change.

} 2.1) When using #compdef tag
} 
} Then 'words' can be modified like any other variable.

I thought you'd decided you were unable to reproduce this?  If you now
*are* able to reproduce it, then *that* might be a bug.
 
} 3) With 'typedef -h'
} 
} Then 'words' can be modified like any other variable.
} 
} I was expecting an explanation of these discrepancies, and a way to
} globally make 'words' work as expected without having to replace each
} use of 'local' by 'typedef -h'.

I apologize for not understanding the source of the mis-understanding.
I was focusing on why #compdef might make a difference, not on the
details of the actual scoping problem you perceived.
 
} > If what you want to discuss is ways to fix bashcompinit so that the
} > functions in bash completion work as expected, that's one thing.  If
} > what you're asking is for the special-ness of "words" to be undone
} > for completion in general, that's so unlikely to happen as to not be
} > worth continuing.
} 
} You are not good at compartmentalizing aren't you?

I'm really disappointed in the way that mailing list discourse in
general lately (and not just on this list) tends to rapidly descend
into insults and ad-hominem attacks.  I've been watching the CentOS
list collapse under the weight of this sort of thing and I plead that
we try to avoid it here.

} If you fix the issue with my example, you fix the issue with
} bashcompinit. It's the same issue. Or "riddle" if that word makes you
} more comfortable.

No, it's not the same issue, because bashcompinit has responsibility
for loading the bash completion functions in a way that makes them
compatible.  That's not the same as first loading bashcompinit and
then independently defining a new completion, though I lean toward 
the conclusion that use of the bash-compatible "complete" command is
probably the correct place to fix this if it can be managed.

-- 

Re: zle messes up 'words' variable?

[Bart Schaefer]

On May 3,  7:39am, Bart Schaefer wrote:
}
} ... bashcompinit has responsibility
} for loading the bash completion functions in a way that makes them
} compatible.  That's not the same as first loading bashcompinit and
} then independently defining a new completion, though I lean toward 
} the conclusion that use of the bash-compatible "complete" command is
} probably the correct place to fix this if it can be managed.

Try this:

Index: Completion/bashcompinit
===================================================================
diff -c -r1.2 bashcompinit
--- Completion/bashcompinit	22 Jun 2004 07:53:03 -0000	1.2
+++ Completion/bashcompinit	3 May 2011 15:01:09 -0000
@@ -120,7 +120,13 @@
       ;;
       F)
         COMPREPLY=()
-	$OPTARG "${words[0]}" "${words[CURRENT-1]}" "${words[CURRENT-2]}"
+        (){
+          set -- "${words[0]}" "${words[CURRENT-1]}" "${words[CURRENT-2]}"
+          # There may be more things we need to add to this typeset to
+          # protect bash functions from compsys special variable names
+          typeset -h words
+          $OPTARG "$@"
+        }
 	results+=( "${COMPREPLY[@]}" )
       ;;
       G)

Re: zle messes up 'words' variable?

[Felipe Contreras]

On Tue, May 3, 2011 at 6:04 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On May 3,  7:39am, Bart Schaefer wrote:
> }
> } ... bashcompinit has responsibility
> } for loading the bash completion functions in a way that makes them
> } compatible.  That's not the same as first loading bashcompinit and
> } then independently defining a new completion, though I lean toward
> } the conclusion that use of the bash-compatible "complete" command is
> } probably the correct place to fix this if it can be managed.
>
> Try this:
>
> Index: Completion/bashcompinit
> ===================================================================
> diff -c -r1.2 bashcompinit
> --- Completion/bashcompinit     22 Jun 2004 07:53:03 -0000      1.2
> +++ Completion/bashcompinit     3 May 2011 15:01:09 -0000
> @@ -120,7 +120,13 @@
>       ;;
>       F)
>         COMPREPLY=()
> -       $OPTARG "${words[0]}" "${words[CURRENT-1]}" "${words[CURRENT-2]}"
> +        (){
> +          set -- "${words[0]}" "${words[CURRENT-1]}" "${words[CURRENT-2]}"
> +          # There may be more things we need to add to this typeset to
> +          # protect bash functions from compsys special variable names
> +          typeset -h words
> +          $OPTARG "$@"
> +        }
>        results+=( "${COMPREPLY[@]}" )
>       ;;
>       G)

Yeap, that fixes it :)

So I think this should be committed to zsh. Now, that's part a of the problem.

Part b is how to workaround the issue in the current versions of zsh,
which is what the git guys are trying to do:
http://article.gmane.org/gmane.comp.version-control.git/172371

I guess simply setting 'typeset -h words' on the top-level function
(_git) should do the trick.

Cheers.

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Bart Schaefer]

On May 3,  8:41pm, Felipe Contreras wrote:
}
} Part b is how to workaround the issue in the current versions of zsh,
} which is what the git guys are trying to do:
} http://article.gmane.org/gmane.comp.version-control.git/172371
} 
} I guess simply setting 'typeset -h words' on the top-level function
} (_git) should do the trick.

Yes, that should do it; but as zsh already has its own version of
completion for git, I'm surprised that anyone is encountering the
problem with the bash completion.  In fact that's probably why the
confusion over "words" didn't come to our attention before this.

Re: zle messes up 'words' variable?

[Felipe Contreras]

On Wed, May 4, 2011 at 3:39 AM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On May 3,  8:41pm, Felipe Contreras wrote:
> }
> } Part b is how to workaround the issue in the current versions of zsh,
> } which is what the git guys are trying to do:
> } http://article.gmane.org/gmane.comp.version-control.git/172371
> }
> } I guess simply setting 'typeset -h words' on the top-level function
> } (_git) should do the trick.
>
> Yes, that should do it; but as zsh already has its own version of
> completion for git, I'm surprised that anyone is encountering the
> problem with the bash completion.  In fact that's probably why the
> confusion over "words" didn't come to our attention before this.

It has been a known issue in git for a while.

The problem with the the zsh completion is that it's so slows that is
basically unusable. And apparently you guys want it this way [1].

[1] http://mid.gmane.org/BANLkTi=iJXX_gBxnGFmDmF4goLeK0OKL3Q@mail.gmail.com

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Bart Schaefer]

On May 5,  5:39pm, Felipe Contreras wrote:
}
} The problem with the the zsh completion is that it's so slows that is
} basically unusable. And apparently you guys want it this way [1].

The problem with hopping in to a list where you aren't a subscriber
is that you miss nuance and follow-ups.

Re: zle messes up 'words' variable?

[Felipe Contreras]

On Thu, May 5, 2011 at 7:24 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On May 5,  5:39pm, Felipe Contreras wrote:
> }
> } The problem with the the zsh completion is that it's so slows that is
> } basically unusable. And apparently you guys want it this way [1].
>
> The problem with hopping in to a list where you aren't a subscriber
> is that you miss nuance and follow-ups.

True. But I asked the question very clearly[1], and nobody seems to be
interested in fixing the slowness issue *now*, and trying to improve
the performance of the more correct approach *later*. I read a few
more mails in the thread online where I wasn't CC'ed and to me it
still that nobody is pushing for a compromise.

If there's will for a compromise, then I can help, even for the
correct approach, but if not, I don't see the point of subscribing;
git's bash completion works perfectly fine for me *now*.

Cheers.

[1] http://article.gmane.org/gmane.comp.shells.zsh.devel/22480

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Bart Schaefer]

On May 5,  7:40pm, Felipe Contreras wrote:
}
} I asked the question very clearly[1], and nobody seems to be
} interested in fixing the slowness issue *now*, and trying to improve
} the performance of the more correct approach *later*.

I don't think there's any useful distinction.  "Now" is still relative
to how soon a new version is installable, and the level of effort
required to gut the existing code and produce a faster "seems to work
but is less correct" version appears comparable to the effort of making
performance improvements on the correct version.

Nikolai put a lot of effort into the current version and as we're all
volunteers here I can't fault him for not wanting to expend effort on
gutting it.  In the same way that the bash variant is good enough for
you, what he's contributed is presently working the way he prefers.  If
no one else is available to produce an alternative, well, that happens
sometimes with a volunteer work force.

If you're not willing to volunteer either, it's not helpful to drive by
with throwaway denigrations of what "you guys want."

Re: zle messes up 'words' variable?

[Felipe Contreras]

On Thu, May 5, 2011 at 10:38 PM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On May 5,  7:40pm, Felipe Contreras wrote:
> }
> } I asked the question very clearly[1], and nobody seems to be
> } interested in fixing the slowness issue *now*, and trying to improve
> } the performance of the more correct approach *later*.
>
> I don't think there's any useful distinction.  "Now" is still relative
> to how soon a new version is installable,

I don't care when the new version will be installable. I can use the
the latest zsh completion on my current system.

> and the level of effort
> required to gut the existing code and produce a faster "seems to work
> but is less correct" version appears comparable to the effort of making
> performance improvements on the correct version.

Perhaps, but that's not how I develop; first make it usable, then
improve it (without breaking it).

> Nikolai put a lot of effort into the current version and as we're all
> volunteers here I can't fault him for not wanting to expend effort on
> gutting it.  In the same way that the bash variant is good enough for
> you, what he's contributed is presently working the way he prefers.  If
> no one else is available to produce an alternative, well, that happens
> sometimes with a volunteer work force.

That's not the case, I volunteered myself to do that, but what's the
point if he is going to reject the patches?

> If you're not willing to volunteer either, it's not helpful to drive by
> with throwaway denigrations of what "you guys want."

Again, I did volunteer, didn't you see the link on the mail you just replied?
> [1] http://article.gmane.org/gmane.comp.shells.zsh.devel/22480

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Nikolai Weibull]

On Thu, May 5, 2011 at 22:14, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> On Thu, May 5, 2011 at 10:38 PM, Bart Schaefer
> <schaefer@brasslantern.com> wrote:

>> Nikolai put a lot of effort into the current version and as we're all
>> volunteers here I can't fault him for not wanting to expend effort on
>> gutting it.  In the same way that the bash variant is good enough for
>> you, what he's contributed is presently working the way he prefers.  If
>> no one else is available to produce an alternative, well, that happens
>> sometimes with a volunteer work force.

> That's not the case, I volunteered myself to do that, but what's the
> point if he is going to reject the patches?

> Again, I did volunteer, didn't you see the link on the mail you just replied?
>> [1] http://article.gmane.org/gmane.comp.shells.zsh.devel/22480

Please stop trying to rewrite history.  It’s not Git-like.  I can
quote history too:

http://article.gmane.org/gmane.comp.shells.zsh.devel/22488

I don’t know how much of the rest of the thread you didn’t read, but I
did actually solve the problem:

http://article.gmane.org/gmane.comp.shells.zsh.devel/22491

To be clear, I didn’t do it to appease you.

I am still open for suggestions on how to improve git-diff-index usage:

http://article.gmane.org/gmane.comp.shells.zsh.devel/22544

Re: zle messes up 'words' variable?

[Felipe Contreras]

On Tue, May 3, 2011 at 5:39 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On May 3, 11:05am, Felipe Contreras wrote:
> }
> } The fact that it's documented as a special variable doesn't mean that
> } the behavior should be totally unexpected.
> }
> } 1) Without modifier
> }
> } Then 'words' can be re-used like any other variable.
> }
> } 2) With 'local' modifier
> }
> } Then modifying 'words' has an impact only on the current function, not
> } on the scope where the variable was defined as local.
>
> Ahh, I see the problem.  You haven't read the entire documentation.
> At the very top of (what in the info file is) section 19.2 is this
> paragraph (caps mine for emphasis):
>
>    Inside completion widgets, AND ANY FUNCTIONS CALLED FROM THEM, some
>    parameters have special meaning; outside these functions they are
>    not special to the shell in any way. These parameters are used to
>    pass information between the completion code and the completion
>    widget. Some of the builtin commands and the condition codes use or
>    change the current values of these parameters. Any existing values
>    will be hidden during execution of completion widgets; EXCEPT FOR
>    COMPSTATE, THE PARAMETERS ARE RESET ON EACH FUNCTION EXIT (INCLUDING
>    NESTED FUNCTION CALLS FROM WITHIN THE COMPLETION WIDGET) to the
>    values they had when the function was entered.
>
> In other words, these variable are always local in all scopes, unless
> they are explicitly declared otherwise (which in this case requires
> the use of typeset -h).  That's part of their special-ness, and is one
> of the things I assert is extremely unlikely to change.

Except that 'words' is not really acting as local, and 'local' should
be considered 'declared otherwise'. That's part of the weirdness.

> } 2.1) When using #compdef tag
> }
> } Then 'words' can be modified like any other variable.
>
> I thought you'd decided you were unable to reproduce this?  If you now
> *are* able to reproduce it, then *that* might be a bug.

That's true.

> } 3) With 'typedef -h'
> }
> } Then 'words' can be modified like any other variable.
> }
> } I was expecting an explanation of these discrepancies, and a way to
> } globally make 'words' work as expected without having to replace each
> } use of 'local' by 'typedef -h'.
>
> I apologize for not understanding the source of the mis-understanding.
> I was focusing on why #compdef might make a difference, not on the
> details of the actual scoping problem you perceived.

I see.

> } > If what you want to discuss is ways to fix bashcompinit so that the
> } > functions in bash completion work as expected, that's one thing.  If
> } > what you're asking is for the special-ness of "words" to be undone
> } > for completion in general, that's so unlikely to happen as to not be
> } > worth continuing.
> }
> } You are not good at compartmentalizing aren't you?
>
> I'm really disappointed in the way that mailing list discourse in
> general lately (and not just on this list) tends to rapidly descend
> into insults and ad-hominem attacks.  I've been watching the CentOS
> list collapse under the weight of this sort of thing and I plead that
> we try to avoid it here.

I didn't intend that as an insult; some people are good are
compartmentalizing, some people are not.

> } If you fix the issue with my example, you fix the issue with
> } bashcompinit. It's the same issue. Or "riddle" if that word makes you
> } more comfortable.
>
> No, it's not the same issue, because bashcompinit has responsibility
> for loading the bash completion functions in a way that makes them
> compatible.  That's not the same as first loading bashcompinit and
> then independently defining a new completion, though I lean toward
> the conclusion that use of the bash-compatible "complete" command is
> probably the correct place to fix this if it can be managed.

Well, bashcompinit is also a script. If my script can be fixed without
modifying _foo() or set_vars(), then the same fix can be applied to
bashcompinit:

+typeset -h words
 complete -F _foo foo

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Bart Schaefer]

On May 3,  8:36pm, Felipe Contreras wrote:
} Subject: Re: zle messes up 'words' variable?
}
} On Tue, May 3, 2011 at 5:39 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
} >    change the current values of these parameters. Any existing values
} >    will be hidden during execution of completion widgets; EXCEPT FOR
} >    COMPSTATE, THE PARAMETERS ARE RESET ON EACH FUNCTION EXIT (INCLUDING
} >    NESTED FUNCTION CALLS FROM WITHIN THE COMPLETION WIDGET) to the
} >    values they had when the function was entered.
} >
} > In other words, these variable are always local in all scopes, unless
} > they are explicitly declared otherwise (which in this case requires
} > the use of typeset -h).  That's part of their special-ness, and is one
} > of the things I assert is extremely unlikely to change.
} 
} Except that 'words' is not really acting as local, and 'local' should
} be considered 'declared otherwise'. That's part of the weirdness.

"local" without -h has never changed the special properties of a zsh
variable.  It would be "weird" in a large number of other cases if it
were to be "fixed" for this case.

} +typeset -h words
}  complete -F _foo foo

No, that's making words non-special at the wrong scope; it may work
for your example because _foo never attempts to examine the special
value of $words, but in practice you can't create a useful completion
function without some reference to that special value.

(However, I think it still fails for your example, because the
typeset at global scope will be supplanted by the special local at
completion scope.  You have to wait until the local scope exists
before you can hide it with "typeset -h".)

Once the special "words" is hidden, the bash completion widgets can
ignore it because for them, the relevant bits are passed in "$@"; but
the _bash_complete wrapper that's installed by "complete -F" needs
access to $words in order to create the arguments it passes to _foo.

Hence the scope where my patch added typeset -h is only correct place
outside the the function called as $OPTARG where words can become non-
special without changing the arguments passed by that function call.

Re: zle messes up 'words' variable?

[Felipe Contreras]

On Wed, May 4, 2011 at 3:56 AM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On May 3,  8:36pm, Felipe Contreras wrote:
> } Subject: Re: zle messes up 'words' variable?
> }
> } On Tue, May 3, 2011 at 5:39 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> } >    change the current values of these parameters. Any existing values
> } >    will be hidden during execution of completion widgets; EXCEPT FOR
> } >    COMPSTATE, THE PARAMETERS ARE RESET ON EACH FUNCTION EXIT (INCLUDING
> } >    NESTED FUNCTION CALLS FROM WITHIN THE COMPLETION WIDGET) to the
> } >    values they had when the function was entered.
> } >
> } > In other words, these variable are always local in all scopes, unless
> } > they are explicitly declared otherwise (which in this case requires
> } > the use of typeset -h).  That's part of their special-ness, and is one
> } > of the things I assert is extremely unlikely to change.
> }
> } Except that 'words' is not really acting as local, and 'local' should
> } be considered 'declared otherwise'. That's part of the weirdness.
>
> "local" without -h has never changed the special properties of a zsh
> variable.  It would be "weird" in a large number of other cases if it
> were to be "fixed" for this case.
>
> } +typeset -h words
> }  complete -F _foo foo
>
> No, that's making words non-special at the wrong scope; it may work
> for your example because _foo never attempts to examine the special
> value of $words, but in practice you can't create a useful completion
> function without some reference to that special value.
>
> (However, I think it still fails for your example, because the
> typeset at global scope will be supplanted by the special local at
> completion scope.  You have to wait until the local scope exists
> before you can hide it with "typeset -h".)
>
> Once the special "words" is hidden, the bash completion widgets can
> ignore it because for them, the relevant bits are passed in "$@"; but
> the _bash_complete wrapper that's installed by "complete -F" needs
> access to $words in order to create the arguments it passes to _foo.
>
> Hence the scope where my patch added typeset -h is only correct place
> outside the the function called as $OPTARG where words can become non-
> special without changing the arguments passed by that function call.

You are right, however, it does work in git[1].

Here's a more complete example to what git is doing, and my proposed
'typeset -h' workaround.

---
#!bash

autoload -U +X bashcompinit && bashcompinit

set_vars ()
{
	cur=${COMP_WORDS[COMP_CWORD]}
	words=("${COMP_WORDS[@]}")
	cwords=$COMP_CWORD
}

_foo_bar ()
{
	local cur words cwords
	set_vars

	echo "foo bar: cur=${cur} words=${words} cwords=${cwords}" >>
/tmp/comp_test.txt
}

_foo ()
{
	typeset -h words

	local cur words cwords
	set_vars

	echo "foo: cur=${cur} words=${words} cwords=${cwords}" >> /tmp/comp_test.txt

	_foo_bar
}

complete -F _foo foo
---

[1] http://mid.gmane.org/1304605458-1483-1-git-send-email-felipe.contreras@gmail.com

-- 
Felipe Contreras

Re: zle messes up 'words' variable?

[Bart Schaefer]

On May 5,  5:48pm, Felipe Contreras wrote:
} Subject: Re: zle messes up 'words' variable?
}
} On Wed, May 4, 2011 at 3:56 AM, Bart Schaefer <schaefer@brasslantern.com> wrote:
} > Hence the scope where my patch added typeset -h is only correct place
} > outside the the function called as $OPTARG where words can become non-
} > special without changing the arguments passed by that function call.
} 
} You are right, however, it does work in git[1].

Just to be sure no one is confused by that "however" ...

It works in the referenced git example and in your new example because
in those new examples it's _inside_ the function called as $OPTARG.

This thread already has enough miscommunication over scoping. :-)

Re: zle messes up 'words' variable?

[Felipe Contreras]

On Thu, May 5, 2011 at 6:50 PM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On May 5,  5:48pm, Felipe Contreras wrote:
> } Subject: Re: zle messes up 'words' variable?
> }
> } On Wed, May 4, 2011 at 3:56 AM, Bart Schaefer <schaefer@brasslantern.com> wrote:
> } > Hence the scope where my patch added typeset -h is only correct place
> } > outside the the function called as $OPTARG where words can become non-
> } > special without changing the arguments passed by that function call.
> }
> } You are right, however, it does work in git[1].
>
> Just to be sure no one is confused by that "however" ...
>
> It works in the referenced git example and in your new example because
> in those new examples it's _inside_ the function called as $OPTARG.
>
> This thread already has enough miscommunication over scoping. :-)

Yes, that's what I meant :)

-- 
Felipe Contreras