Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Marlon Richert]

The following line in _main_complete

[[ -n "$_comp_mesg" ]] && break

has the effect that, whenever _message has been called (with only few
exceptions), the next completer won't be tried, _even when
$compstate[nmatches] is zero._

Why? What is the reason for this?

And is there a convenient way to work around this behavior? I want
_history to be tried when _complete fails, but this behavior often
prevents it. For example, if I try `grep \t`, then I get only the
message `pattern`, whereas I would like to get history words.

And if there is not an easy workaround, would you accept a patch from
me that changes this?

—Marlon

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Mikael Magnusson]

On 3/28/21, Marlon Richert <marlon.richert@gmail.com> wrote:
> The following line in _main_complete
>
> [[ -n "$_comp_mesg" ]] && break
>
> has the effect that, whenever _message has been called (with only few
> exceptions), the next completer won't be tried, _even when
> $compstate[nmatches] is zero._
>
> Why? What is the reason for this?
>
> And is there a convenient way to work around this behavior? I want
> _history to be tried when _complete fails, but this behavior often
> prevents it. For example, if I try `grep \t`, then I get only the
> message `pattern`, whereas I would like to get history words.
>
> And if there is not an easy workaround, would you accept a patch from
> me that changes this?

I haven't tested this, but I suspect that if this check were not
there, then loops due to approximate matching would print the message
multiple times.

-- 
Mikael Magnusson

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Marlon Richert]

On Sun, Mar 28, 2021 at 8:10 PM Mikael Magnusson <mikachu@gmail.com> wrote:
> On 3/28/21, Marlon Richert <marlon.richert@gmail.com> wrote:
> > The following line in _main_complete
> >
> > [[ -n "$_comp_mesg" ]] && break
> >
> > has the effect that, whenever _message has been called (with only few
> > exceptions), the next completer won't be tried, _even when
> > $compstate[nmatches] is zero._
> >
> > Why? What is the reason for this?
>
> I haven't tested this, but I suspect that if this check were not
> there, then loops due to approximate matching would print the message
> multiple times.

Nope:

% autoload -Uz compinit; compinit
% zstyle '*' format '%d'
% _tst() { repeat 10; do _message -e 'TEST'; done }
% compdef _tst tst
% tst \t
TEST
%

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Bart Schaefer]

On Sun, Mar 28, 2021 at 9:34 AM Marlon Richert <marlon.richert@gmail.com> wrote:
>
> The following line in _main_complete
>
> [[ -n "$_comp_mesg" ]] && break
>
> has the effect that, whenever _message has been called (with only few
> exceptions), the next completer won't be tried, _even when
> $compstate[nmatches] is zero._

This is documented, though not precisely accurately:

     ... Note that if there
     are no matches at the time this function is called,
     compstate[insert] is cleared, so additional matches generated later
     are not inserted on the command line.

> Why? What is the reason for this?

_message is intended to be called as a last resort when the completion
function believes it is impossible to generate matches.

(FWIW, I think this behavior pre-dates the ability to complete words
from history, which obviously means it's never literally impossible to
generate something.)

> And is there a convenient way to work around this behavior?

What does "convenient" mean here?  More convenient than just invoking
a binding for history-complete when you see the message, so that the
_grep completion is bypassed?

You might be able to do something by specifying a different completion
for the tag :completion::complete:grep:argument-1:

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Marlon Richert]

On Sun, Mar 28, 2021 at 8:47 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sun, Mar 28, 2021 at 9:34 AM Marlon Richert <marlon.richert@gmail.com> wrote:
> > Why? What is the reason for this?
>
> _message is intended to be called as a last resort when the completion
> function believes it is impossible to generate matches.
>
> (FWIW, I think this behavior pre-dates the ability to complete words
> from history, which obviously means it's never literally impossible to
> generate something.)

Would there be any drawback to removing this check? A completer can
already return 0 to prevent further completers from being called. This
seems like unnecessary special-casing to me. Correct me if I'm wrong.

> > And is there a convenient way to work around this behavior?
>
> What does "convenient" mean here?  More convenient than just invoking
> a binding for history-complete when you see the message, so that the
> _grep completion is bypassed?

Convenient would be that I could specify through a zstyle that I don't
want that behavior. Though, why anyone would actually want the current
behavior is unclear to me.

> You might be able to do something by specifying a different completion
> for the tag :completion::complete:grep:argument-1:

_grep was just an example. I'm looking for a more general-purpose solution.

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Bart Schaefer]

On Sun, Mar 28, 2021 at 11:07 AM Marlon Richert
<marlon.richert@gmail.com> wrote:
>
> Would there be any drawback to removing this check?

I think it would break the intended behavior of "_guard" when used in
an _arguments action (which is how we're getting into _message in the
first place).

There is some divergence between the doc for _guard and the actual
implementation.  The doc says:

_guard [ OPTIONS ] PATTERN DESCR
     This function displays DESCR if PATTERN matches the string to be
     completed.  It is intended to be used in the ACTION for the
     specifications passed to _arguments and similar functions.

     The return status is zero if the message was displayed and the word
     to complete is not empty, and non-zero otherwise.

In fact the return status only depends on the word being non-empty,
not on the message at all.

Another curiousity is that _message is supposed to take an optional tag

_message -e [ TAG ] DESCR

but _guard never passes it one, it just passes all it's non-option
arguments as a string:

_message -e "$*"

This has the effect that anything looked up as a style down-stack from
_message repeats the context as the tag, e.g. from _complete_debug:

       +_description:15> zstyle -s
:completion::complete:grep:argument-1:argument-1 group-name gname

So there may very well be something more needed here than just
removing that check.

> Convenient would be that I could specify through a zstyle that I don't
> want that behavior. Though, why anyone would actually want the current
> behavior is unclear to me.

In this case the author of the _grep completion function determined
that to be the desirable behavior, otherwise _guard would not have
been used in that context.

Whether an author should ever be able to "know better" than the user
is an entirely different question.  But there are tags generated for
both _guard and _message that could be used to look up additional
styles within those functions, if we as a group can agree on the
semantics.

Meanwhile, what about putting this in your completer style?

_complete_or_history () {
  _complete "$@"
  local ret=$? _mesg=$_comp_mesg
  if [[ $ret -ne 0 || -n "$_comp_mesg" ]]
  then
    _comp_mesg=''
    _history "$@" && ret=0
  fi
  (( $compstate[nmatches] )) || _comp_mesg=$mesg
  return ret
}

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Oliver Kiddle]

Marlon Richert wrote:
> The following line in _main_complete
>
> [[ -n "$_comp_mesg" ]] && break
>
> has the effect that, whenever _message has been called (with only few
> exceptions), the next completer won't be tried, _even when
> $compstate[nmatches] is zero._
>
> Why? What is the reason for this?

This allows you to see a hint of the command argument expected at the
cursor position. When you use commands that you're not especially
familiar with, getting prompts telling you what sort of value is
expected along with units and defaults can be very useful.

_messages -e/compadd -x was a later addition that allowed it to prompt
as-if there were matches which is useful for things like patterns and
numbers that can't be usefully completed. This also tied in with the
fake styles for adding fake matches.

I've always had a separate key bound for history completion.

> And is there a convenient way to work around this behavior? I want
> _history to be tried when _complete fails, but this behavior often
> prevents it. For example, if I try `grep \t`, then I get only the
> message `pattern`, whereas I would like to get history words.

You could create a wrapper for _complete that unsets it, something like:
  _complete_nomesg() {
    _complete
    local ret=$?
    _comp_mesg=
    return ret
  }

It might be better to experiment with something like that first. You'll
find that _message changing compstate[insert] is also affecting it.

> And if there is not an easy workaround, would you accept a patch from
> me that changes this?

That would break things that I've got too accustomed to over many years
so I'd not be enthusiastic about a patch that "changes this". Many
completion functions rely on the fact that they don't need to care so
much about return status if _message is called.

It could be possible to add some style or perhaps change the completer
style to allow for different criteria as to whether it continues. A
problem with trying to change this at the level of _main_complete is
that that is very generic and has no detailed information about the
context so any style lookup would be a basic on/off rather than being
context-sensitive.

If you want to experiment with it, guarding the first _comp_mesg
assignment in _message with [[ -n "$3" ]] && allows the format style to
be set to the empty string to disable the setting.

Oliver

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Bart Schaefer]

On Sun, Mar 28, 2021 at 2:42 PM Oliver Kiddle <opk@zsh.org> wrote:
>
> If you want to experiment with it, guarding the first _comp_mesg
> assignment in _message with [[ -n "$3" ]] && allows the format style to
> be set to the empty string to disable the setting.

I don't follow how the format style gets involved here?

Adding on to my previous partial dissection of _guard, it appears that
_main_complete does some messing about with compstate[insert] when
_comp_mesg is set which in a way which is contradictory to what
_message does with it:  _main_complete sets it unconditionally to the
empty string, but _message only does so if the existing value contains
"unambiguous".

What am I missing?

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Marlon Richert]

On Mon, Mar 29, 2021 at 12:42 AM Oliver Kiddle <opk@zsh.org> wrote:
> Marlon Richert wrote:
> > And is there a convenient way to work around this behavior? I want
> > _history to be tried when _complete fails, but this behavior often
> > prevents it. For example, if I try `grep \t`, then I get only the
> > message `pattern`, whereas I would like to get history words.
>
> You could create a wrapper for _complete that unsets it, something like:
>   _complete_nomesg() {
>     _complete
>     local ret=$?
>     _comp_mesg=
>     return ret
>   }
>
> It might be better to experiment with something like that first. You'll
> find that _message changing compstate[insert] is also affecting it.

I've been using some variation of the code below in my
[zsh-autocomplete plugin]
(https://github.com/marlonrichert/zsh-autocomplete) since last July
(so, for about 9 months now) and I have yet to experience any issues
from it, nor have any of my end users reported any issues from it.
That’s why I’m asking whether the behavior upstream is actually
necessary.

   autoload +X -Uz _complete
   functions[_autocomplete._complete]=$functions[_complete]
   _complete() {
     _autocomplete._complete "$@"
     local -i ret=$?
     (( ret )) && _comp_mesg=''
     return ret
   }

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Oliver Kiddle]

Marlon Richert wrote:
> I've been using some variation of the code below in my
> [zsh-autocomplete plugin]
> (https://github.com/marlonrichert/zsh-autocomplete) since last July
> (so, for about 9 months now) and I have yet to experience any issues
> from it, nor have any of my end users reported any issues from it.
> That’s why I’m asking whether the behavior upstream is actually
> necessary.

Perhaps I wasn't clear in my reply but yes I do regard it as necessary.

Maybe that code doesn't cause "issues" but it does change behaviour and
stop completion from working in the way I've come to expect it.

There are some related aspects of the current behaviour I'm not
especially keen on like _guard not taking a tag and situations where you
get both the description and the message constructed from the format
style with the warnings tag. Those should be two distinct cases and to
cover your use-case there might be a way to control whether completion
continues.

What does your completer style look like out-of-interest? Do you set
zstyle ':completion:*:warnings' format ?

Oliver

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Marlon Richert]

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

On Wed, Mar 31, 2021 at 12:25 PM Oliver Kiddle <opk@zsh.org> wrote:
> What does your completer style look like out-of-interest? Do you set
> zstyle ':completion:*:warnings' format ?

zstyle -e ':completion:*' completer _autocomplete.config.completer
_autocomplete.config.completer() {
  reply=( _expand _complete _correct _complete:-fuzzy _history
_history:-fuzzy _ignored )
  [[ CURRENT -eq 1 && -z $PREFIX$SUFFIX && $compstate[context] == command
]] &&
    reply=( _complete )
}

zstyle -e ':completion:*:warnings' format
_autocomplete.config.warnings-format
_autocomplete.config.warnings-format() {
  local d=${${(j:, :)_lastdescr[@]:#}/(#m)*, /$MATCH[1,-3] or }
  reply=( $'%{\e[01;02;39m%}'"No ${${:-$PREFIX$SUFFIX}:+matching }$d
found."$'%{\e[0m%}' )
}

[-- Attachment #2: Type: text/html, Size: 1033 bytes --]

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Daniel Shahaf]

Marlon Richert wrote on Wed, Mar 31, 2021 at 09:29:00 +0300:
> On Mon, Mar 29, 2021 at 12:42 AM Oliver Kiddle <opk@zsh.org> wrote:
> > Marlon Richert wrote:
> > > And is there a convenient way to work around this behavior? I want
> > > _history to be tried when _complete fails, but this behavior often
> > > prevents it. For example, if I try `grep \t`, then I get only the
> > > message `pattern`, whereas I would like to get history words.
> >
> > You could create a wrapper for _complete that unsets it, something like:
> >   _complete_nomesg() {
> >     _complete
> >     local ret=$?
> >     _comp_mesg=
> >     return ret
> >   }
> >
> > It might be better to experiment with something like that first. You'll
> > find that _message changing compstate[insert] is also affecting it.
> 
> I've been using some variation of the code below in my
> [zsh-autocomplete plugin]
> (https://github.com/marlonrichert/zsh-autocomplete) since last July
> (so, for about 9 months now) and I have yet to experience any issues
> from it, nor have any of my end users reported any issues from it.
> That’s why I’m asking whether the behavior upstream is actually
> necessary.
> 
>    autoload +X -Uz _complete
>    functions[_autocomplete._complete]=$functions[_complete]
>    _complete() {
>      _autocomplete._complete "$@"
>      local -i ret=$?
>      (( ret )) && _comp_mesg=''
>      return ret

Considered using an «{ } always { }» block?

>    }
> 

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Marlon Richert]

> On 1. Apr 2021, at 2.20, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
>> 
>>   autoload +X -Uz _complete
>>   functions[_autocomplete._complete]=$functions[_complete]
>>   _complete() {
>>     _autocomplete._complete "$@"
>>     local -i ret=$?
>>     (( ret )) && _comp_mesg=''
>>     return ret
> 
> Considered using an «{ } always { }» block?

What for? I have no need to ever restore the original _complete().

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Daniel Shahaf]

Marlon Richert wrote on Thu, Apr 01, 2021 at 08:11:35 +0300:
> 
> > On 1. Apr 2021, at 2.20, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> >> 
> >>   autoload +X -Uz _complete
> >>   functions[_autocomplete._complete]=$functions[_complete]
> >>   _complete() {
> >>     _autocomplete._complete "$@"
> >>     local -i ret=$?
> >>     (( ret )) && _comp_mesg=''
> >>     return ret
> > 
> > Considered using an «{ } always { }» block?
> 
> What for?

For clarity, since it seems that's the pattern you're shooting for.
Also, it would take care of preserving $?.

> I have no need to ever restore the original _complete().

An always block wouldn't do this.

Cheers,

Daniel

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Marlon Richert]

> On 1. Apr 2021, at 20.08, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
>>> Considered using an «{ } always { }» block?
>> 
>> What for?
> 
> For clarity, since it seems that's the pattern you're shooting for.
> Also, it would take care of preserving $?.
> 
>> I have no need to ever restore the original _complete().
> 
> An always block wouldn't do this.

OK, now I’m really lost. I have no idea what you’re talking about. Could you post some code to illustrate what you’re talking about?

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Daniel Shahaf]

Daniel Shahaf wrote on Thu, Apr 01, 2021 at 17:08:00 +0000:
> Marlon Richert wrote on Thu, Apr 01, 2021 at 08:11:35 +0300:
> > 
> > > On 1. Apr 2021, at 2.20, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> > >> 
> > >>   autoload +X -Uz _complete
> > >>   functions[_autocomplete._complete]=$functions[_complete]
> > >>   _complete() {
> > >>     _autocomplete._complete "$@"
> > >>     local -i ret=$?
> > >>     (( ret )) && _comp_mesg=''
> > >>     return ret
> > > 
> > > Considered using an «{ } always { }» block?
> > 
> > What for?
> 
> For clarity, since it seems that's the pattern you're shooting for.
> Also, it would take care of preserving $?.

I was thinking of this:

_complete() {
  {
    _autocomplete._complete "$@"
  } always {
    (( $? )) && _comp_mesg=''
  }
}

Note that always blocks don't affect the overall exit code:

% { (exit 42) } always { (exit 0) }; echo $?
42
% 

> > I have no need to ever restore the original _complete().
> 
> An always block wouldn't do this.

Re: Why does _main_complete not try the next completer when $_comp_mesg is non-zero?

[Marlon Richert]

> On 3. Apr 2021, at 6.12, Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> 
> I was thinking of this:
> 
> _complete() {
>  {
>    _autocomplete._complete "$@"
>  } always {
>    (( $? )) && _comp_mesg=''
>  }
> }
> 
> Note that always blocks don't affect the overall exit code:
> 
> % { (exit 42) } always { (exit 0) }; echo $?
> 42
> %

Aha. I wasn’t aware that `always` behaves that way with regards to exit status. Thanks for the tip!