_arguments not works as expected. Bug?

_arguments not works as expected. Bug?

[Vasiliy Ivanov]

I noticed when I use _arguments with -s ("more than one option per word") I see different behaviour
for "single-option" words completion and "multi-option":

#compdef testcmd
_arguments -s \
    "(-q --quiet -v --verbose)"{-v,--verbose} \
    "(-v --verbose)"{\*-q,\*--quiet}

# *-q used for better clearness

% testcmd -q <tab> producing only -q completion as expected (as -q excludes only -v), but:
% testcmd -q<tab> offered both -q and -v (why?)

Maybe I miss something, but I see no reason to allow -qv in this case, but to not allow -q -v.

-- 
Regards,
  Vasiliy Ivanov <beelzebubbie.logs@gmail.com>

Re: _arguments not works as expected. Bug?

[Bart Schaefer]

On Sep 25, 10:51pm, Vasiliy Ivanov wrote:
}
} % testcmd -q <tab> producing only -q completion as expected (as -q excludes only -v), but:
} % testcmd -q<tab> offered both -q and -v (why?)

As you might expect, it has to do with the implementation of -s and
the way completion always operates on a single word.

In order to complete from "x" to "xy" or "xyz", compadd has to be told
the set of all full words that might appear in that position.  It then
filters out the words that don't match.

Because of this, when you use "_arguments -s" and the context is the
middle of a word that appears to be multiple single letters, the words
passed to compadd have to be constructed by pasting together letters
from the set of all possible single letter arguments.  Thus if there
are single letter options -m, -n, and -o, and -m already appears, then
_arguments constructs the words -mm (if -m is repeatable), -mn, and
-mo, and passes those to compadd, because those are all the possible
words that begin with -m in that context. [*]

The complication is that it is the script that constructs -mn and -mo,
but the "exclusion list" definition is parsed by the "comparguments"
builtin and isn't available to the script; instead it's applied by
the internals when "compadd" is finally called.  So the script does
not "know" that it could drop some of those letters when generating
the possible permutations; it just generates all of them.

Down in compadd, the exclusion list operates on full words, not on
substrings.  (-v)-q doesn't mean to exclude "v" from words beginning
with "-" of which it is a substring, it means to exclude the word
"-v".  Since the words passed to compadd in this example are "-qq"
and "-qv", neither matches "-v" and so both are offered as valid
completions.

So now you have the answer to "why?".  Preventing it from happening
would require new C code (another option to "comparguments" perhaps?)
and/or some hairy script work to explode the word on the line out to
its individual letters, filter them, and glue them back together.

The particular feature of having exlusions work on multiple options
packed into the same word has not so far been considered important
enough to attract anyone willing to do that work.


[*] It would also work for it to ignore the fact that -m is already on
the line and generate -no, -om, etc., and allow comparison to the word
on the line to filter out the ones not starting with -m, but it tries
to do a small amount of optimization.

Re: _arguments not works as expected. Bug?

[Vasiliy Ivanov]

On 26.09.2014 13:02, Bart Schaefer wrote:
> On Sep 25, 10:51pm, Vasiliy Ivanov wrote:
> }
> } % testcmd -q <tab> producing only -q completion as expected (as -q excludes only -v), but:
> } % testcmd -q<tab> offered both -q and -v (why?)
> 
> As you might expect, it has to do with the implementation of -s and
> the way completion always operates on a single word.
> 
> In order to complete from "x" to "xy" or "xyz", compadd has to be told
> the set of all full words that might appear in that position.  It then
> filters out the words that don't match.
> 
> Because of this, when you use "_arguments -s" and the context is the
> middle of a word that appears to be multiple single letters, the words
> passed to compadd have to be constructed by pasting together letters
> from the set of all possible single letter arguments.  Thus if there
> are single letter options -m, -n, and -o, and -m already appears, then
> _arguments constructs the words -mm (if -m is repeatable), -mn, and
> -mo, and passes those to compadd, because those are all the possible
> words that begin with -m in that context. [*]
> 
> The complication is that it is the script that constructs -mn and -mo,
> but the "exclusion list" definition is parsed by the "comparguments"
> builtin and isn't available to the script; instead it's applied by
> the internals when "compadd" is finally called.  So the script does
> not "know" that it could drop some of those letters when generating
> the possible permutations; it just generates all of them.
> 
> Down in compadd, the exclusion list operates on full words, not on
> substrings.  (-v)-q doesn't mean to exclude "v" from words beginning
> with "-" of which it is a substring, it means to exclude the word
> "-v".  Since the words passed to compadd in this example are "-qq"
> and "-qv", neither matches "-v" and so both are offered as valid
> completions.
> 
> So now you have the answer to "why?".  Preventing it from happening
> would require new C code (another option to "comparguments" perhaps?)
> and/or some hairy script work to explode the word on the line out to
> its individual letters, filter them, and glue them back together.
> 
> The particular feature of having exlusions work on multiple options
> packed into the same word has not so far been considered important
> enough to attract anyone willing to do that work.
> 
> 
> [*] It would also work for it to ignore the fact that -m is already on
> the line and generate -no, -om, etc., and allow comparison to the word
> on the line to filter out the ones not starting with -m, but it tries
> to do a small amount of optimization.
> 

Thanks for detailed clarification. I see all complications, but also I see that very powerful
_arguments function looks slightly «incompleted» itself. Lot of people used only «basic» _arguments'
functionality – maybe because of lack of consistency in «extended» functional. Maybe  this
inconsistency is not «important enough» because of hard to use?

For instance, I tried to implement simple scenario:
command have a couple of «common» options: -a and -b; a few of mutually exclusive options: -k,-l,-m;
and a -h option that could be used only alone. Exclusion sets are not nested, so I can't implement
logic as «-h or (-a,-b (-k or -l or -m))». I can use prepending exclusion lists, but they aren't
worked with -s so I have to deny multi-option words though exclusions sets do their work even in
multi-option words. Very pity.

As a result, I failed to implement proper completion logic and forced to fallback to «simple» way –
all options in one heap. In my opinion, full absence of «exclusion» logic is better that *partial*
implementation because it is more predictable for user.

Sorry for my English :)

-- 
Regards,
  Vasiliy Ivanov <beelzebubbie.logs@gmail.com>

Re: _arguments not works as expected. Bug?

[Bart Schaefer]

On Sep 28,  4:10pm, Vasiliy Ivanov wrote:
}
} Thanks for detailed clarification. I see all complications, but
} also I see that very powerful _arguments function looks slightly
} "incompleted" itself. Lot of people used only "basic" _arguments'
} functionality - maybe because of lack of consistency in "extended"
} functional. Maybe this inconsistency is not "important enough" because
} of hard to use?

Whether it's "important" would depend on whether many people complained
about it when *using* the completions, not when writing the completion
functions.  I don't recall this complaint coming up before, though that
doesn't mean it hasn't.

However, in the context "important enough to attract anyone" I'm making
somewhat euphemistic reference to the fact that zsh is entirely maintained
by volunteers, so the things that get fixed are serious problems and easy
cosmetic fixes, unless the issue actually interests someone who wants to
dive into a difficult but mostly cosmetic problem.  When the completion
system was designed many years ago, there was such a person, but it's not
so easy to find anyone now.

} For instance, I tried to implement simple scenario:
} command have a couple of "common" options: -a and -b; a few of
} mutually exclusive options: -k,-l,-m; and a -h option that could be
} used only alone. Exclusion sets are not nested, so I can't implement
} logic as "-h or (-a,-b (-k or -l or -m))". I can use prepending
} exclusion lists, but they aren't worked with -s so I have to deny
} multi-option words though exclusions sets do their work even in
} multi-option words. Very pity.

Even in the best cases, completion is only supposed to provide reminders,
help avoid typo errors, and fill in long strings such as file paths; it
isn't meant to be able to enforce correctness of the command.

I think something like

  _arguments '(-)-h' '(-h)'{-a,-b} - '(exclusive)' '(-h)'{-k,-l,-m}

expresses all of that except the part about multi-option words.  I would
have thought that

  _arguments : '(-h)'{-a,-b} - '(exclusive)' {-k,-l,-m} - help -h

would also have worked, but for some reason the (-h) exclusion doesn't
work when formulated that way; perhaps someone would be interested in
tracking *that* down.