completion, sorting of options

completion, sorting of options

[Thomas Lauer]

So I'm writing my first completion file (gory thing, just the right
thing for a rainy Sat afternoon). The completion file itself is more or
less working, but I've run into trouble with the options display after
"-<TAB>".

Basically what I want to achieve is that the options are listed in
precisely the order given in the completion file (there are many options
and some are much more probable to be used than others, so I want these
at the top (I know that further typing will limit the shown completions
but not all people using this will be up to that... a simple up/down
might work better for some)).

So I created a zstyle like this:

zstyle ':completion:*:*:mycmd:*:*' sort false

And this *does* change the order but not as expected. Rather, the
completion now first displays all single-letter options (-X), followed
by the -X=value options. Those two sublists are indeed shown in the same
order as in the completion file but in the completion the -X options
*follow* the -X=value options. So there's still something going on...
but I can't see what. (I've also tried "sort nosort"... no change).

Probably the zstyle still needs some more massaging but I haven't a clue
where to look... but one of the z-gurus might know?!

Thanks for helping out  T

Re: completion, sorting of options

[Mikael Magnusson]

On 11/4/23, Thomas Lauer <thomas.lauer@virgin.net> wrote:
> So I'm writing my first completion file (gory thing, just the right
> thing for a rainy Sat afternoon). The completion file itself is more or
> less working, but I've run into trouble with the options display after
> "-<TAB>".
>
> Basically what I want to achieve is that the options are listed in
> precisely the order given in the completion file (there are many options
> and some are much more probable to be used than others, so I want these
> at the top (I know that further typing will limit the shown completions
> but not all people using this will be up to that... a simple up/down
> might work better for some)).

If you give -V foo it will create an unsorted group, but it is a bit
unconventional to do this for sorting regular options by usage. You
could also consider adding the common options with a -J common and the
other ones with a -J rare, or something to that effect.

-- 
Mikael Magnusson

Re: completion, sorting of options

[Thomas Lauer]

From: Mikael Magnusson <mikachu@gmail.com>
Date: Sat, 4 Nov 2023 16:49:25 +0100

> On 11/4/23, Thomas Lauer <thomas.lauer@virgin.net> wrote:
> > So I'm writing my first completion file (gory thing, just the right
> > thing for a rainy Sat afternoon). The completion file itself is more or
> > less working, but I've run into trouble with the options display after
> > "-<TAB>".
> >
> > Basically what I want to achieve is that the options are listed in
> > precisely the order given in the completion file (there are many options
> > and some are much more probable to be used than others, so I want these
> > at the top (I know that further typing will limit the shown completions
> > but not all people using this will be up to that... a simple up/down
> > might work better for some)).
> 
> If you give -V foo it will create an unsorted group, but it is a bit
> unconventional to do this for sorting regular options by usage. You
> could also consider adding the common options with a -J common and the
> other ones with a -J rare, or something to that effect.

Thanks, Mikael. Alas, I am not going to pretend that I understand what
you've written.

As I wrote
> first completion file
and I threw together (w/ some help from big G) something that works (ie
it lists the options on -<TAB>, completes two directories and adds a
final command as to what to do). The options are the problem as there
are many and for the reasons given I'd like to list them in a specific
order.

Anyway, my naive interpretation of the "sort false" bit in the zstyle is
that sort is false, ie no sorting is done ("sort nosort" seems equally
clear but doesn't not sort).

Apparently I could try to do something about that but I've not really
understood what. Could you please elaborate?

Again thanks  T

Re: completion, sorting of options

[Bart Schaefer]

On Sun, Nov 5, 2023 at 7:03 AM Thomas Lauer <thomas.lauer@virgin.net> wrote:
>
> From: Mikael Magnusson <mikachu@gmail.com>
> Date: Sat, 4 Nov 2023 16:49:25 +0100
> >
> > If you give -V foo it will create an unsorted group, but it is a bit
> > unconventional to do this for sorting regular options by usage. You
> > could also consider adding the common options with a -J common and the
> > other ones with a -J rare, or something to that effect.
>
> Thanks, Mikael. Alas, I am not going to pretend that I understand what
> you've written.

He's referring to completion grouping controls, passed to compadd.  If
you're not calling compadd directly, you'll need to show us what you
are doing to be able to get a clear explanation of what to do
differently.

Re: completion, sorting of options

[Thomas Lauer]

From: Bart Schaefer <schaefer@brasslantern.com>
Date: Sun, 5 Nov 2023 07:58:59 -0800

> On Sun, Nov 5, 2023 at 7:03?AM Thomas Lauer <thomas.lauer@virgin.net> wrote:
> >
> > From: Mikael Magnusson <mikachu@gmail.com>
> > Date: Sat, 4 Nov 2023 16:49:25 +0100
> > >
> > > If you give -V foo it will create an unsorted group, but it is a bit
> > > unconventional to do this for sorting regular options by usage. You
> > > could also consider adding the common options with a -J common and the
> > > other ones with a -J rare, or something to that effect.
> >
> > Thanks, Mikael. Alas, I am not going to pretend that I understand what
> > you've written.
> 
> He's referring to completion grouping controls, passed to compadd.  If
> you're not calling compadd directly, you'll need to show us what you
> are doing to be able to get a clear explanation of what to do
> differently.

I am not calling compadd directly. Should I?

I have a file called _cmd with basically this (cut down to the important
bits):

#compdef cmd
_arguments \
	"-a=[value can be a b c d]" \
	"-b=[value can be e f g h]" \
	.... many more options
	"-y[do this]" \
	"-z[do that]" \
	....
	"1: :_files -/" \
	"2: :_files -/" \
	"3: :_cmd_commands"

_cmd_commands(){
	local cmds=("aa:do aa" "bb:do bb" ....)
	_describe -t cmds "commands" cmds
}

This file sits in usr/share/zsh/vendor-completions/. Everything but the
sort order for the options works as expected.

HTH. T

Re: completion, sorting of options

[Thomas Lauer]

From: Thomas Lauer <thomas.lauer@virgin.net>
Date: Sat, 04 Nov 2023 15:42:25 +0000

> So I'm writing my first completion file (gory thing, just the right
> thing for a rainy Sat afternoon). The completion file itself is more or
> less working, but I've run into trouble with the options display after
> "-<TAB>".
> 
> Basically what I want to achieve is that the options are listed in
> precisely the order given in the completion file (there are many options
> and some are much more probable to be used than others, so I want these
> at the top (I know that further typing will limit the shown completions
> but not all people using this will be up to that... a simple up/down
> might work better for some)).
> 
> So I created a zstyle like this:
> 
> zstyle ':completion:*:*:mycmd:*:*' sort false
> 
> And this *does* change the order but not as expected. Rather, the
> completion now first displays all single-letter options (-X), followed
> by the -X=value options. Those two sublists are indeed shown in the same
> order as in the completion file but in the completion the -X options
> *follow* the -X=value options. So there's still something going on...
> but I can't see what. (I've also tried "sort nosort"... no change).
> 
> Probably the zstyle still needs some more massaging but I haven't a clue
> where to look... but one of the z-gurus might know?!
> 
> Thanks for helping out  T

Apparently there's no obvious solution to the zstyle "sort false/nosort"
thing. How about creating yet another value for this style:

zstyle ':completion:*:*:mycmd:*:*' sort nosort_and_i_really_mean_nosort

:-)

Cheers  T

Re: completion, sorting of options

[Bart Schaefer]

On Wed, Nov 8, 2023 at 5:25 AM Thomas Lauer <thomas.lauer@virgin.net> wrote:
>
> Apparently there's no obvious solution to the zstyle "sort false/nosort"
> thing.

The short answer here is that you can't do this with a single call to
_arguments.  The option parsing in _arguments creates two lists, one
of options that don't take values and one of options that do, and
subsequently it passes them to compadd in two separate calls, so
compadd sees them in that order even though _arguments does not.

The "simplest" thing to do here might be to make an array of the
options that must be in a particular order, and then loop that array
calling _arguments once for each, and then make a final call to
_arguments with all the other specs.

> zstyle ':completion:*:*:mycmd:*:*' sort nosort_and_i_really_mean_nosort

Not likely to happen as it would mean a significant change to the deep
guts of _arguments.

Re: completion, sorting of options

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 575 bytes --]


On 2023-11-08 12:09, Bart Schaefer wrote:
> Not likely to happen as it would mean a significant change to the deep
> guts of _arguments.
>
Purely theoretically, is this the sort of thing where a hook might be  
used?  I have only the most cursory understanding of hooks, but my 
minimal exposure to the idea leads me to think that zsh lets you sorta 
grab some output -- hook it -- and redirect it to some custom code which 
would do whatever one might want.  Or, in the spirit of precmd() it's 
very cool that one can very easily add any sort of code one wants in there.

[-- Attachment #2: Type: text/html, Size: 1037 bytes --]

Re: completion, sorting of options

[Bart Schaefer]

On Wed, Nov 8, 2023 at 12:44 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Purely theoretically, is this the sort of thing where a hook might be  used?

Not really.  Hooks are typically for things driven from the C code,
where an escape to shell control extends the default behavior, whereas
_arguments is entirely shell code -- you can either call it
differently (as I've suggested) or replace it with a different
function.

The next-closest thing to "hooks" is using the -e option of zstyle to
cause code to run when a style is looked up, but of course for the
most typical cases the implementation will look up a style as few
times as possible and just store the result.

Re: completion, sorting of options

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 392 bytes --]


On 2023-11-08 13:34, Bart Schaefer wrote:
> Purely theoretically, is this the sort of thing where a hook might be 
> used?
> Not really.  Hooks are typically for things driven from the C code,
> where an escape to shell control extends the default behavior, whereas
> _arguments is entirely shell code
Thanks.  No way of getting one's hands on _arguments then in the way I 
was speculating.

[-- Attachment #2: Type: text/html, Size: 1007 bytes --]

Re: completion, sorting of options

[Thomas Lauer]

From: Bart Schaefer <schaefer@brasslantern.com>
Date: Wed, 8 Nov 2023 12:09:20 -0800

> On Wed, Nov 8, 2023 at 5:25?AM Thomas Lauer <thomas.lauer@virgin.net> wrote:
> >
> > Apparently there's no obvious solution to the zstyle "sort false/nosort"
> > thing.
> 
> The short answer here is that you can't do this with a single call to
> _arguments.  The option parsing in _arguments creates two lists, one
> of options that don't take values and one of options that do, and
> subsequently it passes them to compadd in two separate calls, so
> compadd sees them in that order even though _arguments does not.
> 
> The "simplest" thing to do here might be to make an array of the
> options that must be in a particular order, and then loop that array
> calling _arguments once for each, and then make a final call to
> _arguments with all the other specs.
> 
> > zstyle ':completion:*:*:mycmd:*:*' sort nosort_and_i_really_mean_nosort
> 
> Not likely to happen as it would mean a significant change to the deep
> guts of _arguments.

As the :-) in my message showed (or perhaps didn't) this was not really
a serious suggestion. It's just a bit frustrating that what I'd call an
implementation detail within _arguments makes this call perform in a way
a (probably naive) user doesn't expect and can't easily grok or rectify.

Anyway, thanks for the clarification.  T

Re: completion, sorting of options

[Bart Schaefer]

On Thu, Nov 9, 2023 at 5:32 AM Thomas Lauer <thomas.lauer@virgin.net> wrote:
>
> From: Bart Schaefer <schaefer@brasslantern.com>
> Date: Wed, 8 Nov 2023 12:09:20 -0800
>
> > On Wed, Nov 8, 2023 at 5:25?AM Thomas Lauer <thomas.lauer@virgin.net> wrote:
> > >
> > > zstyle ':completion:*:*:mycmd:*:*' sort nosort_and_i_really_mean_nosort
> >
> > Not likely to happen
>
> As the :-) in my message showed (or perhaps didn't) this was not really
> a serious suggestion.

Oh, I got that, just felt it deserved further comment for the audience.

> It's just a bit frustrating that what I'd call an
> implementation detail within _arguments makes this call perform in a way
> a (probably naive) user doesn't expect and can't easily grok or rectify.

Combination of an unanticipated use case and the order in which
features like unsorted groups were added to "compadd" relative to the
implementation of _arguments.

Re: completion, sorting of options

[Thomas Lauer]

From: Bart Schaefer <schaefer@brasslantern.com>
Date: Thu, 9 Nov 2023 10:07:16 -0800

> On Thu, Nov 9, 2023 at 5:32?AM Thomas Lauer <thomas.lauer@virgin.net> wrote:
> 
> > It's just a bit frustrating that what I'd call an
> > implementation detail within _arguments makes this call perform in a way
> > a (probably naive) user doesn't expect and can't easily grok or rectify.
> 
> Combination of an unanticipated use case and the order in which
> features like unsorted groups were added to "compadd" relative to the
> implementation of _arguments.

Weeeellllll, unanticipated use case... that's debatable IMO. Obviously
somebody thought that a no-sort option would be a good idea, in fact
such a good idea that there are two ways to do it. But what they then
proceeded to do was to have the nosort option sort just a tiny little
bit anyway, for good measure. In my book that's not an unanticipated use
case but questionable design.

I am an old-fashioned git and think that an option should do what it
says on the tin. I'd readily accept that "sort false" does the tiny
little bit anyway, but then nosort should definitely switch sorting off
completely (or vice versa).

Anyway, horse dead, flogging ends here.  T

Re: completion, sorting of options

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 745 bytes --]


On 2023-11-10 06:04, Thomas Lauer wrote:
> But what they then
> proceeded to do was to have the nosort option sort just a tiny little
> bit anyway, for good measure. In my book that's not an unanticipated use
> case but questionable design.
>
> I am an old-fashioned git and think that an option should do what it
> says on the tin.

Anyway, horse dead, flogging ends here. T

One more lash to the dead horse: It's another example of something being 
'too helpful'. But it's all water under the bridge now. As Bart says, 
too dangerous to tinker in the deepest entrails of the code. Still I do 
dream of zsh 6.0 where old traditions might be questioned, dead code cut 
out, compatibility with extinct shells no longer worried about, and so on.

[-- Attachment #2: Type: text/html, Size: 1415 bytes --]

Re: completion, sorting of options

[Bart Schaefer]

On Fri, Nov 10, 2023 at 6:04 AM Thomas Lauer <thomas.lauer@virgin.net> wrote:
>
> From: Bart Schaefer <schaefer@brasslantern.com>
> Date: Thu, 9 Nov 2023 10:07:16 -0800
>
> > Combination of an unanticipated use case and the order in which
> > features like unsorted groups were added to "compadd" relative to the
> > implementation of _arguments.
>
> Weeeellllll, unanticipated use case... that's debatable IMO.

Unanticipated use of _arguments, not of sorting.  The anticipated case
was that not-sorting would only apply to completions that would not be
implemented by generic option-parsing in the first place.

> Obviously
> somebody thought that a no-sort option would be a good idea, in fact
> such a good idea that there are two ways to do it. But what they then
> proceeded to do was to have the nosort option sort just a tiny little
> bit anyway

Pretty much everything in completion is based on grouping related
results -- files can be grouped separately from directories, etc.
Thus sorting or not-sorting applies at the group level.  That
_arguments makes separate internal sets of options with and without
arguments has nothing to do with that, and everything to do with the
way "compadd" has to be called in each case so that in the second case
it can match up the arguments with their options.  It would otherwise
have to call compadd once for every option, rather than twice covering
the whole list of options.  It was not anticipated that anyone would
care.