"fake" style requires at least one real match?

"fake" style requires at least one real match?

[Bart Schaefer]

This is essentially the same Q-and-A as from users/10988 almost exactly
ten years ago.  The difference is that the solution in that message no
longer works.

zstyle ':completion::complete:cd:*' tag-order \
  'named-directories:-mine:extra\ directories
named-directories:-normal:named\ directories *'
mydirs=(mydir1 mydir2)
zstyle -e ':completion::complete:cd:*:named-directories-mine' fake \
  'reply=($mydirs)'
zstyle -e ':completion::complete:cd:*:named-directories-mine' \
ignored-patterns 'reply=("(${(qkj.|.)nameddirs})")'

Unless at least one of the real matches is excluded from ignored-patterns,
the "comptags" loop in _all_labels makes another pass and re-adds all the
matches that it ignored the first time around.  This also affects the
fake-always style, which was intended to explicitly allow ignoring all
the real matches.

There don't appear to have been any changes to _description or _all_labels
in all that time that would affect this, so it must be something in the
internals?  There also haven't been any changes to bin_comptags() that
look like they could be the culprit, I don't know where to look next.

Also, an aside, _cd doesn't offer named-directories unless there is a
prefix of the word.  I don't recall if that's a change from the past or
not.

Re: "fake" style requires at least one real match?

[Oliver Kiddle]

Bart wrote:
> This is essentially the same Q-and-A as from users/10988 almost exactly
> ten years ago.  The difference is that the solution in that message no
> longer works.
>
> zstyle -e ':completion::complete:cd:*:named-directories-mine' \
> ignored-patterns 'reply=("(${(qkj.|.)nameddirs})")'

In my testing, the (q) flag in there is preventing ignored-patterns
from working because the | separators get quoted. My guess is that
(q) was added as a hurried after-thought at the time of users/10988
to allow for special characters in a named directory name. Either that,
or it was tested with only one named directory.
${(kj.|.)nameddirs:q} will work instead.

Also seems odd to restrict the style to the complete completer.

> Also, an aside, _cd doesn't offer named-directories unless there is a
> prefix of the word.  I don't recall if that's a change from the past or
> not.

This is down to the explicit -n "$PREFIX" condition. It was there in _cd
from 3.1.7 so is not a change.

Oliver

Re: "fake" style requires at least one real match?

[Bart Schaefer]

On Nov 4,  3:00pm, Oliver Kiddle wrote:
} Subject: Re: "fake" style requires at least one real match?
}
} Bart wrote:
} > This is essentially the same Q-and-A as from users/10988 almost exactly
} > ten years ago.  The difference is that the solution in that message no
} > longer works.
} >
} > zstyle -e ':completion::complete:cd:*:named-directories-mine' \
} > ignored-patterns 'reply=("(${(qkj.|.)nameddirs})")'
} 
} In my testing, the (q) flag in there is preventing ignored-patterns
} from working because the | separators get quoted.

That may be true, but I can reproduce with a different pattern that is
neither eval'd nor (q)'d.

With this:

setopt cdablevars
mydirs=( mydir1 mydir2 )
zstyle -e ':completion::complete:cd:*:named-directories-mine' \
    fake 'reply=($mydirs)'
zstyle ':completion::complete:cd:*:named-directories-mine' \ 
    ignored-patterns '?[^y]*'
zstyle ':completion::complete:cd:*' tag-order \
	'named-directories:-mine:extra\ directories'

That should ignore anything that doesn't have a "y" as the second
character, and show only the corresponding subset of nameddirs.

% hash -d move=$PWD
% hash -d home=$HOME
% cd m<TAB>
move    mydir1  mydir2

Why wasn't "move" ignored here?  (I had to have at least one character
of prefix to get it to offer named-directories at all.)

After adding a few more styles to format the listing:

torch% cd m
Completing extra directories
move    mydir1  mydir2

So it's using the named-directories-mine context, but it's including
a word that it shouldn't.

This doesn't always happen.  If I switch to local-directories in the
above, it works as expected.  So maybe there's some interaction with
using the -a or -k options to compadd.

Re: "fake" style requires at least one real match?

[Bart Schaefer]

One more example -- if I change this to the fake-always style with an
ignored-patterns of '*' as recommended in the manual --

setopt cdablevars
mydirs=( mydir1 mydir2 )
zstyle -e ':completion::complete:cd:*:named-directories-mine' \
    fake-always 'reply=($mydirs)'
zstyle ':completion::complete:cd:*:named-directories-mine' \ 
    ignored-patterns '*'
zstyle ':completion::complete:cd:*' tag-order \
    'named-directories:-mine:extra\ directories'

-- then it ignores tag-order entirely:

torch% hash -d move=$PWD
torch% cd m
Completing extra directories
mydir1              mydir2            
Completing user
mach      mail      mailnull  mysql   
Completing named directory
move

But if allow a single other named directory to pass:

torch% zstyle ':completion::complete:cd:*:named-directories-mine' \
    ignored-patterns '*~?o*'
torch% cd m
Completing extra directories
move    mydir1  mydir2

What?

Re: "fake" style requires at least one real match?

[Bart Schaefer]

On Nov 4, 10:02pm, Bart Schaefer wrote:
}
} -- then it ignores tag-order entirely:

OK, not entirely because it does construct the named-directories-mine
tag and use its description -- but it doesn't apply the ordering, it
populates all the tags.

Really long diagnosis follows, working sort of backwards up the call
stack.  Skip to the last three paragraphs if you don't want to know
all the details.

This happens because _tilde calls _users and _directory_stack as
well as passing compadd down to _requested for the named-directories
tag.  So users and directory stack entries look up ignored-patterns
entirely separately from the named-directories context.  There's no
other context for _tilde when it is called from _cd (as opposed to
when there is actually a tilde on the line, and you get -tilde- as
the context), so you can't apply ignored-patterns to all of it in
one go unless you apply it to :cd:*:*.

There are probably other cases like this that selectively break the
"supplemented context" usage of tag-order, it's just difficult to see
unless you're also trying to use ignored-patterns in such a context.

The next question is why "_requested users" etc. succeded even though
the tag-order style does not include "users".

Looking at _complete_debug output, we enter _tilde and it calls

    _tags users named-directories directory-stack

It then does

    while _tags

and tries _requested for all three tags in the first pass of the loop.
None of these work because only named-directories-mine is being sought
as the current tag in _all_labels.

So the loop goes around again, comptags -N moves on to the next tag
group, _requested calls "comptags -R" which succeeds, but now $# -eq 1
so neither _all_labels nor _description is called, and _requested
returns 0.  Therefore both _users and _directory_stack get called,
no ignored-patterns are applied, and too much stuff ends up getting
passed to compadd.

This all bubbles up to _alternative, which is calling

    while _next_label

and inside _next_label is where the supplemented context is built.

_tilde gets called, _all_labels reconstructs named-directories-mine,
we do one pass that does the right thing with fake and fake-always,
but then compadd ($4) on line 35 of _all_labels fails so _all_labels
itself fails even though matches were added by _description on line
32; and _alternative goes around the _next_labels loop another time
resulting in all the aforementioned breakage in _tilde.

The upshot of all this is that _description is never expected to
*call* compadd, it's only supposed to generate arguments for compadd
that its caller can pass to compadd; but fake and fake-always are in 
fact compadd-ed directly in _description.  _all_labels (in this case)
doesn't know they are there, and tells *its* caller to keep looking.

I have no immediate idea what to do about this.  We can't rewrite
every caller of _description to delta $compstate[nmatches] just
in case a fake style was used.  And in fact it only matters when
ONLY the fake matches are added, as the Subject: says ... except:

As an additional complication, there's a comment in _describe about
bad things happening if _describe is called more than once for the
same completion, but that's what may happen if fake or fake-always
use the value:description format and there are also descriptions
for the other matches in the context.

Re: "fake" style requires at least one real match?

[Bart Schaefer]

On Nov 5, 11:42pm, Bart Schaefer wrote:
}
} I have no immediate idea what to do about this.  We can't rewrite
} every caller of _description to delta $compstate[nmatches] just
} in case a fake style was used.  And in fact it only matters when
} ONLY the fake matches are added, as the Subject: says ...

I looked through all the functions that call _description and for
the vast majority of them this only matters indirectly to the caller
of the function; that is, the usage is equivalent to

    _description ...
    compadd ...
    return

which means that unless the caller is testing the return value to
decide whether to proceed with another completion, adding fake matches
in _description does not affect anything.  Ignoring for the moment
functions that might appear directly in the 'completer' style, that
leaves the following callers of _description:

_requested and _next_label both ignore the result of _description and
always return zero after calling it.

_alternative does the "right thing" and uses a delta of nmatches to
decide what to return.

_values uses _all_labels and a _next_label loop.

_next_tags redefines _all_labels and _next_label, so I'm going to
ignore it for this purpose.

And then there's _arguments, which has a _tags loop but is a wrapper
that calls back to whatever was passed into it; so there's really no
reasonable way to analyze it.

I think, therefore, the only reasonable thing is to fix _all_labels,
which should by proxy fix _values.  However, it's a little more
complicated than I first thought.

A compounding problem is that we're calling _tilde from _alternative.
_alternative does its own call to _description on line 28, inside
the _tags loop.  Inside _tilde, we start another _tags loop, which
eventually calls _all_labels, which calls _description *again*.  The
fake matches were already added by the first call in _alternative,
so compstate[nmatches] is unchanged and can't work as a test for the
appropriate return value for _all_labels.

For the moment I'm stuck.  Sorry for the play-by-play.  Ideas welcome.

Probably not worth holding up a release, if one is being considered.