[bug] Completion functions _files/_path_files -F filter on escaped file names

[bug] Completion functions _files/_path_files -F filter on escaped file names

[Johan Grande]

Hello maintainers,

I just recently started writing completion functions and I'm bringing to 
you what I think is a bug:

% cat ~/.zshrc
% zsh --version
zsh 5.8.1 (x86_64-ubuntu-linux-gnu)

% foo() { echo "$@"; };
% _foo() { local exclusions=('*a *'); _path_files -F exclusions; }
% compdef _foo foo
% touch a 'a a' b
% foo <tab>
a     a\ a  b
% foo '<tab>
a  b

When completion with _files or _path_files is called at the beginning of 
an argument, exclusion patterns given with `-F` are applied to 
already-escaped file names. This breaks patterns that contain escapable 
characters such as a space or [].

With a \ added to the pattern, 'a a' is filtered out.

% _foo() { local exclusions=('*a\ *'); _path_files -F exclusions; }
% compdef _foo foo
% foo <tab>
a  b

But this is very brittle as it depends on the implementation of the 
escaping instead of the file names themselves.

My use case is a tool that works on TagSpaces tags, i.e., 
space-separated tags surrounded by brackets in file names such as 
IMG-2653[vacation alps].jpg. I want to offer, as completion, files that 
have or don't have a certain tag.

Do you think that this behavior could be fixed?

I appreciate the work you do in maintaining zsh which I use as my daily 
shell.

Cheers

-- 
Johan Grande

Re: [bug] Completion functions _files/_path_files -F filter on escaped file names

[Bart Schaefer]

On Wed, Jun 21, 2023 at 12:29 PM Johan Grande <nahoj@crans.org> wrote:
>
> When completion with _files or _path_files is called at the beginning of
> an argument, exclusion patterns given with `-F` are applied to
> already-escaped file names. This breaks patterns that contain escapable
> characters such as a space or [].
>
> With a \ added to the pattern, 'a a' is filtered out.

Upon closer examination, the documentation of the -F option is
imprecise.  -F is a substitute for the "ignored-patterns" zstyle, and
the description of that style clearly states that it filters out trial
completions.  Thus the -F option does not ignore files, it ignores the
strings that are candidates to be inserted on the command line when
completing files.

> But this is very brittle as it depends on the implementation of the
> escaping instead of the file names themselves.

Unfortunately this is implemented in the internals of the "compadd"
builtin and other forms of completion rely on it.

> Do you think that this behavior could be fixed?

We could try this.  However, it's not backward-compatible for people
who are already quoting spaces etc. for _files -F patterns.

diff --git a/Completion/Unix/Type/_path_files b/Completion/Unix/Type/_path_files
index d46dcbe5a..c471ad08e 100644
--- a/Completion/Unix/Type/_path_files
+++ b/Completion/Unix/Type/_path_files
@@ -100,6 +100,12 @@ if (( $#ignore )); then
   fi
 fi

+# Adjust the patterns for the effects of "compquote" on file names.
+# What's needed here is the inverse of ${(b)...}, that is, quote only
+# characters that are NOT pattern characters but might need quoting.
+# Backslashes appearing in a file name remain a bugaboo here.
+ignore=( ${ignore//(#b)([\!\#\$\&[:space:];\"\'])/\\${match[1]}} )
+
 # If we were given no file selection option, we behave as if we were given
 # a `-f'.

Re: [bug] Completion functions _files/_path_files -F filter on escaped file names

[Johan Grande]

Le 22/06/2023 à 05:54, Bart Schaefer a écrit :
> On Wed, Jun 21, 2023 at 12:29 PM Johan Grande <nahoj@crans.org> wrote:
>> Do you think that this behavior could be fixed?
> 
> We could try this.  However, it's not backward-compatible for people
> who are already quoting spaces etc. for _files -F patterns.

After looking at your suggestion and other, existing completions such as 
_git, I get the impression that I shouldn't use _files but rather mimic 
its basic behavior with _multi_parts and have more flexibility in 
filtering matches. I assume I'll lose some perks of _files but it'll 
hopefully fit my needs.

If you have any tips or a good example to study, I'll be grateful.

What I'll be aiming to do is:
- complete will all files under a path filtered with a regex or an 
arbitrary command
- contrary to _git, I won't have a root dir so if I'm working in a 
directory with >10k files under it, I might want to suggest only 
immediate children in the first instance rather than browse everything 
at once.

Thanks

-- 
Johan

Re: [bug] Completion functions _files/_path_files -F filter on escaped file names

[Bart Schaefer]

[-- Attachment #1: Type: text/plain, Size: 1050 bytes --]

Took me a while to get back to this, went on vacation and veered to other
topics.

On Sat, Jun 24, 2023 at 12:55 PM Johan Grande <nahoj@crans.org> wrote:

>
> After looking at your suggestion and other, existing completions such as
> _git, I get the impression that I shouldn't use _files but rather mimic
> its basic behavior with _multi_parts [...]
> If you have any tips or a good example to study, I'll be grateful.
>

The shortest possible example would be something like

local expl # set by _wanted
local tree=(**/*)
_wanted files expl 'files in tree' _multi_parts "$@" -f - / tree

The stuff you describe here ...


> - complete will all files under a path filtered with a regex or an
> arbitrary command
> - contrary to _git, I won't have a root dir so if I'm working in a
> directory with >10k files under it, I might want to suggest only
> immediate children in the first instance rather than browse everything
> at once.
>

... all goes into how you populate the "tree" array before calling
_multi_parts.

[-- Attachment #2: Type: text/html, Size: 1841 bytes --]