Completion/correction and _path_files

Completion/correction and _path_files

[Bart Schaefer]

Starting in the root of the zsh source tree, if I do this:

schaefer<501> ls Scr<TAB>

It gets corrected:

schaefer<501> ls Src/
Completing corrections
Src/
Completing original
Scr

If instead I start with:

schaefer<502> ls s/a/m<TAB>

That too gets corrected (lower -> upper) and completed:

schaefer<502> ls Src/Aliases/Makefile

However, if I start with this:

schaefer<502> ls Scr/
No matches for `files' or `corrections'

Looking at _complete_debug output, _path_files appears to have found the
correction and at line 372 populates tmp1 with all the files in the Src
subdirectory, but then discards the "Src/" prefix from all those files
at about line 669 and attempts to compare the files themselves to "Scr"
(compfiles -r at line 514) which of course fails.

This seems like a bug.  At some earlier point "Src/" should have been
added as a possible completion; the descent into the directory was not
necessary.  Even with

schaefer<502> ls Scr/A
No matches for `files' or `corrections'

I *think* it's comparing "Aliases" to the entire string "Scr/A", but
it begins to get difficult to follow.

Re: Completion/correction and _path_files

[Peter Stephenson]

On Wed, 21 May 2008 07:34:12 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> However, if I start with this:
> 
> schaefer<502> ls Scr/
> No matches for `files' or `corrections'

This has been annoying me for ages, although if I made a list of things
that had been annoying me for ages I'd be so busy I'd never get a chance to
be annoyed by them.

The problem in the cases I've tried is right at the last stage, when
compadd is run to add the matches.  The prefixes are added as "-p Src/ -W
Src/" and compadd doesn't like those because it's more than it's job's
worth to pass anything that doesn't begin with "Scr/".

We can convince it by passing the -U option to compadd, and the following
does the job in this case (thought whether *every* occurrence of compadd
I've given it to needs it beats me---this isn't even the whole list, since
I didn't add it to a number of occurrences of compadd that don't appear to
be there to add file matches).

However, there's a good chance this has got some side effect.  (You might
hope there wasn't, and that _path_files was smart enough only to add things
it already knew matched, so -U was harmless, but things in the underworld
aren't that easy, and I have a vague feeling I already tried this and found
a problem once.)  If we can find out what that is we might be able to work
around it, and if it doesn't affect approximation we can make the -U only
apply then (I haven't done that here because that would mask any side
effects that I want to tease out).  I will try it for a bit and see what
happens.

I've already noticed this doesn't play well with partial path completion,
though that appears to be separate problem.  Hence Sec/Zie/zle_main<TAB>
completes properly, but S/Zie/zle_main<TAB> gives you options for S without
properly matching the rest.  You have to stop and restart the completion
when it offers you Src.

Index: Completion/Unix/Type/_path_files
===================================================================
RCS file: /cvsroot/zsh/zsh/Completion/Unix/Type/_path_files,v
retrieving revision 1.28
diff -u -r1.28 _path_files
--- Completion/Unix/Type/_path_files	18 May 2008 20:51:48 -0000	1.28
+++ Completion/Unix/Type/_path_files	2 Jun 2008 16:32:27 -0000
@@ -595,7 +595,7 @@
 	    # back up the path.
 	    tmp1=("${(@)tmp1%%/*}")
 	    _list_files tmp1 "$prepath$realpath$testpath"
-	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" -s "/${tmp3#*/}" \
+	    compadd -U -Qf "$mopts[@]" -p "$linepath$tmp2" -s "/${tmp3#*/}" \
 	            -W "$prepath$realpath$testpath" \
 		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
 		    $listopts \
@@ -604,7 +604,7 @@
 	    # Same with a non-empty suffix
 	    tmp1=("${(@)^tmp1%%/*}/${tmp3#*/}")
 	    _list_files tmp1 "$prepath$realpath$testpath"
-	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
+	    compadd -U -Qf "$mopts[@]" -p "$linepath$tmp2" \
 	            -W "$prepath$realpath$testpath" \
 		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
 	            $listopts \
@@ -612,7 +612,7 @@
           fi
 	else
 	  _list_files tmp1 "$prepath$realpath$testpath"
-	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
+	  compadd -U -Qf "$mopts[@]" -p "$linepath$tmp2" \
 	          -W "$prepath$realpath$testpath" \
 		   "$pfxsfx[@]" -M "r:|/=* r:|=*" \
 	           $listopts \
@@ -621,7 +621,7 @@
       else
 	# We are inserting the match into the command line.
         if [[ "$tmp3" = */* ]]; then
-	  tmp4=( -Qf "$mopts[@]" -p "$linepath$tmp2"
+	  tmp4=( -U -Qf "$mopts[@]" -p "$linepath$tmp2"
 	         -W "$prepath$realpath$testpath"
 	         "$pfxsfx[@]" -M "r:|/=* r:|=*" )
 	  if [[ -z "$listsfx" ]]; then
@@ -640,7 +640,7 @@
           fi
         else
 	  _list_files tmp1 "$prepath$realpath$testpath"
-	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
+	  compadd -U -Qf "$mopts[@]" -p "$linepath$tmp2" \
                   -W "$prepath$realpath$testpath" \
 		  "$pfxsfx[@]" -M "r:|/=* r:|=*" \
                   $listopts \
@@ -708,7 +708,7 @@
       compquote tmp4 tmp2 tmp1
       for i in "$tmp1[@]"; do
 	_list_files tmp2 "$prepath$realpath${mid%/*/}"
-        compadd -Qf "$mopts[@]" -p "$linepath$tmp3/" -s "/$tmp4$i" \
+        compadd -U -Qf "$mopts[@]" -p "$linepath$tmp3/" -s "/$tmp4$i" \
                 -W "$prepath$realpath${mid%/*/}/" \
 	        "$pfxsfx[@]" -M "r:|/=* r:|=*" $listopts - "$tmp2"
       done
@@ -733,12 +733,12 @@
             "${PREFIX#\~}$SUFFIX" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
 	tmp1=("$linepath$tmp4${(@)^tmp1}")
 	_list_files tmp1 "$prepath$realpath"
-        compadd -Qf -W "$prepath$realpath" "$pfxsfx[@]" "$mopts[@]" \
+        compadd -U -Qf -W "$prepath$realpath" "$pfxsfx[@]" "$mopts[@]" \
                 -M "r:|/=* r:|=*" $listopts -a tmp1
       else
 	# Not a pattern match
 	_list_files tmp1 "$prepath$realpath$testpath"
-        compadd -Qf -p "$linepath$tmp4" -W "$prepath$realpath$testpath" \
+        compadd -U -Qf -p "$linepath$tmp4" -W "$prepath$realpath$testpath" \
 	        "$pfxsfx[@]" "$mopts[@]" -M "r:|/=* r:|=*" $listopts -a tmp1
       fi
     fi




-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: Completion/correction and _path_files

[Peter Stephenson]

On Mon, 2 Jun 2008 17:40:30 +0100
Peter Stephenson <pws@csr.com> wrote:
> However, there's a good chance this has got some side effect.

One was quite easy to find.  -U apparently overrides the use of the ignored
prefix and suffix, too (the bits of the current word you've already
considered and don't need as part of matching).  I found this by trying
to complete "scan +" which should skip over the "+" and complete
subdirectories of my mail directory; the "+" disappeared.  That much can be
fixed by adding the ignored parts explicitly back in.  This is a replacement
version.

Index: Completion/Unix/Type/_path_files
===================================================================
RCS file: /cvsroot/zsh/zsh/Completion/Unix/Type/_path_files,v
retrieving revision 1.28
diff -u -r1.28 _path_files
--- Completion/Unix/Type/_path_files	18 May 2008 20:51:48 -0000	1.28
+++ Completion/Unix/Type/_path_files	2 Jun 2008 16:57:57 -0000
@@ -595,7 +595,8 @@
 	    # back up the path.
 	    tmp1=("${(@)tmp1%%/*}")
 	    _list_files tmp1 "$prepath$realpath$testpath"
-	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" -s "/${tmp3#*/}" \
+	    compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2"
+	    -s "/${tmp3#*/}$ISUFFIX" \
 	            -W "$prepath$realpath$testpath" \
 		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
 		    $listopts \
@@ -604,16 +605,18 @@
 	    # Same with a non-empty suffix
 	    tmp1=("${(@)^tmp1%%/*}/${tmp3#*/}")
 	    _list_files tmp1 "$prepath$realpath$testpath"
-	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
-	            -W "$prepath$realpath$testpath" \
+	    compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2" \
+	            -s "$ISUFFIX" \
+	            -W "$prepath$realpath$testpath$ISUFFIX" \
 		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
 	            $listopts \
 		    -a tmp1
           fi
 	else
 	  _list_files tmp1 "$prepath$realpath$testpath"
-	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
-	          -W "$prepath$realpath$testpath" \
+	  compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2" \
+	          -s "$ISUFFIX"
+	          -W "$prepath$realpath$testpath$ISUFFIX" \
 		   "$pfxsfx[@]" -M "r:|/=* r:|=*" \
 	           $listopts \
 		   -a tmp1
@@ -621,7 +624,8 @@
       else
 	# We are inserting the match into the command line.
         if [[ "$tmp3" = */* ]]; then
-	  tmp4=( -Qf "$mopts[@]" -p "$linepath$tmp2"
+	  tmp4=( -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2"
+	         -s "$ISUFFIX"
 	         -W "$prepath$realpath$testpath"
 	         "$pfxsfx[@]" -M "r:|/=* r:|=*" )
 	  if [[ -z "$listsfx" ]]; then
@@ -640,7 +644,8 @@
           fi
         else
 	  _list_files tmp1 "$prepath$realpath$testpath"
-	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
+	  compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2" \
+	          -s "$ISUFFIX" \
                   -W "$prepath$realpath$testpath" \
 		  "$pfxsfx[@]" -M "r:|/=* r:|=*" \
                   $listopts \
@@ -708,7 +713,8 @@
       compquote tmp4 tmp2 tmp1
       for i in "$tmp1[@]"; do
 	_list_files tmp2 "$prepath$realpath${mid%/*/}"
-        compadd -Qf "$mopts[@]" -p "$linepath$tmp3/" -s "/$tmp4$i" \
+        compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp3/" \
+	        -s "/$tmp4$i$ISUFFIX" \
                 -W "$prepath$realpath${mid%/*/}/" \
 	        "$pfxsfx[@]" -M "r:|/=* r:|=*" $listopts - "$tmp2"
       done
@@ -738,7 +744,9 @@
       else
 	# Not a pattern match
 	_list_files tmp1 "$prepath$realpath$testpath"
-        compadd -Qf -p "$linepath$tmp4" -W "$prepath$realpath$testpath" \
+        compadd -U -Qf -p "$IPREFIX$linepath$tmp4" \
+	        -s "$ISUFFIX" \
+	        -W "$prepath$realpath$testpath" \
 	        "$pfxsfx[@]" "$mopts[@]" -M "r:|/=* r:|=*" $listopts -a tmp1
       fi
     fi

Re: Completion/correction and _path_files

[Peter Stephenson]

On Mon, 2 Jun 2008 18:02:45 +0100
Peter Stephenson <pws@csr.com> wrote:
> @@ -604,16 +605,18 @@
>  	    # Same with a non-empty suffix
>  	    tmp1=("${(@)^tmp1%%/*}/${tmp3#*/}")
>  	    _list_files tmp1 "$prepath$realpath$testpath"
> -	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
> -	            -W "$prepath$realpath$testpath" \
> +	    compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2" \
> +	            -s "$ISUFFIX" \
> +	            -W "$prepath$realpath$testpath$ISUFFIX" \
                                                  ^^^^^^^^
>  		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
>  	            $listopts \
>  		    -a tmp1
>            fi
>  	else
>  	  _list_files tmp1 "$prepath$realpath$testpath"
> -	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
> -	          -W "$prepath$realpath$testpath" \
> +	  compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2" \
> +	          -s "$ISUFFIX"
> +	          -W "$prepath$realpath$testpath$ISUFFIX" \
                                                ^^^^^^^^
>  		   "$pfxsfx[@]" -M "r:|/=* r:|=*" \
>  	           $listopts \
>  		   -a tmp1

Sorry, you don't want the ignored suffix in the -W which
refers to a real directory, only in -p and -s which refer to
the displayed prefix and suffix.

Index: Completion/Unix/Type/_path_files
===================================================================
RCS file: /cvsroot/zsh/zsh/Completion/Unix/Type/_path_files,v
retrieving revision 1.28
diff -u -r1.28 _path_files
--- Completion/Unix/Type/_path_files	18 May 2008 20:51:48 -0000	1.28
+++ Completion/Unix/Type/_path_files	2 Jun 2008 17:06:40 -0000
@@ -595,7 +595,8 @@
 	    # back up the path.
 	    tmp1=("${(@)tmp1%%/*}")
 	    _list_files tmp1 "$prepath$realpath$testpath"
-	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" -s "/${tmp3#*/}" \
+	    compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2"
+	    -s "/${tmp3#*/}$ISUFFIX" \
 	            -W "$prepath$realpath$testpath" \
 		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
 		    $listopts \
@@ -604,7 +605,8 @@
 	    # Same with a non-empty suffix
 	    tmp1=("${(@)^tmp1%%/*}/${tmp3#*/}")
 	    _list_files tmp1 "$prepath$realpath$testpath"
-	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
+	    compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2" \
+	            -s "$ISUFFIX" \
 	            -W "$prepath$realpath$testpath" \
 		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
 	            $listopts \
@@ -612,7 +614,8 @@
           fi
 	else
 	  _list_files tmp1 "$prepath$realpath$testpath"
-	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
+	  compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2" \
+	          -s "$ISUFFIX"
 	          -W "$prepath$realpath$testpath" \
 		   "$pfxsfx[@]" -M "r:|/=* r:|=*" \
 	           $listopts \
@@ -621,7 +624,8 @@
       else
 	# We are inserting the match into the command line.
         if [[ "$tmp3" = */* ]]; then
-	  tmp4=( -Qf "$mopts[@]" -p "$linepath$tmp2"
+	  tmp4=( -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2"
+	         -s "$ISUFFIX"
 	         -W "$prepath$realpath$testpath"
 	         "$pfxsfx[@]" -M "r:|/=* r:|=*" )
 	  if [[ -z "$listsfx" ]]; then
@@ -640,7 +644,8 @@
           fi
         else
 	  _list_files tmp1 "$prepath$realpath$testpath"
-	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
+	  compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp2" \
+	          -s "$ISUFFIX" \
                   -W "$prepath$realpath$testpath" \
 		  "$pfxsfx[@]" -M "r:|/=* r:|=*" \
                   $listopts \
@@ -708,7 +713,8 @@
       compquote tmp4 tmp2 tmp1
       for i in "$tmp1[@]"; do
 	_list_files tmp2 "$prepath$realpath${mid%/*/}"
-        compadd -Qf "$mopts[@]" -p "$linepath$tmp3/" -s "/$tmp4$i" \
+        compadd -U -Qf "$mopts[@]" -p "$IPREFIX$linepath$tmp3/" \
+	        -s "/$tmp4$i$ISUFFIX" \
                 -W "$prepath$realpath${mid%/*/}/" \
 	        "$pfxsfx[@]" -M "r:|/=* r:|=*" $listopts - "$tmp2"
       done
@@ -738,7 +744,9 @@
       else
 	# Not a pattern match
 	_list_files tmp1 "$prepath$realpath$testpath"
-        compadd -Qf -p "$linepath$tmp4" -W "$prepath$realpath$testpath" \
+        compadd -U -Qf -p "$IPREFIX$linepath$tmp4" \
+	        -s "$ISUFFIX" \
+	        -W "$prepath$realpath$testpath" \
 	        "$pfxsfx[@]" "$mopts[@]" -M "r:|/=* r:|=*" $listopts -a tmp1
       fi
     fi

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: Completion/correction and _path_files

[Peter Stephenson]

On Mon, 2 Jun 2008 18:07:35 +0100
Peter Stephenson <pws@csr.com> wrote:
> Index: Completion/Unix/Type/_path_files

Apart from more typos, I haven't seen any new problems with this, so I will
commit it and you can let me know.

I've decided I don't want to restrict this change just to approximation;
that would create yet another special case in which difficulties can hide.