Re: still confused about completion and matching

Re: still confused about completion and matching

[Sven Wischnowsky]

E. Jay Berkenbilt wrote:

> ...
> 
> I don't think it's really true.... I've never had anything odd about
> smbclient's argument ordering.  I was going to use cvs as a model
> since it is smart enough to use cvs with some of the existing
> arguments to get the module list.  The same type of idea could be used
> do completion for share names once the server has been specified.

After a very cursory look at the (synopsis in the) smbclient manual,
I'd sayy that you probably were better of copying not too much from
_cvs. That is pretty complicated because of the sub-command handling
which doesn't seem to be needed for smbclient.

And I'd like to point you to Peter's User Guide again. It contains a
short introduction to using _arguments that might be helpful
(together, as Andrej said, with the example functions from the
distribution).

> I'll use this as incentive to get my current project finished -- my
> reward will be letting myself play with zsh completion functions. :-)
> It will probably be at least a week before I can start it.
> 
> >   Also, if you could send comments (or even patches) for _arguments
> >   documentation ...
> 
> You can count on it!  Once I get up to a certain level of competence
> I'd like to do a thorough cover-to-cover read of the zsh documentation
> with red pen in hand.  (I'll probably print the .yo files directly --
> easier to generate patches and make comments that way.  Besides, I've
> been considering whether to use yodl for some of my own documentation.
> This way I could find out whether I like it or not. :-]) I have
> written enough documentation to know how important it is to have a
> fresh set of eyes read it.  One eventually becomes immune to errors in
> one's own writing. :-)

Indeed. Any help will be greatly appreciated.

> I don't know when I'll get to this but
> hopefully before the next non-development release which I guess is
> supposed to be 4.0?

Yes.


For some of the things you wrote in 13052 I know or almost know the
reason, but I don't have enough time now. Maybe tomorrow...

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: still confused about completion and matching

[E. Jay Berkenbilt]

I've been using zsh from the repository with the patches that enabled
my fancy glob completion behavior for a few weeks now and I have had
no problems with that or any other aspect of completion behavior.  I
get a little thrill every time I type a shell regexp and hit TAB
now. :-)  It seems that the changes have still not been committed to
the repository.  Is this because we're still waiting for reactions
from people who are not using the new patches or because we forgot or
for some other reason?  The part of me that is sending this message is
the part that monitors for unresolved issues and yearns for the day
when I can use an unpatched zsh. :-)

I still haven't had time to write my smbclient completion function.
I've also added rewriting _rcs to use _arguments to my "eventually"
list unless someone else gets to it first.  As brought up by another
user (Vincent Lefevre) the current behavior doesn't handle the more
subtle aspects of rcs's behavior very well.  I've run into this
several times myself either when looking at RCS directories that are
really on Windows machines or using rcs commands directly to cheat
while looking at CVS repositories.  I think I know rcs well enough to
(with the help of the rcs manual pages of course) rewrite _rcs....

                                Jay

RE: still confused about completion and matching

[Sven Wischnowsky]

Andrej Borsenkow wrote:

> ...
> 
> This condition predates styles. Now we do have suitable means for users to
> configure desired behaviour.

True.

> As I said long ago, I did not want Zsh to decide for me when to start menu
> completion if I did not request it. I do not want to invent another condition.
> 
> > As I sais in one of the previous mails, I wasn't completely happy with
> > that condition myself. The problem is that we certainly don't want to
> > insert the unambiguous string unconditionally even if insert-unambig
> > is set, because that string might often be empty.
> >
> 
> First, why are you _so_ sure there won't be any common prefix? For people that
> know shell glob patterns by heart it is sometimes easier to type pattern than
> to invent matching specification. And matches for foobar[1-9] are always
> shorter than pattern itself.

I'm not sure that it is, I'm sure that it might be...

> Second, how does it differ from ordinary completion? The sole thing I wanted -
> let's treat completion and matching equally. Both give you a set of matches to
> select from. Let's use common rules to decide when and how these matches are
> inserted. (After all, "ordinary" completion is just matching with pattern
> $PREFIX*$SUFFIX ... even if not implemented this way. I do not see why
> $PREFIX?#$SUFFIX should be treated differently :-)

... different from ordinary completion because there we have a lot of
code that can deal with the types of matching (match specs) allowed
there to build unambiguous strings without losing characters typed by
the user.

But I didn't want it to sound as if I'm religiously attached to the
current behaviour. I'm not. We can also change it to look up the
insert-unambiguous style lately, after the completions have been
generated. People can then use `zstyle -e' to add conditions if they
want.

I guess that would be acceptable to everyone? Especially if we add
a/some example(s) to the manual showing clever things one might want
to try there?

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

RE: still confused about completion and matching

[Andrej Borsenkow]

>
> To Bart and Andrej: the lines Jay is referring to are:
>
> +    [[ $compstate[unambiguous_cursor] -gt $#compstate[unambiguous] ]] &&
> +        ins=yes compstate[insert]="$ocsi"
> compstate[pattern_insert]="$ocspi"
>
> These are new and don't have anything to do with the
> insert-unambiguous style.
>

O.K. I have not followed these patches ...
> >
> > That is the problem. It means, that if you have long pattern that
> results in
> > short match, menu completion is started.
> >
> > I was never happy about it as well. But I switched to menu
> selection some time
> > ago :-)
>
> Any suggestions about making this more clever or the results more
> intuitive are welcome ;-)
>

This condition predates styles. Now we do have suitable means for users to
configure desired behaviour.

As I said long ago, I did not want Zsh to decide for me when to start menu
completion if I did not request it. I do not want to invent another condition.

> As I sais in one of the previous mails, I wasn't completely happy with
> that condition myself. The problem is that we certainly don't want to
> insert the unambiguous string unconditionally even if insert-unambig
> is set, because that string might often be empty.
>

First, why are you _so_ sure there won't be any common prefix? For people that
know shell glob patterns by heart it is sometimes easier to type pattern than
to invent matching specification. And matches for foobar[1-9] are always
shorter than pattern itself.

Second, how does it differ from ordinary completion? The sole thing I wanted -
let's treat completion and matching equally. Both give you a set of matches to
select from. Let's use common rules to decide when and how these matches are
inserted. (After all, "ordinary" completion is just matching with pattern
$PREFIX*$SUFFIX ... even if not implemented this way. I do not see why
$PREFIX?#$SUFFIX should be treated differently :-)

cheers

-andrej

Re: still confused about completion and matching

[Sven Wischnowsky]

E. Jay Berkenbilt wrote:

> ...
> 
> The most recent change to _match seems to cause it to revert to menu
> completion in many cases.  If I just completely remove lines 56 and 57
> (dealing with unambiguous_cursor) then all my test cases work just the
> way I want them to. 

To Bart and Andrej: the lines Jay is referring to are:

+    [[ $compstate[unambiguous_cursor] -gt $#compstate[unambiguous] ]] &&
+        ins=yes compstate[insert]="$ocsi" compstate[pattern_insert]="$ocspi"

These are new and don't have anything to do with the
insert-unambiguous style.

> The only thing I lose is earlier expansion in
> some cases, but the result doesn't change the behavior or the amount
> of typing required. 

And this is exactly what they were supposed to achieve. Inserting the
right expansions a bit earlier. The test is a bit weird, I admit, but
worked for the cases I tested, but probably not for other cases, so I
have don't have any problems whatsoever with removing them (or later
trying to come up with a better test.

> ...
> 
> Would the
> next step be committing these changes and waiting for fallout? :-)

I'd like to ask what the other people around here think about
it. Without setting one of the new styles, the patch should only have
one visible effect: allowing to complete paths with multiple pattern-
containing components.

Should we give it a try? Has anyone else tried it yet? (Andrej? I
/think/ you once asked for this multi-component-pattern completion.)


Andrej Borsenkow wrote:

> > insert-unambiguous
> >      This is used by the _match and _approximate completer functions,
> >      where the possible completions may not have a common prefix so
> >      that menu completion is often the most useful may of choosing
> >      completions.  If the style is set to `true', the completer will
> >      start menu completion only if no unambiguous string could be
> >      generated that is at least as long as the original string typed by
> 
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >      the user.
>         ^^^^^^^^^^
> 
> That is the problem. It means, that if you have long pattern that results in
> short match, menu completion is started.
> 
> I was never happy about it as well. But I switched to menu selection some time
> ago :-)

Any suggestions about making this more clever or the results more
intuitive are welcome ;-)

As I sais in one of the previous mails, I wasn't completely happy with 
that condition myself. The problem is that we certainly don't want to
insert the unambiguous string unconditionally even if insert-unambig
is set, because that string might often be empty.

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

RE: still confused about completion and matching

[Andrej Borsenkow]

>
> insert-unambiguous
>      This is used by the _match and _approximate completer functions,
>      where the possible completions may not have a common prefix so
>      that menu completion is often the most useful may of choosing
>      completions.  If the style is set to `true', the completer will
>      start menu completion only if no unambiguous string could be
>      generated that is at least as long as the original string typed by

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>      the user.
        ^^^^^^^^^^

That is the problem. It means, that if you have long pattern that results in
short match, menu completion is started.

I was never happy about it as well. But I switched to menu selection some time
ago :-)

-andrej

Re: still confused about completion and matching

[Bart Schaefer]

On Oct 24, 11:00am, E. Jay Berkenbilt wrote:
}
} The most recent change to _match seems to cause it to revert to menu
} completion in many cases.  If I just completely remove lines 56 and 57
} (dealing with unambiguous_cursor) then all my test cases work just the
} way I want them to.  The only thing I lose is earlier expansion in
} some cases, but the result doesn't change the behavior or the amount
} of typing required.  I don't quite understand what the purpose of
} those two lines are.  I believe I understand what they literally mean,
} but I don't understand why the behavior is desirable.  Do they do
} something important in a case that is excluded by my style settings?
} What is the exact purpose of those lines?

Unless I'm mistaken about which lines you mean -- I don't have all the
patches applied, so I'm inferring line numbers by looking at the diff and
my own source file -- those lines implement the insert-unambiguous style:

insert-unambiguous
     This is used by the _match and _approximate completer functions,
     where the possible completions may not have a common prefix so
     that menu completion is often the most useful may of choosing
     completions.  If the style is set to `true', the completer will
     start menu completion only if no unambiguous string could be
     generated that is at least as long as the original string typed by
     the user.  Note that the _approximate completer uses it after
     setting the completer field in the context name to one of
     correct-NUM or approximate-NUM, where NUM is the number of errors
     that were accepted.

Do you have insert-unambiguous set to true?  If not, perhaps you should.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: still confused about completion and matching

[E. Jay Berkenbilt]

>   > ...
>   > 
>   > If I omit "suffix" from :completion:*:paths expand, although 
>   > 
>   > ls u?/ TAB
>   > 
>   > doesn't generate a superfluous / anymore,
>   > 
>   > ls u?/q/ TAB
>   > 
>   > still does.
>
>   This is a different case. These are the automatically added slashes we
>   get due to LIST_TYPES. I've got a patch for the C-code to fix it there.
>   I'll commit it. We won't want it to ever add a file-type character 
>   after a slash anyway or do we?

Yes, I see.  I can't imagine why you would ever want to add path
characters after /.  The committed patch to the C code does indeed
seem to solve that problem.

>   > ls u?/q/d TAB
>   > 
>   > lists u1/q/devel and u3/q/dark as it should.  However,
>   > 
>   > ls u?/q/de TAB
>   > 
>   > doesn't list anything.  Similarly,
>
>   As you found out, this happened in cases where there was only one match.
>   This was caused by the yet unchanged compadd around line 624 in _path_files
>   which used pattern matching even if there was a pattern in a component in
>   the -p-prefix (which is used using the match specs).

Okay, seems fine now.

>   > ...
>   > 
>   > Finally,
>   > 
>   > ls u? TAB
>   > 
>   > works fine, but
>   > 
>   > ls ./u? TAB
>   > 
>   > makes the ? disappear from the commandline.
>
>   A small problem with an attempt at additional cleverness in _match.

The most recent change to _match seems to cause it to revert to menu
completion in many cases.  If I just completely remove lines 56 and 57
(dealing with unambiguous_cursor) then all my test cases work just the
way I want them to.  The only thing I lose is earlier expansion in
some cases, but the result doesn't change the behavior or the amount
of typing required.  I don't quite understand what the purpose of
those two lines are.  I believe I understand what they literally mean,
but I don't understand why the behavior is desirable.  Do they do
something important in a case that is excluded by my style settings?
What is the exact purpose of those lines?

I'm running now with the latest CVS release (including your path
characters after slash patch) and the latest round of patches here
with the unambiguous cursor code commented out.  Everything is working
just fine.

The only thing that differs from my original ideal description is that
if you have u1/q/something and u2/q/something, then u?/q/s TAB will
show both choices but will not complete out the something and show
u?/q/something.  You already explained why that can't be done right
now though and it has an easy workaround since now u?/q/s*/ does
exactly the same thing as that would have done.  The result can be
achieved in a maximum of two additional keystrokes.

So I guess what I'm saying is that the latest patches, after excluding
the unambiguous_cursor stuff, give me what I want.  Assuming the
unambiguous_cursor stuff is actually serving a purpose that I fail to
see, can we make it conditional upon something I can turn off?  I'd
suggest something but I don't have enough of a conceptual grasp of
what that code is doing to give the style a suitable name.  Would the
next step be committing these changes and waiting for fallout? :-)

Thanks again for all your work on this!

                                Jay

Re: still confused about completion and matching

[Sven Wischnowsky]

E. Jay Berkenbilt wrote:

> ...
> 
> If I omit "suffix" from :completion:*:paths expand, although 
> 
> ls u?/ TAB
> 
> doesn't generate a superfluous / anymore,
> 
> ls u?/q/ TAB
> 
> still does.

This is a different case. These are the automatically added slashes we
get due to LIST_TYPES. I've got a patch for the C-code to fix it there.
I'll commit it. We won't want it to ever add a file-type character 
after a slash anyway or do we?

> ls u?/q/d TAB
> 
> lists u1/q/devel and u3/q/dark as it should.  However,
> 
> ls u?/q/de TAB
> 
> doesn't list anything.  Similarly,

As you found out, this happened in cases where there was only one match.
This was caused by the yet unchanged compadd around line 624 in _path_files
which used pattern matching even if there was a pattern in a component in
the -p-prefix (which is used using the match specs).

> ...
> 
> Finally,
> 
> ls u? TAB
> 
> works fine, but
> 
> ls ./u? TAB
> 
> makes the ? disappear from the commandline.

A small problem with an attempt at additional cleverness in _match.

> ...
> 
> Now to make a gratuitous personal observation.... zsh is about the
> only open source software I've used recently where it hasn't been just
> as easy for me to send a patch implementing my requested changes as it
> is to describe them in detail.  I'm not sure exactly why this is, but
> I think it's just because there is so much power that can be expressed
> so compactly that the density of code that I don't understand due to
> lack of zsh knowledge is increased.  Perhaps also the software has so
> much potential that I'm thinking of changes that just wouldn't occur
> to me if the software couldn't already almost handle them. 

I sometimes have the impression that this is a trap even we've run
into before ;-)

> I have a
> rule against modifying code I don't understand (at least locally).
> Reading the code in the zsh completion system, I find that there is
> something mysterious in just about every line though the documentation
> seems remarkably complete.  Should I decide to spend the time required
> to really understand what's going on in code like that in _path_files,
> looking back at these discussions and patches will no doubt be an
> invaluable tool. 

_path_files certainly wasn't the best starting point when trying to
learn the ways of the completion system. Writing a new completion
function, looking at older ones, starting with a simple and incomplete
solution and improving it step by step is almost certainly the best
way. As Andrej already said, you should begin by using only _arguments,
continue by using its advanced features as you go along, using (i.e.
calling) other utility functions if needed (I'm referring to functions
like _hosts, for which we still don't have a complete list, sigh). If
needed, you could then have a look at the compadd and compset builtins
and the special completion parameters (and from those $compstate is
normally the last one needed). And if your completion function can
reasonably support different modes, learn using zstyle to look up
styles.



Bye
 Sven

diff -u -r ../oz/Completion/Builtins/_zstyle ./Completion/Builtins/_zstyle
--- ../oz/Completion/Builtins/_zstyle	Sun Oct 22 15:17:57 2000
+++ ./Completion/Builtins/_zstyle	Mon Oct 23 19:28:26 2000
@@ -44,7 +44,7 @@
   ignored-patterns	 c:
   insert-ids             c:insert-ids
   insert-tab             c:bool
-  insert-unambiguous	 c:bool
+  insert-unambiguous	 c:insunambig
   keep-prefix		 c:keep-prefix
   last-prompt		 c:bool
   list			 c:listwhen
@@ -52,6 +52,7 @@
   list-packed		 c:bool
   list-prompt            c:
   list-rows-first	 c:bool
+  list-suffixes		 c:bool
   local			 c:
   match-original	 c:match-orig
   matcher		 c:
@@ -290,6 +291,10 @@
 
     oldmatches) 
       _wanted values expl 'use list of old matches' compadd true false only
+      ;;
+
+    insunambig) 
+      _wanted values expl 'insert unambiguous string compadd true false pattern
       ;;
 
     urgh) 
diff -u -r ../oz/Completion/Core/_match ./Completion/Core/_match
--- ../oz/Completion/Core/_match	Sun Oct 22 15:17:57 2000
+++ ./Completion/Core/_match	Mon Oct 23 20:33:15 2000
@@ -11,41 +11,56 @@
 
 ### Shouldn't be needed any more: [[ _matcher_num -gt 1 ]] && return 1
 
-local tmp opm="$compstate[pattern_match]" ret=0 orig ins
+local tmp opm="$compstate[pattern_match]" ret=1 orig ins
+local oms="$_old_match_string"
+local ocsi="$compstate[insert]" ocspi="$compstate[pattern_insert]"
 
 # Do nothing if we don't have a pattern.
 
 tmp="${${:-$PREFIX$SUFFIX}#[~=]}"
 [[ "$tmp:q" = "$tmp" ]] && return 1
 
+_old_match_string="$PREFIX$SUFFIX$HISTNO"
+
 zstyle -s ":completion:${curcontext}:" match-original orig
-zstyle -b ":completion:${curcontext}:" insert-unambiguous ins
+zstyle -s ":completion:${curcontext}:" insert-unambiguous ins
 
 # Try completion without inserting a `*'?
 
 if [[ -n "$orig" ]]; then
   compstate[pattern_match]='-'
-  _complete && ret=1
+  _complete && ret=0
   compstate[pattern_match]="$opm"
 
-  if (( ret )); then
-    [[ "$ins" = yes &&
-       $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 
-        compstate[pattern_insert]=unambiguous
-    return 0
-  fi
+  # No completion with inserting `*'?
+
+  [[ ret -eq 1 && "$orig" = only ]] && return 1
+fi
+
+if (( ret )); then
+  compstate[pattern_match]='*'
+  _complete && ret=0
+  compstate[pattern_match]="$opm"
 fi
 
-# No completion with inserting `*'?
+if (( ! ret )); then
+
+  if [[ "$ins" = pattern && $compstate[nmatches] -gt 1 ]]; then
 
-[[ "$orig" = only ]] && return 1
+    [[ "$oms" = "$PREFIX$SUFFIX$HISTNO" &&
+       "$compstate[insert]" = automenu-unambiguous ]] &&
+        compstate[insert]=automenu
+    [[ "$compstate[insert]" != *menu ]] &&
+        compstate[pattern_insert]= compstate[insert]=
 
-compstate[pattern_match]='*'
-_complete && ret=1
-compstate[pattern_match]="$opm"
+    [[ $compstate[unambiguous_cursor] -gt $#compstate[unambiguous] ]] &&
+        ins=yes compstate[insert]="$ocsi" compstate[pattern_insert]="$ocspi"
+  fi
 
-[[ ret -eq 1 && "$ins" = yes &&
-   $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 
-    compstate[pattern_insert]=unambiguous
+  [[ "$ins" = (true|yes|on|1) &&
+     $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 
+      compstate[pattern_insert]=unambiguous
+
+fi
 
-return 1-ret
+return ret
diff -u -r ../oz/Completion/Core/_path_files ./Completion/Core/_path_files
--- ../oz/Completion/Core/_path_files	Sun Oct 22 15:17:57 2000
+++ ./Completion/Core/_path_files	Mon Oct 23 20:56:33 2000
@@ -5,7 +5,7 @@
 
 local linepath realpath donepath prepath testpath exppath skips skipped
 local tmp1 tmp2 tmp3 tmp4 i orig eorig pre suf tpre tsuf opre osuf cpre
-local pats haspats ignore pfxsfx sopt gopt opt sdirs ignpar cfopt
+local pats haspats ignore pfxsfx sopt gopt opt sdirs ignpar cfopt listsfx
 local nm=$compstate[nmatches] menu matcher mopts sort match mid accex fake
 
 typeset -U prepaths exppaths
@@ -137,6 +137,8 @@
 fi
 
 zstyle -s ":completion:${curcontext}:paths" special-dirs sdirs
+zstyle -t ":completion:${curcontext}:paths" list-suffixes &&
+    listsfx=yes
 
 [[ "$pats" = ((|*[[:blank:]])\*(|[[:blank:]]*)|*\([^[:blank:]]#/[^[:blank:]]#\)*) ]] &&
     sopt=$sopt/
@@ -460,9 +462,11 @@
       SUFFIX="${tsuf}"
     fi
 
-    if (( tmp4 )) ||
-       [[ -n "$compstate[pattern_match]" &&
-          "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+    # This once tested `|| [[ -n "$compstate[pattern_match]" &&
+    # "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]' but it should now be smart
+    # enough to handle multiple components with patterns.
+
+    if (( tmp4 )); then
       # It is. For menucompletion we now add the possible completions
       # for this component with the unambigous prefix we have built
       # and the rest of the string from the line as the suffix.
@@ -480,15 +484,33 @@
         compquote tmp1 tmp2
       fi
 
+      if [[ -z "$_comp_correct" &&
+            "$compstate[pattern_match]" = \*  && -n "$listsfx" &&
+            "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+        PREFIX="$opre"
+        SUFFIX="$osuf"
+      fi
+
       if [[ -n $menu || -z "$compstate[insert]" ]] ||
-         ! zstyle -t ":completion:${curcontext}:paths" expand suffix; then
+         ! zstyle -t ":completion:${curcontext}:paths" expand suffix ||
+           [[ -z "$listsfx" &&
+              ( -n "$_comp_correct" ||
+                -z "$compstate[pattern_match]" || "$SUFFIX" != */* ||
+                "${SUFFIX#*/}" = (|*[^\\])[][*?#~^\|\<\>]* ) ]]; then
         (( tmp4 )) && zstyle -t ":completion:${curcontext}:paths" ambiguous &&
             compstate[to_end]=
         if [[ "$tmp3" = */* ]]; then
-	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" -s "/${tmp3#*/}" \
-	          -W "$prepath$realpath$testpath" \
-		  "$pfxsfx[@]" -M "r:|/=* r:|=*" \
-		  - "${(@)tmp1%%/*}"
+	  if [[ -z "$listsfx" || "$tmp3" != */?* ]]; then
+	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" -s "/${tmp3#*/}" \
+	            -W "$prepath$realpath$testpath" \
+		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
+		    - "${(@)tmp1%%/*}"
+          else
+	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
+	            -W "$prepath$realpath$testpath" \
+		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
+		    - "${(@)^tmp1%%/*}/${tmp3#*/}"
+          fi
 	else
 	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
 	          -W "$prepath$realpath$testpath" \
@@ -497,12 +519,20 @@
 	fi
       else
         if [[ "$tmp3" = */* ]]; then
-	  tmp3=( -Qf "$mopts[@]" -p "$linepath$tmp2"
+	  tmp4=( -Qf "$mopts[@]" -p "$linepath$tmp2"
 	         -W "$prepath$realpath$testpath"
 	         "$pfxsfx[@]" -M "r:|/=* r:|=*" )
-          for i in "$tmp1[@]"; do
-	    compadd "$tmp3[@]" -s "/${i#*/}" - "${i%%/*}"
-	  done
+	  if [[ -z "$listsfx" ]]; then
+            for i in "$tmp1[@]"; do
+	      compadd "$tmp4[@]" -s "/${i#*/}" - "${i%%/*}"
+	    done
+          else
+            [[ -n "$compstate[pattern_match]" ]] && SUFFIX="${SUFFIX:s./.*/}*"
+
+            for i in "$tmp1[@]"; do
+	      compadd "$tmp4[@]" - "$i"
+	    done
+          fi
         else
 	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
                   -W "$prepath$realpath$testpath" \
@@ -526,22 +556,33 @@
     # take it from the filenames.
 
     testpath="${testpath}${tmp1[1]%%/*}/"
-    tmp1=( "${(@)tmp1#*/}" )
 
     tmp3="${tmp3#*/}"
 
     if [[ "$tpre" = */* ]]; then
-      cpre="${cpre}${tpre%%/*}/"
+      if [[ -z "$_comp_correct" && -n "$compstate[pattern_match]" &&
+            "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+        cpre="${cpre}${tmp1[1]%%/*}/"
+      else
+        cpre="${cpre}${tpre%%/*}/"
+      fi
       tpre="${tpre#*/}"
     elif [[ "$tsuf" = */* ]]; then
       [[ "$tsuf" != /* ]] && mid="$testpath"
-      cpre="${cpre}${tpre}/"
+      if [[ -z "$_comp_correct" && -n "$compstate[pattern_match]" &&
+            "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+        cpre="${cpre}${tmp1[1]%%/*}/"
+      else
+        cpre="${cpre}${tpre}/"
+      fi
       tpre="${tsuf#*/}"
       tsuf=
     else
       tpre=
       tsuf=
     fi
+
+    tmp1=( "${(@)tmp1#*/}" )
   done
 
   if [[ -z "$tmp4" ]]; then
@@ -580,8 +621,14 @@
       else
         compquote tmp4 tmp1
       fi
-      compadd -Qf "$mopts[@]" -p "$linepath$tmp4" -W "$prepath$realpath$testpath" \
-	      "$pfxsfx[@]" -M "r:|/=* r:|=*" -a tmp1
+      if [[ -z "$_comp_correct" && -n "$compstate[pattern_match]" &&
+            "$PREFIX$SUFFIX" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+        compadd -Qf -W "$prepath$realpath" "$pfxsfx[@]" "$mopts[@]" \
+                -M "r:|/=* r:|=*" - "$linepath$tmp4${(@)^tmp1}"
+      else
+        compadd -Qf -p "$linepath$tmp4" -W "$prepath$realpath$testpath" \
+	        "$pfxsfx[@]" "$mopts[@]" -M "r:|/=* r:|=*" -a tmp1
+      fi
     fi
   fi
 done
diff -u -r ../oz/Doc/Zsh/compsys.yo ./Doc/Zsh/compsys.yo
--- ../oz/Doc/Zsh/compsys.yo	Sun Oct 22 15:17:51 2000
+++ ./Doc/Zsh/compsys.yo	Mon Oct 23 19:28:27 2000
@@ -1367,6 +1367,10 @@
 in the context name to one of tt(correct-)var(num) or
 tt(approximate-)var(num), where var(num) is the number of errors that
 were accepted.
+
+When used for the tt(_match) completer, the style may also be set to
+the string `tt(pattern)'.  This makes the pattern on the line be left
+unchanged if it didn't match unambiguously.
 )
 kindex(keep-prefix, completion style)
 item(tt(keep-prefix))(
@@ -1466,6 +1470,14 @@
 determines if matches are to be listed in a rows-first fashion, as for the
 tt(LIST_ROWS_FIRST) option.
 )
+kindex(list-suffixes, completion style)
+item(tt(list-suffixes))(
+This style is used by the function used to complete filenames.  If
+completion is attempted on a string containing multiple partially
+typed pathname components and this style is set to `true', all
+components starting with the first one for which more than one match
+could be generated will be shown.
+)
 kindex(local, completion style)
 item(tt(local))(
 This style is used by completion functions which generate URLs as
@@ -2416,6 +2428,9 @@
 tt(insert-unambiguous) style is set to `true'.  In
 this case menu completion will only be started if no unambiguous string
 could be generated that is at least as long as the original string.
+The style may also be set to the string `tt(pattern)'.  This will keep 
+the pattern on the line intact as long as there isn't an unambiguous
+completion with which it could be replaced.
 
 Note that the matcher specifications defined globally or used by the
 completion functions will not be used.
@@ -3474,7 +3489,7 @@
 `tt(-r)', and `tt(-R)' options from the tt(compadd) builtin.
 
 Finally, the tt(_path_files) function  uses the styles tt(expand),
-tt(ambiguous) and tt(special-dirs) and tt(file-sort).
+tt(ambiguous), tt(special-dirs), tt(list-suffixes) and tt(file-sort).
 )
 findex(_regex_arguments)
 item(tt(_regex_arguments) var(name) var(specs) ...)(

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: still confused about completion and matching

[E. Jay Berkenbilt]

>   > >   Good move I say. Well, look at the _arguments, it should be
>   > >   enough.
>
>   Of course, I immediately realized, that there are two completions
>   here. One for smbclient arguments (command line) and another one
>   for smbclient commands (that is, inside of interactive
>   session). Can tcsh do the latter? :-))

No, it can't. :-)  However, years ago, I had a program called "ile"
(input line editor) that could add editing capabilities to arbitrary
commands.  It wouldn't be hard to write a modern version of this
command using something like gnu readline attached to a pty in which
was running some interactive command....  I wonder how hard it would
be to get zsh to do something like this.

>   If you are really motivated, look at the nslookup function and its usage of
>   zsh/zpty for controlling interactive command. Theoretically, you could
>   complete remote filenames, print jobs ... I love zsh :-))

Wow.... me too. :-)

RE: still confused about completion and matching

[Andrej Borsenkow]

> >
> >   Good move I say. Well, look at the _arguments, it should be
> >   enough.

Of course, I immediately realized, that there are two completions here. One
for smbclient arguments (command line) and another one for smbclient commands
(that is, inside of interactive session). Can tcsh do the latter? :-))

If you are really motivated, look at the nslookup function and its usage of
zsh/zpty for controlling interactive command. Theoretically, you could
complete remote filenames, print jobs ... I love zsh :-))


-andrej

Re: still confused about completion and matching

[E. Jay Berkenbilt]

>   >			      The single remaining command I used to have a
>   > programmable completion for in tcsh that I don't in zsh is smbclient.
>   > I plan to eventually write a completion function for smbclient and
>   > submit it.  This should be much simpler than this _path_files stuff,
>   > especially if I read Etc/completion-style-guide.
>   >
>
>   Good move I say. Well, look at the _arguments, it should be
>   enough. There are plenty of examples - most of completion functions
>   now-a-days are implemented using this great tool. The only problem
>   I forsee is, that smbclient has options after arguments (at least,
>   that is how they are listed in man pages).  _arguments normally
>   expect reverse order.
>
>   But I do not know, if smbclient really requires it, and am too lazy
>   to look in sources :-)

I don't think it's really true.... I've never had anything odd about
smbclient's argument ordering.  I was going to use cvs as a model
since it is smart enough to use cvs with some of the existing
arguments to get the module list.  The same type of idea could be used
do completion for share names once the server has been specified.

I'll use this as incentive to get my current project finished -- my
reward will be letting myself play with zsh completion functions. :-)
It will probably be at least a week before I can start it.

>   Also, if you could send comments (or even patches) for _arguments
>   documentation ...

You can count on it!  Once I get up to a certain level of competence
I'd like to do a thorough cover-to-cover read of the zsh documentation
with red pen in hand.  (I'll probably print the .yo files directly --
easier to generate patches and make comments that way.  Besides, I've
been considering whether to use yodl for some of my own documentation.
This way I could find out whether I like it or not. :-]) I have
written enough documentation to know how important it is to have a
fresh set of eyes read it.  One eventually becomes immune to errors in
one's own writing. :-) I don't know when I'll get to this but
hopefully before the next non-development release which I guess is
supposed to be 4.0?

>   Of course, completing server and share names is a bit demanding
>   ... but it is well possible in Zsh.

See above.  Thanks.

--
E. Jay Berkenbilt (ejb@ql.org)  |  http://www.ql.org/q/

RE: still confused about completion and matching

[Andrej Borsenkow]

                           The single remaining command I used to have a
> programmable completion for in tcsh that I don't in zsh is smbclient.
> I plan to eventually write a completion function for smbclient and
> submit it.  This should be much simpler than this _path_files stuff,
> especially if I read Etc/completion-style-guide.
>

Good move I say. Well, look at the _arguments, it should be enough. There are
plenty of examples - most of completion functions now-a-days are implemented
using this great tool. The only problem I forsee is, that smbclient has
options after arguments (at least, that is how they are listed in man pages).
_arguments normally expect reverse order.

But I do not know, if smbclient really requires it, and am too lazy to look in
sources :-)

Also, if you could send comments (or even patches) for _arguments
documentation ...

Of course, completing server and share names is a bit demanding ... but it is
well possible in Zsh.

-andrej

Re: still confused about completion and matching

[E. Jay Berkenbilt]

With this latest patch, the behavior is very close to what I was
looking for.  It's close enough in fact that even if none of the
problems I mention below are fixed, I would still find the existing
functionality from the latest patch to be quite useful.  I really
appreciate all your work on this.

Here are a few cases that don't work quite as I would expect.  None of
these cases are particularly important, but you may be interested in
them, and some of them may be trivial to fix.

Here's a copy of my testing startup environment with a few new mkdir
commands added:

   PS1='zsh%% '
   setopt noautomenu
   autoload -U compinit
   compinit
   bindkey "^I" complete-word
   rm -rf /tmp/z
   mkdir /tmp/z
   cd /tmp/z
   mkdir u{1,2,3,4,5}
   mkdir u{1,2,3,4}/q
   mkdir u5/q1
   mkdir u1/q/e1
   mkdir u1/q/devel
   mkdir u1/q/e1/b{1,2}
   mkdir u2/q/e2
   mkdir u2/q/e2/a{1,2}
   mkdir u3/q/dark
   zstyle ':completion:*' completer _complete _match
   zstyle ':completion:*:match:*' insert-unambiguous pattern
   zstyle ':completion:*:paths' list-suffixes yes
   zstyle ':completion:*:paths' expand prefix suffix

If I omit "suffix" from :completion:*:paths expand, although 

ls u?/ TAB

doesn't generate a superfluous / anymore,

ls u?/q/ TAB

still does.

ls u?/q/d TAB

lists u1/q/devel and u3/q/dark as it should.  However,

ls u?/q/de TAB

doesn't list anything.  Similarly,

ls u?/q/e TAB

lists u1/q/e1 and u2/q/e2, but

ls u?/q/e2 TAB

lists nothing.  However,

ls u?/q/e2/ TAB

actually expands to u2/q/e2/a as would be ideal.  If you

rmdir u2/q/e2/a2

though then

ls u?/q/e2/ TAB

no longer lists anything.  It seems as though there are some cases in
which there being only a single possible completion causes nothing to
be listed at all whereas the case of no ambiguity on what has been
typed but multiple possible completions works fine.  Both the
devel/dark case and the removal of a2 breaking things illustrate this.

Finally,

ls u? TAB

works fine, but

ls ./u? TAB

makes the ? disappear from the commandline.

As always, should you decide to attempt to fix the above issues, I
will happily test the patches.

I will use zsh for my normal activities with those patches applied
just to make sure nothing else breaks.  I guess it would be good for
people who don't want this new style options to also work with these
patches to make sure that they don't break anything when not turned
on.

>  The problem really is that the completion can only: 1) leave the
>  command line unchanged, 2) insert a match (menu completion like or if
>  the match is unambiguous) or 3) insert the unambiguous part of a
>  match. That last one can't work if the matches were generated using
>  pattern matching -- how should it detect which wildcards should be
>  left unchanged and which have been used to match something
>  unambiguously. And note that the stuff I'm talking about is in the
>  C-code, at the very heart of the completion code.

Thanks for this explanation.  My thoughts on the "how should it detect
which wildcards should be left unchanged...." would be to use an
approach similar to backreferencing in regular expression matches.  If
a given wildcard character (or group of contiguous wildcard
characters) matched the same string in all expansions of the overall
expression, that single group of contiguous wildcard characters could
be replaced.  The completion system could then retry if anything it
did changed the commandline.  I may be totally off the mark here since
my understanding of these internals is minimal, and I'm not really
suggesting implementing anything like that.  For me this has passed
into the realm of academic discussion on what could possibly be done.
The existing patches are really quite enough for me to get what I want
in just about every case I ever really run into.  For example, if I
want to check in all changed files except for Makefile, I can now type

cvs ci -m"my comment here" *~M* TAB

and if the list looks right I can invoke insert-all-matches.  If not,
I can edit the expression I've typed and try again.  Also, if I forget
which of my /u?/q directories something is in, I can use /u?/q/ TAB to
look at all of them at once.  If the completion system ends up being
able to help me further great; if not, I can back over to the ? (with
C-r ?), change it, and continue.  This all seems to work fine both for
my contrived examples in /tmp/z and for my real directory structure,
and is really 99% of what I want in the first place.

I really appreciate all your help in trying to accommodate my esoteric
requests.  I've found this all quite educational as well.

Now to make a gratuitous personal observation.... zsh is about the
only open source software I've used recently where it hasn't been just
as easy for me to send a patch implementing my requested changes as it
is to describe them in detail.  I'm not sure exactly why this is, but
I think it's just because there is so much power that can be expressed
so compactly that the density of code that I don't understand due to
lack of zsh knowledge is increased.  Perhaps also the software has so
much potential that I'm thinking of changes that just wouldn't occur
to me if the software couldn't already almost handle them.  I have a
rule against modifying code I don't understand (at least locally).
Reading the code in the zsh completion system, I find that there is
something mysterious in just about every line though the documentation
seems remarkably complete.  Should I decide to spend the time required
to really understand what's going on in code like that in _path_files,
looking back at these discussions and patches will no doubt be an
invaluable tool.  In the mean time, I plan to start with something a
bit more modest.  The single remaining command I used to have a
programmable completion for in tcsh that I don't in zsh is smbclient.
I plan to eventually write a completion function for smbclient and
submit it.  This should be much simpler than this _path_files stuff,
especially if I read Etc/completion-style-guide.

Thanks again for all your help and for your work on zsh in general.  I
am continually impressed with how much I can do so easily now since
I've switched to zsh!

                                Jay

Re: still confused about completion and matching

[Sven Wischnowsky]

E. Jay Berkenbilt wrote:

> ...
> 
> zsh% ls u?/q/e?/ TAB
> 
> finds no matches.  The problem has to do with the code that is in the
> if (( tmp4 )) block in _path_files starting at line 463.

Yes, it couldn't match the paths with the way $PREFIX and $SUFFIX were 
set up.

> ...
> 
> The functionality I was able to get differs from what I originally
> described in two ways: if partial expansion is possible (i.e.,
> expansion of the first metacharacter but not the second), it is still
> not done.  Expansion is done only when *all* metacharacters match
> unambiguously.  This is fine -- in fact, it's probably better than
> what I originally specified. 

The patch below should be a bit better (also fixing the `u1// ...'
thing you reported).

> The other thing is that the code doesn't
> complete as far as possible.  For example, ls u?/ TAB should complete
> through u?/q since all the choices start with q.  I can live without
> this too though I don't see exactly why it doesn't work.

It's because of the change in _match. That keeps the command line from 
being changed if completion is ambiguous (btw., the patch below
slightly changes that test, too, giving a better behaviour, but it
can't be perfect, I'm sorry).

The problem really is that the completion can only: 1) leave the
command line unchanged, 2) insert a match (menu completion like or if
the match is unambiguous) or 3) insert the unambiguous part of a
match. That last one can't work if the matches were generated using
pattern matching -- how should it detect which wildcards should be
left unchanged and which have been used to match something
unambiguously. And note that the stuff I'm talking about is in the
C-code, at the very heart of the completion code.

We could change the strings generated as matches in _path_files to
contain the patterns in the ambiguous components, but that would break 
menu completion (in the list this could be fixed using display
strings, but with menu completion: no way).

So, sorry, I don't see a way to implement what you want exactly,
without breaking one style of completion or the other.

Bye
 Sven

diff -u -r ../oz/Completion/Builtins/_zstyle ./Completion/Builtins/_zstyle
--- ../oz/Completion/Builtins/_zstyle	Wed Oct 18 20:54:45 2000
+++ ./Completion/Builtins/_zstyle	Wed Oct 18 21:47:59 2000
@@ -44,7 +44,7 @@
   ignored-patterns	 c:
   insert-ids             c:insert-ids
   insert-tab             c:bool
-  insert-unambiguous	 c:bool
+  insert-unambiguous	 c:insunambig
   keep-prefix		 c:keep-prefix
   last-prompt		 c:bool
   list			 c:listwhen
@@ -52,6 +52,7 @@
   list-packed		 c:bool
   list-prompt            c:
   list-rows-first	 c:bool
+  list-suffixes		 c:bool
   local			 c:
   match-original	 c:match-orig
   matcher		 c:
@@ -290,6 +291,10 @@
 
     oldmatches) 
       _wanted values expl 'use list of old matches' compadd true false only
+      ;;
+
+    insunambig) 
+      _wanted values expl 'insert unambiguous string compadd true false pattern
       ;;
 
     urgh) 
diff -u -r ../oz/Completion/Core/_match ./Completion/Core/_match
--- ../oz/Completion/Core/_match	Wed Oct 18 20:54:45 2000
+++ ./Completion/Core/_match	Wed Oct 18 21:57:34 2000
@@ -11,41 +11,55 @@
 
 ### Shouldn't be needed any more: [[ _matcher_num -gt 1 ]] && return 1
 
-local tmp opm="$compstate[pattern_match]" ret=0 orig ins
+local tmp opm="$compstate[pattern_match]" ret=1 orig ins
+local oms="$_old_match_string" ocsi="$compstate[insert]"
 
 # Do nothing if we don't have a pattern.
 
 tmp="${${:-$PREFIX$SUFFIX}#[~=]}"
 [[ "$tmp:q" = "$tmp" ]] && return 1
 
+_old_match_string="$PREFIX$SUFFIX$HISTNO"
+
 zstyle -s ":completion:${curcontext}:" match-original orig
-zstyle -b ":completion:${curcontext}:" insert-unambiguous ins
+zstyle -s ":completion:${curcontext}:" insert-unambiguous ins
 
 # Try completion without inserting a `*'?
 
 if [[ -n "$orig" ]]; then
   compstate[pattern_match]='-'
-  _complete && ret=1
+  _complete && ret=0
   compstate[pattern_match]="$opm"
 
-  if (( ret )); then
-    [[ "$ins" = yes &&
-       $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 
-        compstate[pattern_insert]=unambiguous
-    return 0
-  fi
+  # No completion with inserting `*'?
+
+  [[ ret -eq 1 && "$orig" = only ]] && return 1
+fi
+
+if (( ret )); then
+  compstate[pattern_match]='*'
+  _complete && ret=0
+  compstate[pattern_match]="$opm"
 fi
 
-# No completion with inserting `*'?
+if (( ! ret )); then
+
+  if [[ "$ins" = pattern && $compstate[nmatches] -gt 1 ]]; then
 
-[[ "$orig" = only ]] && return 1
+    [[ "$oms" = "$PREFIX$SUFFIX$HISTNO" &&
+       "$compstate[insert]" = automenu-unambiguous ]] &&
+        compstate[insert]=automenu
+    [[ "$compstate[insert]" != *menu ]] &&
+        compstate[pattern_insert]= compstate[insert]=
 
-compstate[pattern_match]='*'
-_complete && ret=1
-compstate[pattern_match]="$opm"
+    [[ $compstate[unambiguous_cursor] -gt $#compstate[unambiguous] ]] &&
+        ins=yes compstate[insert]="$ocsi"
+  fi
 
-[[ ret -eq 1 && "$ins" = yes &&
-   $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 
-    compstate[pattern_insert]=unambiguous
+  [[ "$ins" = (true|yes|on|1) &&
+     $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 
+      compstate[pattern_insert]=unambiguous
+
+fi
 
-return 1-ret
+return ret
diff -u -r ../oz/Completion/Core/_path_files ./Completion/Core/_path_files
--- ../oz/Completion/Core/_path_files	Wed Oct 18 20:54:45 2000
+++ ./Completion/Core/_path_files	Wed Oct 18 21:48:57 2000
@@ -5,7 +5,7 @@
 
 local linepath realpath donepath prepath testpath exppath skips skipped
 local tmp1 tmp2 tmp3 tmp4 i orig eorig pre suf tpre tsuf opre osuf cpre
-local pats haspats ignore pfxsfx sopt gopt opt sdirs ignpar cfopt
+local pats haspats ignore pfxsfx sopt gopt opt sdirs ignpar cfopt listsfx
 local nm=$compstate[nmatches] menu matcher mopts sort match mid accex fake
 
 typeset -U prepaths exppaths
@@ -137,6 +137,8 @@
 fi
 
 zstyle -s ":completion:${curcontext}:paths" special-dirs sdirs
+zstyle -t ":completion:${curcontext}:paths" list-suffixes &&
+    listsfx=yes
 
 [[ "$pats" = ((|*[[:blank:]])\*(|[[:blank:]]*)|*\([^[:blank:]]#/[^[:blank:]]#\)*) ]] &&
     sopt=$sopt/
@@ -460,9 +462,11 @@
       SUFFIX="${tsuf}"
     fi
 
-    if (( tmp4 )) ||
-       [[ -n "$compstate[pattern_match]" &&
-          "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+    # This once tested `|| [[ -n "$compstate[pattern_match]" &&
+    # "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]' but it should now be smart
+    # enough to handle multiple components with patterns.
+
+    if (( tmp4 )); then
       # It is. For menucompletion we now add the possible completions
       # for this component with the unambigous prefix we have built
       # and the rest of the string from the line as the suffix.
@@ -480,15 +484,33 @@
         compquote tmp1 tmp2
       fi
 
+      if [[ -z "$_comp_correct" &&
+            "$compstate[pattern_match]" = \*  && -n "$listsfx" &&
+            "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+        PREFIX="$opre"
+        SUFFIX="$osuf"
+      fi
+
       if [[ -n $menu || -z "$compstate[insert]" ]] ||
-         ! zstyle -t ":completion:${curcontext}:paths" expand suffix; then
+         ! zstyle -t ":completion:${curcontext}:paths" expand suffix ||
+           [[ -z "$listsfx" &&
+              ( -n "$_comp_correct" ||
+                -z "$compstate[pattern_match]" || "$SUFFIX" != */* ||
+                "${SUFFIX#*/}" = (|*[^\\])[][*?#~^\|\<\>]* ) ]]; then
         (( tmp4 )) && zstyle -t ":completion:${curcontext}:paths" ambiguous &&
             compstate[to_end]=
         if [[ "$tmp3" = */* ]]; then
-	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" -s "/${tmp3#*/}" \
-	          -W "$prepath$realpath$testpath" \
-		  "$pfxsfx[@]" -M "r:|/=* r:|=*" \
-		  - "${(@)tmp1%%/*}"
+	  if [[ -z "$listsfx" || "$tmp3" != */?* ]]; then
+	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" -s "/${tmp3#*/}" \
+	            -W "$prepath$realpath$testpath" \
+		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
+		    - "${(@)tmp1%%/*}"
+          else
+	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
+	            -W "$prepath$realpath$testpath" \
+		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
+		    - "${(@)^tmp1%%/*}/${tmp3#*/}"
+          fi
 	else
 	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
 	          -W "$prepath$realpath$testpath" \
@@ -497,12 +519,20 @@
 	fi
       else
         if [[ "$tmp3" = */* ]]; then
-	  tmp3=( -Qf "$mopts[@]" -p "$linepath$tmp2"
+	  tmp4=( -Qf "$mopts[@]" -p "$linepath$tmp2"
 	         -W "$prepath$realpath$testpath"
 	         "$pfxsfx[@]" -M "r:|/=* r:|=*" )
-          for i in "$tmp1[@]"; do
-	    compadd "$tmp3[@]" -s "/${i#*/}" - "${i%%/*}"
-	  done
+	  if [[ -z "$listsfx" ]]; then
+            for i in "$tmp1[@]"; do
+	      compadd "$tmp4[@]" -s "/${i#*/}" - "${i%%/*}"
+	    done
+          else
+            [[ -n "$compstate[pattern_match]" ]] && SUFFIX="${SUFFIX:s./.*/}*"
+
+            for i in "$tmp1[@]"; do
+	      compadd "$tmp4[@]" - "$i"
+	    done
+          fi
         else
 	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
                   -W "$prepath$realpath$testpath" \
@@ -526,22 +556,33 @@
     # take it from the filenames.
 
     testpath="${testpath}${tmp1[1]%%/*}/"
-    tmp1=( "${(@)tmp1#*/}" )
 
     tmp3="${tmp3#*/}"
 
     if [[ "$tpre" = */* ]]; then
-      cpre="${cpre}${tpre%%/*}/"
+      if [[ -z "$_comp_correct" && -n "$compstate[pattern_match]" &&
+            "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+        cpre="${cpre}${tmp1[1]%%/*}/"
+      else
+        cpre="${cpre}${tpre%%/*}/"
+      fi
       tpre="${tpre#*/}"
     elif [[ "$tsuf" = */* ]]; then
       [[ "$tsuf" != /* ]] && mid="$testpath"
-      cpre="${cpre}${tpre}/"
+      if [[ -z "$_comp_correct" && -n "$compstate[pattern_match]" &&
+            "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+        cpre="${cpre}${tmp1[1]%%/*}/"
+      else
+        cpre="${cpre}${tpre}/"
+      fi
       tpre="${tsuf#*/}"
       tsuf=
     else
       tpre=
       tsuf=
     fi
+
+    tmp1=( "${(@)tmp1#*/}" )
   done
 
   if [[ -z "$tmp4" ]]; then
diff -u -r ../oz/Doc/Zsh/compsys.yo ./Doc/Zsh/compsys.yo
--- ../oz/Doc/Zsh/compsys.yo	Wed Oct 18 20:54:38 2000
+++ ./Doc/Zsh/compsys.yo	Wed Oct 18 21:48:15 2000
@@ -1367,6 +1367,10 @@
 in the context name to one of tt(correct-)var(num) or
 tt(approximate-)var(num), where var(num) is the number of errors that
 were accepted.
+
+When used for the tt(_match) completer, the style may also be set to
+the string `tt(pattern)'.  This makes the pattern on the line be left
+unchanged if it didn't match unambiguously.
 )
 kindex(keep-prefix, completion style)
 item(tt(keep-prefix))(
@@ -1466,6 +1470,14 @@
 determines if matches are to be listed in a rows-first fashion, as for the
 tt(LIST_ROWS_FIRST) option.
 )
+kindex(list-suffixes, completion style)
+item(tt(list-suffixes))(
+This style is used by the function used to complete filenames.  If
+completion is attempted on a string containing multiple partially
+typed pathname components and this style is set to `true', all
+components starting with the first one for which more than one match
+could be generated will be shown.
+)
 kindex(local, completion style)
 item(tt(local))(
 This style is used by completion functions which generate URLs as
@@ -2416,6 +2428,9 @@
 tt(insert-unambiguous) style is set to `true'.  In
 this case menu completion will only be started if no unambiguous string
 could be generated that is at least as long as the original string.
+The style may also be set to the string `tt(pattern)'.  This will keep 
+the pattern on the line intact as long as there isn't an unambiguous
+completion with which it could be replaced.
 
 Note that the matcher specifications defined globally or used by the
 completion functions will not be used.
@@ -3474,7 +3489,7 @@
 `tt(-r)', and `tt(-R)' options from the tt(compadd) builtin.
 
 Finally, the tt(_path_files) function  uses the styles tt(expand),
-tt(ambiguous) and tt(special-dirs) and tt(file-sort).
+tt(ambiguous), tt(special-dirs), tt(list-suffixes) and tt(file-sort).
 )
 findex(_regex_arguments)
 item(tt(_regex_arguments) var(name) var(specs) ...)(

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: still confused about completion and matching

[E. Jay Berkenbilt]

This patch does help, but it there are several things about it that
don't quite work.  I spent some time trying to debug it, but I really
don't have time today to spend on this.  I'm quite under the gun on a
work-related deadline.  I'll share what I have so far though.

Start zsh and run the following:

   PS1='zsh%% '
   setopt noautomenu
   autoload -U compinit
   compinit
   bindkey "^I" complete-word
   rm -rf /tmp/z
   mkdir /tmp/z
   cd /tmp/z
   mkdir u{1,2,3,4,5}
   mkdir u{1,2,3,4}/q
   mkdir u5/q1
   mkdir u1/q/e1
   mkdir u2/q/e2
   mkdir u2/q/e2/a{1,2}
   zstyle ':completion:*' completer _complete _match
   zstyle ':completion:*:match:*' insert-unambiguous pattern
   zstyle ':completion:*:paths' list-suffixes yes
   zstyle ':completion:*:paths' expand prefix suffix

Now:

zsh% ls u? TAB

is good.

zsh% ls u?/q TAB

is good.

zsh% ls u?/q/e? TAB

is good.

zsh% ls u?/q/e?/ TAB

finds no matches.  The problem has to do with the code that is in the
if (( tmp4 )) block in _path_files starting at line 463.

In this case, the compfiles -r call at line 452 gets called four
times.

1.  tmp1=(u2/q/e2/a1 u2/q/e2/a2)
    tmp3=(u?/q/e?/)
    tmp4=0

2.  tmp1=(q/e2/a1 q/e2/a2)
    tmp3=(q/e?/)
    tmp4=0

3.  tmp1=(e2/a1 e2/a2)
    tmp3=(e?/)
    tmp4=0

4.  tmp1=(a1 a2)
    tmp3=()
    tmp4=1

If you cheat and set enter the if block even though tmp4=0 (i.e.,
change 469 to "if (( 1 )); then") then you get

zsh% ls u?/q/e?/ TAB
u2/q/e2/a1  u2/q/e2/a2/

When you do

zsh% ls u?/q/e?/a2 TAB

the commandline gets replaced with the expansion done.

Obviously this is an incorrect change...  Entering that block
unconditionally breaks many other cases.  But I think it shows that
compfiles -r returning 1 is no longer a sufficient condition for
entering that block.

By tweaking bits of this code here and there I was able to get
behavior sufficiently close to what I wanted to be satisfying in all
cases, but not in all cases at the same time. :-)  In other words, I
could fix one problem and break another case.  This is just because I
haven't done a systematic job of understanding the code due to lack of
time, but it seems like this is really almost there.

The functionality I was able to get differs from what I originally
described in two ways: if partial expansion is possible (i.e.,
expansion of the first metacharacter but not the second), it is still
not done.  Expansion is done only when *all* metacharacters match
unambiguously.  This is fine -- in fact, it's probably better than
what I originally specified.  The other thing is that the code doesn't
complete as far as possible.  For example, ls u?/ TAB should complete
through u?/q since all the choices start with q.  I can live without
this too though I don't see exactly why it doesn't work.

On a final note, if you start with the above initialization except
omit the "zstyle ':completion:*:paths' expand prefix suffix" line
only, then 

zsh% ls u?/

gives you u1//  u2//  u3//  u4//  u5// as choices (with the extraneous
/).  I don't know whether it would make sense to run in this mode, but
I thought it was worth pointing this out.

Sorry I can't spend more time on this now.  I would be happy to at
least test additional changes though.

                                Jay

Re: still confused about completion and matching

[Sven Wischnowsky]

I've build the patch below over the weekend which should allow one to
come much nearer to what Jay wants. I won't commit it yet because I
haven't tested it that much (having felt slightly sick over the
weekend, too).

There is one unconditional change, namely that the code in _path_files 
should now be able to deal with multiple components containing
patterns. Well, it should be able to deal with them better than
before. Completing ** doesn't work, though (see below).

Other, conditional, changes: _path_files uses a new style,
list-suffixes. If it is set to true, the completion listing will
contain the sub-paths from the first ambiguous component to the ends.
And another change: the list-ambiguous style, when used by _match, may 
also be set to `pattern'. In this case it will keep the pattern on the 
line unchanged if there are multiple matches. Entering menu completion 
via automenu behaviour should work, though (which hopefully gives an
acceptable mixture, because never being able to have the pattern
replaced if one never reaches something unambiguous...).

So, if Jay or anyone else wants to test it, I suggest to use something 
like:

    zstyle ':completion:*' completer _complete _match
    zstyle ':completion:*:paths' expand prefix suffix
    zstyle ':completion:*:paths' list-suffixes yes
    zstyle ':completion:*:match:*' insert-unambiguous pattern

Setting `expand' to contain `suffix' (with or without `prefix') may be 
crucial for the behaviour Jay wants.


Bart Schaefer wrote:

> ...
> 
> It appears that _path_files isn't entirely prepared to deal with ** in
> the input string.  The code after line 411 ("Step over to the next
> component, if any") lops the ** out of the pattern, thus reducing the 
> set of paths that are considered from ( u1/q1 u2/q1 u3/q1 u4/q ) to
> just ( u4/q/a1 u4/q/a2 ).  So as things now stand:
> 
> 	zsh -f
> 	% zstyle ':completion:*' completer _complete _match
> 	% autoload -U compinit; compinit -D
> 	% ls u?/**/a<C-D>
> 	u4/
> 
> Only `u4' is listed, even though `u1' and `u2' also contain matches of
> the recursive pattern.  This means that even if you could get _path_files
> to insert more than just the ambiguous prefix, you still won't see all
> four of the possible completions you desire.

That's right. And having thought about it some more and played with
it... I think this is really an expansion thing, isn't it?

Or maybe it isn't, but it would require lots of changes and might be
terribly slow. Hm.

> On Oct 13,  1:03pm, Sven Wischnowsky wrote:
> > 
> > [...] the path-prefixes and -suffixes [...]
> > are still matched `normally'. And it has to be this way because the
> > completion code (the C-code) has to be able to build a common string
> > for them which it can if they differ only as far as match specs allow, 
> > but it can't do that for arbitrary strings matched by a pattern.
> 
> It seems to me that _path_files (or perhaps even the C code) could employ
> an algorithm similar to that cooked up by PWS for "zmv -w", to identify
> glob patterns in the string from the command line and implicitly put them
> in parens where necessary so that "match references" (there's another
> overloading of "match") are generated, then used to identify "equivalent"
> substrings in each of the potential matches.

Hm, hadn't thought of that. Weaving that into the existing code might
be hard (but I haven't really thought about it yet).

> > That leaves only the possibility to move the suffixes into the words
> > given to compadd. This would be done in lines 488-510 in _path_files.
> > That together with using a non-empty value for compstate[pattern_match]
> > should bring you nearer to what you want. But note that this won't be
> > added to _path_files if it is done unconditionally. Seeing only the
> > ambiguous component is much clearer for most cases.
> 
> I mostly agree; however, with ** involved, there may be more than one
> consecutive ambiguous path component.  This leads to confusing behavior
> in which it's not clear without actually entering menu-completion exactly
> where in the command line the listed completions would be inserted.  (The
> `ambiguous' style notwithstanding.)

... if we once get completion on ** to work. But the list-suffixes
style in the patch would already help, I think.

Bye
 Sven

diff -u -r ../oz/Completion/Builtins/_zstyle ./Completion/Builtins/_zstyle
--- ../oz/Completion/Builtins/_zstyle	Fri Oct 13 19:41:11 2000
+++ ./Completion/Builtins/_zstyle	Sun Oct 15 20:52:39 2000
@@ -44,7 +44,7 @@
   ignored-patterns	 c:
   insert-ids             c:insert-ids
   insert-tab             c:bool
-  insert-unambiguous	 c:bool
+  insert-unambiguous	 c:insunambig
   keep-prefix		 c:keep-prefix
   last-prompt		 c:bool
   list			 c:listwhen
@@ -52,6 +52,7 @@
   list-packed		 c:bool
   list-prompt            c:
   list-rows-first	 c:bool
+  list-suffixes		 c:bool
   local			 c:
   match-original	 c:match-orig
   matcher		 c:
@@ -290,6 +291,10 @@
 
     oldmatches) 
       _wanted values expl 'use list of old matches' compadd true false only
+      ;;
+
+    insunambig) 
+      _wanted values expl 'insert unambiguous string compadd true false pattern
       ;;
 
     urgh) 
diff -u -r ../oz/Completion/Core/_match ./Completion/Core/_match
--- ../oz/Completion/Core/_match	Fri Oct 13 19:41:11 2000
+++ ./Completion/Core/_match	Sun Oct 15 20:44:10 2000
@@ -11,41 +11,51 @@
 
 ### Shouldn't be needed any more: [[ _matcher_num -gt 1 ]] && return 1
 
-local tmp opm="$compstate[pattern_match]" ret=0 orig ins
+local tmp opm="$compstate[pattern_match]" ret=1 orig ins
+local oms="$_old_match_string"
 
 # Do nothing if we don't have a pattern.
 
 tmp="${${:-$PREFIX$SUFFIX}#[~=]}"
 [[ "$tmp:q" = "$tmp" ]] && return 1
 
+_old_match_string="$PREFIX$SUFFIX"
+
 zstyle -s ":completion:${curcontext}:" match-original orig
-zstyle -b ":completion:${curcontext}:" insert-unambiguous ins
+zstyle -s ":completion:${curcontext}:" insert-unambiguous ins
 
 # Try completion without inserting a `*'?
 
 if [[ -n "$orig" ]]; then
   compstate[pattern_match]='-'
-  _complete && ret=1
+  _complete && ret=0
   compstate[pattern_match]="$opm"
 
-  if (( ret )); then
-    [[ "$ins" = yes &&
-       $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 
-        compstate[pattern_insert]=unambiguous
-    return 0
-  fi
+  # No completion with inserting `*'?
+
+  [[ ret -eq 1 && "$orig" = only ]] && return 1
 fi
 
-# No completion with inserting `*'?
+if (( ret )); then
+  compstate[pattern_match]='*'
+  _complete && ret=0
+  compstate[pattern_match]="$opm"
+fi
 
-[[ "$orig" = only ]] && return 1
+if (( ! ret )); then
 
-compstate[pattern_match]='*'
-_complete && ret=1
-compstate[pattern_match]="$opm"
+  [[ "$ins" = (true|yes|on|1) &&
+     $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 
+      compstate[pattern_insert]=unambiguous
+
+  if [[ "$ins" = pattern && $compstate[nmatches] -gt 1 ]]; then
+    [[ "$oms" = "$PREFIX$SUFFIX" &&
+       "$compstate[insert]" = automenu-unambiguous ]] &&
+        compstate[insert]=automenu
+    [[ "$compstate[insert]" != *menu ]] &&
+        compstate[pattern_insert]= compstate[insert]=
+  fi
 
-[[ ret -eq 1 && "$ins" = yes &&
-   $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 
-    compstate[pattern_insert]=unambiguous
+fi
 
-return 1-ret
+return ret
diff -u -r ../oz/Completion/Core/_path_files ./Completion/Core/_path_files
--- ../oz/Completion/Core/_path_files	Fri Oct 13 19:41:11 2000
+++ ./Completion/Core/_path_files	Sun Oct 15 20:18:58 2000
@@ -5,7 +5,7 @@
 
 local linepath realpath donepath prepath testpath exppath skips skipped
 local tmp1 tmp2 tmp3 tmp4 i orig eorig pre suf tpre tsuf opre osuf cpre
-local pats haspats ignore pfxsfx sopt gopt opt sdirs ignpar cfopt
+local pats haspats ignore pfxsfx sopt gopt opt sdirs ignpar cfopt listsfx
 local nm=$compstate[nmatches] menu matcher mopts sort match mid accex fake
 
 typeset -U prepaths exppaths
@@ -137,6 +137,8 @@
 fi
 
 zstyle -s ":completion:${curcontext}:paths" special-dirs sdirs
+zstyle -t ":completion:${curcontext}:paths" list-suffixes &&
+    listsfx=yes
 
 [[ "$pats" = ((|*[[:blank:]])\*(|[[:blank:]]*)|*\([^[:blank:]]#/[^[:blank:]]#\)*) ]] &&
     sopt=$sopt/
@@ -460,9 +462,11 @@
       SUFFIX="${tsuf}"
     fi
 
-    if (( tmp4 )) ||
-       [[ -n "$compstate[pattern_match]" &&
-          "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+    # This once tested `|| [[ -n "$compstate[pattern_match]" &&
+    # "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]' but it should now be smart
+    # enough to handle multiple components with patterns.
+
+    if (( tmp4 )); then
       # It is. For menucompletion we now add the possible completions
       # for this component with the unambigous prefix we have built
       # and the rest of the string from the line as the suffix.
@@ -480,15 +484,30 @@
         compquote tmp1 tmp2
       fi
 
+if [[ "$compstate[pattern_match]" = \*  && -n "$listsfx" ]]; then
+PREFIX="$opre"
+SUFFIX="$osuf"
+fi
+
       if [[ -n $menu || -z "$compstate[insert]" ]] ||
-         ! zstyle -t ":completion:${curcontext}:paths" expand suffix; then
+         ! zstyle -t ":completion:${curcontext}:paths" expand suffix ||
+           [[ -z "$listsfx" &&
+              ( -z "$compstate[pattern_match]" || "$SUFFIX" != */* ||
+                "${SUFFIX#*/}" = (|*[^\\])[][*?#~^\|\<\>]* ) ]]; then
         (( tmp4 )) && zstyle -t ":completion:${curcontext}:paths" ambiguous &&
             compstate[to_end]=
         if [[ "$tmp3" = */* ]]; then
-	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" -s "/${tmp3#*/}" \
-	          -W "$prepath$realpath$testpath" \
-		  "$pfxsfx[@]" -M "r:|/=* r:|=*" \
-		  - "${(@)tmp1%%/*}"
+	  if [[ -z "$listsfx" ]]; then
+	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" -s "/${tmp3#*/}" \
+	            -W "$prepath$realpath$testpath" \
+		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
+		    - "${(@)tmp1%%/*}"
+          else
+	    compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
+	            -W "$prepath$realpath$testpath" \
+		    "$pfxsfx[@]" -M "r:|/=* r:|=*" \
+		    - "${(@)^tmp1%%/*}/${tmp3#*/}"
+          fi
 	else
 	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
 	          -W "$prepath$realpath$testpath" \
@@ -500,9 +519,15 @@
 	  tmp3=( -Qf "$mopts[@]" -p "$linepath$tmp2"
 	         -W "$prepath$realpath$testpath"
 	         "$pfxsfx[@]" -M "r:|/=* r:|=*" )
-          for i in "$tmp1[@]"; do
-	    compadd "$tmp3[@]" -s "/${i#*/}" - "${i%%/*}"
-	  done
+	  if [[ -z "$listsfx" ]]; then
+            for i in "$tmp1[@]"; do
+	      compadd "$tmp3[@]" -s "/${i#*/}" - "${i%%/*}"
+	    done
+          else
+            for i in "$tmp1[@]"; do
+	      compadd "$tmp3[@]" - "$i"
+	    done
+          fi
         else
 	  compadd -Qf "$mopts[@]" -p "$linepath$tmp2" \
                   -W "$prepath$realpath$testpath" \
@@ -526,22 +551,33 @@
     # take it from the filenames.
 
     testpath="${testpath}${tmp1[1]%%/*}/"
-    tmp1=( "${(@)tmp1#*/}" )
 
     tmp3="${tmp3#*/}"
 
     if [[ "$tpre" = */* ]]; then
-      cpre="${cpre}${tpre%%/*}/"
+      if [[ -n "$compstate[pattern_match]" &&
+            "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+        cpre="${cpre}${tmp1[1]%%/*}/"
+      else
+        cpre="${cpre}${tpre%%/*}/"
+      fi
       tpre="${tpre#*/}"
     elif [[ "$tsuf" = */* ]]; then
       [[ "$tsuf" != /* ]] && mid="$testpath"
-      cpre="${cpre}${tpre}/"
+      if [[ -n "$compstate[pattern_match]" &&
+            "$tmp2" = (|*[^\\])[][*?#~^\|\<\>]* ]]; then
+        cpre="${cpre}${tmp1[1]%%/*}/"
+      else
+        cpre="${cpre}${tpre}/"
+      fi
       tpre="${tsuf#*/}"
       tsuf=
     else
       tpre=
       tsuf=
     fi
+
+    tmp1=( "${(@)tmp1#*/}" )
   done
 
   if [[ -z "$tmp4" ]]; then
diff -u -r ../oz/Doc/Zsh/compsys.yo ./Doc/Zsh/compsys.yo
--- ../oz/Doc/Zsh/compsys.yo	Fri Oct 13 19:41:04 2000
+++ ./Doc/Zsh/compsys.yo	Sun Oct 15 21:23:34 2000
@@ -1367,6 +1367,11 @@
 in the context name to one of tt(correct-)var(num) or
 tt(approximate-)var(num), where var(num) is the number of errors that
 were accepted.
+
+When used for the tt(_match) completer, the style may also be set to
+the string `tt(pattern)'.  This makes the pattern on the line be left
+unchanged if there are multiple matches, so that it is only replaced
+when completion is unambiguous.
 )
 kindex(keep-prefix, completion style)
 item(tt(keep-prefix))(
@@ -1466,6 +1471,14 @@
 determines if matches are to be listed in a rows-first fashion, as for the
 tt(LIST_ROWS_FIRST) option.
 )
+kindex(list-suffixes, completion style)
+item(tt(list-suffixes))(
+This style is used by the function used to complete filenames.  If
+completion is attempted on a string containing multiple partially
+typed pathname components and this style is set to `true', all
+components starting with the first one for which more than one match
+could be generated will be shown.
+)
 kindex(local, completion style)
 item(tt(local))(
 This style is used by completion functions which generate URLs as
@@ -2416,6 +2429,9 @@
 tt(insert-unambiguous) style is set to `true'.  In
 this case menu completion will only be started if no unambiguous string
 could be generated that is at least as long as the original string.
+The style may also be set to the string `tt(pattern)'.  This will keep 
+the pattern on the line intact as long as there isn't an unambiguous
+completion with which it could be replaced.
 
 Note that the matcher specifications defined globally or used by the
 completion functions will not be used.
@@ -3474,7 +3490,7 @@
 `tt(-r)', and `tt(-R)' options from the tt(compadd) builtin.
 
 Finally, the tt(_path_files) function  uses the styles tt(expand),
-tt(ambiguous) and tt(special-dirs) and tt(file-sort).
+tt(ambiguous), tt(special-dirs), tt(list-suffixes) and tt(file-sort).
 )
 findex(_regex_arguments)
 item(tt(_regex_arguments) var(name) var(specs) ...)(

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: still confused about completion and matching

[Bart Schaefer]

On Oct 12,  3:56pm, E. Jay Berkenbilt wrote:
} 
} What I want is effectively for the completion system to wait as long
} as possible before replacing a pattern with a match and to leave my
} cursor where it is when I hit TAB.

It took me at least as long to decide whether to answer this, and then
how much I had to add to what Sven said already, as Jay reported that
it took to compose his message in the first place; on top of that I've
been feverish all weekend, so I hope what I've finally ended up saying
makes at least a little bit of sense.

} If I type
} 
} zsh% ls u?/**/a_TAB
} 
} ideally, I like to see
} 
} zsh% ls u?/**/a_
} u2/q1/e2/a1  u2/q1/e2/a2  u4/q/a1  u4/q/a2
} 
} if I have extended_glob set (which is actually the thing that first
} made me switch to zsh).  Does this make sense?

One random aside:  You don't need extended_glob to use **.

It appears that _path_files isn't entirely prepared to deal with ** in
the input string.  The code after line 411 ("Step over to the next
component, if any") lops the ** out of the pattern, thus reducing the 
set of paths that are considered from ( u1/q1 u2/q1 u3/q1 u4/q ) to
just ( u4/q/a1 u4/q/a2 ).  So as things now stand:

	zsh -f
	% zstyle ':completion:*' completer _complete _match
	% autoload -U compinit; compinit -D
	% ls u?/**/a<C-D>
	u4/

Only `u4' is listed, even though `u1' and `u2' also contain matches of
the recursive pattern.  This means that even if you could get _path_files
to insert more than just the ambiguous prefix, you still won't see all
four of the possible completions you desire.

On Oct 13,  1:03pm, Sven Wischnowsky wrote:
> 
> [...] the path-prefixes and -suffixes [...]
> are still matched `normally'. And it has to be this way because the
> completion code (the C-code) has to be able to build a common string
> for them which it can if they differ only as far as match specs allow, 
> but it can't do that for arbitrary strings matched by a pattern.

It seems to me that _path_files (or perhaps even the C code) could employ
an algorithm similar to that cooked up by PWS for "zmv -w", to identify
glob patterns in the string from the command line and implicitly put them
in parens where necessary so that "match references" (there's another
overloading of "match") are generated, then used to identify "equivalent"
substrings in each of the potential matches.

> That leaves only the possibility to move the suffixes into the words
> given to compadd. This would be done in lines 488-510 in _path_files.
> That together with using a non-empty value for compstate[pattern_match]
> should bring you nearer to what you want. But note that this won't be
> added to _path_files if it is done unconditionally. Seeing only the
> ambiguous component is much clearer for most cases.

I mostly agree; however, with ** involved, there may be more than one
consecutive ambiguous path component.  This leads to confusing behavior
in which it's not clear without actually entering menu-completion exactly
where in the command line the listed completions would be inserted.  (The
`ambiguous' style notwithstanding.)

I had a bit more to ask about, but I came to the conclustion that I'm not
coherent enough yet to write it down, so I guess I'll leave it at that.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: still confused about completion and matching

[Sven Wischnowsky]

You may find messages about completion on patterns, including **/ in
the archive (sorry, can't remember the message numbers, maybe Adam or
Andrej can help). One thing I can remember us saying about the **
thing is that this might, urgh, be terribly slow in some cases.


I'll first try to clarify some things and then comment on some of your 
remarks (I guess you already figured out most of the following).

The most important thing you have to know about completion and
matching is that here are two ways to do it. You get the `normal' one
with glob_complete unset, which is the same as
compstate[pattern_match] unset or empty. In this case the completion
system only uses the prefix and suffix from the line and the match
specs (set with the matcher-list and matcher styles, given to the -M
option of compadd).

The other way is selected with compstate[pattern_match] set to a
non-empty string. In that case the string from the line is taken as a
pattern and compared to the possible completions. BUT without the
prefixes and suffixes given with the -[psiI] options to compadd. In
filename completion, these are the path-prefixes and -suffixes. They
are still matched `normally'. And it has to be this way because the
completion code (the C-code) has to be able to build a common string
for them which it can if they differ only as far as match specs allow, 
but it can't do that for arbitrary strings matched by a pattern.
Also, since patterns allow for completely different matches, menu
completion is normally started. After all, the user started
`completion', not `listing'. The pattern_insert key of compstate is
expected to be only seldom of any use.

That leaves only the possibility to move the suffixes into the words
given to compadd. This would be done in lines 488-510 in _path_files.
That together with using a non-empty value for compstate[pattern_match]
should bring you nearer to what you want. But note that this won't be
added to _path_files if it is done unconditionally. Seeing only the
ambiguous component is much clearer for most cases.


Some remarks:

> if I type
> 
> zsh% ls u?/q/_TAB
> 
> I get menu completion with the choices /u1, /u2, and /u4 and with my
> command line cycling through u1/q/, u2/q/, and /u4/q/, each time I hit
> TAB.  If when it says u2/q/ I hit space, backspace, TAB, then u2/q/
> becomes u2/q1/e2/.  It is as if the completion system is imagining a *
> after the q and before the /.  I thought this would happen only if I
> had compstate[pattern_match]='*', not ='-'.  Am I confused?

The whole point of _path_files is to do exactly that. Completing
multiple pathname components at once. And, as explained above, this
has *nothing* to do with compstate[pattern_match].

> As I was researching all this, I was looking at _match to see whether
> I could get that to do what I wanted.  I am very confused about line
> 47:
> 
>    $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 

compstate[unambiguous] is the unambiguous string the completion code
was able to build for all matches added. The test just checks if that
string is longer than the original string (which is split at the
cursor position into $PREFIX and $SUFFIX). You have the feeling that
this is a hack, you are right. It was my first and, until now, last
attempt to come up with a sensible condition for deciding if an
unambiguous string may make sense. That's all.


So... there are two completely different things one could do. First,
using only strings without patterns (all your exampleds except the **
thing used only a ? at the end, given the same as an * at the end,
which is normal for _path_files anyway). One could then add a style to 
control if the whole ambiguous suffixes should be shown in the
list. Another style could be used to say that the cursor should stay
where it is if there are ambiguities both at the end and in a previous 
component. For both styles there might be other things one could
reasonably make them conditional upon, but I think only experience
could show what they are.

Or one could try to add this multi-component pattern behaviour. One
would need to add code that decides if for a certain component the
original string or the one generated should be used. The original
string would keep the pattern unchanged, of course. This is already
not trivial to decide because only later components say if a previous
one should be left alone or not. Then, when the prefix is unambiguous, 
i.e. if one or more components should be replaced with the only matching
directory names, there are two possibilities. First, one could use the
-U option of compadd (although I would generally not suggest to ever
use -U, at least not in a normal completion context). Or one could
trick the completion code into believing that everything is fine by
setting PREFIX and SUFFIX to the string(s) that are inserted. That
would make it use matching, but since the strings are the same... As
far as can think, this should even make it possible to modify the
prefix and list possible matches for a later component (listing them
without the unambiguous prefix) at the same time (on the same
completion attempt). In other words, this might allow you to do what
you wanted.


Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: still confused about completion and matching

[E. Jay Berkenbilt]

Let me add to my previous message by stating that I see that what I'm
really talking about is generating a list of completions.  This seems
pretty much separate from whether menu completion is used or not
(other than that I don't want it to just happen automagically if I
don't have auto_menu set or some other setting that makes it happen).
Given the behavior that I outlined, with auto_menu on, the
multi-component choices should still be presented with menu
completion......  Does that make sense?

still confused about completion and matching

[E. Jay Berkenbilt]

Thanks for the tips on _all_matches.  That part is now working
perfectly for me.  Since I now have ^Xa to insert all matches, I don't
want hitting TAB with a pattern to do that anymore.  This means that I
am free to go after what I really want. :-)

So I'm now back to trying to customize pattern matching behavior.  I
can't quite get it to do what I want.  I think I may be running into
to a combination of a bug and some misunderstanding, but probably
mostly just not knowing quite enough yet.

This is all with the current CVS version...

First, I'll explain what I want.  Then I can ramble for a while about
what I've tried and what I've found.

What I want is effectively for the completion system to wait as long
as possible before replacing a pattern with a match and to leave my
cursor where it is when I hit TAB.  It should only replace a match
when it can do so unambiguously, and, when it shows choices, it should
show as many directory components as required starting from the first
one where there is an ambiguity and ending with where the cursor is.
It should not complete past a /.  It should never fall back to menu
completion.

I'll try to say this more algorithmically.  I haven't coded this for
real, so there may still be a logic error lurking here.  I hope not.

First of all, we want to do this only if we are completing filenames.
This logic probably needs to go in _path_files itself perhaps
conditionally upon some compstate[pattern_match] setting or on some
style.

I know that _path_files has lots of code to deal with filtering the
list of matches based on various criteria such as looking at only
files with certain types or attributes or that match certain
patterns.  I am going to oversimplify what I'm looking for by ignoring
this very important functionality in hopes that doing so will make it
possible for me to communicate my goal.

What I'm looking for goes something like this:

  cur_word = the word currently being completed
  tmp1 = list of matches of ${cur_word}*
  tmp2 = longest prefix common to all matches
  if tmp2 contains a /
    # Some pattern characters can be unambiguously replaced
    tmp3 = everything up to and including the last / in tmp2
    replace "the appropriate part of cur_word" with tmp3
  else
    tmp3 = ""
  fi
  the list of choices is the portion of each word in tmp1 that
      follows the common tmp3 prefix (if any)

Then the completion system should still insert after the cursor
whatever can be inserted unambiguously.  (This is the part I can't
figure out how to do in the current system.)

Figuring out what is meant by the phrase "the appropriate part of
cur_word" is obviously very complex and I see that a great deal of
code in _path_files is devoted to this problem.  It gets especially
hairy if multiple components may be matched by any pattern.

Now I'll try clarify with examples.  I haven't written this into code
so there's a chance I may be making a mistake somewhere and not saying
what I mean but I've tried very hard to avoid that.

Suppose I have the directory structure created by the following
commands:

   rm -rf /tmp/z
   mkdir /tmp/z
   cd /tmp/z
   mkdir u{1,2,3,4}
   mkdir u{1,2,3}/q1
   mkdir u4/q
   mkdir u1/q1/e1
   mkdir u2/q1/e2
   mkdir u2/q1/e2/a{1,2}
   mkdir u4/q/a{1,2}

This is the following structure:

u1
 q1
  e1
u2
 q1
  e2
   a1
   a2
u3
 q1
u4
 q
  a1
  a2

For clarity, throughout this message, I am using _ to represent the
cursor position and TAB to mean that I am hitting TAB at that point.

If I now type

zsh% ls u?_TAB

I want

zsh% ls u?/_
u1/  u2/  u3/  u4/

The ? cannot be replaced unambiguously (tmp3 above is empty) so we
leave it alone.  However, all the matches to this pattern can be
followed by a /.

Then if I hit TAB again, I want to see

zsh% ls u?/q_
u1/q  u2/q  u3/q  u4/q/

because, again it is possible to complete through to the q for all
choices without replacing the ? since all possible choices start with
q.  Note that the u4/q/ case gets a trailing / because u4/q/ is
complete and is a directory.

If I type 

zsh% ls u?/q/_TAB

I want

zsh% ls u4/q/a_
a1/  a2/

since there is now only one way to replace the ?.  After this, we no
longer have a pattern so subsequent tabs should behave "normally".  (I
don't want the completion system to imagine that there is a * after
the q.  It seems to do that now, even with
compstate[pattern_match]='-', as I discuss below.)

Now suppose I type

zsh% ls u?/q1/_TAB

I should see 

zsh% ls u?/q1/_
u1/q1/  u2/q1/  u3/q1/

since these are the possible choices.  If I now say

zsh% ls u?/q1/e_TAB

I should see

zsh% ls u?/q1/e_
u1/q1/e1/  u2/q1/e2/

because these are the choices that match this pattern.  If I type

zsh% ls u?/q1/e?/a_TAB

I should see

zsh% ls u2/q1/e2/a_
a1/  a2/

It's okay if I have to hit TAB more than once to get this output.

If I type

zsh% ls u?/**/a_TAB

ideally, I like to see

zsh% ls u?/**/a_
u2/q1/e2/a1  u2/q1/e2/a2  u4/q/a1  u4/q/a2

if I have extended_glob set (which is actually the thing that first
made me switch to zsh).  Does this make sense?  Am I asking for too
much? :-)



Now, you can ALMOST get this behavior.  With the following definition
of _qcomp:

function _qcomp {
   local tmp opm="$compstate[pattern_match]" found=0

   tmp="${${:-$PREFIX$SUFFIX}#[~=]}"
   [[ "$tmp:q" = "$tmp" ]] && return 1

   compstate[pattern_match]='-'
   _complete && found=1
   compstate[pattern_match]="$opm"

   if (( found )); then
     compstate[insert]=
   fi

   return 1-found
}

and the following completion style setting:

zstyle ':completion:*' completer _complete _qcomp _ignored

If you look at the traces, _path_files does actually have the complete
list of matches.  The choices you see are all the first components of
the choices I want to see.  If you replace 

compstate[insert]=

with

compstate[insert]=unambiguous

above then you get another behavior that is almost right.  It breaks
for the ** case though.  Also, it falls back to menu completion for
some reason that I don't understand.  With this, there is some other
behavior I get which I think is a feature but I don't really like it.
I think it has to do with compstate[pattern_match].  Given the above,
if I type

zsh% ls u?/q/_TAB

I get menu completion with the choices /u1, /u2, and /u4 and with my
command line cycling through u1/q/, u2/q/, and /u4/q/, each time I hit
TAB.  If when it says u2/q/ I hit space, backspace, TAB, then u2/q/
becomes u2/q1/e2/.  It is as if the completion system is imagining a *
after the q and before the /.  I thought this would happen only if I
had compstate[pattern_match]='*', not ='-'.  Am I confused?


As I was researching all this, I was looking at _match to see whether
I could get that to do what I wanted.  I am very confused about line
47:

   $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 

I can't figure out its purpose.  It seems to always be false.  This
line prevented me from getting compstate[pattern_insert]=unambiguous
even if I had 

zstyle ':completion:*' insert-unambiguous true

which is why I ultimately went to _qcomp again.


The reason that I would like this is based on the way I tend to
arrange my directories.  On my machine, I have various partitions
which I call /u1, /u2, etc.  On each partition, I create a directory
called q which I own and which I use as scratch space.  I don't always
remember which directory something is in.  I have a directory called
/u1/q/devel where I check out stuff from the CVS repository at the
office.  I have /u1/q/zsh where I currently have zsh checked out.
However, on some other system, maybe devel is in /u2/q.  I would like
to be able to say

zsh% ls /u?/q/d_TAB 

and get

zsh% ls /u1/q/devel/

On the other hand, sometimes I have a /u1/q/devel and a /u2/q/devel
with different side branches.  Maybe job 900 is in one and job 904 is
in another.  In that case, I should be able to type

zsh% ls /u?/q/d_TAB

and get 

zsh% ls /u?/q/devel/
u1/q/devel/  u2/q/devel/

and then I should hit TAB again and get

zsh% ls /u?/q/devel/
u1/q/devel/900  u2/q/devel/904

I should then be able to type 904 so that i have

zsh% ls /u?/q/devel/904

and get the shell to replace the ? with a 2 since it can now do so
unambiguously.



I hope this makes sense.  Would you believe that it took me four hours
to compose this mail message?  I've gone through about 10 different
ways of presenting this with some earlier versions referencing trace
output from ^x? with completion functions set various ways,
referencing different parts of _path_files, etc....  I hope that the
message hasn't gotten completely lost.  This would be so much easier
to communicate interactively.

The main thing that would currently stop me from being able to
implement this in a reasonable amount of time is that I still don't
really fully understand all the internals of completion widgets.  I
don't know all the options to compadd, some of the compstate keys,
compquote, etc., and I still don't fully understand how to use styles.
It seems though that it shouldn't be that hard to get _path_files to
do this at least conditionally.  What do you think?

My brain hurts and I have to get back to my real work.... :-/

Hopefully this will start some interesting discussion. :-)

                                Jay