5.0.3 +* -> git completion regression

5.0.3 +* -> git completion regression

[Phil Pennock]

% git push origin <TAB>
__git_complete_remote_or_refspec:33: bad pattern: +*

Does not occur in a build from 5.0.2, does with 5.0.3; this completion
comes from the git project.

The git project has a _git file which ends up finding a
git-completion.bash file and sourcing that with:

  ZSH_VERSION='' . "$script"

----------------------------8< cut here >8------------------------------
__git_complete_remote_or_refspec ()
{
        local cur_="$cur" cmd="${words[1]}"
[...]
        case "$cur_" in
        *:*)
                case "$COMP_WORDBREAKS" in
                *:*) : great ;;
                *)   pfx="${cur_%%:*}:" ;;
                esac
                cur_="${cur_#*:}"
                lhs=0
                ;;
        +*)
                pfx="+"
                cur_="${cur_#+}"
                ;;
        esac
[...]
----------------------------8< cut here >8------------------------------

That case matching pattern +* is on the 33rd line of the function.

So this appears to be bash not treating the + as special where zsh does.
AFAICT, this seems like perfectly reasonable behaviour on zsh's part,
but nonetheless something which used to work no longer does.

git bisect says this is 68d0d76db55c0b8778f0b68d3eda54060b576c41 :

    31441: use array to decide which forms of pattern are enabled

I think the right approach might be to file a git bug instead, to modify
the _git wrapper from:

  ZSH_VERSION='' . "$script"

to:

  emulate sh -c 'ZSH_VERSION="" . "$script"'

since that fixes it and the zsh behavious appears correct.

Thoughts?  Is there anything which zsh should be doing differently?

-Phil

Re: 5.0.3 +* -> git completion regression

[Peter Stephenson]

On Mon, 16 Dec 2013 03:14:37 -0500
Phil Pennock <zsh-workers+phil.pennock@spodhuis.org> wrote:
> % git push origin <TAB>
> __git_complete_remote_or_refspec:33: bad pattern: +*

I can't think of any circumstance where +* should be a bad pattern.  Is
this easy to reproduce in a simple example?

pws

Re: 5.0.3 +* -> git completion regression

[Phil Pennock]

On 2013-12-16 at 09:47 +0000, Peter Stephenson wrote:
> On Mon, 16 Dec 2013 03:14:37 -0500
> Phil Pennock <zsh-workers+phil.pennock@spodhuis.org> wrote:
> > % git push origin <TAB>
> > __git_complete_remote_or_refspec:33: bad pattern: +*
> 
> I can't think of any circumstance where +* should be a bad pattern.  Is
> this easy to reproduce in a simple example?

Finally managed it.  It seems that the _git wrapper has a function
__git_zsh_bash_func() which does "emulate -L ksh" and somehow that
breaks when calling into the functions, while sourcing the file under
sticky sh emulation (via "emulate sh -c") overrides that ksh context and
fixes this.

% zsh -f
ilmenite% emulate ksh
ilmenite% case "$foo" in +*) echo snert;; esac
zsh: bad pattern: +*
%


I have git-1.8.5.1 from HomeBrew on a Mac, and zsh 5.0.3 from MacPorts
(yeah, mixing those two isn't great, but not relevant here).  I used git
bisect to find the triggering commit in zsh, doing a
configure/make/make-install cycle with each, invoking that shell,
switching to a git repo dir and typing "git push origin <TAB>".

I didn't use "zsh -f" during that cycle, but for completeness:

    % zsh -f
    ilmenite% fpath=(/usr/local/share/zsh/site-functions $fpath)
    ilmenite% autoload compinit ; compinit
    ilmenite% cd ~/path/to/a/git/repo/
    ilmenite% git push origin 
    __git_complete_remote_or_refspec:33: bad pattern: +*
    ilmenite%

Without the fpath adjustment, I get the _git completion from zsh and
everything works fine.  It's only the git-supplied zsh completion
configuration which breaks until I force emulation mode for the .bash
inclusion.

----------------------------8< cut here >8------------------------------
+__git_zsh_main:25> case arg (command)
+__git_zsh_main:25> case arg (arg)
+__git_zsh_main:33> local 'command=push' __git_dir
+__git_zsh_main:35> ((  0  ))
+__git_zsh_main:38> __git_dir=''·
+__git_zsh_main:41> ((  0  ))
+__git_zsh_main:43> words=( git push origin )·
+__git_zsh_main:45> __git_zsh_bash_func push
+__git_zsh_bash_func:2> emulate -L ksh
+__git_zsh_bash_func:4> local 'command=push'
+__git_zsh_bash_func:6> local 'completion_func=_git_push'
+__git_zsh_bash_func:7> declare -f _git_push
+__git_zsh_bash_func:7> _git_push
+_git_push:2> case origin (--repo)
+_git_push:7> case  (--repo=*)
+_git_push:7> case  (--*)
+_git_push:20> __git_complete_remote_or_refspec
+__git_complete_remote_or_refspec:2> local 'cur_=' 'cmd=push'
+__git_complete_remote_or_refspec:3> local i 'c=2' 'remote=' 'pfx=' 'lhs=1' 'no_complete_refspec=0'
+__git_complete_remote_or_refspec:4> [ push '=' remote ']'
+__git_complete_remote_or_refspec:7> [ 2 -lt 3 ']'
+__git_complete_remote_or_refspec:8> i=origin·
+__git_complete_remote_or_refspec:9> case origin (--mirror)
+__git_complete_remote_or_refspec:9> case origin (--all)
+__git_complete_remote_or_refspec:9> case origin (-*)
+__git_complete_remote_or_refspec:9> case origin (*)
+__git_complete_remote_or_refspec:21> remote=origin·
+__git_complete_remote_or_refspec:21> break
+__git_complete_remote_or_refspec:25> [ -z origin ']'
+__git_complete_remote_or_refspec:29> [ 0 '=' 1 ']'
+__git_complete_remote_or_refspec:32> [ origin '=' . ']'
+__git_complete_remote_or_refspec:33> case  (*:*)
+__git_complete_remote_or_refspec:33> case  (+*)
__git_complete_remote_or_refspec:33: bad pattern: +*
+_dispatch:64> [[ patterns == (all|*patterns*) ]]
+_dispatch:64> return ret
----------------------------8< cut here >8------------------------------

Re: 5.0.3 +* -> git completion regression

[Bart Schaefer]

On Dec 16,  5:37am, Phil Pennock wrote:
}
} % zsh -f
} ilmenite% emulate ksh
} ilmenite% case "$foo" in +*) echo snert;; esac
} zsh: bad pattern: +*
} %

Here's what I get when compiled with debugging turned on:

$ case foo in +*) echo bar;; esac
 BUG: character not handled in patcomppiece
zsh: bad pattern: +*
$ 

(Not sure why there's a leading space on the "BUG:" line, seems odd.)

Anyway it looks like we've triggered lookahead on "+" because of ksh
emulation, but then when the next character is not a paren it fails to
back up and reprocess the "+", or something to that effect.

This is all tangled up with "disable -p" changes, but selectively baking
out chunks that involve the kshchar flag hasn't so far shown anything.

Re: 5.0.3 +* -> git completion regression

[Peter Stephenson]

On Mon, 16 Dec 2013 05:37:26 -0500
Phil Pennock <zsh-workers+phil.pennock@spodhuis.org> wrote:
> On 2013-12-16 at 09:47 +0000, Peter Stephenson wrote:
> > On Mon, 16 Dec 2013 03:14:37 -0500
> > Phil Pennock <zsh-workers+phil.pennock@spodhuis.org> wrote:
> > > % git push origin <TAB>
> > > __git_complete_remote_or_refspec:33: bad pattern: +*
> > 
> > I can't think of any circumstance where +* should be a bad pattern.  Is
> > this easy to reproduce in a simple example?
> 
> Finally managed it.  It seems that the _git wrapper has a function
> __git_zsh_bash_func() which does "emulate -L ksh" and somehow that
> breaks when calling into the functions, while sourcing the file under
> sticky sh emulation (via "emulate sh -c") overrides that ksh context and
> fixes this.
> 
> % zsh -f
> ilmenite% emulate ksh
> ilmenite% case "$foo" in +*) echo snert;; esac
> zsh: bad pattern: +*

Thanks.  This is an efficient fix, so I hope it's good enough --- I've
added a regression test, and it seems to be.

Another good workout with some sh scripts or functions would be useful.

diff --git a/Src/pattern.c b/Src/pattern.c
index a7ef125..b79c3b4 100644
--- a/Src/pattern.c
+++ b/Src/pattern.c
@@ -1265,12 +1265,18 @@ patcomppiece(int *flagp, int paren)
 	     * the character following is an end-of-segment character.  Thus
 	     * tildes are not special if there is nothing following to
 	     * be excluded.
+	     *
+	     * Don't look for X()-style kshglobs at this point; we've
+	     * checked above for the case with parentheses and we don't
+	     * want to match without parentheses.
 	     */
-	    if (kshchar || (memchr(zpc_special, *patparse, ZPC_COUNT) &&
-			    (*patparse != zpc_special[ZPC_TILDE] ||
-			     patparse[1] == '/' ||
-			     !memchr(zpc_special, patparse[1], ZPC_SEG_COUNT))))
+	    if (kshchar ||
+		(memchr(zpc_special, *patparse, ZPC_NO_KSH_GLOB) &&
+		 (*patparse != zpc_special[ZPC_TILDE] ||
+		  patparse[1] == '/' ||
+		  !memchr(zpc_special, patparse[1], ZPC_SEG_COUNT)))) {
 		break;
+	    }
     	}
 
 	/* Remember the previous character for backtracking */
diff --git a/Src/zsh.h b/Src/zsh.h
index a935d23..c86d2a6 100644
--- a/Src/zsh.h
+++ b/Src/zsh.h
@@ -1417,7 +1417,11 @@ enum zpc_chars {
     ZPC_HAT,			/* ^ for exclusion (extended glob) */
     ZPC_HASH,			/* # for repetition (extended glob) */
     ZPC_BNULLKEEP,		/* Special backslashed null not removed */
-    ZPC_KSH_QUEST,              /* ? for ?(...) in KSH_GLOB */
+    /*
+     * These characters are only valid before a parenthesis
+     */
+    ZPC_NO_KSH_GLOB,
+    ZPC_KSH_QUEST = ZPC_NO_KSH_GLOB, /* ? for ?(...) in KSH_GLOB */
     ZPC_KSH_STAR,               /* * for *(...) in KSH_GLOB */
     ZPC_KSH_PLUS,               /* + for +(...) in KSH_GLOB */
     ZPC_KSH_BANG,               /* ! for !(...) in KSH_GLOB */
diff --git a/Test/D02glob.ztst b/Test/D02glob.ztst
index 81b0021..1f8f652 100644
--- a/Test/D02glob.ztst
+++ b/Test/D02glob.ztst
@@ -499,3 +499,30 @@
   )
 0:No error with empty null glob with (N).
 >
+
+  (setopt kshglob
+   test_array=(
+     '+fours'    '+*'
+     '@titude'   '@*'
+     '!bang'     '!*'
+     # and check they work in the real kshglob cases too...
+     '+bus+bus'  '+(+bus|-car)'
+     '@sinhats'  '@(@sinhats|wrensinfens)'
+     '!kerror'   '!(!somethingelse)'
+     # and these don't match, to be sure
+     '+more'      '+(+less)'
+     '@all@all'   '@(@all)'
+     '!goesitall' '!(!goesitall)'
+   )
+   for str pat in $test_array; do
+     eval "[[ $str = $pat ]]" && print "$str matches $pat"
+   done
+   true
+  )
+0:kshglob option does not break +, @, ! without following open parenthesis
+>+fours matches +*
+>@titude matches @*
+>!bang matches !*
+>+bus+bus matches +(+bus|-car)
+>@sinhats matches @(@sinhats|wrensinfens)
+>!kerror matches !(!somethingelse)

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: 5.0.3 +* -> git completion regression

[Phil Pennock]

On 2013-12-16 at 22:08 +0000, Peter Stephenson wrote:
> Thanks.  This is an efficient fix, so I hope it's good enough --- I've
> added a regression test, and it seems to be.
> 
> Another good workout with some sh scripts or functions would be useful.

Appears to work for me in practice; while I haven't tested extensively
(I don't have a test suite of all the things I do), one thing did crop
up.

I was confused over zsh versions after installing with this patch, but
config.log shows:

configure:14406: WARNING: unrecognized options: --enable-local-patchlevel

It looks like it's spelt --enable-custom-patchlevel so the INSTALL file
is wrong.

>From 6335389a59f5ec79154923fe8105aad4d846df25 Mon Sep 17 00:00:00 2001
From: Phil Pennock <pdpennock@users.sourceforge.net>
Date: Tue, 17 Dec 2013 12:43:17 -0500
Subject: [PATCH] Fix --enable-custom-patchlevel name in INSTALL

---
 INSTALL | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/INSTALL b/INSTALL
index 00791cd..99895bd 100644
--- a/INSTALL
+++ b/INSTALL
@@ -297,7 +297,7 @@ Modified versions of zsh
 If you are making local modifications to zsh, you are strongly
 advised to configure with the option
 
-  --enable-local-patchlevel="<my-mod-string>"
+  --enable-custom-patchlevel="<my-mod-string>"
 
 so that the variable $ZSH_PATCHLEVEL indicates this is not a standard
 version of the shell.  The argument is arbitrary, but should indicate
-- 
1.8.5.1