Weird tab complete problem with long directory names with a dash and unicode characters

Weird tab complete problem with long directory names with a dash and unicode characters

[Mikael Magnusson]

If you do
mkdir -p るるるるるるるるるるるるるるるるるるるるるるるるる-/foo
and then attempt to complete foo from the current location, it doesn't
work with compinit loaded. If you take off one of the るs it's fine, if
you take out the - it's fine (though the position of the - doesn't
seem to matter), if you use a million "a"s instead it's fine, the る
isn't super special though, it's also broken with a bunch of け or any
other combination I tried (I originally found it because of a real
directory name). My guess is this is somehow related to the - / Dash
stuff Peter did recently, but reverting them randomly also didn't seem
to have much effect. I'm also puzzled as to why the length of the
string matters and why 26?

-- 
Mikael Magnusson

Re: Weird tab complete problem with long directory names with a dash and unicode characters

[Daniel Shahaf]

Mikael Magnusson wrote on Sat, Dec 31, 2016 at 21:44:28 +0100:
> mkdir -p るるるるるるるるるるるるるるるるるるるるるるるるる-/foo
> and then attempt to complete foo from the current location, it doesn't
> work with compinit loaded. If you take off one of the るs it's fine, if
> you take out the - it's fine (though the position of the - doesn't
> seem to matter), if you use a million "a"s instead it's fine, the る
> isn't super special though, it's also broken with a bunch of け or any
> other combination I tried (I originally found it because of a real
> directory name). My guess is this is somehow related to the - / Dash
> stuff Peter did recently, but reverting them randomly also didn't seem
> to have much effect. I'm also puzzled as to why the length of the
> string matters and why 26?

Bisected this down to these three:

* commit c2b6bd51f98a5871acd9fcbeddad67216a07907c
| Author: Peter Stephenson <pws@zsh.org>
| Date:   Tue Jan 19 17:38:59 2016 +0000
| 
|     37693: turn Dash into '-' in glob qualifiers
|  
* commit ad16356e1923ec1b4daf97b27b10a835cfe73ba7
| Author: Peter Stephenson <pws@zsh.org>
| Date:   Tue Jan 19 17:24:12 2016 +0000
| 
|     37689: ! and ^ need to be tokenised in character sets
|  
* commit 8eb9070d6785f423dd9bdbbb0513aa47c8a08d62
  Author: Peter Stephenson <pws@zsh.org>
  Date:   Tue Jan 19 09:55:46 2016 +0000
  
      37678: Now possible to quote "-" in pattern range

c2b6bd51 gives the same symptoms as in HEAD.  The two commits before
that give "compaudit:151: unknown file attribute: -".  The parent of
8eb9070d completes '.../foo' correctly.

These revisions are new in 5.3.

Cheers,

Daniel

Re: Weird tab complete problem with long directory names with a dash and unicode characters

[Bart Schaefer]

On Jan 1,  3:47am, Daniel Shahaf wrote:
}
} c2b6bd51 gives the same symptoms as in HEAD.  The two commits before
} that give "compaudit:151: unknown file attribute: -".

You should try "compinit -u" to get past this and see if the completion
itself still works.  It seems unlikely that c2b6bd51 broke this because
there are no glob qualifiers anywhere in the failing completion.

Re: Weird tab complete problem with long directory names with a dash and unicode characters

[Daniel Shahaf]

Bart Schaefer wrote on Mon, Jan 02, 2017 at 13:52:06 -0800:
> On Jan 1,  3:47am, Daniel Shahaf wrote:
> }
> } c2b6bd51 gives the same symptoms as in HEAD.  The two commits before
> } that give "compaudit:151: unknown file attribute: -".
> 
> You should try "compinit -u" to get past this and see if the completion
> itself still works.  It seems unlikely that c2b6bd51 broke this because
> there are no glob qualifiers anywhere in the failing completion.

That narrows it down further:

ad16356e1923ec1b4daf97b27b10a835cfe73ba7 is the first bad commit
commit ad16356e1923ec1b4daf97b27b10a835cfe73ba7
Author: Peter Stephenson <pws@zsh.org>
Date:   Tue Jan 19 17:24:12 2016 +0000

    37689: ! and ^ need to be tokenised in character sets

Re: Weird tab complete problem with long directory names with a dash and unicode characters

[Peter Stephenson]

On Sat, 31 Dec 2016 21:44:28 +0100
Mikael Magnusson <mikachu@gmail.com> wrote:
> If you do
> mkdir -p るるるるるるるるるるるるるるるるるるるるるるるるる-/foo
> and then attempt to complete foo from the current location, it doesn't
> work with compinit loaded.

Got somewhere with this: adding a bogus assignment of tmp1 back to
itself inside _path_files (first of the two lines below), I get

       +_path_files:467> tmp1=( 'xるるるるるるるるるるるるる-/*' ) 
       +_path_files:468> tmp1=( ) 

The string in the first line seems to be correct, unless I'm
not seeing / copying everything that's there.

The next line is an expansion using $~tmp1 that should just glob.  So
it's going wrong there --- command line tests seem to confirm this.

pws

Re: Weird tab complete problem with long directory names with a dash and unicode characters

[Peter Stephenson]

On Tue, 3 Jan 2017 12:57:06 +0000
Peter Stephenson <p.stephenson@samsung.com> wrote:
> The next line is an expansion using $~tmp1 that should just glob.  So
> it's going wrong there --- command line tests seem to confirm this.

Oh, it's that hack for pure strings in a pattern with Meta characters
again (hi from Cambridge).  So it depends on metafication and memory
management and in the affected cases it was only ever working by luck.

Time we got just rid of it completely (bye).

pws

diff --git a/Src/pattern.c b/Src/pattern.c
index 1f2e94b..928790f 100644
--- a/Src/pattern.c
+++ b/Src/pattern.c
@@ -668,15 +668,9 @@ patcompile(char *exp, int inflags, char **endexp)
 			    if (imeta(*mtest))
 				nmeta++;
 			if (nmeta) {
-			    char *oldpatout = patout;
-			    ptrdiff_t pd;
 			    patadd(NULL, 0, nmeta, 0);
-			    /*
-			     * Yuk.
-			     */
 			    p = (Patprog)patout;
-			    pd = patout - oldpatout;
-			    opnd += pd;
+			    opnd = dupstring_wlen(opnd, oplen);
 			    dst = patout + startoff;
 			}