zsh deep freeze (probably autocompletion-related)

zsh deep freeze (probably autocompletion-related)

[Einar Lielmanis]

Steps to reproduce:

"rm ./\" [enter] [tab]

zsh freezes hard and cannot be terminated; kill -9 ir required.

#zsh tells me that it can be reproduced on a recent
CVS build as well.

Cheers,

Einar

Re: zsh deep freeze (probably autocompletion-related)

[Bart Schaefer]

On Dec 7,  5:03pm, Einar Lielmanis wrote:
}
} Steps to reproduce:
} 
} "rm ./\" [enter] [tab]
} 
} zsh freezes hard and cannot be terminated; kill -9 ir required.

I get a crash rather than a freeze.

Program received signal SIGSEGV, Segmentation fault.
0x080bfe09 in chuck (str=0xb7d37000 "$\003\001 \020")
    at ../../zsh-4.0/Src/utils.c:1795
1795        while ((str[0] = str[1]))
(gdb) where
#0  0x080bfe09 in chuck (str=0xb7d37000 "$\003\001 \020")
    at ../../zsh-4.0/Src/utils.c:1795
#1  0x08131568 in get_comp_string ()
    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:1259
#2  0x0812ff58 in docomplete (lst=0)
    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:659
#3  0x0812f395 in completeword (args=0x816d01c)
    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:232
#4  0x0812f2b9 in completecall (args=0x816d01c)
    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:208
#5  0x08121bbc in execzlefunc (func=0x815b3f0, args=0x816d01c, set_bindk=0)
    at ../../../zsh-4.0/Src/Zle/zle_main.c:1312
#6  0x08121263 in zlecore () at ../../../zsh-4.0/Src/Zle/zle_main.c:1050
#7  0x08121817 in zleread (lp=0x816ddc8, rp=0x0, flags=3, context=1)
    at ../../../zsh-4.0/Src/Zle/zle_main.c:1213
#8  0x081235a8 in zle_main_entry (cmd=1, ap=0xbfeee6e4 "\221áC")
    at ../../../zsh-4.0/Src/Zle/zle_main.c:1866
#9  0x0807df9c in zleentry (cmd=1) at ../../zsh-4.0/Src/init.c:1305
#10 0x0807e7c7 in inputline () at ../../zsh-4.0/Src/input.c:278

(gdb) up
#1  0x08131568 in get_comp_string ()
    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:1259
1259                    chuck(tt + zlemetacs - wb - qsub);
(gdb) p tt
$3 = 0xb7d33898 "./x"
(gdb) p zlemetacs
$4 = 7
(gdb) p wb
$5 = 3
(gdb) p qsub
$6 = 0

Looks like zlemetacs is too large.

Re: zsh deep freeze (probably autocompletion-related)

[Mikael Magnusson]

2009/12/7 Bart Schaefer <schaefer@brasslantern.com>:
> On Dec 7,  5:03pm, Einar Lielmanis wrote:
> }
> } Steps to reproduce:
> }
> } "rm ./\" [enter] [tab]
> }
> } zsh freezes hard and cannot be terminated; kill -9 ir required.
>
> I get a crash rather than a freeze.
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x080bfe09 in chuck (str=0xb7d37000 "$\003\001 \020")
>    at ../../zsh-4.0/Src/utils.c:1795
> 1795        while ((str[0] = str[1]))
> (gdb) where
> #0  0x080bfe09 in chuck (str=0xb7d37000 "$\003\001 \020")
>    at ../../zsh-4.0/Src/utils.c:1795
> #1  0x08131568 in get_comp_string ()
>    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:1259
> #2  0x0812ff58 in docomplete (lst=0)
>    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:659
> #3  0x0812f395 in completeword (args=0x816d01c)
>    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:232
> #4  0x0812f2b9 in completecall (args=0x816d01c)
>    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:208
> #5  0x08121bbc in execzlefunc (func=0x815b3f0, args=0x816d01c, set_bindk=0)
>    at ../../../zsh-4.0/Src/Zle/zle_main.c:1312
> #6  0x08121263 in zlecore () at ../../../zsh-4.0/Src/Zle/zle_main.c:1050
> #7  0x08121817 in zleread (lp=0x816ddc8, rp=0x0, flags=3, context=1)
>    at ../../../zsh-4.0/Src/Zle/zle_main.c:1213
> #8  0x081235a8 in zle_main_entry (cmd=1, ap=0xbfeee6e4 "\221áC")
>    at ../../../zsh-4.0/Src/Zle/zle_main.c:1866
> #9  0x0807df9c in zleentry (cmd=1) at ../../zsh-4.0/Src/init.c:1305
> #10 0x0807e7c7 in inputline () at ../../zsh-4.0/Src/input.c:278
>
> (gdb) up
> #1  0x08131568 in get_comp_string ()
>    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:1259
> 1259                    chuck(tt + zlemetacs - wb - qsub);
> (gdb) p tt
> $3 = 0xb7d33898 "./x"
> (gdb) p zlemetacs
> $4 = 7
> (gdb) p wb
> $5 = 3
> (gdb) p qsub
> $6 = 0
>
> Looks like zlemetacs is too large.

I get a loop, so i pressed ctrl-c

Program received signal SIGINT, Interrupt.
mb_metacharinit () at utils.c:449
(gdb) bt
#0  mb_metacharinit () at utils.c:449
#1  0x080cee63 in itype_end (ptr=0x819bde2 "", itype=128, once=1) at
utils.c:3312
#2  0x6f92bf00 in get_comp_string () at zle_tricky.c:1450
#3  0x6f929784 in docomplete (lst=4) at zle_tricky.c:659
#4  0x6f928c27 in expandorcomplete (args=0x6f93ed60) at zle_tricky.c:315
#5  0x6f9287ca in completecall (args=0x6f93ed60) at zle_tricky.c:208
#6  0x6f9181a8 in execzlefunc (func=0x6f93cd40, args=0x6f93ed60, set_bindk=0)
    at zle_main.c:1312
#7  0x6f917613 in zlecore () at zle_main.c:1050
#8  0x6f917d45 in zleread (lp=0x80efb58, rp=0x0, flags=3, context=1)
at zle_main.c:1213
#9  0x6f919e93 in zle_main_entry (cmd=1, ap=0x77ffe3f4 "\016") at
zle_main.c:1866
#10 0x08086fc6 in zleentry (cmd=1) at init.c:1304
#11 0x08087889 in inputline () at input.c:278
#12 0x080876f8 in ingetc () at input.c:214
#13 0x0807d881 in ihgetc () at hist.c:263
#14 0x08090743 in gettokstr (c=10, sub=0) at lex.c:1245
#15 0x0808fefa in gettok () at lex.c:929
#16 0x0808edf4 in zshlex () at lex.c:364
#17 0x080ad75a in par_simple (complex=0x77ffe720, nr=0) at parse.c:1661
#18 0x080ab887 in par_cmd (complex=0x77ffe720) at parse.c:879
#19 0x080ab354 in par_pline (complex=0x77ffe720) at parse.c:728
#20 0x080ab304 in par_sublist2 (complex=0x77ffe720) at parse.c:709
#21 0x080ab16e in par_sublist (complex=0x77ffe758) at parse.c:664
#22 0x080aabc7 in par_event () at parse.c:477
#23 0x080aab3e in parse_event () at parse.c:454
#24 0x080844f5 in loop (toplevel=1, justonce=0) at init.c:132
#25 0x0808733f in zsh_main (argc=2, argv=0x77ffe8b4) at init.c:1454
#26 0x080553d6 in main (argc=Cannot access memory at address 0x0


(gdb) bt full 3
#0  mb_metacharinit () at utils.c:449
No locals.
#1  0x080cee63 in itype_end (ptr=0x819bde2 "", itype=128, once=1) at
utils.c:3312
No locals.
#2  0x6f92bf00 in get_comp_string () at zle_tricky.c:1450
        nclen = 0
        nnb = 0x819bde2 ""
        nb = 0x0
        ne = 0x0
        t0 = 34
        tt0 = 34
        i = 0
        j = 0
        k = 0
        cp = 0
        rd = 0
        sl = 3
        ocs = 2013257460
        ins = 0
        oins = 0
        ia = 0
        parct = 0
        varq = 0
        ona = 0
        qsub = 0
        zlemetacs_qsub = 9
        redirpos = 0
        s = 0x819bde0 "./"
        tmp = 0x817c648 "echo ./\\\n"
        p = 0x0
        tt = 0x819bde2 ""
---Type <return> to continue, or q <return> to quit---
        rdop = "\000\066\220o\350\336\377w<\032\000p\005\000\000\000\200\065\020\b"
        linptr = 0x6ffd6860 "echo ./\\\nx"
        u = 0x817c651 ""


and stepping forward a bit, all the counters used for += things are 0,
since tt is just an empty string.

3313		while (*ptr) {
(gdb)
3378	    return (char *)ptr;
(gdb)
3379	}
(gdb)
get_comp_string () at zle_tricky.c:1451
1451			    nnb = tt + nclen;
(gdb)
1452			tt += nclen;
(gdb) print nclen
$2 = 0

-- 
Mikael Magnusson

Re: zsh deep freeze (probably autocompletion-related)

[Peter Stephenson]

Bart Schaefer wrote:
> Looks like zlemetacs is too large.

The backslash-newline is being swallowed in the input / lex code,
causing ZLE to miscount the size of the string it's got back by 2.
Ideally it should be included in qsub in the same way it handles
RC_QUOTES.  I'm not sure how easy this is.  Anyone should feel free to
look at the underlying problem before I get round to it, otherwise
there's no need to waste time / energy / me-too messages on this.

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom

Re: zsh deep freeze (probably autocompletion-related)

[Peter Stephenson]

On Mon, 07 Dec 2009 17:13:51 +0000
Peter Stephenson <pws@csr.com> wrote:
> The backslash-newline is being swallowed in the input / lex code,
> causing ZLE to miscount the size of the string it's got back by 2.
> Ideally it should be included in qsub in the same way it handles
> RC_QUOTES.

I think the following is OK as far as it goes: it does fix up the string
that's being completed, since I can see that both uses of chuck(), on
the string from the lexer and the string going into the words array, are
coming out right.  Also, I haven't messed up completion later on in the
second line, i.e. in words after the one with the backslash-newline in
it.

However, it doesn't actually make completion work at that point.  That
doesn't surprise me much or worry me.  If anyone really has enough time
on their hands to get that to work they're welcome to look, but I think
the way the command line is split in two makes it virtually impossible
to do this without rewriting the line editor, nor do I consider it
within the spirit of completion to require it to work there.
Familiarise yourself with push-line-or-edit.

Most of the bulk of the patch is because I finally got fed up with "i"
being used sometimes as an internal loop variable and sometimes as the
index of the word being completed.

Index: Src/Zle/zle_tricky.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_tricky.c,v
retrieving revision 1.100
diff -u -r1.100 zle_tricky.c
--- Src/Zle/zle_tricky.c	15 Aug 2009 16:59:22 -0000	1.100
+++ Src/Zle/zle_tricky.c	7 Dec 2009 21:25:10 -0000
@@ -1067,6 +1067,10 @@
     int t0, tt0, i, j, k, cp, rd, sl, ocs, ins, oins, ia, parct, varq = 0;
     int ona = noaliases;
     /*
+     * Index of word being considered
+     */
+    int wordpos;
+    /*
      * qsub fixes up the offset into the current completion word
      * for changes made by the lexer.  That currently means the
      * effect of RCQUOTES on embedded pairs of single quotes.
@@ -1141,7 +1145,7 @@
     lexsave();
     inpush(dupstrspace(linptr), 0, NULL);
     strinbeg(0);
-    i = tt0 = cp = rd = ins = oins = linarr = parct = ia = redirpos = 0;
+    wordpos = tt0 = cp = rd = ins = oins = linarr = parct = ia = redirpos = 0;
 
     /* This loop is possibly the wrong way to do this.  It goes through *
      * the previously massaged command line using the lexer.  It stores *
@@ -1154,8 +1158,9 @@
     do {
         qsub = 0;
 
-	lincmd = ((incmdpos && !ins && !incond) || (oins == 2 && i == 2) ||
-		  (ins == 3 && i == 1));
+	lincmd = ((incmdpos && !ins && !incond) ||
+		  (oins == 2 && wordpos == 2) ||
+		  (ins == 3 && wordpos == 1));
 	linredir = (inredir && !ins);
 	oins = ins;
 	/* Get the next token. */
@@ -1198,7 +1203,7 @@
                 strcpy(rdop, tokstrings[tok]);
             strcpy(rdstr, rdop);
 	    /* Record if we haven't had the command word yet */
-	    if (i == redirpos)
+	    if (wordpos == redirpos)
 		redirpos++;
         }
 	if (tok == DINPAR)
@@ -1208,7 +1213,7 @@
 	if (tok == ENDINPUT)
 	    break;
 	if ((ins && (tok == DOLOOP || tok == SEPER)) ||
-	    (ins == 2 && i == 2) || (ins == 3 && i == 3) ||
+	    (ins == 2 && wordpos == 2) || (ins == 3 && wordpos == 3) ||
 	    tok == BAR    || tok == AMPER     ||
 	    tok == BARAMP || tok == AMPERBANG ||
 	    ((tok == DBAR || tok == DAMPER) && !incond) ||
@@ -1226,7 +1231,7 @@
 	    if (tt)
 		break;
 	    /* Otherwise reset the variables we are collecting data in. */
-	    i = tt0 = cp = rd = ins = redirpos = 0;
+	    wordpos = tt0 = cp = rd = ins = redirpos = 0;
 	}
 	if (lincmd && (tok == STRING || tok == FOR || tok == FOREACH ||
 		       tok == SELECT || tok == REPEAT || tok == CASE)) {
@@ -1236,15 +1241,46 @@
 	    zsfree(cmdstr);
 	    cmdstr = ztrdup(tokstr);
 	    /* If everything before is a redirection, don't reset the index */
-	    if (i != redirpos)
-		i = redirpos = 0;
+	    if (wordpos != redirpos)
+		wordpos = redirpos = 0;
 	}
 	if (!zleparse && !tt0) {
 	    /* This is done when the lexer reached the word the cursor is on. */
 	    tt = tokstr ? dupstring(tokstr) : NULL;
 
+	    /*
+	     * If there was a proper interface between this
+	     * function and the lexical analyser, we wouldn't
+	     * have to fix things up.
+	     *
+	     * Fix up backslash-newline pairs in zlemetaline
+	     * that the lexer will have removed.  As we're
+	     * looking back at the zlemetaline version it's
+	     * still using untokenized quotes.
+	     */
+	    for (i = j = k = 0, u = zlemetaline + wb;
+		 u < zlemetaline + we; u++) {
+		if (*u == '`' && !(k & 1))
+		    i++;
+		else if (*u == '\"' && !(k & 1) && !(i & 1))
+		    j++;
+		else if (*u == '\'' && !(j & 1))
+		    k++;
+		else if (*u == '\\' && u[1] && !(k & 1))
+		{
+		    if (u[1] == '\n') {
+			/* Removed by lexer in tt */
+			qsub += 2;
+		    }
+		    u++;
+		}
+	    }
+	    /*
+	     * Fix up RCQUOTES quotes that the
+	     * the lexer will also have removed.
+	     */
             if (isset(RCQUOTES) && tt) {
-		char *tt1, *e = tt + zlemetacs - wb;
+		char *tt1, *e = tt + zlemetacs - wb - qsub;
 		for (tt1 = tt; *tt1; tt1++) {
 		    if (*tt1 == Snull) {
 			char *p;
@@ -1259,7 +1295,7 @@
 		chuck(tt + zlemetacs - wb - qsub);
 	    tt0 = tok;
 	    /* Store the number of this word. */
-	    clwpos = i;
+	    clwpos = wordpos;
 	    cp = lincmd;
 	    rd = linredir;
 	    ia = linarr;
@@ -1279,39 +1315,39 @@
 	if (!tokstr)
 	    continue;
 	/* Hack to allow completion after `repeat n do'. */
-	if (oins == 2 && !i && !strcmp(tokstr, "do"))
+	if (oins == 2 && !wordpos && !strcmp(tokstr, "do"))
 	    ins = 3;
 	/* We need to store the token strings of all words (for some of *
 	 * the more complicated compctl -x things).  They are stored in *
 	 * the clwords array.  Make this array big enough.              */
-	if (i + 1 == clwsize) {
+	if (wordpos + 1 == clwsize) {
 	    int n;
 	    clwords = (char **)realloc(clwords,
 				       (clwsize *= 2) * sizeof(char *));
-	    for(n = clwsize; --n > i; )
+	    for(n = clwsize; --n > wordpos; )
 		clwords[n] = NULL;
 	}
-	zsfree(clwords[i]);
+	zsfree(clwords[wordpos]);
 	/* And store the current token string. */
-	clwords[i] = ztrdup(tokstr);
+	clwords[wordpos] = ztrdup(tokstr);
 	sl = strlen(tokstr);
 	/* Sometimes the lexer gives us token strings ending with *
 	 * spaces we delete the spaces.                           */
-	while (sl && clwords[i][sl - 1] == ' ' &&
-	       (sl < 2 || (clwords[i][sl - 2] != Bnull &&
-			   clwords[i][sl - 2] != Meta)))
-	    clwords[i][--sl] = '\0';
+	while (sl && clwords[wordpos][sl - 1] == ' ' &&
+	       (sl < 2 || (clwords[wordpos][sl - 2] != Bnull &&
+			   clwords[wordpos][sl - 2] != Meta)))
+	    clwords[wordpos][--sl] = '\0';
 	/* If this is the word the cursor is in and we added a `x', *
 	 * remove it.                                               */
-	if (clwpos == i++ && addedx) {
+	if (clwpos == wordpos++ && addedx) {
 	    zlemetacs_qsub = zlemetacs - qsub;
-	    chuck(&clwords[i - 1][((zlemetacs_qsub - wb) >= sl) ?
+	    chuck(&clwords[wordpos - 1][((zlemetacs_qsub - wb) >= sl) ?
 				 (sl - 1) : (zlemetacs_qsub - wb)]);
 	}
     } while (tok != LEXERR && tok != ENDINPUT &&
 	     (tok != SEPER || (zleparse && !tt0)));
     /* Calculate the number of words stored in the clwords array. */
-    clwnum = (tt || !i) ? i : i - 1;
+    clwnum = (tt || !wordpos) ? wordpos : wordpos - 1;
     zsfree(clwords[clwnum]);
     clwords[clwnum] = NULL;
     t0 = tt0;



-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: zsh deep freeze (probably autocompletion-related)

[Bart Schaefer]

On Dec 7,  9:32pm, Peter Stephenson wrote:
}
} I think the following is OK as far as it goes: it does fix up the string
} that's being completed ...

} However, it doesn't actually make completion work at that point.
} Familiarise yourself with push-line-or-edit.

Well, OK ... with 27479 applied and with \eq bound to push-line-or-edit,
either one of

schaefer<501> rm ./\<RET><ESC>q<TAB>

or

schaefer<501> rm ./\<ESC><RET><TAB>

leads to

schaefer<501> rm ./ÿÿÿÿ./

Needless to say there is no directory named ÿÿÿÿ. in the current
directory.  So something is still not quite right here, but it's
differently not quite right when you have an actual multi-line
buffer at PS1 as opposed to a PS2 prompt.