completion after < at the start of the line

completion after < at the start of the line

[Stephane Chazelas]

Hiya,

some weird behavior, I suspect something wrong here.

It all started with zsh taking 100% for some time (time being
big) when doing

$ < ~/loc<TAB> some command

That is invoking completion on a file after an input redirection
added as an afterthought at the beggining of a command line.

Now, some things easier to reproduce:

$ mkdir 1
$ cd 1
$ touch foo bar
$ zsh -f
sc% < b<TAB> foo

(That is type "< b foo" move the cursor back to 4rd character,
then type <Tab>)

Then zsh completes that to:

sc% < foo foo

(instead of expected < bar foo).

(don't know if that's related to the 100% CPU case).

With either of 4.3.2-dev-1+20070322-1 or 4.3.2-25 debian
packages.

-- 
Stéphane

Re: completion after < at the start of the line

[Peter Stephenson]

Stephane Chazelas <Stephane_Chazelas@yahoo.fr> wrote:
> $ mkdir 1
> $ cd 1
> $ touch foo bar
> $ zsh -f
> sc% < b<TAB> foo
> 
> (That is type "< b foo" move the cursor back to 4rd character,
> then type <Tab>)
> 
> Then zsh completes that to:
> 
> sc% < foo foo
> 
> (instead of expected < bar foo).

Good news:  I know why this happens.  Bad news: it's in get_comp_string(),
one of the world's most impenetrable functions (but you should see
set_comp_sep()).

We reset the index into the completion words to zero when the command word
arrives.  Obviously this is wrong in this case since the argument to the
redirection is already there and this overwrites it.  However, not setting it to zero at that point is too simple a fix since it messes up handling of multiple commands on a line separated by ";" etc. etc.

Unfortunately the handling of the index into the array of words is obscured
by one of those typical variables (here called "ins") which gets set to
apparently random values in random places in the function, is used in tests
all over the place so you're never quite sure what's going on, and is
completely undocumented.  If we could work out that was doing (and,
ideally, document it, although I'm not sure the words don't magically drift
off the editor into space if you try) we might be able to work out how to
reset the start of line index.

An alternative strategy is to record redirections before the command word,
but I'd rather understand what's there at present than bolt something new
on regardless.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


To access the latest news from CSR copy this link into a web browser:  http://www.csr.com/email_sig.php

To get further information regarding CSR, please visit our Investor Relations page at http://ir.csr.com/csr/about/overview

Re: completion after < at the start of the line

[Peter Stephenson]

Peter Stephenson <pws@csr.com> wrote:
> Stephane Chazelas <Stephane_Chazelas@yahoo.fr> wrote:
>> $ mkdir 1
>> $ cd 1
>> $ touch foo bar
>> $ zsh -f
>> sc% < b<TAB> foo
>> 
>> (That is type "< b foo" move the cursor back to 4rd character,
>> then type <Tab>)
>> 
>> Then zsh completes that to:
>> 
>> sc% < foo foo
>> 
>> (instead of expected < bar foo).
>
> An alternative strategy is to record redirections before the command word,
> but I'd rather understand what's there at present than bolt something new
> on regardless.

I've gone for this strategy.  I thought about it in the bath:  the
number of cases where you might need to reset the index because you've
got a new command word is quite large and it would be nastier to try
to detect them all than to work around the problem.  The fix using a
special index for initial redirections isn't particularly horrible by
the standards of this code.

I've been seeing something screwy to do with the complist module when
trying this.  It doesn't seem to be related; it happens with or without
an initial redirection, but I happened to fall over it when testing
this.  I've been getting it with "ls --<TAB>" with menu selection turned
on.

Index: Src/Zle/zle_tricky.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_tricky.c,v
retrieving revision 1.84
diff -u -r1.84 zle_tricky.c
--- Src/Zle/zle_tricky.c	13 Feb 2007 21:54:41 -0000	1.84
+++ Src/Zle/zle_tricky.c	28 Mar 2007 21:17:34 -0000
@@ -1072,6 +1072,12 @@
      * still use zlemetacs.
      */
     int qsub, zlemetacs_qsub = 0;
+    /*
+     * redirpos is used to record string arguments for redirection
+     * when they occur at the start of the line.  In this case
+     * the command word is not at index zero in the array.
+     */
+    int redirpos;
     char *s = NULL, *tmp, *p, *tt = NULL, rdop[20];
     char *linptr, *u;
 
@@ -1130,7 +1136,7 @@
     lexsave();
     inpush(dupstrspace(linptr), 0, NULL);
     strinbeg(0);
-    i = tt0 = cp = rd = ins = oins = linarr = parct = ia = 0;
+    i = tt0 = cp = rd = ins = oins = linarr = parct = ia = redirpos = 0;
 
     /* This loop is possibly the wrong way to do this.  It goes through *
      * the previously massaged command line using the lexer.  It stores *
@@ -1186,6 +1192,9 @@
             else
                 strcpy(rdop, tokstrings[tok]);
             strcpy(rdstr, rdop);
+	    /* Record if we haven't had the command word yet */
+	    if (i == redirpos)
+		redirpos++;
         }
 	if (tok == DINPAR)
 	    tokstr = NULL;
@@ -1204,7 +1213,7 @@
 	    if (tt)
 		break;
 	    /* Otherwise reset the variables we are collecting data in. */
-	    i = tt0 = cp = rd = ins = 0;
+	    i = tt0 = cp = rd = ins = redirpos = 0;
 	}
 	if (lincmd && (tok == STRING || tok == FOR || tok == FOREACH ||
 		       tok == SELECT || tok == REPEAT || tok == CASE)) {
@@ -1213,7 +1222,9 @@
 	    ins = (tok == REPEAT ? 2 : (tok != STRING));
 	    zsfree(cmdstr);
 	    cmdstr = ztrdup(tokstr);
-	    i = 0;
+	    /* If everything before is a redirection, don't reset the index */
+	    if (i != redirpos)
+		i = redirpos = 0;
 	}
 	if (!zleparse && !tt0) {
 	    /* This is done when the lexer reached the word the cursor is on. */
@@ -1399,9 +1410,9 @@
      * foo[_ wrong (note no $).  If we are in a subscript, treat it   *
      * as being in math.                                              */
     if (inwhat != IN_MATH) {
-	int i = 0;
 	char *nnb, *nb = NULL, *ne = NULL;
 
+	i = 0;
 	MB_METACHARINIT();
 	if (itype_end(s, IIDENT, 1) == s)
 	    nnb = s + MB_METACHARLEN(s);


-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: completion after < at the start of the line

[Peter Stephenson]

On Wed, 28 Mar 2007 22:25:25 +0100
Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> I've been seeing something screwy to do with the complist module when
> trying this.  It doesn't seem to be related; it happens with or without
> an initial redirection, but I happened to fall over it when testing
> this.  I've been getting it with "ls --<TAB>" with menu selection turned
> on.

I think I've tracked this down after a couple of hours of frustration:
if the last line in a completion list was just short of the width,
complist() turned off printing early, but didn't increment over the
character it wasn't printing, which could be just enough to force the
string to need an extra line.  This was quite a special case.

I added some extra debugging aids to help track this down; in
particular, if zsh is compiled with debugging, ZSH_DEBUG_LOG can be used
to give a file to which dputs output is appended.  I've left this in.

Index: INSTALL
===================================================================
RCS file: /cvsroot/zsh/zsh/INSTALL,v
retrieving revision 1.28
diff -u -r1.28 INSTALL
--- INSTALL	1 Nov 2006 12:25:21 -0000	1.28
+++ INSTALL	29 Mar 2007 21:18:14 -0000
@@ -326,6 +326,9 @@
 To add some sanity checks and generate debugging information for debuggers
 you can use the following option.  This also disables optimization.
  --enable-zsh-debug            # use it if you want to debug zsh
+In this mode, zsh may output extra information about internal errors
+to stderr.  The shell variable ZSH_DEBUG_LOG may be set to another file
+to which errors will be appended.
 
 Startup/shutdown files
 ----------------------
Index: Src/utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
retrieving revision 1.159
diff -u -r1.159 utils.c
--- Src/utils.c	26 Feb 2007 18:47:25 -0000	1.159
+++ Src/utils.c	29 Mar 2007 21:18:16 -0000
@@ -129,7 +129,7 @@
 	fputc((unsigned char)':', stderr);
     }
 
-    zerrmsg(fmt, ap);
+    zerrmsg(stderr, fmt, ap);
 }
 
 
@@ -218,14 +218,20 @@
 dputs(VA_ALIST1(const char *message))
 VA_DCL
 {
+    char *filename;
+    FILE *file;
     va_list ap;
     VA_DEF_ARG(const char *message);
 
     VA_START(ap, message);
     VA_GET_ARG(ap, message, const char *);
-    zerrmsg(message, ap);
+    if ((filename = getsparam("ZSH_DEBUG_LOG")) != NULL &&
+	(file = fopen(filename, "a")) != NULL) {
+	zerrmsg(file, message, ap);
+	fclose(file);
+    } else
+	zerrmsg(stderr, message, ap);
     va_end(ap);
-    fflush(stderr);
 }
 
 #endif /* DEBUG */
@@ -245,15 +251,15 @@
 
 /**/
 void
-zerrmsg(const char *fmt, va_list ap)
+zerrmsg(FILE *file, const char *fmt, va_list ap)
 {
     const char *str;
     int num;
 
     if ((unset(SHINSTDIN) || locallevel) && lineno)
-	fprintf(stderr, "%ld: ", (long)lineno);
+	fprintf(file, "%ld: ", (long)lineno);
     else
-	fputc((unsigned char)' ', stderr);
+	fputc((unsigned char)' ', file);
 
     while (*fmt)
 	if (*fmt == '%') {
@@ -261,7 +267,7 @@
 	    switch (*fmt++) {
 	    case 's':
 		str = va_arg(ap, const char *);
-		nicezputs(str, stderr);
+		nicezputs(str, file);
 		break;
 	    case 'l': {
 		char *s;
@@ -271,50 +277,50 @@
 		s = zhalloc(num + 1);
 		memcpy(s, str, num);
 		s[num] = '\0';
-		nicezputs(s, stderr);
+		nicezputs(s, file);
 		break;
 	    }
 	    case 'd':
 		num = va_arg(ap, int);
-		fprintf(stderr, "%d", num);
+		fprintf(file, "%d", num);
 		break;
 	    case '%':
-		putc('%', stderr);
+		putc('%', file);
 		break;
 	    case 'c':
 		num = va_arg(ap, int);
 #ifdef MULTIBYTE_SUPPORT
 		mb_metacharinit();
-		zputs(wcs_nicechar(num, NULL, NULL), stderr);
+		zputs(wcs_nicechar(num, NULL, NULL), file);
 #else
-		zputs(nicechar(num), stderr);
+		zputs(nicechar(num), file);
 #endif
 		break;
 	    case 'e':
 		/* print the corresponding message for this errno */
 		num = va_arg(ap, int);
 		if (num == EINTR) {
-		    fputs("interrupt\n", stderr);
+		    fputs("interrupt\n", file);
 		    errflag = 1;
 		    return;
 		}
 		/* If the message is not about I/O problems, it looks better *
 		 * if we uncapitalize the first letter of the message        */
 		if (num == EIO)
-		    fputs(strerror(num), stderr);
+		    fputs(strerror(num), file);
 		else {
 		    char *errmsg = strerror(num);
-		    fputc(tulower(errmsg[0]), stderr);
-		    fputs(errmsg + 1, stderr);
+		    fputc(tulower(errmsg[0]), file);
+		    fputs(errmsg + 1, file);
 		}
 		break;
 	    }
 	} else {
-	    putc(*fmt == Meta ? *++fmt ^ 32 : *fmt, stderr);
+	    putc(*fmt == Meta ? *++fmt ^ 32 : *fmt, file);
 	    fmt++;
 	}
-    putc('\n', stderr);
-    fflush(stderr);
+    putc('\n', file);
+    fflush(file);
 }
 
 /* Output a single character, for the termcap routines.     *
Index: Src/zsh.h
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/zsh.h,v
retrieving revision 1.111
diff -u -r1.111 zsh.h
--- Src/zsh.h	19 Mar 2007 12:21:55 -0000	1.111
+++ Src/zsh.h	29 Mar 2007 21:18:17 -0000
@@ -1925,10 +1925,12 @@
 # define DPUTS(X,Y) if (!(X)) {;} else dputs(ERRMSG(Y))
 # define DPUTS1(X,Y,Z1) if (!(X)) {;} else dputs(ERRMSG(Y), Z1)
 # define DPUTS2(X,Y,Z1,Z2) if (!(X)) {;} else dputs(ERRMSG(Y), Z1, Z2)
+# define DPUTS3(X,Y,Z1,Z2,Z3) if (!(X)) {;} else dputs(ERRMSG(Y), Z1, Z2, Z3)
 #else
 # define DPUTS(X,Y)
 # define DPUTS1(X,Y,Z1)
 # define DPUTS2(X,Y,Z1,Z2)
+# define DPUTS3(X,Y,Z1,Z2,Z3)
 #endif
 
 /**************************/
Index: Src/Zle/complist.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/complist.c,v
retrieving revision 1.99
diff -u -r1.99 complist.c
--- Src/Zle/complist.c	6 Feb 2007 10:12:20 -0000	1.99
+++ Src/Zle/complist.c	29 Mar 2007 21:18:18 -0000
@@ -1139,6 +1139,10 @@
 	    if (dopr == 1) {
 		if (ml == mlend - 1 && (cc % columns) == columns - 1) {
 		    dopr = 0;
+		    if (*p == Meta)
+			p += 2;
+		    else
+			p++;
 		    continue;
 		}
 		while (len--) {
@@ -1580,6 +1584,8 @@
     Cmatch m;
     int len, subcols = 0, stop = 0, ret = 0;
 
+    DPUTS2(ml >= mlines, "clprintm called with ml too large (%d/%d)",
+	   ml, mlines);
     if (g != last_group)
         *last_cap = '\0';
 

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/