Command line bug since zsh 3.1.7...

Command line bug since zsh 3.1.7...

[Sebastian Rasmussen]

Hi!

Today I discovered a bug in zsh 4.2.1 that's been
present in zsh since 3.1.7!

The problem was discovered when I accidentally
deleted a continued line continaing a pipe in a
makefile that I was building. zsh appeared to
segfault so I tried to make a simple testcase that
reproduced the bug and ended up with:

SHELL = zsh

foo:
        echo bar | \
        # this line is intentionally missing a command

When I run make I get

hal9000% make
echo bar | \
# this line is intentionally missing a command
make: *** [foo] Segmentation fault
hal9000%

which surely can't be right. By persuading make to
print what commands were exec():ed I discovered
that zsh was invoked like this:

zsh -c 'echo bar |'

Testing this gives:

hal9000% zsh -c 'echo bar |'
zsh: segmentation fault  zsh -c 'echo bar |'
hal9000%

Which seems consistent with the results from make.
By compiling and testing several versions I nailed
down the last working version to 3.1.6 which
appears not to segfault:

hal9000% ./zsh-3.1.6 -c 'echo bar |'
hal9000%

The first crashing version is 3.1.7. Running this
through gdb gives:

hal9000% gdb zsh-3.1.7
GNU gdb 6.1-debian
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-linux"...Using host libthread_db library 
"/lib/

(gdb) run -c 'echo bar |'
Starting program: /var/tmp/sebras/zsh/bin/zsh-3.1.7 -c 'echo bar |'
Detaching after fork from child process 4193.

Program received signal SIGSEGV, Segmentation fault.
0x40116a5b in strlen () from /lib/tls/libc.so.6
(gdb) bt
#0  0x40116a5b in strlen () from /lib/tls/libc.so.6
#1  0x080a0622 in taddstr (s=0x5bfe8419 <Address 0x5bfe8419 out of bounds>)
    at text.c:60
#2  0x080a0746 in taddlist (state=0xbffff9f0, num=29204875) at text.c:86
#3  0x080a0d4b in gettext2 (state=0xbffff9f0) at text.c:326
#4  0x080a08cd in getjobtext (prog=0x402265c0, c=0x40226600) at text.c:160
#5  0x0805f2d1 in execpline2 (state=0xbffffbb0, pcode=0, how=18, input=12,
    output=0, last1=1) at exec.c:1175
#6  0x0805f568 in execpline2 (state=0xbffffbb0, pcode=35, how=18, input=0,
    output=0, last1=1) at exec.c:1233
#7  0x0805e94f in execpline (state=0xbffffbb0, slcode=5122, how=18, last1=1)
    at exec.c:976
#8  0x0805e3ba in execlist (state=0xbffffbb0, dont_change_job=0, exiting=1)
    at exec.c:821
#9  0x0805e0c3 in execode (p=0x402265c0, dont_change_job=0, exiting=1)
    at exec.c:727
#10 0x0805e085 in execstring (s=0xbffffde8 "echo bar |", dont_change_job=0,
    exiting=1) at exec.c:713
#11 0x08073ea2 in init_misc () at init.c:888
#12 0x080514a9 in main (argc=3, argv=0xbffffcc4) at main.c:82
(gdb)

Or, the equivalent backtrace in zsh 4.2.1:

hal9000% gdb zsh-4.2.1
GNU gdb 6.1-debian
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-linux"...Using host libthread_db library 
"/lib/

(gdb) run -c 'echo bar |'
Starting program: /var/tmp/sebras/zsh/bin/zsh-4.2.1 -c 'echo bar |'
Detaching after fork from child process 16004.

Program received signal SIGSEGV, Segmentation fault.
0x40116a5b in strlen () from /lib/tls/libc.so.6
(gdb) bt
#0  0x40116a5b in strlen () from /lib/tls/libc.so.6
#1  0x080b468e in taddstr (s=0x502af91e <Address 0x502af91e out of bounds>)
    at text.c:60
#2  0x080b47b2 in taddlist (state=0xbffff9d0, num=14602435) at text.c:86
#3  0x080b4e52 in gettext2 (state=0xbffff9d0) at text.c:335
#4  0x080b495a in getjobtext (prog=0x40226098, c=0x402260e0) at text.c:164
#5  0x0806624a in execpline2 (state=0xbffffba0, pcode=0, how=18, input=12,
    output=0, last1=1) at exec.c:1268
#6  0x08066509 in execpline2 (state=0xbffffba0, pcode=163, how=18, input=0,
    output=0, last1=1) at exec.c:1326
#7  0x080657d3 in execpline (state=0xbffffba0, slcode=5122, how=18, last1=1)
    at exec.c:1065
#8  0x08065120 in execlist (state=0xbffffba0, dont_change_job=0, exiting=1)
    at exec.c:871
#9  0x08064e2c in execode (p=0x40226098, dont_change_job=0, exiting=1)
    at exec.c:771
#10 0x08064de3 in execstring (s=0xbffffdf8 "echo bar |", dont_change_job=0,
    exiting=1) at exec.c:756
#11 0x0807d506 in init_misc () at init.c:984
#12 0x0807e2bd in zsh_main (argc=3, argv=0xbffffcd4) at init.c:1263
#13 0x080523f6 in main (argc=3, argv=0xbffffcd4) at main.c:93
(gdb)

I hope that this is enough information for you to
nail down and fix the bug. If not, don't hesitate
to contact me and I'll try to provide you with the
relevant info.

/ Sebastian Rasmussen

_________________________________________________________________
The new MSN 8: smart spam protection and 2 months FREE*  
http://join.msn.com/?page=features/junkmail

Re: Command line bug since zsh 3.1.7...

[Bart Schaefer]

On Wed, 29 Sep 2004, Sebastian Rasmussen wrote:

> hal9000% zsh -c 'echo bar |'
> zsh: segmentation fault  zsh -c 'echo bar |'

Seems to be this easy (and it should have been obvious that it's wrong
to ignore the result of the recursive parse):


Index: parse.c
===================================================================
RCS file: /extra/cvsroot/zsh/zsh-4.0/Src/parse.c,v
retrieving revision 1.20
diff -c -r1.20 parse.c
--- parse.c	14 Aug 2004 05:01:23 -0000	1.20
+++ parse.c	30 Sep 2004 03:50:06 -0000
@@ -729,9 +729,9 @@
 	ecbuf[p] = WCB_PIPE(WC_PIPE_MID, (line >= 0 ? line + 1 : 0));
 	ecispace(p + 1, 1);
 	ecbuf[p + 1] = ecused - 1 - p;
-	par_pline(complex);
+	p = par_pline(complex);
 	cmdpop();
-	return 1;
+	return p;
     } else if (tok == BARAMP) {
 	int r;
 
@@ -750,9 +750,9 @@
 	ecbuf[p] = WCB_PIPE(WC_PIPE_MID, (line >= 0 ? line + 1 : 0));
 	ecispace(p + 1, 1);
 	ecbuf[p + 1] = ecused - 1 - p;
-	par_pline(complex);
+	p = par_pline(complex);
 	cmdpop();
-	return 1;
+	return p;
     } else {
 	ecbuf[p] = WCB_PIPE(WC_PIPE_END, (line >= 0 ? line + 1 : 0));
 	return 1;

Re: Command line bug since zsh 3.1.7...

[Bart Schaefer]

On Wed, 29 Sep 2004, Bart Schaefer wrote:

> On Wed, 29 Sep 2004, Sebastian Rasmussen wrote:
> 
> > hal9000% zsh -c 'echo bar |'
> > zsh: segmentation fault  zsh -c 'echo bar |'
> 
> Seems to be this easy

The following might be a more palatable fix, as it produces a parse error
rather than silently treating the '|' as a ';'.  This is still not the
same as bash (which says "unexpected end of file") but arguably "syntax
error near `|'" is a more helpful error anyway.

There are still other cases where zsh doesn't crash, but doesn't produce
an error either.  E.g. (even without the following patch):

schaefer[643] bash -c 'echo |{;}'
bash: syntax error near unexpected token `;}'
bash: -c: line 1: `echo |{;}'
schaefer[644] Src/zsh -fc 'echo |{;}'
schaefer[645] 

(Perhaps the patch should set tok = LEXERR only when tok == ENDINPUT ?
I couldn't decide.)

Index: Src/parse.c
===================================================================
RCS file: /extra/cvsroot/zsh/zsh-4.0/Src/parse.c,v
retrieving revision 1.20
diff -c -r1.20 parse.c
--- parse.c	14 Aug 2004 05:01:23 -0000	1.20
+++ parse.c	30 Sep 2004 04:01:06 -0000
@@ -729,7 +729,9 @@
 	ecbuf[p] = WCB_PIPE(WC_PIPE_MID, (line >= 0 ? line + 1 : 0));
 	ecispace(p + 1, 1);
 	ecbuf[p + 1] = ecused - 1 - p;
-	par_pline(complex);
+	if (!par_pline(complex)) {
+	    tok = LEXERR;
+	}
 	cmdpop();
 	return 1;
     } else if (tok == BARAMP) {
@@ -750,7 +752,9 @@
 	ecbuf[p] = WCB_PIPE(WC_PIPE_MID, (line >= 0 ? line + 1 : 0));
 	ecispace(p + 1, 1);
 	ecbuf[p + 1] = ecused - 1 - p;
-	par_pline(complex);
+	if (!par_pline(complex)) {
+	    tok = LEXERR;
+	}
 	cmdpop();
 	return 1;
     } else {

Re: Command line bug since zsh 3.1.7...

[Peter Stephenson]

Bart Schaefer wrote:
> There are still other cases where zsh doesn't crash, but doesn't produce
> an error either.  E.g. (even without the following patch):
> 
> schaefer[643] bash -c 'echo |{;}'
> bash: syntax error near unexpected token `;}'
> bash: -c: line 1: `echo |{;}'
> schaefer[644] Src/zsh -fc 'echo |{;}'
> schaefer[645] 

zsh tends to be a bit more lenient with this sort of thing anyway.
Here, the ";" unambiguously indicates the end of a command line, which
just happens to be empty.  I'm not completely sure it's a bug.  Compare,
for example, the behaviour of "for f in ; do; done".  It's quite useful
that this simply does nothing.  (Whether it's valid according to the
various standards is another matter.)

> (Perhaps the patch should set tok = LEXERR only when tok == ENDINPUT ?
> I couldn't decide.)

It looks safer the way it is.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************