zsh malloc bug

zsh malloc bug

[Dave Yost]

Hi.

Z% echo $ZSH_VERSION 
4.1.1
Z% uname -a 
Darwin ip2 7.4.0 Darwin Kernel Version 7.4.0: Wed May 12 16:58:24 PDT 2004; root:xnu/xnu-517.7.7.obj~7/RELEASE_PPC  Power Macintosh powerpc

I ran the script below and 1 second later hit ^C, eliciting this output:

Z% ./bug
^Cint
*** malloc[21559]: Deallocation of a pointer not malloced: 0x10f3c0; This could be a double free(), or free() called with the middle of an allocated block; Try setting environment variable MallocHelp to see tools to help debug
Z% 

Here's the script

==============
#!/bin/zsh

TRAPEXIT() {
	echo exit
}

TRAPINT() {
	echo int
	exit
}

sleep 5
# Now type control-c

# or wait til here for a bus error
while true
do
	sleep 1
done
==============

I have a slightly more complicated script that elicits two malloc debug printouts.  I'll send it to whoever fixes this.

See also: my email with the subject "zsh needs try-finally".

Dave

Re: zsh malloc bug

[Peter Stephenson]

Dave Yost wrote:
> Z% echo $ZSH_VERSION 
> 4.1.1
> Z% uname -a 
> Darwin ip2 7.4.0 Darwin Kernel Version 7.4.0: Wed May 12 16:58:24 PDT 2004; r
> oot:xnu/xnu-517.7.7.obj~7/RELEASE_PPC  Power Macintosh powerpc

Thanks for the report... could you try 4.2.0?  Traps have changed in
various ways and this doesn't happen after a short test on my latest
code.

pws


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

Re: zsh malloc bug

[Dave Yost]

At 10:32 AM +0100 2004-06-02, Peter Stephenson wrote:
>Dave Yost wrote:
>> Z% echo $ZSH_VERSION
>> 4.1.1
>> Z% uname -a
>> Darwin ip2 7.4.0 Darwin Kernel Version 7.4.0: Wed May 12 16:58:24 PDT 2004; r
>> oot:xnu/xnu-517.7.7.obj~7/RELEASE_PPC  Power Macintosh powerpc
>
>Thanks for the report... could you try 4.2.0?  Traps have changed in
>various ways and this doesn't happen after a short test on my latest
>code.

zsh 4.2.0 hangs on my cygwin system when I try some of these scenarios.

BTW, the TRAPxxx documentation should be more explicit about whether trapping the event prevents exit and what happens if you exit explicitly from within the trap.

Thanks

Dave

==============================
#!/bin/zsh

reason=

TRAPINT() {
	#echo int
	reason=int
	exit 100
}

TRAPZERR() {
	#echo zerr
	reason=zerr
	exit 101 # is this necessary?
}

TRAPEXIT() {
	echo exit $reason
	# exit?
}

sleep $1

# Try ^C here sometimes.

echo after sleep

set -e

cp

echo after command

Re: zsh malloc bug

[Peter Stephenson]

Dave Yost wrote:
> zsh 4.2.0 hangs on my cygwin system when I try some of these scenarios.

I couldn't get this to happen on Solaris 8 or Fedora Core 1.  I would be
a bit suspicious about Cygwin, since they had to jump through lots of
hoops to make it look like a UNIX environment.

> BTW, the TRAPxxx documentation should be more explicit about whether trapping
>  the event prevents exit

Yes, it should say how the return status is handled and apparently it
doesn't.  That's a big omission.

> and what happens if you exit explicitly from within the trap.

I'm not quite sure what this means, since if you exit explicitly with
the trap it exits, but the recent behaviour is not to run exit traps at
that point.

Index: Doc/Zsh/func.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/func.yo,v
retrieving revision 1.7
diff -u -r1.7 func.yo
--- Doc/Zsh/func.yo	10 Jul 2001 08:59:18 -0000	1.7
+++ Doc/Zsh/func.yo	8 Jun 2004 13:17:22 -0000
@@ -193,6 +193,24 @@
 
 If a function of this form is defined and null,
 the shell and processes spawned by it will ignore tt(SIG)var(NAL).
+
+The return value from the function is handled specially.  If it is
+zero, the signal is assumed to have been handled, and execution continues
+normally.  Otherwise, the normal effect of the signal is produced;
+if this causes execution to terminate, the status returned to the shell is
+the status returned from the function.
+
+Programs terminated by uncaught signals typically return the status 128
+plus the signal number.  Hence the following causes the handler for
+tt(SIGINT) to print a message, then mimic the usual effect of the signal.
+
+example(TRAPINT() {
+  print "Caught SIGINT, aborting."
+  return $(( 128 + $1 ))
+})
+
+The functions tt(TRAPZERR), tt(TRAPDEBUG) and tt(TRAPEXIT) are never
+executed inside other traps.
 )
 findex(TRAPDEBUG)
 item(tt(TRAPDEBUG))(

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

RANDOMSMALL <range> [ <seed> ]

[Dave Yost]

Zsh's RANDOM function is deficient.  I propose a new builtin a la the lesson learned in the Java world.

http://java.sun.com/docs/books/effective/excursion-random.html

Dave

Re: RANDOMSMALL <range> [ <seed> ]

[Peter Stephenson]

> Zsh's RANDOM function is deficient.

It's limited by POSIX, though I haven't checked how much.  Look at the
math function rand48 in the zsh/mathfunc package --- search zshmodules.

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>
Work: pws@csr.com
Web: http://www.pwstephenson.fsnet.co.uk

PATCH: Re: RANDOMSMALL <range> [ <seed> ]

[Peter Stephenson]

Peter Stephenson wrote:
> Look at the math function rand48 in the zsh/mathfunc package.

Ooops, this turns out to be badly broken when used with a seed
parameter.  After the second time you always get the same number.
Not actually all that random.

Index: Src/Modules/mathfunc.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Modules/mathfunc.c,v
retrieving revision 1.4
diff -u -r1.4 mathfunc.c
--- Src/Modules/mathfunc.c	2 Jun 2004 22:15:00 -0000	1.4
+++ Src/Modules/mathfunc.c	8 Jun 2004 21:42:14 -0000
@@ -500,7 +500,7 @@
 		     * to each unsigned short.
 		     */
 		    for (i = 0; i < 3 && !do_init; i++) {
-			unsigned short *seedptr = seedbuf + i;
+			unsigned short *seedptr = seedbufptr + i;
 			*seedptr = 0;
 			for (j = 0; j < 4; j++) {
 			    if (*seedstr >= '0' && *seedstr <= '9')

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>
Work: pws@csr.com
Web: http://www.pwstephenson.fsnet.co.uk

Re: RANDOMSMALL <range> [ <seed> ]

[Larry Bakst]

I would suggest Mersenne Twister for all random numbers which is located here:

http://www.math.sci.hiroshima-u.ac.jp/%7Em-mat/MT/emt.html

leb


>Zsh's RANDOM function is deficient.  I propose a new builtin a la the lesson learned in the Java world.
>
>http://java.sun.com/docs/books/effective/excursion-random.html
>
>Dave


-- 
Lawrence E. Bakst
pleb@iridescent.org

Re: zsh malloc bug

[Dave Yost]

At 2:20 PM +0100 2004-06-08, Peter Stephenson wrote:
>Dave Yost wrote:
>> zsh 4.2.0 hangs on my cygwin system when I try some of these scenarios.
>
>I couldn't get this to happen on Solaris 8 or Fedora Core 1.  I would be
>a bit suspicious about Cygwin, since they had to jump through lots of
>hoops to make it look like a UNIX environment.

It happens on Mac OS X.

>
>> BTW, the TRAPxxx documentation should be more explicit about whether trapping
>>  the event prevents exit
>
>Yes, it should say how the return status is handled and apparently it
>doesn't.  That's a big omission.

Not anymore!  Thanks for updating the doc and for sending the update along.

Dave

============== revised source
==============================
#!/bin/zsh

TRAPINT() {
	echo Interrupted
	exit 2
}

TRAPZERR() {
	reason=$?
	echo Failed command.
	exit $reason
}

TRAPEXIT() {
	echo exited normally
}

sleep $1

# Try ^C here sometimes.

echo after sleep

set -e

cp

echo after command

Re: zsh malloc bug

[Peter Stephenson]

Bart Schaefer wrote:
> Does that really do any good?

No, it doesn't.

pws


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited. 
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: zsh malloc bug

[Bart Schaefer]

On Mon, 26 Jul 2004, Peter Stephenson wrote:

> +# Allow this to be passed down.
> +export MODULE_PATH

Does that really do any good?

Src/params.c:/* MODULE_PATH is not imported for security reasons */

Re: zsh malloc bug

[Peter Stephenson]

Dave Yost wrote:
> A real example follows.

Finaly got around to looking, thanks for the script.

Rather embarassingly, it turns out EXIT traps on scripts are completely
broken.  You just need to do:

zsh -fc 'TRAPEXIT() { print Ooh-er.; }'

It's now documented that special traps such as EXIT don't run inside
other traps, so the following simple fix ought to be fine.

Are there any remaining trap problems after this?

I added a variable to give the name of the zsh executable inside tests,
so we can test this kind of thing.  As you'll see it's a bit imperfect.

Index: Src/exec.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/exec.c,v
retrieving revision 1.67
diff -u -r1.67 exec.c
--- Src/exec.c	7 Jul 2004 15:00:57 -0000	1.67
+++ Src/exec.c	26 Jul 2004 13:02:33 -0000
@@ -3495,7 +3495,8 @@
 	memcpy(oldpipestats, pipestats, bytes);
     }
 
-    starttrapscope();
+    if (!intrap)
+	starttrapscope();
 
     tab = pparams;
     if (!(flags & PM_UNDEFINED))
@@ -3595,7 +3596,8 @@
 	opts[LOCALOPTIONS] = saveopts[LOCALOPTIONS];
     }
 
-    endtrapscope();
+    if (!intrap)
+	endtrapscope();
 
     if (trapreturn < -1)
 	trapreturn++;
Index: Src/signals.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/signals.c,v
retrieving revision 1.28
diff -u -r1.28 signals.c
--- Src/signals.c	2 May 2004 19:55:58 -0000	1.28
+++ Src/signals.c	26 Jul 2004 13:02:34 -0000
@@ -939,6 +939,10 @@
  * with non-standard sigtrapped & sigfuncs values
  */
 
+/* Are we already executing a trap? */
+/**/
+int intrap;
+
 /**/
 void
 dotrapargs(int sig, int *sigtr, void *sigfn)
@@ -949,9 +953,6 @@
     int obreaks = breaks;
     int isfunc;
 
-    /* Are we already executing a trap? */
-    static int intrap;
-
     /* if signal is being ignored or the trap function		      *
      * is NULL, then return					      *
      *								      *
Index: Test/C03traps.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/C03traps.ztst,v
retrieving revision 1.3
diff -u -r1.3 C03traps.ztst
--- Test/C03traps.ztst	10 Mar 2004 10:50:03 -0000	1.3
+++ Test/C03traps.ztst	26 Jul 2004 13:02:34 -0000
@@ -57,6 +57,13 @@
 >Function 1 going
 >Function 2 going
 
+# $ZTST_exe is relative to the parent directory.
+# We ought to fix this in ztst.zsh...
+  cd ..
+  $ZTST_exe -fc 'TRAPEXIT() { print Exited.; }'
+0:EXIT traps on a script
+>Exited.
+
   fn1() {
     trap
     trap 'print INT1' INT
Index: Test/Makefile.in
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/Makefile.in,v
retrieving revision 1.7
diff -u -r1.7 Makefile.in
--- Test/Makefile.in	12 Aug 2001 19:43:13 -0000	1.7
+++ Test/Makefile.in	26 Jul 2004 13:02:34 -0000
@@ -46,7 +46,8 @@
 	  $(MAKE) MODDIR=`pwd`/$(subdir)/Modules install.modules > /dev/null; \
 	fi
 	-for f in $(sdir)/$(TESTNUM)*.ztst; do \
-	  $(dir_top)/Src/zsh +Z -f $(sdir)/ztst.zsh $$f; \
+	  ZTST_exe=$(dir_top)/Src/zsh \
+	    $(dir_top)/Src/zsh +Z -f $(sdir)/ztst.zsh $$f; \
 	done
 	rm -rf Modules .zcompdump
 
Index: Test/ztst.zsh
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/ztst.zsh,v
retrieving revision 1.18
diff -u -r1.18 ztst.zsh
--- Test/ztst.zsh	5 Aug 2002 13:10:03 -0000	1.18
+++ Test/ztst.zsh	26 Jul 2004 13:02:35 -0000
@@ -31,6 +31,8 @@
 # Set the module load path to correspond to this build of zsh.
 # This Modules directory should have been created by "make check".
 [[ -d Modules/zsh ]] && module_path=( $PWD/Modules )
+# Allow this to be passed down.
+export MODULE_PATH
 
 # We need to be able to save and restore the options used in the test.
 # We use the $options variable of the parameter module for this.


-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

Re: zsh malloc bug

[Dave Yost]

A real example follows.

1283 Z% touch ,x
1284 Z% echo '  
import blah.*;
' | ./prepend ,x      
,x.TMP-prepend-21041
zsh: done       echo ' import '$package'.*; ' | 
zsh: bus error  ./prepend ,x
1285 Z% 

======= prepend
#!/bin/zsh

commandName=${0##*/}

usage() {
	if [[ $1 != '' ]] ; then echo 1>&2 "\n$1" ; fi
	echo 1>&2 "
Usage: $commandName [ -t ] file ...

Prepends stdin to each of the given files.
Reads stdin only once.

The -t option preserves each file's modify time.
"
	exit 2
}

zparseopts -D -K - -help=argHelp v=argVerbose t=argPreserveMtime

if [[ $#argHelp != 0 ]] ; then
	usage
fi

case $1 in
-*) usage "Unknown option: $1"
	;;
esac

#-----------------------------------------

sourceTemp=/tmp/$commandName.$$
cat > $sourceTemp
# I would have preferred this, but it discards trailing newlines:
# source=`cat`

TRAPINT() {
	echo 1>&2 "$commandName aborting: interrupted at file $file"
	rm -f $tmp
	exit 2
}

TRAPEXIT() {
	rm -f $sourceTemp
}

for file in $*
do
	if [[ -d $file ]] ; then
		echo 1>&2 "$commandName: skipping directory $file"
	else
		fileDir=${0%/*}
		fileName=${0##*/}
		tmp=$file.TMP-$commandName-$$
		echo $tmp
		cat $sourceTemp >  $tmp \
		&& cat $file	>> $tmp
		if [[ $? != 0 ]] ; then
			echo 1>&2 "$commandName aborting: trouble at file $file"
			rm -f $tmp
			exit 2
		else
			if [[ $#argPreserveMtime != 0 ]] ; then
				cpt -N $file $tmp
			fi
			mv -f $tmp $file
			if [[ $? != 0 ]] ; then
				echo 1>&2 "$commandName aborting: trouble at file $file"
				rm -f $tmp
				exit 2
			fi
			if [[ $#argVerbose != 0 ]] ; then
				echo $file
			fi
		fi
	fi
done