Problems with source command in precmd()

Problems with source command in precmd()

[Daniel Friesel]

[-- Attachment #1: Type: text/plain, Size: 541 bytes --]

Hey all,

there appear to be some crashes when using the source command in the
precmd function.

It only happens in zsh 4.3.10, zsh 4.3.9 works fine.
So far, it could be reproduced on various Linux distributions (Gentoo,
Debian, Ubuntu), FreeBSD and OpenBSD.

Steps to reproduce:
Write
> precmd () { source somefile }
into .zshrc
Then
> touch somefile
(so it exists, does not need to have any content)

Then start zsh and hit return.

The first attachment is the glibc backtrace I get, the second attachment
a gdb backtrace by someone else.

[-- Attachment #2: glibc-error --]
[-- Type: text/plain, Size: 4805 bytes --]

*** glibc detected *** zsh: double free or corruption (!prev): 0x08821038 ***
======= Backtrace: =========
/lib/i686/cmov/libc.so.6[0xb7f10824]
/lib/i686/cmov/libc.so.6[0xb7f120b3]
/lib/i686/cmov/libc.so.6(cfree+0x6d)[0xb7f150ad]
zsh(hend+0x42e)[0x807c8de]
zsh(loop+0x1e4)[0x807f4f4]
zsh(zsh_main+0x1d6)[0x8080126]
zsh(main+0x1b)[0x8054cbb]
/lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb7ebbb55]
zsh[0x8054c01]
======= Memory map: ========
08048000-080ca000 r-xp 00000000 fe:01 306775     /bin/zsh4
080ca000-080ce000 rw-p 00081000 fe:01 306775     /bin/zsh4
080ce000-080e1000 rw-p 00000000 00:00 0 
08813000-0895d000 rw-p 00000000 00:00 0          [heap]
b7b00000-b7b21000 rw-p 00000000 00:00 0 
b7b21000-b7c00000 ---p 00000000 00:00 0 
b7c59000-b7c75000 r-xp 00000000 fe:01 371035     /lib/libgcc_s.so.1
b7c75000-b7c76000 rw-p 0001c000 fe:01 371035     /lib/libgcc_s.so.1
b7c82000-b7ca1000 r--s 00000000 fe:01 181125     /usr/share/zsh/functions/Completion/Base.zwc
b7ca5000-b7cbb000 r--s 00000000 fe:01 181316     /usr/share/zsh/functions/Completion.zwc
b7cbb000-b7cc1000 r-xp 00000000 fe:01 179771     /usr/lib/zsh/4.3.10/zsh/zutil.so
b7cc1000-b7cc2000 rw-p 00005000 fe:01 179771     /usr/lib/zsh/4.3.10/zsh/zutil.so
b7cc2000-b7ce0000 r-xp 00000000 fe:01 180338     /usr/lib/zsh/4.3.10/zsh/complete.so
b7ce0000-b7ce1000 rw-p 0001e000 fe:01 180338     /usr/lib/zsh/4.3.10/zsh/complete.so
b7ce1000-b7d13000 r-xp 00000000 fe:01 179767     /usr/lib/zsh/4.3.10/zsh/zle.so
b7d13000-b7d18000 rw-p 00032000 fe:01 179767     /usr/lib/zsh/4.3.10/zsh/zle.so
b7d18000-b7d1a000 r-xp 00000000 fe:01 180344     /usr/lib/zsh/4.3.10/zsh/terminfo.so
b7d1a000-b7d1b000 rw-p 00001000 fe:01 180344     /usr/lib/zsh/4.3.10/zsh/terminfo.so
b7d1b000-b7d21000 r-xp 00000000 fe:01 179761     /usr/lib/zsh/4.3.10/zsh/parameter.so
b7d21000-b7d22000 rw-p 00006000 fe:01 179761     /usr/lib/zsh/4.3.10/zsh/parameter.so
b7d23000-b7d2d000 r-xp 00000000 fe:01 379886     /lib/i686/cmov/libnss_files-2.10.2.so
b7d2d000-b7d2e000 r--p 00009000 fe:01 379886     /lib/i686/cmov/libnss_files-2.10.2.so
b7d2e000-b7d2f000 rw-p 0000a000 fe:01 379886     /lib/i686/cmov/libnss_files-2.10.2.so
b7d2f000-b7d38000 r-xp 00000000 fe:01 379943     /lib/i686/cmov/libnss_nis-2.10.2.so
b7d38000-b7d39000 r--p 00008000 fe:01 379943     /lib/i686/cmov/libnss_nis-2.10.2.so
b7d39000-b7d3a000 rw-p 00009000 fe:01 379943     /lib/i686/cmov/libnss_nis-2.10.2.so
b7d3a000-b7d4d000 r-xp 00000000 fe:01 379946     /lib/i686/cmov/libnsl-2.10.2.so
b7d4d000-b7d4e000 r--p 00012000 fe:01 379946     /lib/i686/cmov/libnsl-2.10.2.so
b7d4e000-b7d4f000 rw-p 00013000 fe:01 379946     /lib/i686/cmov/libnsl-2.10.2.so
b7d4f000-b7d51000 rw-p 00000000 00:00 0 
b7d51000-b7d57000 r-xp 00000000 fe:01 379911     /lib/i686/cmov/libnss_compat-2.10.2.so
b7d57000-b7d58000 r--p 00006000 fe:01 379911     /lib/i686/cmov/libnss_compat-2.10.2.so
b7d58000-b7d59000 rw-p 00007000 fe:01 379911     /lib/i686/cmov/libnss_compat-2.10.2.so
b7d59000-b7e9e000 r--p 00000000 fe:01 129089     /usr/lib/locale/locale-archive
b7e9e000-b7ea0000 rw-p 00000000 00:00 0 
b7ea0000-b7ea4000 r-xp 00000000 fe:01 370994     /lib/libattr.so.1.1.0
b7ea4000-b7ea5000 rw-p 00003000 fe:01 370994     /lib/libattr.so.1.1.0
b7ea5000-b7fe6000 r-xp 00000000 fe:01 379925     /lib/i686/cmov/libc-2.10.2.so
b7fe6000-b7fe8000 r--p 00141000 fe:01 379925     /lib/i686/cmov/libc-2.10.2.so
b7fe8000-b7fe9000 rw-p 00143000 fe:01 379925     /lib/i686/cmov/libc-2.10.2.so
b7fe9000-b7fec000 rw-p 00000000 00:00 0 
b7fec000-b8010000 r-xp 00000000 fe:01 379910     /lib/i686/cmov/libm-2.10.2.so
b8010000-b8011000 r--p 00023000 fe:01 379910     /lib/i686/cmov/libm-2.10.2.so
b8011000-b8012000 rw-p 00024000 fe:01 379910     /lib/i686/cmov/libm-2.10.2.so
b8012000-b8053000 r-xp 00000000 fe:01 370970     /lib/libncursesw.so.5.7
b8053000-b8056000 rw-p 00041000 fe:01 370970     /lib/libncursesw.so.5.7
b8056000-b8057000 rw-p 00000000 00:00 0 
b8057000-b8059000 r-xp 00000000 fe:01 379941     /lib/i686/cmov/libdl-2.10.2.so
b8059000-b805a000 r--p 00001000 fe:01 379941     /lib/i686/cmov/libdl-2.10.2.so
b805a000-b805b000 rw-p 00002000 fe:01 379941     /lib/i686/cmov/libdl-2.10.2.so
b805b000-b805e000 r-xp 00000000 fe:01 370990     /lib/libcap.so.2.17
b805e000-b805f000 rw-p 00002000 fe:01 370990     /lib/libcap.so.2.17
b8060000-b8064000 rw-p 00000000 00:00 0 
b8064000-b806b000 r--s 00000000 fe:01 56608      /usr/lib/gconv/gconv-modules.cache
b806b000-b806d000 rw-p 00000000 00:00 0 
b806d000-b806e000 r-xp 00000000 00:00 0          [vdso]
b806e000-b808a000 r-xp 00000000 fe:01 374994     /lib/ld-2.10.2.so
b808a000-b808b000 r--p 0001b000 fe:01 374994     /lib/ld-2.10.2.so
b808b000-b808c000 rw-p 0001c000 fe:01 374994     /lib/ld-2.10.2.so
bfc72000-bfc87000 rw-p 00000000 00:00 0          [stack]
zsh: abort      zsh

[-- Attachment #3: gdb-bt --]
[-- Type: text/plain, Size: 1439 bytes --]

Core was generated by `-zsh'.
Program terminated with signal 11, Segmentation fault.
#0  _int_malloc (av=0x7f282daa7e60, bytes=1024) at malloc.c:4436
4436    malloc.c: No such file or directory.
        in malloc.c
(gdb) bt
#0  _int_malloc (av=0x7f282daa7e60, bytes=1024) at malloc.c:4436
#1  0x00007f282d7ca290 in *__GI___libc_malloc (bytes=1024) at malloc.c:3660
#2  0x0000000000446b14 in zalloc (size=139810541567584) at mem.c:583
#3  0x00007f282cafdbf0 in initundo () at zle_utils.c:982
#4  0x00007f282cae9d07 in zleread (lp=<value optimized out>, rp=<value optimized out>, flags=<value optimized out>, context=0) at zle_main.c:1190
#5  0x0000000000436bf6 in zleentry (cmd=1) at init.c:1258
#6  0x000000000043a05e in inputline () at input.c:278
#7  ingetc () at input.c:214
#8  0x000000000043579a in ihgetc () at hist.c:263
#9  0x0000000000440fb6 in gettok () at lex.c:677
#10 zshlex () at lex.c:364
#11 0x000000000045a9c4 in parse_event () at parse.c:451
#12 0x000000000043809a in loop (toplevel=1, justonce=0) at init.c:131
#13 0x000000000043961e in zsh_main (argc=<value optimized out>, argv=<value optimized out>) at init.c:1409
#14 0x00007f282d771bbd in __libc_start_main (main=<value optimized out>, argc=<value optimized out>, ubp_av=<value optimized out>, init=<value optimized out>, fini=<value optimized out>, rtld_fini=<value optimized out>, stack_end=0x7fffadc67508) at libc-start.c:220
#15 0x0000000000410319 in _start ()

Re: Problems with source command in precmd()

[Bart Schaefer]

On Dec 19, 10:49pm, Daniel Friesel wrote:
}
} there appear to be some crashes when using the source command in the
} precmd function.

I can reproduce this.  It's sufficient to run zsh -f and enter

precmd() { source /dev/null }

at the prompt, then accept-line a second time.

I get a different backtrace with --enable-zsh-mem.

*** glibc detected *** double free or corruption (!prev): 0x09e79ae8 ***
(gdb) where
#0  0x003047a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1  0x00345825 in raise () from /lib/tls/libc.so.6
#2  0x00347289 in abort () from /lib/tls/libc.so.6
#3  0x00379d2a in __libc_message () from /lib/tls/libc.so.6
#4  0x0038072f in _int_free () from /lib/tls/libc.so.6
#5  0x00380baa in free () from /lib/tls/libc.so.6
#6  0x0808f7e3 in zfree (p=0x9e79ae8, sz=64) at ../../zsh-4.0/Src/mem.c:1500
#7  0x080777b5 in hend (prog=0x0) at ../../zsh-4.0/Src/hist.c:1271
#8  0x0807b3f7 in loop (toplevel=1, justonce=0) at ../../zsh-4.0/Src/init.c:133
#9  0x0807e476 in zsh_main (argc=2, argv=0xbfea3854)
    at ../../zsh-4.0/Src/init.c:1455
#10 0x0804cbea in main (argc=2, argv=0xbfea3854) at ../../zsh-4.0/Src/main.c:93


I'm not sure that's useful, as the first free() has already occurred by
then.  No code around there has changed especially recently. Possibly a
heap is getting popped while a global still has a pointer into it?

Re: Problems with source command in precmd()

[Peter Stephenson]

On Sat, 19 Dec 2009 15:57:32 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Dec 19, 10:49pm, Daniel Friesel wrote:
> }
> } there appear to be some crashes when using the source command in the
> } precmd function.
> 
> I can reproduce this.  It's sufficient to run zsh -f and enter
> 
> precmd() { source /dev/null }
> 
> at the prompt, then accept-line a second time.

This took some hunting down.  valgrind was silent until the point where the
crash happened.

There is a recently added execsave() / execrestore() in the file execution
loop.  This is a good thing to do, but it wasn't sanitising enough of the
state that it was saving to ensure it could be restored cleanly.  In
particular there appears to be a ghastly hack whereby setting "stophist ==
3" is being used in precmd functions to influence a future iteration
(instead of doing its proper job of controlling the history mechanism for
each iteration), and this was causing nameless horrors to happen to the
data referred to by the saved exec information.

This is hard to test for without a handler for interactive tests.  I
couldn't provoke the problem with zsh -ci.

Index: Src/lex.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/lex.c,v
retrieving revision 1.52
diff -u -r1.52 lex.c
--- Src/lex.c	3 Mar 2009 17:26:07 -0000	1.52
+++ Src/lex.c	4 Jan 2010 12:00:45 -0000
@@ -248,8 +248,11 @@
     ls->histactive = histactive;
     ls->histdone = histdone;
     ls->stophist = stophist;
+    stophist = 0;
     ls->hline = chline;
+    chline = NULL;
     ls->hptr = hptr;
+    hptr = NULL;
     ls->hlinesz = hlinesz;
     ls->cstack = cmdstack;
     ls->csp = cmdsp;
@@ -259,7 +262,9 @@
     ls->tokstr = tokstr;
     ls->zshlextext = zshlextext;
     ls->bptr = bptr;
+    tokstr = zshlextext = bptr = NULL;
     ls->bsiz = bsiz;
+    bsiz = 256;
     ls->len = len;
     ls->chwords = chwords;
     ls->chwordlen = chwordlen;

-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom

Re: Problems with source command in precmd()

[Peter Stephenson]

On Mon, 4 Jan 2010 12:15:12 +0000
Peter Stephenson <pws@csr.com> wrote:
> There is a recently added execsave() / execrestore() in the file execution
> loop.

It's not, it's a lexsave() / lexrestore().  This explains why I modified
lexsave().

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/