Zsh parser malloc corruption

Zsh parser malloc corruption

[Eduardo Bustamante]

dualbus@debian:~/bash-fuzzing/zsh-parser$ cat -v malloc-corruption
0000000000000000000000000000000000000000${0#0000000000000000^@000000000000000000000000000000000000000000000000000^@^@000M-^GM-^O0000000$000000#000000000000$$$0}000000000000&0000000000000000000000000000000000000000000000000000000000000000&00000000

dualbus@debian:~/bash-fuzzing/zsh-parser$ base64 malloc-corruption
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMCR7MCMwMDAwMDAwMDAwMDAw
MDAwADAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMAAA
MDAwh48wMDAwMDAwJDAwMDAwMCMwMDAwMDAwMDAwMDAkJCQwfTAwMDAwMDAwMDAwMCYwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
JjAwMDAwMDAwCg==

dualbus@debian:~/bash-fuzzing/zsh-parser$ ~/src/zsh/zsh/Src/zsh -n
malloc-corruption
*** Error in `/home/dualbus/src/zsh/zsh/Src/zsh': malloc(): memory
corruption: 0x0000000000aca090 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x70bcb)[0x7f47ad245bcb]
/lib/x86_64-linux-gnu/libc.so.6(+0x76f96)[0x7f47ad24bf96]
/lib/x86_64-linux-gnu/libc.so.6(+0x78f69)[0x7f47ad24df69]
/lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7f47ad24fd84]
/home/dualbus/src/zsh/zsh/Src/zsh(zalloc+0x3c)[0x4798dc]
/home/dualbus/src/zsh/zsh/Src/zsh(setunderscore+0xa2)[0x435892]
/home/dualbus/src/zsh/zsh/Src/zsh[0x43d6b5]
/home/dualbus/src/zsh/zsh/Src/zsh[0x43b804]
/home/dualbus/src/zsh/zsh/Src/zsh[0x433f6e]
/home/dualbus/src/zsh/zsh/Src/zsh(execlist+0x64e)[0x432dfe]
/home/dualbus/src/zsh/zsh/Src/zsh(execode+0x11e)[0x43277e]
/home/dualbus/src/zsh/zsh/Src/zsh(loop+0x416)[0x45e366]
/home/dualbus/src/zsh/zsh/Src/zsh(zsh_main+0x366)[0x4627d6]
/home/dualbus/src/zsh/zsh/Src/zsh(main+0x22)[0x411a32]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f47ad1f52b1]
/home/dualbus/src/zsh/zsh/Src/zsh(_start+0x2a)[0x41193a]
======= Memory map: ========
00400000-004e9000 r-xp 00000000 fe:01 18487233
  /home/dualbus/src/zsh/zsh/Src/zsh
006e9000-006ea000 r--p 000e9000 fe:01 18487233
  /home/dualbus/src/zsh/zsh/Src/zsh
006ea000-006f1000 rw-p 000ea000 fe:01 18487233
  /home/dualbus/src/zsh/zsh/Src/zsh
006f1000-00704000 rw-p 00000000 00:00 0
00ab3000-00ad4000 rw-p 00000000 00:00 0                                  [heap]
7f47a8000000-7f47a8021000 rw-p 00000000 00:00 0
7f47a8021000-7f47ac000000 ---p 00000000 00:00 0
7f47ac563000-7f47ac579000 r-xp 00000000 fe:01 1310769
  /lib/x86_64-linux-gnu/libgcc_s.so.1
7f47ac579000-7f47ac778000 ---p 00016000 fe:01 1310769
  /lib/x86_64-linux-gnu/libgcc_s.so.1
7f47ac778000-7f47ac779000 r--p 00015000 fe:01 1310769
  /lib/x86_64-linux-gnu/libgcc_s.so.1
7f47ac779000-7f47ac77a000 rw-p 00016000 fe:01 1310769
  /lib/x86_64-linux-gnu/libgcc_s.so.1
7f47ac77a000-7f47ac784000 r-xp 00000000 fe:01 1311265
  /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f47ac784000-7f47ac984000 ---p 0000a000 fe:01 1311265
  /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f47ac984000-7f47ac985000 r--p 0000a000 fe:01 1311265
  /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f47ac985000-7f47ac986000 rw-p 0000b000 fe:01 1311265
  /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f47ac986000-7f47ac98c000 rw-p 00000000 00:00 0
7f47ac98c000-7f47ac997000 r-xp 00000000 fe:01 1311269
  /lib/x86_64-linux-gnu/libnss_nis-2.24.so
7f47ac997000-7f47acb96000 ---p 0000b000 fe:01 1311269
  /lib/x86_64-linux-gnu/libnss_nis-2.24.so
7f47acb96000-7f47acb97000 r--p 0000a000 fe:01 1311269
  /lib/x86_64-linux-gnu/libnss_nis-2.24.so
7f47acb97000-7f47acb98000 rw-p 0000b000 fe:01 1311269
  /lib/x86_64-linux-gnu/libnss_nis-2.24.so
7f47acb98000-7f47acbac000 r-xp 00000000 fe:01 1311178
  /lib/x86_64-linux-gnu/libnsl-2.24.so
7f47acbac000-7f47acdac000 ---p 00014000 fe:01 1311178
  /lib/x86_64-linux-gnu/libnsl-2.24.so
7f47acdac000-7f47acdad000 r--p 00014000 fe:01 1311178
  /lib/x86_64-linux-gnu/libnsl-2.24.so
7f47acdad000-7f47acdae000 rw-p 00015000 fe:01 1311178
  /lib/x86_64-linux-gnu/libnsl-2.24.so
7f47acdae000-7f47acdb0000 rw-p 00000000 00:00 0
7f47acdb0000-7f47acdb7000 r-xp 00000000 fe:01 1311180
  /lib/x86_64-linux-gnu/libnss_compat-2.24.so
7f47acdb7000-7f47acfb6000 ---p 00007000 fe:01 1311180
  /lib/x86_64-linux-gnu/libnss_compat-2.24.so
7f47acfb6000-7f47acfb7000 r--p 00006000 fe:01 1311180
  /lib/x86_64-linux-gnu/libnss_compat-2.24.so
7f47acfb7000-7f47acfb8000 rw-p 00007000 fe:01 1311180
  /lib/x86_64-linux-gnu/libnss_compat-2.24.so
7f47acfb8000-7f47acfd0000 r-xp 00000000 fe:01 1311335
  /lib/x86_64-linux-gnu/libpthread-2.24.so
7f47acfd0000-7f47ad1cf000 ---p 00018000 fe:01 1311335
  /lib/x86_64-linux-gnu/libpthread-2.24.so
7f47ad1cf000-7f47ad1d0000 r--p 00017000 fe:01 1311335
  /lib/x86_64-linux-gnu/libpthread-2.24.so
7f47ad1d0000-7f47ad1d1000 rw-p 00018000 fe:01 1311335
  /lib/x86_64-linux-gnu/libpthread-2.24.so
7f47ad1d1000-7f47ad1d5000 rw-p 00000000 00:00 0
7f47ad1d5000-7f47ad36a000 r-xp 00000000 fe:01 1311151
  /lib/x86_64-linux-gnu/libc-2.24.so
7f47ad36a000-7f47ad569000 ---p 00195000 fe:01 1311151
  /lib/x86_64-linux-gnu/libc-2.24.so
7f47ad569000-7f47ad56d000 r--p 00194000 fe:01 1311151
  /lib/x86_64-linux-gnu/libc-2.24.so
7f47ad56d000-7f47ad56f000 rw-p 00198000 fe:01 1311151
  /lib/x86_64-linux-gnu/libc-2.24.so
7f47ad56f000-7f47ad573000 rw-p 00000000 00:00 0
7f47ad573000-7f47ad676000 r-xp 00000000 fe:01 1311172
  /lib/x86_64-linux-gnu/libm-2.24.so
7f47ad676000-7f47ad875000 ---p 00103000 fe:01 1311172
  /lib/x86_64-linux-gnu/libm-2.24.so
7f47ad875000-7f47ad876000 r--p 00102000 fe:01 1311172
  /lib/x86_64-linux-gnu/libm-2.24.so
7f47ad876000-7f47ad877000 rw-p 00103000 fe:01 1311172
  /lib/x86_64-linux-gnu/libm-2.24.so
7f47ad877000-7f47ad87e000 r-xp 00000000 fe:01 1313974
  /lib/x86_64-linux-gnu/librt-2.24.so
7f47ad87e000-7f47ada7d000 ---p 00007000 fe:01 1313974
  /lib/x86_64-linux-gnu/librt-2.24.so
7f47ada7d000-7f47ada7e000 r--p 00006000 fe:01 1313974
  /lib/x86_64-linux-gnu/librt-2.24.so
7f47ada7e000-7f47ada7f000 rw-p 00007000 fe:01 1313974
  /lib/x86_64-linux-gnu/librt-2.24.so
7f47ada7f000-7f47adaa4000 r-xp 00000000 fe:01 1310814
  /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f47adaa4000-7f47adca4000 ---p 00025000 fe:01 1310814
  /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f47adca4000-7f47adca8000 r--p 00025000 fe:01 1310814
  /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f47adca8000-7f47adca9000 rw-p 00029000 fe:01 1310814
  /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f47adca9000-7f47adcac000 r-xp 00000000 fe:01 1311170
  /lib/x86_64-linux-gnu/libdl-2.24.so
7f47adcac000-7f47adeab000 ---p 00003000 fe:01 1311170
  /lib/x86_64-linux-gnu/libdl-2.24.so
7f47adeab000-7f47adeac000 r--p 00002000 fe:01 1311170
  /lib/x86_64-linux-gnu/libdl-2.24.so
7f47adeac000-7f47adead000 rw-p 00003000 fe:01 1311170
  /lib/x86_64-linux-gnu/libdl-2.24.so
7f47adead000-7f47aded0000 r-xp 00000000 fe:01 1310733
  /lib/x86_64-linux-gnu/ld-2.24.so
7f47adf32000-7f47adf37000 rw-p 00000000 00:00 0
7f47adf37000-7f47adf88000 r--p 00000000 fe:01 26351510
  /usr/lib/locale/aa_DJ.utf8/LC_CTYPE
7f47adf88000-7f47ae0b8000 r--p 00000000 fe:01 26351509
  /usr/lib/locale/aa_DJ.utf8/LC_COLLATE
7f47ae0b8000-7f47ae0bc000 rw-p 00000000 00:00 0
7f47ae0bc000-7f47ae0bd000 r--p 00000000 fe:01 26351533
  /usr/lib/locale/aa_ET/LC_NUMERIC
7f47ae0bd000-7f47ae0be000 r--p 00000000 fe:01 26480725
  /usr/lib/locale/en_US.utf8/LC_TIME
7f47ae0be000-7f47ae0bf000 r--p 00000000 fe:01 26355066
  /usr/lib/locale/chr_US/LC_MONETARY
7f47ae0bf000-7f47ae0c0000 r--p 00000000 fe:01 26355282
  /usr/lib/locale/en_AG/LC_MESSAGES/SYS_LC_MESSAGES
7f47ae0c0000-7f47ae0c1000 r--p 00000000 fe:01 26355068
  /usr/lib/locale/chr_US/LC_PAPER
7f47ae0c1000-7f47ae0c2000 r--p 00000000 fe:01 26355067
  /usr/lib/locale/chr_US/LC_NAME
7f47ae0c2000-7f47ae0c3000 r--p 00000000 fe:01 26480723
  /usr/lib/locale/en_US.utf8/LC_ADDRESS
7f47ae0c3000-7f47ae0c4000 r--p 00000000 fe:01 26355069
  /usr/lib/locale/chr_US/LC_TELEPHONE
7f47ae0c4000-7f47ae0c5000 r--p 00000000 fe:01 26355064
  /usr/lib/locale/chr_US/LC_MEASUREMENT
7f47ae0c5000-7f47ae0cc000 r--s 00000000 fe:01 25449459
  /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
7f47ae0cc000-7f47ae0cd000 r--p 00000000 fe:01 26480724
  /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION
7f47ae0cd000-7f47ae0d0000 rw-p 00000000 00:00 0
7f47ae0d0000-7f47ae0d1000 r--p 00023000 fe:01 1310733
  /lib/x86_64-linux-gnu/ld-2.24.so
7f47ae0d1000-7f47ae0d2000 rw-p 00024000 fe:01 1310733
  /lib/x86_64-linux-gnu/ld-2.24.so
7f47ae0d2000-7f47ae0d3000 rw-p 00000000 00:00 0
7ffd82d8d000-7ffd82dae000 rw-p 00000000 00:00 0                          [stack]
7ffd82de7000-7ffd82de9000 r--p 00000000 00:00 0                          [vvar]
7ffd82de9000-7ffd82deb000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
  [vsyscall]
Aborted

(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff71353fa in __GI_abort () at abort.c:89
#2  0x00007ffff7171bd0 in __libc_message (do_abort=do_abort@entry=2,
    fmt=fmt@entry=0x7ffff7266bd0 "*** Error in `%s': %s: 0x%s ***\n")
at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007ffff7177f96 in malloc_printerr (action=3,
str=0x7ffff72637cb "malloc(): memory corruption", ptr=<optimized out>,
    ar_ptr=<optimized out>) at malloc.c:5046
#4  0x00007ffff7179f69 in _int_malloc (av=av@entry=0x7ffff7499b00
<main_arena>, bytes=bytes@entry=96) at malloc.c:3509
#5  0x00007ffff717bd84 in __GI___libc_malloc (bytes=96) at malloc.c:2925
#6  0x00000000004798dc in zalloc (size=96) at mem.c:966
#7  0x0000000000435892 in setunderscore (str=0x7ffff7e5bc18 '0'
<repeats 40 times>, "malloc-corruption", '0' <repeats 12 times>)
    at exec.c:2518
#8  0x000000000043d6b5 in execcmd_exec (state=0x7fffffffde20,
eparams=0x7fffffffcce0, input=0, output=0, how=4, last1=2)
    at exec.c:3183
#9  0x000000000043b804 in execpline2 (state=0x7fffffffde20, pcode=131,
how=4, input=0, output=0, last1=0) at exec.c:1873
#10 0x0000000000433f6e in execpline (state=0x7fffffffde20,
slcode=3074, how=4, last1=0) at exec.c:1602
#11 0x0000000000432dfe in execlist (state=0x7fffffffde20,
dont_change_job=0, exiting=0) at exec.c:1360
#12 0x000000000043277e in execode (p=0x7ffff7e5b5c0,
dont_change_job=0, exiting=0, context=0x4d90c4 "toplevel") at
exec.c:1141
#13 0x000000000045e366 in loop (toplevel=1, justonce=0) at init.c:208
#14 0x00000000004627d6 in zsh_main (argc=3, argv=0x7fffffffe448) at init.c:1692
#15 0x0000000000411a32 in main (argc=3, argv=0x7fffffffe448) at ./main.c:93

Re: Zsh parser malloc corruption

[Peter Stephenson]

I think it would probably help if you attach the files causing these
problems.

Thanks
pws

Re: Zsh parser malloc corruption

[Eduardo Bustamante]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]

On Mon, May 8, 2017 at 9:11 AM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
> I think it would probably help if you attach the files causing these
> problems.

Woops, sorry.

Attached.

dualbus@debian:~/bash-fuzzing/zsh-parser$ md5sum malloc-corruption
5978fcf3851937267d2fa4e9656b7502  malloc-corruption

[-- Attachment #2: malloc-corruption --]
[-- Type: application/octet-stream, Size: 238 bytes --]

Re: Zsh parser malloc corruption

[Eduardo Bustamante]

On Mon, May 8, 2017 at 9:11 AM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
> I think it would probably help if you attach the files causing these
> problems.

By the way, I obtained all of these crashing inputs by running `Zsh
-n' under the AFL fuzzer. I'm trying to go through the crashes I got
and sort / classify. I don't know if you'd prefer for me to just
upload the crashes in a compressed tarball somewhere and send you the
link.

Re: Zsh parser malloc corruption

[Peter Stephenson]

On Mon, 8 May 2017 09:20:38 -0500
Eduardo Bustamante <dualbus@gmail.com> wrote:
> On Mon, May 8, 2017 at 9:11 AM, Peter Stephenson
> <p.stephenson@samsung.com> wrote:
> > I think it would probably help if you attach the files causing these
> > problems.
> 
> By the way, I obtained all of these crashing inputs by running `Zsh
> -n' under the AFL fuzzer. I'm trying to go through the crashes I got
> and sort / classify. I don't know if you'd prefer for me to just
> upload the crashes in a compressed tarball somewhere and send you the
> link.

Separate reports are fine as they're probably typically different
issues.

pws

Re: Zsh parser malloc corruption

[Daniel Shahaf]

Peter Stephenson wrote on Mon, May 08, 2017 at 16:10:24 +0100:
> On Mon, 8 May 2017 09:20:38 -0500
> Eduardo Bustamante <dualbus@gmail.com> wrote:
> > On Mon, May 8, 2017 at 9:11 AM, Peter Stephenson
> > <p.stephenson@samsung.com> wrote:
> > > I think it would probably help if you attach the files causing these
> > > problems.
> > 
> > By the way, I obtained all of these crashing inputs by running `Zsh
> > -n' under the AFL fuzzer. I'm trying to go through the crashes I got
> > and sort / classify. I don't know if you'd prefer for me to just
> > upload the crashes in a compressed tarball somewhere and send you the
> > link.
> 
> Separate reports are fine as they're probably typically different
> issues.

Agreed.  Also, thank you Eduardo for the reports!  Much appreciated.

Re: Zsh parser malloc corruption

[Bart Schaefer]

PWS, I'm going to ask you to please look at this after all, because it
seems to be related to 

    36682: expand pattern interface to optimise unmetafication

Valgrind says:

==19116== Invalid write of size 1
==19116==    at 0x4A2E0D: patcompile (pattern.c:679)
==19116==    by 0x456846: compgetmatch (glob.c:2623)
==19116==    by 0x4568FA: getmatch (glob.c:2663)
==19116==    by 0x4BA2D9: paramsubst (subst.c:3045)
==19116==    by 0x4B486A: stringsubst (subst.c:247)
==19116==    by 0x4B3BED: prefork (subst.c:85)
==19116==    by 0x4437D5: execcmd_getargs (exec.c:2659)
==19116==    by 0x443BCF: execcmd_exec (exec.c:2765)
==19116==    by 0x4414B5: execpline2 (exec.c:1873)
==19116==    by 0x43FCDA: execpline (exec.c:1602)
==19116==    by 0x43EEA5: execlist (exec.c:1360)
==19116==    by 0x43E5A3: execode (exec.c:1141)

This repeats several times, and eventually kills valgrind itself:

valgrind: the 'impossible' happened:
   Killed by fatal signal
==19116==    at 0x38058236: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)

It appears that the "patalloc" space is not large enough to hold the
metafied pattern, maybe because there are NUL bytes in the pattern
prior to it being metafied?

Also I can reproduce my crash with a shorter input; delete everything
from (including) the first "&" through the end of the malloc-corruption
file.

Re: Zsh parser malloc corruption

[Peter Stephenson]

On Tue, 9 May 2017 23:21:41 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> PWS, I'm going to ask you to please look at this after all, because it
> seems to be related to 
> 
>     36682: expand pattern interface to optimise unmetafication
> 
> Valgrind says:
> 
> ==19116== Invalid write of size 1
> ==19116==    at 0x4A2E0D: patcompile (pattern.c:679)

Doesn't seem easy to reproduce --- probably due to malloc library
variabilities and/or uninitialised memory --- but when I managed to get
it to happen it reported an error a few lines earlier.

Does the following help?  I think it's needed in any case --- we can't
use strcpy() on unmetafied strings as they are there partly to allow us
to treat embedded nulls as normal characters.  It's also pointless
and inconsistent testing for a NULL termination in a function with the
length passed in.

pws

diff --git a/Src/string.c b/Src/string.c
index a8da14f..9e14ef9 100644
--- a/Src/string.c
+++ b/Src/string.c
@@ -52,7 +52,8 @@ dupstring_wlen(const char *s, unsigned len)
     if (!s)
 	return NULL;
     t = (char *) zhalloc(len + 1);
-    strcpy(t, s);
+    memcpy(t, s, len);
+    t[len] = '\0';
     return t;
 }
 

Re: Zsh parser malloc corruption

[Bart Schaefer]

On May 11,  5:15pm, Peter Stephenson wrote:
}
} On Tue, 9 May 2017 23:21:41 -0700
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > PWS, I'm going to ask you to please look at this after all
} 
} Doesn't seem easy to reproduce --- 
} 
} Does the following help?

That stops the crashes that I was able to reproduce, in any case.

Thanks.