Fwd: Bug#924736: zsh 5.7.1 segfaults when three setopt options are in play [origin: wesley@schwengle.net]

Fwd: Bug#924736: zsh 5.7.1 segfaults when three setopt options are in play [origin: wesley@schwengle.net]

[Axel Beckert]

Hi,

we at Debian received the following bug report at
https://bugs.debian.org/924736

I can confirm that this issue is present in zsh 5.7.1 as well as git
HEAD as of commit 947e26fe5a0083b42ef5db9cb0f8c46923602ae1:

----- Forwarded message from Wesley Schwengle <wesley@schwengle.net> -----
Date: Sat, 16 Mar 2019 18:54:27 +0100
From: Wesley Schwengle <wesley@schwengle.net>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: [Pkg-zsh-devel] Bug#924736: zsh 5.7.1 segfaults when three setopt options are in play
Reply-To: Wesley Schwengle <wesley@schwengle.net>, 924736@bugs.debian.org

Package: zsh
Version: 5.7.1-1
Severity: important

Dear Maintainer,

*** Reporter, please consider answering these questions, where appropriate ***

Have a zshrc with the following setopts:

setopt hist_reduce_blanks
setopt hist_ignore_space
setopt interactivecomments

* Run zsh -f
* Now enter `     #`
* You get a command not found error
* Now source your zshrc
* Again entery `     #`
* Segfault

I've reproduced it with a docker image from debian testing.
https://gist.github.com/waterkip/ab532e8dc65ad948046b6848dcfacffa

It does work on Debian stable (zsh 5.3.1).

Dockerfile contents:

FROM debian:testing
WORKDIR /root
RUN apt-get update && apt-get install --no-install-recommends -y zsh
COPY zsh/.zsh/minimal-zshrc .zshrc

$ dpkg -l zsh
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  zsh            5.7.1-1      amd64        shell with lots of features

-- Package-specific info:

Packages which provide vendor completions:

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version                     Architecture Description
+++-==============-===========================-============-========================================================
ii  curl           7.64.0-1                    amd64        command line tool for transferring data with URL syntax
ii  docker-ce-cli  5:18.09.3~3-0~debian-buster amd64        Docker CLI: the open-source application container engine
ii  mpv            0.29.1-1                    amd64        video player based on MPlayer/mplayer2
ii  pulseaudio     12.2-4                      amd64        PulseAudio sound server
ii  systemd        241-1                       amd64        system and service manager
ii  udev           241-1                       amd64        /dev/ and hotplug management daemon
ii  vlc-bin        3.0.6-1                     amd64        binaries from VLC
ii  youtube-dl     2019.01.17-1                all          downloader of videos from YouTube and other sites

dpkg-query: no path found matching pattern /usr/share/zsh/vendor-functions/


-- System Information:
Debian Release: buster/sid
  APT prefers testing
  APT policy: (500, 'testing'), (100, 'unstable'), (50, 'experimental'), (10, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.19.0-2-amd64 (SMP w/8 CPU cores)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages zsh depends on:
ii  libc6       2.28-8
ii  libcap2     1:2.25-2
ii  libtinfo6   6.1+20181013-2
ii  zsh-common  5.7.1-1

Versions of packages zsh recommends:
ii  libc6         2.28-8
ii  libncursesw6  6.1+20181013-2
ii  libpcre3      2:8.39-11

Versions of packages zsh suggests:
pn  zsh-doc  <none>

-- no debconf information
----- End forwarded message -----

I can as well confirm that zsh 5.3.1 is not affected.

----- Forwarded message from wesleys@euronet.nl -----
Date: Sat, 16 Mar 2019 19:24:31 +0100 (CET)
From: wesleys@euronet.nl
To: 924736@bugs.debian.org
Subject: [Pkg-zsh-devel] Bug#924736: Acknowledgement (zsh 5.7.1 segfaults when three setopt options are
	in play)
Reply-To: wesleys@euronet.nl, 924736@bugs.debian.org



on #zsh there was some confusion about the reproduction path
`     #` should be typed *without* the backticks. Spaces are hard to show on a text only medium.

FWIW, it seems like an upstream bug, I can also reproduce it on Arch

Cheers,
Wesley
----- End forwarded message -----

Haven't had time to bisect this, but I got this backtrace from git
HEAD:

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `./Src/zsh -f'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000558b2df7b10c in histreduceblanks ()
(gdb) bt
#0  0x0000558b2df7b10c in histreduceblanks ()
#1  0x0000558b2df80ecb in hend ()
#2  0x0000558b2df814bf in loop ()
#3  0x0000558b2df84be6 in zsh_main ()
#4  0x00007f4441f6109b in __libc_start_main (main=0x558b2df4ac90 <main>, argc=2, argv=0x7ffc57da4588, 
    init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc57da4578)
    at ../csu/libc-start.c:308
#5  0x0000558b2df4acca in _start ()
(gdb) 

		Kind regards, Axel
-- 
PGP: 2FF9CD59612616B5      /~\  Plain Text Ribbon Campaign, http://arc.pasp.de/
Mail: abe@deuxchevaux.org  \ /  Say No to HTML in E-Mail and Usenet
Mail+Jabber: abe@noone.org  X
https://axel.beckert.ch/   / \  I love long mails: https://email.is-not-s.ms/

Re: Bug#924736: zsh 5.7.1 segfaults when three setopt options are in play [origin: wesley@schwengle.net]

[Wesley Schwengle]

Hello all,

<snip>

> Haven't had time to bisect this, but I got this backtrace from git
> HEAD:

On #zsh (thanks okdana, Mikachu and DHowett) we noticed that the bug
was introduced between 5.3.1 and 5.6.2.
I managed to bisect and the bug seems to originate from
https://github.com/zsh-users/zsh/commit/758966502caa6f91abcbaaebf2610609250de1fb
Once that commit is reverted the bug disappears. Mikachu noticed that
if one applies the patch below the problem is also resolved. My
understanding of the zsh code base is limited, so I don't feel
confident to submit the patch for actual inclusion in the code base:

diff --git i/Src/hist.c w/Src/hist.c
index dbdc1e4e5..972a36db2 100644
--- i/Src/hist.c
+++ w/Src/hist.c
@@ -1489,7 +1489,7 @@ hend(Eprog prog)
} else
save = 0;
}
-       if (chwordpos <= 2 && !hist_keep_comment)
+       if (chwordpos <= 2)
save = 0;
else if (should_ignore_line(prog))
save = -1;

Cheers,
Wesley

-- 
Wesley Schwengle, Developer
Mintlab B.V., https://www.zaaksysteem.nl
E: wesley@mintlab.nl
T:  +31 20 737 00 05

Re: Fwd: Bug#924736: zsh 5.7.1 segfaults when three setopt options are in play [origin: wesley@schwengle.net]

[Peter Stephenson]

On Sat, 2019-03-16 at 22:41 +0100, Axel Beckert wrote:
> Have a zshrc with the following setopts:
> 
> setopt hist_reduce_blanks
> setopt hist_ignore_space
> setopt interactivecomments
> 
> * Run zsh -f
> * Now enter `     #`
> * You get a command not found error
> * Now source your zshrc
> * Again entery `     #`
> * Segfault

Yes, that's completely reproducible.

I think it's the logic within histreduceblanks() that's flaky in this
case, where there's a comment at the end of a line with no commands and
hence no words.  The final comment is a special case because the
positions of words aren't marked.  It can't possibly be correct to do
that copy at the end if the destination pointer is after the source
pointer, can it?  So I think the following ought to be safe.

If anyone else thinks the code here is trying to do something cleverer that this
may stop --- your guess is as good as mine at this point --- let me know
(but I think that's a much lesser problem).

pws

diff --git a/Src/hist.c b/Src/hist.c
index f7e53de..901cd3b 100644
--- a/Src/hist.c
+++ b/Src/hist.c
@@ -1198,8 +1198,9 @@ histreduceblanks(void)
 	chline[pos] = '\0';
     } else {
 	ptr = chline + pos;
-	while ((*ptr++ = *lastptr++))
-	    ;
+	if (ptr < lastptr)
+	    while ((*ptr++ = *lastptr++))
+		;
     }
 }