4.3.9: frequent crashes in completion menu selection

4.3.9: frequent crashes in completion menu selection

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 5285 bytes --]

I hit it quite often on Mandriva cooker; I do not see any obvious 
post-4.3.9 commits dealing with crashes in this area. This usually 
happens during menu selection browsing file system.

Stack trace:

Program terminated with signal 11, Segmentation fault.
[New process 4310]
#0  0xb7b8f4d2 in domenuselect (dummy=0xb7c24374, dat=0xbff4af10)
    at complist.c:2433
2433                        if (*p && !mmarked(*p) && **p && mselect == 
(**p)->gnum)
(gdb) bt
#0  0xb7b8f4d2 in domenuselect (dummy=0xb7c24374, dat=0xbff4af10)
    at complist.c:2433
#1  0x0808f09a in runhookdef (h=0xb7c24374, d=0xbff4af10) at 
module.c:990
#2  0xb7c0ef0d in after_complete (dummy=0xb7bd397c, dat=0xbff4b060)
    at compcore.c:515
#3  0xb7bc03fb in docomplete (lst=0) at zle_tricky.c:869
#4  0xb7bbc171 in completecall (args=0xb7bd3ca0) at zle_tricky.c:208
#5  0xb7bacd5d in execzlefunc (func=0xb7bd0cd8, args=0xb7bd3ca0, 
set_bindk=0)
    at zle_main.c:1292
#6  0xb7bacf8a in zlecore () at zle_main.c:1043
#7  0xb7bad5a8 in zleread (lp=0x80e97dc, rp=0x0, flags=7, context=0)
    at zle_main.c:1206
#8  0xb7baf427 in zle_main_entry (cmd=80, ap=0x95c5c00 "�\001")
    at zle_main.c:1836
#9  0x0807ca02 in zleentry (cmd=1) at init.c:1255
#10 0x080801c3 in ingetc () at input.c:278
#11 0x0807c779 in ihgetc () at hist.c:263
#12 0x08087926 in yylex () at lex.c:677
#13 0x080a3737 in parse_event () at parse.c:451
#14 0x0807e03c in loop (toplevel=1, justonce=0) at init.c:129
#15 0x0807f726 in zsh_main (argc=1, argv=0xbff4b4e4) at init.c:1406
#16 0x08052a8f in main (argc=50332124, argv=0x131) at ./main.c:93

Completion configuration:

{pts/1}% zstyle -L
zstyle ':completion:*:paths' accept-exact true
zstyle ':completion:*' ambiguous true         
zstyle ':completion:*' auto-description specify: %d
zstyle ':completion:*' completer _oldlist _complete _match
zstyle ':completion:*:*:insmod:*' file-patterns '*.ko:modules 
*(-/):directories' '*:all-files'                                                                  
zstyle ':completion:*:*:modprobe:*' file-patterns '*.ko:modules 
*(-/):directories' '*:all-files'                                                                
zstyle ':completion:*:*:modinfo:*' file-patterns '*.ko:modules 
*(-/):directories' '*:all-files'                                                                 
zstyle :compinstall filename /home/bor/.zshrc                                   
zstyle ':completion:*' format Completing %d                                     
zstyle ':completion:*' group-name ''                                            
zstyle ':completion:*:rsync:*' hosts ftp.uio.no ftp.sunet.se                    
zstyle ':completion:*' list-colors 'rs=0' 'di=01;34' 'ln=01;36' 
'pi=40;33' 'so=01;35' 'do=01;35' 'bd=40;33;01' 'cd=40;33;01' 
'or=01;05;37;41' 'mi=01;05;37;41' 'su=37;41' 'sg=30;43' 'tw=30;42' 
'ow=34;42' 'st=37;44' 'ex=01;32' '*.cmd=01;32' '*.exe=01;32' 
'*.com=01;32' '*.btm=01;32' '*.bat=01;32' '*.sh=01;32' '*.csh=01;32' 
'*.tar=01;31' '*.tgz=01;31' '*.arj=01;31' '*.taz=01;31' '*.lzh=01;31' 
'*.lzma=01;31' '*.zip=01;31' '*.z=01;31' '*.Z=01;31' '*.dz=01;31' 
'*.gz=01;31' '*.bz2=01;31' '*.bz=01;31' '*.tbz2=01;31' '*.tz=01;31' 
'*.lzma=01;31' '*.tlz=01;31' '*.deb=01;31' '*.rpm=01;31' '*.cpio=01;31' 
'*.jar=01;31' '*.rar=01;31' '*.ace=01;31' '*.zoo=01;31' '*.cpio=01;31' 
'*.7z=01;31' '*.rz=01;31' '*.iso=01;31' '*.jpg=01;35' '*.jpeg=01;35' 
'*.JPG=01;35' '*.JPEG=01;35' '*.gif=01;35' '*.bmp=01;35' '*.pbm=01;35' 
'*.pgm=01;35' '*.ppm=01;35' '*.tga=01;35' '*.xbm=01;35' '*.xpm=01;35' 
'*.tif=01;35' '*.tiff=01;35' '*.png=01;35' '*.svg=01;35' '*.svgz=01;35' 
'*.mng=01;35' '*.pcx=01;35' '*.mov=01;35' '*.mpg=01;35' '*.mpeg=01;35' 
'*.m2v=01;35' '*.mkv=01;35' '*.ogm=01;35' '*.mp4=01;35' '*.m4v=01;35' 
'*.mp4v=01;35' '*.vob=01;35' '*.qt=01;35' '*.nuv=01;35' '*.wmv=01;35' 
'*.asf=01;35' '*.rm=01;35' '*.rmvb=01;35' '*.flc=01;35' '*.avi=01;35' 
'*.fli=01;35' '*.flv=01;35' '*.gl=01;35' '*.dl=01;35' '*.xcf=01;35' 
'*.xwd=01;35' '*.yuv=01;35' '*.divx=01;35' '*.xvid=01;35' '*.asf=01;35' 
'*.wmv=01;35' '*.mp4=01;35' '*.3gp=01;35' '*.flv=01;35' '*.ico=01;35' 
'*.aac=00;36' '*.au=00;36' '*.flac=00;36' '*.mid=00;36' '*.midi=00;36' 
'*.mka=00;36' '*.mp3=00;36' '*.mpc=00;36' '*.ogg=00;36' '*.ra=00;36' 
'*.wav=00;36' '*.mp2=00;36' '*.mod=00;36' '*.xm=00;36' '*.s3m=00;36' 
'*.it=00;36' '*.wma=00;36' '*~=47;30' '*.bak=47;30' '*.swp=47;30' 
'*.bck=47;30' '*.bk=47;30' '*.old=47;30' '*.tmp=47;30' '*.save=47;30' 
'*.rpmsave=47;30' '*.rpmnew=47;30'
zstyle ':completion:*' list-prompt '%SAt %p: Hit TAB for more, or the 
characterto insert%s'
zstyle ':completion:*' list-rows-first true
zstyle ':completion:*' match-original both
zstyle ':completion:*' matcher-list '' 'm:{a-z}={A-Z}' '+r:|[._-]=** r:|
=**'
zstyle ':completion:*' menu 'select=0'
zstyle ':completion:*' verbose true
{pts/1}% setopt
autopushd
nobanghist
cdablevars
extendedglob
histexpiredupsfirst
histignorealldups
histignoredups
histreduceblanks
histsavenodups
ignoreeof
incappendhistory
interactive
nolistambiguous
nolistbeep
menucomplete
monitor
printeightbit
promptsubst
pushdminus
shinstdin
zle

Crash dump is there if more information is required.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: 4.3.9: frequent crashes in completion menu selection

[Peter Stephenson]

Andrey Borzenkov wrote:
> I hit it quite often on Mandriva cooker; I do not see any obvious
> post-4.3.9 commits dealing with crashes in this area. This usually
> happens during menu selection browsing file system.

There's probably not a cat's chance in hell of getting anywhere with
this (given the code it affects) unless it can be reproduced easily.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: 4.3.9: frequent crashes in completion menu selection

[Richard Hartmann]

[-- Attachment #1: Type: text/plain, Size: 421 bytes --]

On Thu, Jan 8, 2009 at 13:26, Andrey Borzenkov <arvidjaar@newmail.ru> wrote:
> I hit it quite often on Mandriva cooker; I do not see any obvious
> post-4.3.9 commits dealing with crashes in this area. This usually
> happens during menu selection browsing file system.

So when I start zsh -f and source the file attached, I should get a
crash? Does not happen with Debian's 4.3.9-1 nor with
4.3.9-dev-1-cvs1230


Richard

[-- Attachment #2: zsh.test.090108.sh --]
[-- Type: application/x-sh, Size: 3251 bytes --]

Re: 4.3.9: frequent crashes in completion menu selection

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 624 bytes --]

On Четверг 08 января 2009 18:58:12 Richard Hartmann wrote:
> On Thu, Jan 8, 2009 at 13:26, Andrey Borzenkov <arvidjaar@newmail.ru> 
wrote:
> > I hit it quite often on Mandriva cooker; I do not see any obvious
> > post-4.3.9 commits dealing with crashes in this area. This usually
> > happens during menu selection browsing file system.
>
> So when I start zsh -f and source the file attached, I should get a
> crash? Does not happen with Debian's 4.3.9-1 nor with
> 4.3.9-dev-1-cvs1230
>

If it were as easy to reproduce ... try to work with these settings for 
longer time (likely several days).

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: 4.3.9: frequent crashes in completion menu selection

[Richard Hartmann]

On Thu, Jan 8, 2009 at 17:02, Andrey Borzenkov <arvidjaar@newmail.ru> wrote:

> If it were as easy to reproduce ... try to work with these settings for
> longer time (likely several days).

What version are you running? Can you upgrade and use that version
for some time?

Can you try to use one shell per task? Does only only shell crash?
Do the crashes stop?

I don't know if it's feasible to run zsh within Valgrind for your daily
work, but that might help.


Also, _when_ does this happen? Randomly, or..?



Richard

Re: 4.3.9: frequent crashes in completion menu selection

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 1267 bytes --]

On 8 января 2009 15:30:33 Peter Stephenson wrote:
> Andrey Borzenkov wrote:
> > I hit it quite often on Mandriva cooker; I do not see any obvious
> > post-4.3.9 commits dealing with crashes in this area. This usually
> > happens during menu selection browsing file system.
>
> There's probably not a cat's chance in hell of getting anywhere with
> this (given the code it affects) unless it can be reproduced easily.

The following seems to do it here:

mkdir /tmp/u{1,2}/w{1,2}
{pts/1}% zsh -f
cooker% autoload compinit; compinit
cooker% zmodload zsh/complist
cooker% zstyle ':completion:*' menu 'select=0'
cooker% bindkey -M menuselect "/" accept-and-infer-next-history
cooker% setopt menucomplete
cooker% zstyle ':completion:*' format Completing %d
cooker% ls /tmp/uTAB

Now you get menu selection; press two times '/' to get "Completing 
'files'" (at /tmp/u1/w1/ line contents); now press undo (^_) until you 
get empty line; press RETURN; press 'ls /tmp/uTAB'. Zsh crashes.

For some reasons crucial point is setting "format". Not only can I not 
make it crash without this; it seems to change the undo behaviour as 
well - pressing undo multiples times with and without "format" set 
cycles through *different* line contents.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: 4.3.9: frequent crashes in completion menu selection

[Mikael Magnusson]

2009/1/18 Andrey Borzenkov <arvidjaar@newmail.ru>:
> On 8 января 2009 15:30:33 Peter Stephenson wrote:
>> Andrey Borzenkov wrote:
>> > I hit it quite often on Mandriva cooker; I do not see any obvious
>> > post-4.3.9 commits dealing with crashes in this area. This usually
>> > happens during menu selection browsing file system.
>>
>> There's probably not a cat's chance in hell of getting anywhere with
>> this (given the code it affects) unless it can be reproduced easily.
>
> The following seems to do it here:
>
> mkdir /tmp/u{1,2}/w{1,2}
> {pts/1}% zsh -f
> cooker% autoload compinit; compinit
> cooker% zmodload zsh/complist
> cooker% zstyle ':completion:*' menu 'select=0'
> cooker% bindkey -M menuselect "/" accept-and-infer-next-history
> cooker% setopt menucomplete
> cooker% zstyle ':completion:*' format Completing %d
> cooker% ls /tmp/uTAB
>
> Now you get menu selection; press two times '/' to get "Completing
> 'files'" (at /tmp/u1/w1/ line contents); now press undo (^_) until you
> get empty line; press RETURN; press 'ls /tmp/uTAB'. Zsh crashes.
>
> For some reasons crucial point is setting "format". Not only can I not
> make it crash without this; it seems to change the undo behaviour as
> well - pressing undo multiples times with and without "format" set
> cycles through *different* line contents.

While it doesn't explain the crash, I'm pretty sure that should be
zstyle ':completion:*' format 'Completing %d'

-- 
Mikael Magnusson

Re: 4.3.9: frequent crashes in completion menu selection

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 1831 bytes --]

On 18 января 2009 14:52:15 Mikael Magnusson wrote:
> 2009/1/18 Andrey Borzenkov <arvidjaar@newmail.ru>:
> > On 8 января 2009 15:30:33 Peter Stephenson wrote:
> >> Andrey Borzenkov wrote:
> >> > I hit it quite often on Mandriva cooker; I do not see any
> >> > obvious post-4.3.9 commits dealing with crashes in this area.
> >> > This usually happens during menu selection browsing file system.
> >>
> >> There's probably not a cat's chance in hell of getting anywhere
> >> with this (given the code it affects) unless it can be reproduced
> >> easily.
> >
> > The following seems to do it here:
> >
> > mkdir /tmp/u{1,2}/w{1,2}
> > {pts/1}% zsh -f
> > cooker% autoload compinit; compinit
> > cooker% zmodload zsh/complist
> > cooker% zstyle ':completion:*' menu 'select=0'
> > cooker% bindkey -M menuselect "/" accept-and-infer-next-history
> > cooker% setopt menucomplete
> > cooker% zstyle ':completion:*' format Completing %d
> > cooker% ls /tmp/uTAB
> >
> > Now you get menu selection; press two times '/' to get "Completing
> > 'files'" (at /tmp/u1/w1/ line contents); now press undo (^_) until
> > you get empty line; press RETURN; press 'ls /tmp/uTAB'. Zsh
> > crashes.
> >
> > For some reasons crucial point is setting "format". Not only can I
> > not make it crash without this; it seems to change the undo
> > behaviour as well - pressing undo multiples times with and without
> > "format" set cycles through *different* line contents.
>
> While it doesn't explain the crash, I'm pretty sure that should be
> zstyle ':completion:*' format 'Completing %d'

This was copy'n'paste from 'zstyle -L' output. If this is wrong, zstyle 
-L should be fixed.

Besides, it seems to actually work - I do get "Completing 'files'" which 
pretty much complies with format setting :)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: 4.3.9: frequent crashes in completion menu selection

[Richard Hartmann]

On Sun, Jan 18, 2009 at 12:47, Andrey Borzenkov <arvidjaar@newmail.ru> wrote:

> Zsh crashes.

Confirmed for CVS from 13.01.09


Richard

Re: 4.3.9: frequent crashes in completion menu selection

[Bart Schaefer]

On Sun, Jan 18, 2009 at 11:24 AM, Richard Hartmann
<richih.mailinglist@gmail.com> wrote:
> On Sun, Jan 18, 2009 at 12:47, Andrey Borzenkov <arvidjaar@newmail.ru> wrote:
>
>> Zsh crashes.
>
> Confirmed for CVS from 13.01.09

It looks like the loop that tries to locate the highlighted match in
the list of completions is running one line too far, but I can't see
why (a) there is no highlighted match and (b) having a format style
causes this to happen.

Program received signal SIGSEGV, Segmentation fault.
0x08109622 in domenuselect (dummy=0x8156c74, dat=0xbfe3d4b0)
    at ../../../zsh-4.0/Src/Zle/complist.c:2433
2433                        if (*p && !mmarked(*p) && **p && mselect
== (**p)->gnum)
(gdb) p p
$1 = (Cmatch **) 0x85d7a90
(gdb) p *p
$2 = (Cmatch *) 0x148
(gdb) where
#0  0x08109622 in domenuselect (dummy=0x8156c74, dat=0xbfe3d4b0)
    at ../../../zsh-4.0/Src/Zle/complist.c:2433
#1  0x0808f80a in runhookdef (h=0x8156c74, d=0xbfe3d4b0)
    at ../../zsh-4.0/Src/module.c:990
#2  0x080f0231 in after_complete (dummy=0x815a3dc, dat=0xbfe3d540)
    at ../../../zsh-4.0/Src/Zle/compcore.c:515
#3  0x0808f86a in runhookdef (h=0x815a3dc, d=0xbfe3d540)
    at ../../zsh-4.0/Src/module.c:996
#4  0x0812e71e in docomplete (lst=0)
    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:869
#5  0x0812d507 in expandorcomplete (args=0x8169abc)
    at ../../../zsh-4.0/Src/Zle/zle_tricky.c:315

Preceding lines are:
2421        mlines = 999999;
2422        mlbeg = 0;
2423        molbeg = -42;
2424        for (;;) {
2425            METACHECK();
2426
2427            if (mline < 0 || mtab_been_reallocated) {
2428                int x, y;
2429                Cmatch **p = mtab;
2430
2431                for (y = 0; y < mlines; y++) {
2432                    for (x = mcols; x > 0; x--, p++)


(gdb) p mlines
$7 = 999999
(gdb) p y
$8 = 1
(gdb) p mtab_been_reallocated
$9 = 1

The only assignment of mtab_been_reallocated = 1 is in complistmatches().

Re: 4.3.9: frequent crashes in completion menu selection

[Peter Stephenson]

On Sun, 18 Jan 2009 15:47:31 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> It looks like the loop that tries to locate the highlighted match in
> the list of completions is running one line too far, but I can't see
> why (a) there is no highlighted match and (b) having a format style
> causes this to happen.
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0x08109622 in domenuselect (dummy=0x8156c74, dat=0xbfe3d4b0)
>     at ../../../zsh-4.0/Src/Zle/complist.c:2433
> 2433                        if (*p && !mmarked(*p) && **p && mselect

Might be this simple:  mtab_been_reallocated doesn't work the
first time through the loop, because we've just reset some key parameters
before the loop, and Andrey's results clearly show this is happening when
you start a new completion.  That variable was added to fix a bug with
redisplaying within menu completion.

Index: Src/Zle/complist.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/complist.c,v
retrieving revision 1.118
diff -u -r1.118 complist.c
--- Src/Zle/complist.c	6 Aug 2008 02:21:03 -0000	1.118
+++ Src/Zle/complist.c	20 Jan 2009 10:36:46 -0000
@@ -2421,6 +2421,7 @@
     mlines = 999999;
     mlbeg = 0;
     molbeg = -42;
+    mtab_been_reallocated = 0;
     for (;;) {
 	METACHECK();
 
-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070