Bug? in complist when filenames are longer than the screen

Bug? in complist when filenames are longer than the screen

[DervishD]

    Hi all :)

    I'm not sure if this is a bug or if I'm doing anything wrong, but
I can reproduce it even without reading RC's (neither /etc/zshenv
nor /etc/zshrc exist, and I run zsh with "-f").

    $ print $ZSH_VERSION
    4.2.5
    $ touch a b c filename1 filename2 d e
    $ zmodload zsh/complist
    $ COLUMNS=3
    $ ls <TAB><TAB><TAB><TAB><UP><DOWN>
 
    The screen messes up and, depending on the filenames, zsh
sometimes segfaults or hangs. I don't have at hand any zsh compiled
with debug support, but if anyone can reproduce this behaviour I'll
build one and try to post here a backtrace. Anyway, the most common
behaviour is to hang, so the backtrace won't help.

    The problem happens whenever any item in the completion list is
longer or equal to $COLUMNS.

    If more information is needed or whatever, feel free to tell me.

    Thanks a lot in advance :)

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...

Re: Bug? in complist when filenames are longer than the screen

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 1415 bytes --]

On Monday 03 October 2005 18:45, DervishD wrote:
>     Hi all :)
>
>     I'm not sure if this is a bug or if I'm doing anything wrong, but
> I can reproduce it even without reading RC's (neither /etc/zshenv
> nor /etc/zshrc exist, and I run zsh with "-f").
>
>     $ print $ZSH_VERSION
>     4.2.5
>     $ touch a b c filename1 filename2 d e
>     $ zmodload zsh/complist
>     $ COLUMNS=3
>     $ ls <TAB><TAB><TAB><TAB><UP><DOWN>
>
>     The screen messes up and, depending on the filenames, zsh
> sometimes segfaults or hangs. 

It would be nice to have exact recipe how to reproduce a crash preferably 
starting with zsh -f. I have screen corruption but not (OK I am not sure what 
sensibly can be done on 3 column width :)

Jokes aside, I do confirm screen corruption during normal completion under 
konsole and linux console. This happens with long filenames, usually when I 
complete "cd -" and have deeply nested directories. Like

{pts/1}% cd -TAB

======================

{pts/1}% cd -1/TAB
Completing directory stack
1 
-- /home/bor/Download/D-Link/LINUX_GPL_SOURCE/mtd-20020722/drivers/mtd/maps/CVS
2 -- /home/bor/tmp

======================

{pts/1}% cd -1/
Completing directory stack
1 -- /homcd -2/
1 -- /homcd -2/
1 
-- /home/bor/Download/D-Link/LINUX_GPL_SOURCE/mtd-20020722/drivers/mtd/maps/CVS

2 -- /home/bor/tmp

======================

-andrey

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Bug? in complist when filenames are longer than the screen

[DervishD]

    Hi Andrey :)

 * Andrey Borzenkov <arvidjaar@newmail.ru> dixit:
> On Monday 03 October 2005 18:45, DervishD wrote:
> >     Hi all :)
> >
> >     I'm not sure if this is a bug or if I'm doing anything wrong, but
> > I can reproduce it even without reading RC's (neither /etc/zshenv
> > nor /etc/zshrc exist, and I run zsh with "-f").
> >
> >     $ print $ZSH_VERSION
> >     4.2.5
> >     $ touch a b c filename1 filename2 d e
> >     $ zmodload zsh/complist
> >     $ COLUMNS=3
> >     $ ls <TAB><TAB><TAB><TAB><UP><DOWN>
> >
> >     The screen messes up and, depending on the filenames, zsh
> > sometimes segfaults or hangs. 
> It would be nice to have exact recipe how to reproduce a crash preferably 
> starting with zsh -f. I have screen corruption but not (OK I am not sure what 
> sensibly can be done on 3 column width :)

    The above, when run after "zsh -f" and making sure you don't have
any /etc/zshenv not protected by "[[ -o rcs ]]", causes a hangup (not
a crash) in my system. Unfortunately the crash happens randomly, and
seldom. The common case is a hangup, with zsh eating lots of CPU, but
from time to time it crashes.

    With 3 column width you can call "ls" if your PS1 is short enough
XDDDD. BTW, this has happened to me with COLUMNS=100 and a very long
file name, that's why I discovered it. First zsh messed up the
screen, and when I hit <UP> the shell crashed. I haven't been able to
reproduce the crash with that particular example, zsh always hangs
now, but I have had crashes and hangs with the example I provide
above.

> Jokes aside, I do confirm screen corruption during normal completion under 
> konsole and linux console. This happens with long filenames

    Exactly. Any filename whose name length is greater or equal to
$COLUMNS seems to cause the problem.

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...

Re: Bug? in complist when filenames are longer than the screen

[Bart Schaefer]

On Oct 3,  8:48pm, DervishD wrote:
}
}     The above, when run after "zsh -f" and making sure you don't have
} any /etc/zshenv not protected by "[[ -o rcs ]]", causes a hangup (not
} a crash) in my system. Unfortunately the crash happens randomly

See if the following patch reduces the problem to something reproducible.
I don't really know what the problem is, but all of the following are
defensive changes against possible infinite loops.

Index: Src/Zle/complist.c
--- current/Src/Zle/complist.c	Sun Oct  2 18:38:17 2005
+++ Src/Zle/complist.c	Mon Oct  3 19:02:17 2005
@@ -831,7 +831,7 @@
     selectlocalmap(NULL);
     settyinfo(&shttyinfo);
     putc('\r', shout);
-    for (i = columns - 1; i--; )
+    for (i = columns - 1; i-- > 0; )
 	putc(' ', shout);
     putc('\r', shout);
 
@@ -1148,7 +1148,7 @@
 		    if (mselect >= 0) {
 			int mm = (mcols * ml), i;
 
-			for (i = mcols; i--; ) {
+			for (i = mcols; i-- > 0; ) {
 			    mtab[mm + i] = mtmark(NULL);
 			    mgtab[mm + i] = mgmark(NULL);
 			}
@@ -1469,12 +1469,12 @@
 	    int mm = (mcols * ml), i;
 
             if (m->flags & CMF_DUMMY) {
-                for (i = mcols; i--; ) {
+                for (i = mcols; i-- > 0; ) {
                     mtab[mm + i] = mtmark(mp);
                     mgtab[mm + i] = mgmark(g);
                 }
             } else {
-                for (i = mcols; i--; ) {
+                for (i = mcols; i-- > 0; ) {
                     mtab[mm + i] = mp;
                     mgtab[mm + i] = g;
                 }
@@ -1523,12 +1523,12 @@
 	    int mm = mcols * ml, i;
 
             if (m->flags & CMF_DUMMY) {
-                for (i = (width ? width : mcols); i--; ) {
+                for (i = (width ? width : mcols); i-- > 0; ) {
                     mtab[mx + mm + i] = mtmark(mp);
                     mgtab[mx + mm + i] = mgmark(g);
                 }
             } else {
-                for (i = (width ? width : mcols); i--; ) {
+                for (i = (width ? width : mcols); i-- > 0; ) {
                     mtab[mx + mm + i] = mp;
                     mgtab[mx + mm + i] = g;
                 }
@@ -2142,7 +2142,7 @@
 	    Cmatch **p = mtab;
 
 	    for (y = 0; y < mlines; y++) {
-		for (x = mcols; x; x--, p++)
+		for (x = mcols; x > 0; x--, p++)
 		    if (*p && !mmarked(*p) && **p && mselect == (**p)->gnum)
 			break;
 		if (x) {
@@ -2162,7 +2162,7 @@
 	    int c;
 
 	    while (mlbeg) {
-		for (q = p, c = columns; c; q++, c--)
+		for (q = p, c = columns; c > 0; q++, c--)
 		    if (*q && !mmarked(*q))
 			break;
 		if (c)
@@ -2180,7 +2180,7 @@
 	    int c;
 
 	    while (mlbeg < mlines) {
-		for (q = p, c = columns; c; q++, c--)
+		for (q = p, c = columns; c > 0; q++, c--)
 		    if (*q)
 			break;
 		if (c)

PATCH? - Re: Bug? in complist when filenames are longer than the screen

[Bart Schaefer]

On Oct 4,  2:07am, Bart Schaefer wrote:
}
} I don't really know what the problem is

I still don't know what the problem is, but I have a reproducible
infinite loop.  Use a PS1 string at least 10 characters long, and
make the window narrower than 12 characters, so that the first
completion filled in on the command line by menu-completion is
forced to wrap to the next line of the screen.  Then begin doing
menu completion with listing.  When the completion reaches the end
of the list and wraps back to the first entry, zsh goes into an
infinite loop attempting to compute the *line* on which to begin
redrawing.

The following prevents the infinite loop, but is almost certainly
not the best patch.  However, I don't understand this code well
enough to produce a better one.

It should be possible to apply this patch after my "defensive" one,
though all those defenses against looping on the column count were
of no help when the actual loop was on the line count.

--- current/Src/Zle/complist.c	Mon Oct  3 21:15:31 2005
+++ Src/Zle/complist.c	Mon Oct  3 21:25:34 2005
@@ -2154,8 +2154,11 @@
 		mline = y;
 	}
 	while (mline < mlbeg)
-	    if ((mlbeg -= step) < 0)
+	    if ((mlbeg -= step) < 0) {
 		mlbeg = 0;
+		if (mline < 0)
+		    break;
+	    }
 
 	if (mlbeg && lbeg != mlbeg) {
 	    Cmatch **p = mtab + ((mlbeg - 1) * columns), **q;

Re: PATCH? - Re: Bug? in complist when filenames are longer than the screen

[DervishD]

    Hi Bart :)

    First of all, thanks for your patches :)

 * Bart Schaefer <schaefer@brasslantern.com> dixit:
> } I don't really know what the problem is
> I still don't know what the problem is, but I have a reproducible
> infinite loop.  Use a PS1 string at least 10 characters long, and
> make the window narrower than 12 characters, so that the first
> completion filled in on the command line by menu-completion is
> forced to wrap to the next line of the screen.  Then begin doing
> menu completion with listing.  When the completion reaches the end
> of the list and wraps back to the first entry, zsh goes into an
> infinite loop attempting to compute the *line* on which to begin
> redrawing.

    The first patch (the "defensive" patch) does not help here, it
has no effect at least in the way I'm reproducing the problem
(exactly with the code snippet I posted yesterday in the list, using
zsh -f).

    BUT the second patch certainly prevents the infinite loop.
Unfortunately it doesn't avoid the crash nor the screen messup.
I cannot reproduce the crash, unfortunately :(( It happens randomly
after the screen messup, sometimes after the first keypress after the
menu completion ends, other times after playing with the keyboard for
a while :(

    I'm gonna try your third patch and I'll answer about it in that
thread.

    Thanks a lot for being trying to solve my problem.

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...

Re: PATCH? - Re: Bug? in complist when filenames are longer than the screen

[Peter Stephenson]

I haven't been able to see any problem with long prompts, even doing some
fairly hairy things, so I may be using different options.  Are there still
bugs with long prompts or long file names after the patch?

I am aware of a problem, which appears to be long standing, that sometimes
the last line of listing isn't erased properly.  It's fairly minor but some
day someone may understand the code well enough to fix it.

Also, does the long-prompt patch need to be applied to 4.2?


-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com

Re: PATCH? - Re: Bug? in complist when filenames are longer than the screen

[DervishD]

    Hi Peter :)

 * Peter Stephenson <pws@csr.com> dixit:
> I haven't been able to see any problem with long prompts, even doing some
> fairly hairy things, so I may be using different options.  Are there still
> bugs with long prompts or long file names after the patch?

    After the patch, long file names causes screen mess and crashes,
but not infinite loops. Long prompts doesn't seem to be affected,
I've been able to reproduce the problem with and without a very long
PS1. The options doesn't seem to be the culprit, because I've tested
with "zsh -f" and *without* /etc/zshenv.
 
> Also, does the long-prompt patch need to be applied to 4.2?
 
    I first hit the bug in my 4.2.5, so...

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...

Re: PATCH? - Re: Bug? in complist when filenames are longer than the screen

[Peter Stephenson]

DervishD wrote:
>     After the patch, long file names causes screen mess and crashes,
> but not infinite loops.

Still not getting this...

One oddity, which may or may not be related is that I have a description
line at the top 'Completing %d' (via the format style).  When I scroll
down the list so that it's not at the top, that message disappears, and
also the message at the bottom about the position in the listing jumps
up one line.  This may be a completely different bug in the calculation.

pws


This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com

Re: PATCH? - Re: Bug? in complist when filenames are longer than the screen

[DervishD]

    Hi Peter :)

 * Peter Stephenson <pws@csr.com> dixit:
> DervishD wrote:
> >     After the patch, long file names causes screen mess and crashes,
> > but not infinite loops.
> Still not getting this...
> 
> One oddity, which may or may not be related is that I have a
> description line at the top 'Completing %d' (via the format style). 
> When I scroll down the list so that it's not at the top, that
> message disappears, and also the message at the bottom about the
> position in the listing jumps up one line.  This may be a
> completely different bug in the calculation.

    Probably. In the first place, I'm not using "compsys" completion
(for giving it a name...) but "compctl", and I don't have
description lines on top of the completion list.

    The only think I'm inheriting from my /etc/zshenv is PS1, but
I've tested even without it and the problem is the same. With an
empty PS1 the crash is much more difficult to have, and I even doubt
I got any crash at all with an empty PS1 last time I tested Bart's
patches :????

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...

Re: PATCH? - Re: Bug? in complist when filenames are longer than the screen

[Bart Schaefer]

On Oct 4, 11:48am, DervishD wrote:
}
}     The first patch (the "defensive" patch) does not help here, it
} has no effect at least in the way I'm reproducing the problem

Not unexpected; it was sort of a shot in the dark.

}     BUT the second patch certainly prevents the infinite loop.

I'll commit that one, then, with some notation that it's a stopgap and
probably needs further repairs.

} Unfortunately it doesn't avoid the crash nor the screen messup.

It might have helped if you'd reminded us in the first place that you
are not using compsys.  More on this elsewhere.

Re: PATCH? - Re: Bug? in complist when filenames are longer than the screen

[Bart Schaefer]

On Oct 4, 11:38am, Peter Stephenson wrote:
}
} I haven't been able to see any problem with long prompts

Sorry, it's not actually the length of the prompt that matters, it's that
the window is narrow enough that the prompt fills it and completion lists
must thus begin two lines below the prompt instead of one line below.

It may also require that at least one file name is also wider than the
window, or that the listing itself does not fit in the number of lines
remaining (so that complist has to scroll up and down while cycling
through a menu-selection).

} Also, does the long-prompt patch need to be applied to 4.2?

Apparently, though calling it a "long-prompt patch" is a misnomer.

Re: Bug? in complist when filenames are longer than the screen

[Bart Schaefer]

On Oct 3,  4:45pm, DervishD wrote:
}
}     $ touch a b c filename1 filename2 d e
}     $ zmodload zsh/complist
}     $ COLUMNS=3

I didn't look at this all that closely before, but it's not at all
surprising that the display gets messed up when COLUMNS is not the
same as the actual width of the window.  The complist module relies
on the terminal emulator to have wrapped long lines, etc., when they
were output; it does not insert its own line breaks at an imaginary
window boundary.  (ZLE sometimes appears to do so, but it has a lot
of extra smarts regarding terminals that don't auto-scroll, very few
of which smarts were ever incorporated into complist.)

Are you sure you don't mean MENUSELECT=3 in that example?  If you
have never set MENUSELECT, then loading the zsh/complist module is
not going to have any effect, and in this next step ...

}     $ ls <TAB><TAB><TAB><TAB><UP><DOWN>

... the up/down are going to scroll through the history, not through
the completions.  If you're getting a reproducible crash exactly as
described above, then it's not completion that's crashing, and we
need to be looking elsewhere.

Re: PATCH? - Re: Bug? in complist when filenames are longer than the screen

[DervishD]

    Hi Bart :)

 * Bart Schaefer <schaefer@brasslantern.com> dixit:
> On Oct 4, 11:48am, DervishD wrote:
> } Unfortunately it doesn't avoid the crash nor the screen messup.
> It might have helped if you'd reminded us in the first place that you
> are not using compsys.  More on this elsewhere.

    Sorry :(( I supposed that a shell started with -f and without any
/etc/zshenv present didn't have any chance of configuring and
starting compsys. In fact I was not using compctl in my example,
either, because my compctl completions were not loaded.

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...

Re: Bug? in complist when filenames are longer than the screen

[DervishD]

    Hi Bart :)

 * Bart Schaefer <schaefer@brasslantern.com> dixit:
> On Oct 3,  4:45pm, DervishD wrote:
> }     $ touch a b c filename1 filename2 d e
> }     $ zmodload zsh/complist
> }     $ COLUMNS=3
> I didn't look at this all that closely before, but it's not at all
> surprising that the display gets messed up when COLUMNS is not the
> same as the actual width of the window.  The complist module relies
> on the terminal emulator to have wrapped long lines, etc., when
> they were output; it does not insert its own line breaks at an
> imaginary window boundary.  (ZLE sometimes appears to do so, but it
> has a lot of extra smarts regarding terminals that don't
> auto-scroll, very few of which smarts were ever incorporated into
> complist.)

    OK, but I used COLUMNS=3 just to avoid to "touch" very long file
names. The problem is the same here with "COLUMS=100" (the default in
my system) and very long file names, even without a long prompt.

> Are you sure you don't mean MENUSELECT=3 in that example?  If you
> have never set MENUSELECT, then loading the zsh/complist module is
> not going to have any effect, and in this next step ...

    No, I'm sure. When testing, I didn't set up MENUSELECT, but I'm
afraid it got inherited from the login shell :((( with a value of 0.

    I'm going to make another test, using a clean environment. OK,
here are the results with the UNPATCHED version of zsh: if I use very
long names (much longer than COLUMNS), the bug doesn't happen!. It
only happens with a list of files in one directory :??? Mmm,
interesting, it looks like a corner case: the bug only happens when
in the file listing is there a file whose name length is *exactly*
$COLUMNS. Longer or shorter files doesn't cause the bug to happen.
I've tested this with different prompts and in a clean environment
(that is, the login shell doesn't source ANY file, because no RC file
is present).

    I've found the bug to be repeatable with this sequence (try this
in a clean zsh environment, no /etc/zshenv, no any other rcfile, and
it must be a login shell, just to make sure):

    $ rm -rf test
    $ mkdir test ; cd test
    $ PS1=
    $ touch ${(l:$COLUMNS::a:):-} a b c
    $ zmodload zsh/complist
    $ MENUSELECT=0

    It's very important to create the other three files (a, b and c)
in the directory (otherwise you won't get a list ;)).

    This causes a crash, always, without looping, even with your
patches.

> }     $ ls <TAB><TAB><TAB><TAB><UP><DOWN>
> ... the up/down are going to scroll through the history, not through
> the completions.  If you're getting a reproducible crash exactly as
> described above, then it's not completion that's crashing, and we
> need to be looking elsewhere.

    The command above runs through the completions if MENUSELECT is
0, of course. My fault, I should have carried the tests in a fully
clean and default environment, not just "zsh -f", because variables
and the like are inherited. Sorry :(((

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...

Re: Bug? in complist when filenames are longer than the screen

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 3580 bytes --]

On Tuesday 04 October 2005 21:13, DervishD wrote:
[...]

We seem to have two bugs here.

>
>     I'm going to make another test, using a clean environment. OK,
> here are the results with the UNPATCHED version of zsh: if I use very
> long names (much longer than COLUMNS), the bug doesn't happen!. It
> only happens with a list of files in one directory :??? Mmm,
> interesting, it looks like a corner case: the bug only happens when
> in the file listing is there a file whose name length is *exactly*
> $COLUMNS. Longer or shorter files doesn't cause the bug to happen.

Yes, this was claimed to have been fixed at least twice. It has something to 
do with terminals (not) going to next line when getting NL in the last 
position.


>
>     I've found the bug to be repeatable with this sequence (try this
> in a clean zsh environment, no /etc/zshenv, no any other rcfile, and
> it must be a login shell, just to make sure):
>
>     $ rm -rf test
>     $ mkdir test ; cd test
>     $ PS1=
>     $ touch ${(l:$COLUMNS::a:):-} a b c
>     $ zmodload zsh/complist
>     $ MENUSELECT=0
>
>     It's very important to create the other three files (a, b and c)
> in the directory (otherwise you won't get a list ;)).
>
>     This causes a crash, always, without looping, even with your
> patches.
>

Does it happen only in current CVS or in older versions too?

#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7d52ef1 in raise () from /lib/tls/libc.so.6
#2  0xb7d5483b in abort () from /lib/tls/libc.so.6
#3  0xb7d88ff5 in __fsetlocking () from /lib/tls/libc.so.6
#4  0xb7d8f587 in malloc_usable_size () from /lib/tls/libc.so.6
#5  0xb7d8fa02 in free () from /lib/tls/libc.so.6
#6  0xb7bdc272 in unmetafy_line ()
    at /home/bor/src/zsh/Src/Zle/zle_tricky.c:964
#7  0xb7bd4958 in zrefresh () at /home/bor/src/zsh/Src/Zle/zle_refresh.c:366
#8  0xb7bd69c2 in zrefresh () at /home/bor/src/zsh/Src/Zle/zle_refresh.c:769
#9  0xb7b81267 in domenuselect (dummy=0xb7bb8a74, dat=0xbfe015e0)
    at /home/bor/src/zsh/Src/Zle/complist.c:2230
#10 0x08096c37 in runhookdef (h=0xb7bb8a74, d=0xbfe015e0)
    at /home/bor/src/zsh/Src/module.c:1883
#11 0xb7ba0b52 in after_complete (dummy=0xb7bef39c, dat=0xbfe0166c)
    at /home/bor/src/zsh/Src/Zle/compcore.c:513
#12 0x08096c9c in runhookdef (h=0xb7bef39c, d=0xbfe0166c)
    at /home/bor/src/zsh/Src/module.c:1889
#13 0xb7bdbd71 in docomplete (lst=0)
    at /home/bor/src/zsh/Src/Zle/zle_tricky.c:851
#14 0xb7bda7b5 in expandorcomplete (args=0xb7bef5dc)
    at /home/bor/src/zsh/Src/Zle/zle_tricky.c:315
#15 0xb7bcd0e2 in execzlefunc (func=0xb7bed748, args=0xb7bef5dc)
    at /home/bor/src/zsh/Src/Zle/zle_main.c:1094
#16 0xb7bcc602 in zlecore () at /home/bor/src/zsh/Src/Zle/zle_main.c:862
#17 0xb7bcccf2 in zleread (lp=0x80e7940, rp=0x0, flags=3, context=0)
    at /home/bor/src/zsh/Src/Zle/zle_main.c:1016
#18 0x08081527 in inputline () at /home/bor/src/zsh/Src/input.c:278
#19 0x080813dc in ingetc () at /home/bor/src/zsh/Src/input.c:214
#20 0x0807810f in ihgetc () at /home/bor/src/zsh/Src/hist.c:240
#21 0x08088b98 in gettok () at /home/bor/src/zsh/Src/lex.c:628
#22 0x080883d0 in yylex () at /home/bor/src/zsh/Src/lex.c:344
#23 0x080a1d8d in parse_event () at /home/bor/src/zsh/Src/parse.c:451
#24 0x0807e31e in loop (toplevel=1, justonce=0)
    at /home/bor/src/zsh/Src/init.c:128
#25 0x0808101e in zsh_main (argc=2, argv=0xbfe01bf4)
    at /home/bor/src/zsh/Src/init.c:1312
#26 0x08052972 in main (argc=2, argv=0xbfe01bf4)
    at /home/bor/src/zsh/Src/main.c:93

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Bug? in complist when filenames are longer than the screen

[DervishD]

    Hi Andrey :)

 * Andrey Borzenkov <arvidjaar@newmail.ru> dixit:
> On Tuesday 04 October 2005 21:13, DervishD wrote:
> >     I'm going to make another test, using a clean environment. OK,
> > here are the results with the UNPATCHED version of zsh: if I use very
> > long names (much longer than COLUMNS), the bug doesn't happen!. It
> > only happens with a list of files in one directory :??? Mmm,
> > interesting, it looks like a corner case: the bug only happens when
> > in the file listing is there a file whose name length is *exactly*
> > $COLUMNS. Longer or shorter files doesn't cause the bug to happen.
> Yes, this was claimed to have been fixed at least twice. It has
> something to do with terminals (not) going to next line when
> getting NL in the last position.

    That's interesting, because it would explain another problem I'm
having, where the last character of a line exactly $COLUMNS long is
overwritten by the first character of PS1 if it is not followed by a
newline. OTOH, printing such line and printing a character afterwards
leads the correct result, that is, the character is printed in the
following line.
 
> >     I've found the bug to be repeatable with this sequence (try this
> > in a clean zsh environment, no /etc/zshenv, no any other rcfile, and
> > it must be a login shell, just to make sure):
> >
> >     $ rm -rf test
> >     $ mkdir test ; cd test
> >     $ PS1=
> >     $ touch ${(l:$COLUMNS::a:):-} a b c
> >     $ zmodload zsh/complist
> >     $ MENUSELECT=0
> >
> >     It's very important to create the other three files (a, b and c)
> > in the directory (otherwise you won't get a list ;)).
> >
> >     This causes a crash, always, without looping, even with your
> > patches.
>
> Does it happen only in current CVS or in older versions too?

    In fact I don't know if it happens in current CVS, I've just
tested in 4.2.5

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...

Re: Bug? in complist when filenames are longer than the screen

[Bart Schaefer]

On Oct 4, 11:24pm, Andrey Borzenkov wrote:
}
} On Tuesday 04 October 2005 21:13, DervishD wrote:
} [...]
} 
} We seem to have two bugs here.

Yes, but they're closely related.

The *potential* for the infinite loop was introduced way back in
revision 1.6 of complist.c, more than 5 years ago.

} > interesting, it looks like a corner case: the bug only happens when
} > in the file listing is there a file whose name length is *exactly*
} > $COLUMNS. Longer or shorter files doesn't cause the bug to happen.
} 
} Yes, this was claimed to have been fixed at least twice. It has
} something to do with terminals (not) going to next line when getting
} NL in the last position.

I don't think that's actually related to the problem.  It could affect
the display being messed up, but a garbled display is not going to
crash the shell.

What appears to be happening is that a garbage structure is being read
from either mtab[] or mgtab[] (or more likely, both, but it is random
which one contains crash-inducing garbage).  Those are both flat arrays
representing a mapping, for every character position on the display,
of which completion (mtab) and group (mgtab) correspond to the text at
that position.  I say flat because they're not two-dimensional by lines
and columns, rather array index 0 is the top left and index $(($LINES *
$COLUMNS - 1)) is the lower right.

Or, at least, that's my interpretation of what mtab and mgtab are.
There are no comments anywhere, and Sven liked to use extremely short
variable names for the indexes into these arrays.  Note that there are
a lot of NULL elements in these arrays (for screen positions occupied
by spaces or descriptive text).

I think the problem is an interaction among the changes in rev 1.6,
rev 1.12, and rev 1.17, all of which alter the way those tables are
traversed or maintained.  The most likely one is 1.12, which attempts
to prevent the highlighted match from wrapping both horizontally and
vertically at the same time.  The trouble is -- I think -- that when
there is only one column, every time the end of the list is reached
it *appears* to be wrapping both horizontally and vertically (the
destination column never differs from the origin column, as it were).

The commitlog for 1.17 claims it only affects menu scrolling, but I'm
pretty confident from some gdb sessions that the code in the line 1035
hunk [*] and in the line 1678 hunk [*] is being executed in ordinary
menu selection as well.

[*] Line numbers from cvsweb diff display browsing CVS on SourceForge.
These no longer correspond to lines in the current source.

All of this is complicated by the fact that Sven relied on pointers
always referring to even-numbered-byte boundaries, and so bitwise-ORs
the lowest bit with 1 to differentiate "marked" from "unmarked" screen
positions.  This is hopelessly nonportable; some architectures will
crash outright as soon as the bit-flip makes the pointer invalid.
In any case, though, the inifinite loop is caused when scanning the
entire mtab array fails to find any "marked" position at all, and I
suspect (but can't yet be sure) that rev 1.12 is the reason that no
position has been marked when the selection has moved off the end of
the list and ought to wrap back to the start.

Rev 1.17 gets involved because (again: I think, not yet sure) it doesn't
properly distinguish the edge case of whether a line that is exactly the
width of the screen should be considered to "span multiple lines".

If someone else has some spare minutes and eyeballs, please ...

Re: Bug? in complist when filenames are longer than the screen

[DervishD]

    Hi Bart :)

 * Bart Schaefer <schaefer@brasslantern.com> dixit:
> On Oct 4, 11:24pm, Andrey Borzenkov wrote:
> Rev 1.17 gets involved because (again: I think, not yet sure) it doesn't
> properly distinguish the edge case of whether a line that is exactly the
> width of the screen should be considered to "span multiple lines".
> 
> If someone else has some spare minutes and eyeballs, please ...

    I certainly have a couple of spare eyeballs here, and maybe the
spare minutes too, but I'm in no way familiar with zsh source code,
so I don't think that I can discover anything you haven't discovered
yet. In fact I think that you may discover and fix the problem by the
time I end studiying complist.c :( I know it's only 3k lines, but
it's full of one-letter variables and almost no comments, so I would
need much time to read and fully understand it (or maybe not fully
understand, but understand enough to not introduce new and nasty bugs
while trying to fix this one).

    OTOH, what I can for sure do is to test any change you may want
to test, in as many conditions as I can afford, and if I have time
this weekend I can try to pin down the bug with gdb and lots of
patience I suppose.

    Thanks for all your effort, Bart :)

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736 | http://www.dervishd.net
http://www.pleyades.net & http://www.gotesdelluna.net
It's my PC and I'll cry if I want to...