Re: zsh eats 100% CPU with completion in /

[Mikael Magnusson <mikachu@gmail.com>]

2009/11/2 Peter Stephenson <pws@csr.com>:
> On Mon, 2 Nov 2009 02:26:54 +0100
> Mikael Magnusson <mikachu@gmail.com> wrote:
>> Well, I obviously have no idea what I'm doing now, first I just tried
>> breaking on
>> gettok, but it went on and on forever, then i started doing s 100 and
>> it still took
>> a good while until we came to the infinite loop. Then I tried the new reversible
>> debugging thing in gdb 7, breaked on zshlex, then continued backward
>> to gettok, and
>> stepped forward from there. I'm not exactly sure if I went far enough though.
>
> Thanks again.
>
> Somehow you've got tokstr equal to "." rather then either "" or "./x" in
> this case.  That might be consistent---at that point we haven't removed the
> "x", so we might remove what we think is the "x" while it's actually the
> ".", ending up with the "" we see later.  That would imply the end index is
> two too short.
>
> tmp is from zlemetaline:  that's OK, "./", except the "x" has disappeared
> already.  That happens at zle_tricky.c:1259.  I didn't think you'd got that
> far yet, but I'm quite confused about what's actually happening, so it may
> be this is a red herring (it doesn't seem to fit with the other evidence).
>
> The key thing to find out (probably) is how gettokstr() puts together the
> string it's returning, which should be "./x", and how zle fudges it.  There
> are no special characters in that, so we should bypass the switch at
> lex.c:975 each time until we get to the end of the input.  So we should be
> calling add('.'), add('/'), add('x').  I see three calls to add in the
> trace, followed by hitting LX2_BREAK, which would be OK for the end of the
> string but I've no idea if it corresponds to that or not.
>
> Those "*bptr++ = c"s ought to be a help: that's where we're appending to
> tokstr.  I see three of those following by a "*bptr = '\0'", which looks
> right.  It might be worth tracking the value of tokstr after that
> point ("d tokstr" would probably do it).
>
> I think the nasty fix-up-for-zle code in zle is the stuff in exalias()
> around the call to gotwork().  This is grotesque and uncommented, so I'm
> not sure what it's doing, but if it's working it should leave tokstr as
> "./x".  We're returning from inside that code at line 1745, which I presume
> is correct, but I don't know, it's just too opaque.  gotword() should be
> setting "we" (the word end index) to 2 (or maybe 3?  This is full of opaque
> adjustments, too), and "wb" (the word beginning) to 0.
>
> (Hmm... even if tokstr were "./x" an incorrect "we" might screw it up, but
> from other evidence it looks likes it's getting to be "." somehow.)
>
> Anyway, I wouldn't be surprised if the difference from the working case was
> down here, somewhere, with the end index getting fudged wrongly, so if you
> can work out how to break on the right gettok() or gettokstr() you should
> "just" be able to post traces for both cases.
>
> Maybe if we work out what's happening we can even stick some comments in
> the code...

I found something now, but I'm not sure if it's the gdb record log being weird.
It looks like lextok2['/'] is 0, which ends up making tokstr not "."
as we thought, but ".\000x" (maybe). I did try the whole thing without
any recording and this is what I see:

Breakpoint 1, get_comp_string () at zle_tricky.c:1067
1067	    int t0, tt0, i, j, k, cp, rd, sl, ocs, ins, oins, ia, parct, varq = 0;

then a lot of stepping to
1164		ctxtlex();

then
(gdb) print tokstr
$1 = 0x6fb71758 "."
(gdb) print tokstr+1
$2 = 0x6fb71759 ""
(gdb) print tokstr+2
$3 = 0x6fb7175a "x"
(gdb) print tokstr+3
$4 = 0x6fb7175b ""

Here are some other variables
(gdb) print bptr
$10 = 0x6fb7175b ""
(gdb) print bptr-1
$11 = 0x6fb7175a "x"
(gdb) print bptr-2
$12 = 0x6fb71759 ""
(gdb) print bptr-3
$13 = 0x6fb71758 "."
(gdb) print bptr-4
$14 = 0x6fb71757 ""
(gdb) print inbuf
$15 = 0x6fb71750 "./x "
(gdb) print inbufptr
$16 = 0x6fb71753 " "
(gdb) print inbufct
$17 = 1
(gdb) print inbufleft
$18 = 1

Tried that again and stepped into ctxtlex a bit, and
Breakpoint 2, ctxtlex () at lex.c:408
408	    zshlex();
(gdb) print tokstr
$2 = 0x0
(gdb) s
zshlex () at lex.c:361
361	    if (tok == LEXERR)
(gdb) n
364		tok = gettok();
(gdb)
365	    while (tok != ENDINPUT && exalias());
(gdb) print tokstr
$3 = 0x6fb23758 "."
(gdb) print tokstr+1
$4 = 0x6fb23759 ""
(gdb) print tokstr+2
$5 = 0x6fb2375a "x"
(gdb) print tokstr+3
$6 = 0x6fb2375b ""


So I don't think it's exalias() that's breaking it? So far I have no
idea how all this works though, has all this already been called one
(or more) times by the time we call it from get_comp_string()?

Just confirmed ['/'] is set to 0 by doing cd .. in /

Here we are,

(gdb) watch lextok2['/']
Hardware watchpoint 9: lextok2['/']
Continuing.
Hardware watchpoint 9: lextok2['/']

Old value = 47 '/'
New value = 0 '\000'
0x080c962f in xsymlinks (s=0x818e8f1 "..") at utils.c:696
696		    *p = '\0';

(start over and add breakpoints (i'm not in reversible now just to be safe))

692		    if (!strcmp(xbuf, "/"))
(gdb) print xbuf
$22 = '\000' <repeats 8191 times>
(gdb) n
694		    p = xbuf + strlen(xbuf);
(gdb)
695		    while (*--p != '/');
(gdb) n
at this point i pressed ctrl-c because i was curious what was taking
so long. Apparently this loops over all memory until it finds a slash,
which takes a while under gdb. Also apparently, the first / it finds
is in lextok2. Amusingly, ctrl-c and breaking on the next line is
instant, compared to waiting 30 seconds for 'n' to finish.
Breakpoint 11, xsymlinks (s=0x818e8f1 "..") at utils.c:696
696		    *p = '\0';
(gdb) print p
$25 = 0x80e9b6f
"/0123456789:;<=>\225@ABCDEFGHIJKLMNOPQRSTUVWXYZ\217\\]\206_`abcdefghijklmnopqrstuvwxyz\215|}\226\177\200\201\202\203\204\205\206\207\210\211\212\213\214\215\216\217\220\221\222\223\224\225\226\227\230\231\232\233\234\235\236\237\240\241\242\243\244\245\246\247\250\251\252\253\254\255\256\257\260\261\262\263\264\265\266\267\270\271\272\273\274\275\276\277\300\301\302\303\304\305\306\307\310\311\312\313\314\315\316\317\320\321\322\323\324\325\326\327\330\331\332\333\334\335\336\337\340\341\342\343\344\345\346\347\350\351\352\353\354\355\356\357\360\361\362\363\364\365",
<incomplete sequence \366>...

-- 
Mikael Magnusson