PATCH: 3.1.6-pws-6: $compstate[unambiguous] could be garbage

PATCH: 3.1.6-pws-6: $compstate[unambiguous] could be garbage

[Bart Schaefer]

Earlier today I started noticing

_main_complete: 95: bad set of key/value pairs for associative array

Line 95 is copying the entire $compstate assoc int $lastcomp.  I eventually
tracked the problem to the value of `unambiguous' which it seemed was being
dropped from the ${(@kv)compstate} expansion.  Turns out the reason it was
being dropped is that it pointed to a garbage string because unambig_data()
was being called twice and was freeing and reallocating the cached value
each time; the string returned the first time ended up pointing into free
memory.

I don't know whether the `|| !ainfo' is necessary in the patch below, but
it seemed safest to have it there.  It was always nonzero in the bug case,
but I didn't go through every possible call to the function.

Index: Src/Zle/zle_tricky.c
===================================================================
@@ -7963,7 +7963,7 @@
 	    scache = cline_str((ainfo->count ? ainfo->line : fainfo->line),
 			       0, &ccache);
 	}
-    } else {
+    } else if (mnum != unambig_mnum || !ainfo || !scache) {
 	zsfree(scache);
 	scache = ztrdup("");
 	ccache = 0;

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: PATCH: 3.1.6-pws-6: $compstate[unambiguous] could be garbage

[Clint Adams]

> -    } else {
> +    } else if (mnum != unambig_mnum || !ainfo || !scache) {

Still getting things like

_main_complete: bad pattern: x['^H8T^V@ [92]
_main_complete: bad pattern: (\M-d^P^H8T^V@-print= [92]
_main_complete: bad pattern: (^N"^H8T^V@ [92]

when trying to complete a2ps --pretty-print=

Re: PATCH: 3.1.6-pws-6: $compstate[unambiguous] could be garbage

[Bart Schaefer]

On Oct 7,  1:30pm, Clint Adams wrote:
> Subject: Re: PATCH: 3.1.6-pws-6: $compstate[unambiguous] could be garbage
> > -    } else {
> > +    } else if (mnum != unambig_mnum || !ainfo || !scache) {
> 
> Still getting things like
> 
> _main_complete: bad pattern: x['^H8T^V@ [92]

Well, that certainly does appear that it could be caused by this or a
similar problem.  `unambiguous' and `unambiguous_cursor' are the only keys
that have readonly computed values, and as it turns out they're both
computed by the same caching function, which is what caused the bug I saw
in the first place.

So if you get a debugger going and put a breakpoint in unambig_data() and
manage to see it free and reallocate the `scache' pointer twice, let us
know what `mnum', `unambig_mnum', and `ainfo' were when it happened.

One thing I tried while figuring out what was going on for the patch above
was inserting

noglob print -l x"${(@kv)^compstate}"x

immediately before line 92.  That would at least tell you which assoc key
has the bad value.

Re: PATCH: 3.1.6-pws-6: $compstate[unambiguous] could be garbage

[Clint Adams]

> One thing I tried while figuring out what was going on for the patch above
> was inserting
> 
> noglob print -l x"${(@kv)^compstate}"x
> 
> immediately before line 92.  That would at least tell you which assoc key
> has the bad value.

Beware, this is long:

zsh/2 7 % a2ps --prexnormal_nmatchesx
x1x
xnmatchesx
x1x
xrestorex
xautox
xcontextx
xcommandx
xvaredx
xx
xunambiguousx
x--pretty-printx
xlist_maxx
x100x
xunambiguous_cursorx
x15x
xlistx
xambiguousx
xtotal_matchersx
x0x
xexactx
xacceptx
xto_endx
xmatchx
xlast_promptx
xyesx
xpattern_insertx
xmenux
xmatcher_stringx
xx
xmatcherx
x0x
xinsertx
xmenux
tty-print=

and then

tty-print=xnormal_nmatchesx
x0x
xnmatchesx
x0x
xpattern_matchx
x*x
xrestorex
xautox
xcontextx
xcommandx
xvaredx
xx
xunambiguousx
xx
xlist_maxx
x100x
xunambiguous_cursorx
x1x
xlistx
xambiguousx
xtotal_matchersx
x0x
xexactx
xacceptx
xto_endx
xmatchx
xlast_promptx
xyesx
xpattern_insertx
xmenux
xmatcher_stringx
xx
xmatcherx
x0x
xinsertx
xmenux

and then

xnormal_nmatchesx
x0x
xnmatchesx
x0x
xpattern_matchx
x*x
xrestorex
xautox
xcontextx
xcommandx
xvaredx
xx
xunambiguousx
xx
xlist_maxx
x100x
xunambiguous_cursorx
x1x
xlistx
xambiguousx
xtotal_matchersx
x0x
xexactx
xacceptx
xto_endx
xmatchx
xlast_promptx
xyesx
xpattern_insertx
xmenux
xmatcher_stringx
xx
xmatcherx
x0x
xinsertx
xmenux

and then

xnormal_nmatchesx
x0x
xnmatchesx
x0x
xpattern_matchx
x*x
xrestorex
xautox
xcontextx
xcommandx
xvaredx
xx
xunambiguousx
x0Tu@0Tu@ox  <-----------------------------
xlist_maxx
x100x
xunambiguous_cursorx
x1x
xlistx
xambiguousx
xtotal_matchersx
x0x
xexactx
xacceptx
xto_endx
xmatchx
xlast_promptx
xyesx
xpattern_insertx
xmenux
xmatcher_stringx
xx
xmatcherx
x0x
xinsertx
xmenux


And from gdb

a2ps --pre<TAB>

Breakpoint 1, unambig_data (cp=0x0) at zle_tricky.c:7956
7956                zsfree(scache);
(gdb) display mnum
1: mnum = 1
(gdb) display unambig_mnum
2: unambig_mnum = -1
(gdb) display ainfo
3: ainfo = 0x8103020
(gdb) display scache
4: scache = 0x0

(gdb) continue
Continuing.
tty-print=

[dozens of <TAB>s and nothing]


And again with breakpoint set to two lines earlier:

zsh/2 1 % a2ps --pre
Breakpoint 1, unambig_data (cp=0x0) at zle_tricky.c:7954
7954        if (mnum && ainfo) {
3: ainfo = 0x8103020
2: unambig_mnum = -1
1: mnum = 1
(gdb) display scache
4: scache = 0x0
(gdb) 
(gdb) continue
Continuing.

Breakpoint 1, unambig_data (cp=0xbfffe6d4) at zle_tricky.c:7954
7954        if (mnum && ainfo) {
4: scache = 0x823d650 "--pretty-print"
3: ainfo = 0x8103020
2: unambig_mnum = 1
1: mnum = 1
(gdb) continue
Continuing.
tty-print=
[TAB]
Breakpoint 1, unambig_data (cp=0x0) at zle_tricky.c:7954
7954        if (mnum && ainfo) {
4: scache = 0x823d650 "--pretty-print"
3: ainfo = 0x8103038
2: unambig_mnum = -1
1: mnum = 0
(gdb) continue
Continuing.

Breakpoint 1, unambig_data (cp=0xbfffeba4) at zle_tricky.c:7954
7954        if (mnum && ainfo) {
4: scache = 0x81232f0 ""
3: ainfo = 0x8103038
2: unambig_mnum = 0
1: mnum = 0
(gdb) continue
Continuing.
[TAB]

Breakpoint 1, unambig_data (cp=0x0) at zle_tricky.c:7954
7954        if (mnum && ainfo) {
4: scache = 0x81232f0 ""
3: ainfo = 0x8103050
2: unambig_mnum = -1
1: mnum = 0
(gdb) continue
Continuing.

Breakpoint 1, unambig_data (cp=0xbfffeba4) at zle_tricky.c:7954
7954        if (mnum && ainfo) {
4: scache = 0x8122f70 ""
3: ainfo = 0x8103050
2: unambig_mnum = 0
1: mnum = 0
(gdb) continue
Continuing.

Breakpoint 1, unambig_data (cp=0x0) at zle_tricky.c:7954
7954        if (mnum && ainfo) {
4: scache = 0x8122f70 ""
3: ainfo = 0x8103068
2: unambig_mnum = -1
1: mnum = 0

and so on.

I now conclude that I have been stupid and failed to install zle.so after
patching and that the corruption no longer seems to occur.

Nevertheless, --pretty-print still doesn't complete.  See next message.