Re: Weird bug with approximate completion

Re: Weird bug with approximate completion

[Sven Wischnowsky]

Peter Stephenson wrote:

> Sven Wischnowsky wrote:
> >   sticking it after the first slash will make the completion code use
> >   `/users/foo/(#a...)bin/comp' as the expanded prefix and make it
> >   search in that directory -- which doesn't exists, so it will
> >   generate no matches; should we have some special casing in the
> >   C-code or in the shell code here?
> 
> I don't really understand this since I don't know at what point it hasn't
> been treated as a pattern when it should have been.

When doing pattern completion with a pathname the completion code
still looks only at the pathname component the cursor is in. So it
will split the word into `~/(#a...)/bin/comp' and the prefix
`set'. The it tries to get the real path for the first one and that
gets expanded to `/users/foo/(#a...)/bin/comp', with the `(#a...)'
still in place. Later it uses this path to generate possible matches
by trying to open the directory and reading it. But since there is no
directory with a weird name like `...(#a..)...' it can't open it and
produces no matches.

> > I couldn't reproduce your bug, though (and for me the line you quoted
> > is line number 6239).
> 
> It's now moved to line 6238.  I've fixed the symptoms by turning the
> ncalloc() into a zhalloc(), which I shouldn't have to do, but can't hurt
> since the memory is never freed so it should be using zhalloc() anyway.
> This suggests something fairly sinister is going on.  ncalloc() appears to
> be pointing to the right thing, but the wrong argument appears when it's
> called.

Maybe we should put the test below into tricky.c. One of the things on 
my list is sticking several calls to `DPUTS' in there anyway...

Bye
 Sven

diff -u os/Zle/zle_tricky.c Src/Zle/zle_tricky.c
--- os/Zle/zle_tricky.c	Thu Mar  4 10:59:54 1999
+++ Src/Zle/zle_tricky.c	Thu Mar  4 11:15:13 1999
@@ -5930,6 +5930,8 @@
     if (incompfunc != 1 && findnode(ccstack, cc))
 	return;
 
+    MUSTUSEHEAP("complistflags");
+
     addlinknode(ccstack, cc);
 
     if (incompfunc != 1 && allccs) {

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: Weird bug with approximate completion

[Peter Stephenson]

Sven Wischnowsky wrote:
>   sticking it after the first slash will make the completion code use
>   `/users/foo/(#a...)bin/comp' as the expanded prefix and make it
>   search in that directory -- which doesn't exists, so it will
>   generate no matches; should we have some special casing in the
>   C-code or in the shell code here?

I don't really understand this since I don't know at what point it hasn't
been treated as a pattern when it should have been.

> I couldn't reproduce your bug, though (and for me the line you quoted
> is line number 6239).

It's now moved to line 6238.  I've fixed the symptoms by turning the
ncalloc() into a zhalloc(), which I shouldn't have to do, but can't hurt
since the memory is never freed so it should be using zhalloc() anyway.
This suggests something fairly sinister is going on.  ncalloc() appears to
be pointing to the right thing, but the wrong argument appears when it's
called.

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

Re: Weird bug with approximate completion

[Sven Wischnowsky]

Peter Stephenson wrote:

> I'm getting a weird bug with approximate completion:  the shell exits with
> an out-of-memory error.  One command that did this (there were more) was
> % echo ~/bin/comp/set<TAB>
> where the real file is ~/bin/comp/_setopt.
> 
> It looks like zhalloc() is being called with one argument and getting
> another much larger one.  It could be specific to AIX 3.2, and related to
> the fact that my stack backtrace is now 97 frames deep (but another time it
> was `only' 88).  No fewer then 12 of those (10 in the other case) are
> runshfunc(), because of the effect of wrappers.  The limits are all large,
> e.g. stacksize is 32MB.  Maybe it would be nice to flatten out the
> hierarchy for calls to shell functions a bit.
> 
> But it's strange it only happens when correcting, so I can't be sure it
> isn't a shell problem.  I couldn't get it to happen as such on HPUX or
> IRIX, but I did see something weird with correction on IRIX, lots of 255
> characters appeared (which isn't even German).  So maybe there is some
> corruption coming from the shell.  Looks rather subtle though.  In that
> case it may be useful to know that the call to zhalloc() that goes wrong is
> always from
> 
> 		p = (char *)ncalloc(lpl + lsl + 3);
> 
> in make complistflags() around line 6192.  The arguments lpl and lsl
> themselves seem to be sensible, so it's nothing simple.  One clue might be
> that I don't use pattern matching much apart from correction, and this is
> in a (ispattern & 1) section (fairly innocuous looking, however, and
> ordinary glob completion doesn't seem to do this).

Problem #1 (patched):
  `rpre' should be set to the expanded prefix even if called from a
  completion widget

Problem #2:
  sticking it after the first slash will make the completion code use
  `/users/foo/(#a...)bin/comp' as the expanded prefix and make it
  search in that directory -- which doesn't exists, so it will
  generate no matches; should we have some special casing in the
  C-code or in the shell code here?

Problem #3 (patched):
  due to #2, with automatic correction, the `(#a...)' should not be
  put at the beginning of the string if it starts with `~'; this would
  lead to `(#a1)~/bin/comp/set' which matches files in the current
  directory whose name is only one character long (in a certain sense
  this is correct)

I couldn't reproduce your bug, though (and for me the line you quoted
is line number 6239). The bugs fixed by the patch made this fail for
me, but I didn't get any strange allocation behaviour. The 255 you
mention, btw, suggest a memory problem, too, since (at least with
debugging) newly allocated memory is filled with `-1's.

Bye
 Sven

diff -u os/Zle/zle_tricky.c Src/Zle/zle_tricky.c
--- os/Zle/zle_tricky.c	Wed Mar  3 12:51:36 1999
+++ Src/Zle/zle_tricky.c	Wed Mar  3 17:09:41 1999
@@ -6067,7 +6067,7 @@
     noreal = !delit;
     for (p = lpre; *p && *p != String && *p != Tick; p++);
     tt = ic && !ispar ? lpre + 1 : lpre;
-    rpre = (*p || (!incompfunc && (*lpre == Tilde || *lpre == Equals))) ?
+    rpre = (*p || *lpre == Tilde || *lpre == Equals) ?
 	(noreal = 0, getreal(tt)) :
 	dupstring(tt);
 
@@ -6086,6 +6086,7 @@
 	    if (!s1)
 		s1 = p;
 	}
+    rsl = strlen(rsuf);
     for (s2 = NULL, sf2 = t = 0, p = rsuf; *p && (!t || !sf2); p++)
 	if (itok(*p))
 	    t |= sf2 ? 4 : 2;
diff -u -r oc/Core/_main_complete Completion/Core/_main_complete
--- oc/Core/_main_complete	Wed Mar  3 13:00:41 1999
+++ Completion/Core/_main_complete	Wed Mar  3 16:41:18 1999
@@ -104,7 +104,7 @@
 	# original string as a possible match, let it not be shown in
 	# the list, and probably display the `CCPROMPT'.
 
-        (( $+CCORIG )) && builtin compadd -n - "$PREFIX$SUFFIX"
+        (( $+CCORIG )) && builtin compadd -nQ - "$PREFIX$SUFFIX"
 
 	# If you always want to see the list of possible corrections,
 	# set `compstate[list]=list' here.
@@ -145,7 +145,11 @@
       # ignored prefix).
 
       compadd() {
-        PREFIX="(#a${_comp_correct})$PREFIX"
+        if [[ "$PREFIX" = \~*/* ]]; then
+	  PREFIX="${PREFIX%%/*}/(#a${_comp_correct})${PREFIX#*/}"
+	else
+          PREFIX="(#a${_comp_correct})$PREFIX"
+	fi
 	if (( $+CCPROMPT )); then
 	  builtin compadd -X "$CCPROMPT" -J _correct "$@"
 	else
@@ -153,7 +157,11 @@
 	fi
       }
       compgen() {
-        PREFIX="(#a${_comp_correct})$PREFIX"
+        if [[ "$PREFIX" = \~*/* ]]; then
+	  PREFIX="${PREFIX%%/*}/(#a${_comp_correct})${PREFIX#*/}"
+	else
+          PREFIX="(#a${_comp_correct})$PREFIX"
+	fi
 	if (( $+CCPROMPT )); then
 	  builtin compgen "$@" -X "$CCPROMPT" -J _correct
 	else

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Weird bug with approximate completion

[Peter Stephenson]

I'm getting a weird bug with approximate completion:  the shell exits with
an out-of-memory error.  One command that did this (there were more) was
% echo ~/bin/comp/set<TAB>
where the real file is ~/bin/comp/_setopt.

It looks like zhalloc() is being called with one argument and getting
another much larger one.  It could be specific to AIX 3.2, and related to
the fact that my stack backtrace is now 97 frames deep (but another time it
was `only' 88).  No fewer then 12 of those (10 in the other case) are
runshfunc(), because of the effect of wrappers.  The limits are all large,
e.g. stacksize is 32MB.  Maybe it would be nice to flatten out the
hierarchy for calls to shell functions a bit.

But it's strange it only happens when correcting, so I can't be sure it
isn't a shell problem.  I couldn't get it to happen as such on HPUX or
IRIX, but I did see something weird with correction on IRIX, lots of 255
characters appeared (which isn't even German).  So maybe there is some
corruption coming from the shell.  Looks rather subtle though.  In that
case it may be useful to know that the call to zhalloc() that goes wrong is
always from

		p = (char *)ncalloc(lpl + lsl + 3);

in make complistflags() around line 6192.  The arguments lpl and lsl
themselves seem to be sensible, so it's nothing simple.  One clue might be
that I don't use pattern matching much apart from correction, and this is
in a (ispattern & 1) section (fairly innocuous looking, however, and
ordinary glob completion doesn't seem to do this).

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy