globcomplete bug + bug when compctl.so not loaded + init patch

globcomplete bug + bug when compctl.so not loaded + init patch

[Peter Stephenson]

Looks like things go haywire when globcomplete is set with new completion.

% _foo() { complist -k '(one two)'; }
% defcomp _foo foo
% setopt globcomplete
% foo o*e<TAB>

The shell hangs and has to be got rid of with kill -9.  It looks like it's
happening during the complist.  


I tried to reproduce this with zsh -f (and succeeded eventually), but got
another bug --- well two, actually.  First I had to apply the patch at the
end to get the _* files to match properly; plus I needed to fix up some
nullglob flags in init and dump else dump miscounted the number of files; I
also made dump produce an alphabetic order (I never realised `whence' just
dumped the table out without ordering, but maybe I should have).

Then the following happened.

% fpath=(~/bin/comp)
% . ~/bin/comp/init
% setopt glob<TAB>
zsh: 19372 illegal hardware instruction (core dumped)  ./zsh -f

It looks like this is because compctl.so has never been loaded, so that
makecompparamsptr never got defined.  Probably this ought to be checked for
somewhere.

--- Functions/Completion/dump.bk	Fri Feb 19 14:38:12 1999
+++ Functions/Completion/dump	Wed Feb 24 14:39:26 1999
@@ -16,7 +16,7 @@
 
 _d_file=${COMPDUMP-${0:h}/init.dump}
 
-_d_files=( ${^~fpath}/_*~*~ )
+_d_files=( ${^~fpath}/_*~*~(N) )
 
 print "#files: $#_d_files" > $_d_file
 
@@ -26,7 +26,7 @@
 # ensure that a single quote inside a variable is itself correctly quoted.
 
 print "comps=(" >> $_d_file
-for _d_f in ${(k)comps}; do
+for _d_f in ${(ok)comps}; do
     print -r - "'${_d_f//\'/'\\''}'" "'${comps[$_d_f]//\'/'\\''}'"
 done  >> $_d_file
 print ")" >> $_d_file
@@ -66,16 +66,16 @@
 # Autoloads: whence -w produces "_d_foo: function", so look for
 # all functions beginning with `_'.
 
-_d_als=($(whence -wm '_*' |
+_d_als=($(whence -wm '_*' | sort |
 while read -rA _d_line; do
   [[ ${_d_line[2]} = function ]] && print -r - ${_d_line[1]%:}
 done))
 
-# print them out:  about six to a line looks neat
+# print them out:  about five to a line looks neat
 
 while (( $#_d_als )); do
   print -n autoload
-  for (( _i = 0; _i < 6; _i++ )); do
+  for (( _i = 0; _i < 5; _i++ )); do
     if (( $#_d_als )); then
       print -n " $_d_als[1]"
       shift _d_als
--- Functions/Completion/init.bk	Tue Feb 23 14:56:17 1999
+++ Functions/Completion/init	Wed Feb 24 14:39:26 1999
@@ -170,7 +170,12 @@
 
 : ${COMPDUMP:=$0.dump}
 
-_i_files=( ${^~fpath}/_*~*~ )
+if [[ ! -o extendedglob ]]; then
+  _i_noextglob=yes
+  setopt extendedglob
+fi
+
+_i_files=( ${^~fpath}/_*~*~(N) )
 _i_initname=$0
 _i_done=''
 
@@ -185,10 +190,6 @@
   unset _i_line
 fi
 if [[ -z "$_i_done" ]]; then
-  if [[ ! -o extendedglob ]]; then
-    _i_noextglob=yes
-    setopt extendedglob
-  fi
   for _i_dir in $fpath; do
     [[ $_i_dir = . ]] && continue
     for _i_file in $_i_dir/_*~*~(N); do
@@ -222,13 +223,13 @@
       fi
     done
 
-  [[ -z "$_i_noextglob" ]] || unsetopt extendedglob
-
-  unset _i_dir _i_line _i_file _i_tag _i_noextglob
+  unset _i_dir _i_line _i_file _i_tag
 
   # If autodumping was requested, do it now.
 
   (( _i_autodump )) && builtin . ${_i_initname:h}/dump
 fi
 
-unset _i_files _i_initname _i_done _i_autodump
+[[ -z "$_i_noextglob" ]] || unsetopt extendedglob
+
+unset _i_files _i_initname _i_done _i_autodump _i_noextglob

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

RE: globcomplete bug + bug when compctl.so not loaded + init patch

[Andrej Borsenkow]

  First I had to apply the patch at the
> end to get the _* files to match properly; plus I needed to fix up some
> nullglob flags in init and dump else dump miscounted the number
> of files; I
> also made dump produce an alphabetic order (I never realised `whence' just
> dumped the table out without ordering, but maybe I should have).
>


I cannot apply it. I have pws-9 + 5469 patch. What else am I missing?

/andrej

Re: globcomplete bug + bug when compctl.so not loaded + init patch

[Peter Stephenson]

"Andrej Borsenkow" wrote:
> I cannot apply it. I have pws-9 + 5469 patch. What else am I missing?

Those look like the relevant bits, but anyway here's what I think I've got
applied so far.  Unfortunately all my mail disappeared (due to the zsh
equivalent of an explosion in the laboratory whose cause may become
apparent), so it's not easy to check back.  If Sven can't apply it either,
it's probably my fault.


Martin Buchholz: 5448: libc.h can't be included on Debian Linux, so only
include it on NeXT where it's necessary.

Matt: 5330: I've put this back the way it original was.  I hate sed almost
as much as awk.

Sven: 5455: keep track of which matcher specification to use

Sven: 5466: compwid manual for -after and -between

Sven: 5467: expn manual typo

Sven: 5469: init fix and Functions/Completion/_comp_parts

Sven: 5470: new completion conditions didn't handle untokenization
consistently.

Sven: 5471: range code knows not to handle associative arrays

Sven: 5476: quoting of tildes in Functions/Completion/_path_files

Sven: 5483: completeinword fixes

Sven: 5489: control for matching in _path_files and _comp_parts

Sven: 5490: unset test for AA elements when substituting

pws: unposted, see 5503: remove dynamic=no from configure.in when
underscore is needed.

pws: 5508: init and dump, globbing and printing.

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

Re: globcomplete bug + bug when compctl.so not loaded + init patch

[Sven Wischnowsky]

Peter Stephenson wrote:

> I tried to reproduce this with zsh -f (and succeeded eventually), but got
> another bug --- well two, actually.  First I had to apply the patch at the
> end to get the _* files to match properly; plus I needed to fix up some
> nullglob flags in init and dump else dump miscounted the number of files; I
> also made dump produce an alphabetic order (I never realised `whence' just
> dumped the table out without ordering, but maybe I should have).
> 
> Then the following happened.
> 
> % fpath=(~/bin/comp)
> % . ~/bin/comp/init
> % setopt glob<TAB>
> zsh: 19372 illegal hardware instruction (core dumped)  ./zsh -f

I guess it's the same Andrej just reported. The patch below takes some 
code from `bin_zmodload_load' and puts it into a new function
`require_module'. Then `bin_zle_complete' calls this function to try
to make `compctl' loaded. In an environment without dynamically loaded 
modules it just tests if `makecompparamsptr' is NULL.

Bye
 Sven

diff -u os/module.c Src/module.c
--- os/module.c	Wed Feb 24 13:59:11 1999
+++ Src/module.c	Wed Feb 24 15:47:06 1999
@@ -567,6 +567,37 @@
     return m;
 }
 
+/* This ensures that the module with the name given as the second argument
+ * is loaded.
+ * The third argument should be non-zero if the function should complain
+ * about trying to load a module with a full path name in restricted mode.
+ * The last argument should be non-zero if this function should signal an
+ * error if the module is already loaded.
+ * The return value is the module of NULL if the module couldn't be loaded. */
+
+/**/
+Module
+require_module(char *nam, char *module, int res, int test)
+{
+    Module m = NULL;
+    LinkNode node;
+
+    node = find_module(module);
+    if (node && (m = ((Module) getdata(node)))->handle &&
+	!(m->flags & MOD_UNLOAD)) {
+	if (test) {
+	    zwarnnam(nam, "module %s already loaded.", module, 0);
+	    return NULL;
+	}
+    } else if (res && isset(RESTRICTED) && strchr(module, '/')) {
+	zwarnnam(nam, "%s: restricted", module, 0);
+	return NULL;
+    } else
+	return load_module(module);
+
+    return m;
+}
+
 /**/
 void
 add_dep(char *name, char *from)
@@ -963,22 +994,10 @@
 	return 0;
     } else {
 	/* load modules */
-	for (; *args; args++) {
-	    Module m;
-
-	    node = find_module(*args);
-	    if (node && (m = ((Module) getdata(node)))->handle &&
-		!(m->flags & MOD_UNLOAD)) {
-		if (!ops['i']) {
-		    zwarnnam(nam, "module %s already loaded.", *args, 0);
-		    ret = 1;
-		}
-	    } else if (isset(RESTRICTED) && strchr(*args, '/')) {
-		zwarnnam(nam, "%s: restricted", *args, 0);
-		ret = 1;
-	    } else if (!load_module(*args))
+	for (; *args; args++)
+	    if (!require_module(nam, *args, 1, (!ops['i'])))
 		ret = 1;
-	}
+
 	return ret;
     }
 }
diff -u os/Zle/comp1.c Src/Zle/comp1.c
--- os/Zle/comp1.c	Tue Feb 23 11:34:27 1999
+++ Src/Zle/comp1.c	Wed Feb 24 15:52:40 1999
@@ -410,6 +410,7 @@
     cc_first.mask2 = CC_CCCONT;
     compcontext = compcommand = compprefix = compsuffix =
 	compiprefix = NULL;
+    makecompparamsptr = NULL;
     return 0;
 }
 
diff -u os/Zle/zle_thingy.c Src/Zle/zle_thingy.c
--- os/Zle/zle_thingy.c	Tue Feb 23 11:34:29 1999
+++ Src/Zle/zle_thingy.c	Wed Feb 24 15:46:17 1999
@@ -478,6 +478,17 @@
     Thingy t;
     Widget w, cw;
 
+#ifdef DYNAMIC
+    if (!require_module(name, "compctl", 0, 0)) {
+	zerrnam(name, "can't load compctl module", NULL, 0);
+	return 1;
+    }
+#else
+    if (!makecompparamsptr) {
+	zerrnam(name, "compctl module not available", NULL, 0);
+	return 1;
+    }
+#endif
     t = rthingy(args[1]);
     cw = t->widget;
     unrefthingy(t);

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: globcomplete bug + bug when compctl.so not loaded + init patch

[Sven Wischnowsky]

Peter Stephenson wrote:

> Looks like things go haywire when globcomplete is set with new completion.
> 
> % _foo() { complist -k '(one two)'; }
> % defcomp _foo foo
> % setopt globcomplete
> % foo o*e<TAB>
> 
> The shell hangs and has to be got rid of with kill -9.  It looks like it's
> happening during the complist.  

The problem was that the second pointer into the match wasn't modified 
together with the main pointer. This caused a wrong value to be placed 
in one of those cline structs and later this led to an endless loop
(trying to make a negative and decremented value match a positive value).

Bye
 Sven

diff -u os/Zle/zle_tricky.c Src/Zle/zle_tricky.c
--- os/Zle/zle_tricky.c	Wed Feb 24 15:54:41 1999
+++ Src/Zle/zle_tricky.c	Wed Feb 24 16:07:43 1999
@@ -4311,6 +4311,7 @@
 		strcat(tt, s);
 		if (lpsuf)
 		    strcat(tt, lpsuf);
+		e += (tt - s);
 		untokenize(s = tt);
 		sl = strlen(s);
 	    }

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: globcomplete bug + bug when compctl.so not loaded + init patch

[Peter Stephenson]

Sven Wischnowsky wrote:
> > Looks like things go haywire when globcomplete is set with new completion.
> 
> The problem was that the second pointer into the match wasn't modified 
> together with the main pointer. This caused a wrong value to be placed 
> in one of those cline structs and later this led to an endless loop
> (trying to make a negative and decremented value match a positive value).

This doesn't seem to fix it for me; there's still something wrong in
addmatch() on the same test:

% setopt globcomplete
% setopt glob*e<TAB>

e is still not being set, so ee is wrong, so the second call to str_cline
after `if (ee != ss + sl || (lpsuf && *lspuf))' hangs.

Further up, the code is avoiding the whole file handling branch and
following the

  test = domatch(s, patcomp, 0)

branch of the next main condition around 4344, and is not following the

  if (!test && lpre && lpsuf) {

around 4359, so there's no opportunity for e to be set.

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

Re: globcomplete bug + bug when compctl.so not loaded + init patch

[Sven Wischnowsky]

Peter Stephenson wrote:

> "Andrej Borsenkow" wrote:
> > I cannot apply it. I have pws-9 + 5469 patch. What else am I missing?
> 
> Those look like the relevant bits, but anyway here's what I think I've got
> applied so far.  Unfortunately all my mail disappeared (due to the zsh
> equivalent of an explosion in the laboratory whose cause may become
> apparent), so it's not easy to check back.  If Sven can't apply it either,
> it's probably my fault.

I could, and I think the relevant patch is 5491 (and probably 5489).

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: globcomplete bug + bug when compctl.so not loaded + init patch

[Sven Wischnowsky]

Peter Stephenson wrote:

> This doesn't seem to fix it for me; there's still something wrong in
> addmatch() on the same test:
> 
> % setopt globcomplete
> % setopt glob*e<TAB>

Sorrysorrysorry, tested it only with files (ahem).

Bye
 Sven

--- os/Zle/zle_tricky.c	Wed Feb 24 16:09:57 1999
+++ Src/Zle/zle_tricky.c	Thu Feb 25 13:08:49 1999
@@ -4272,8 +4272,10 @@
 	    mpl = fpl; msl = fsl;
 	} else {
 	    if ((cp = filecomp)) {
-		if ((test = domatch(s, filecomp, 0)))
+		if ((test = domatch(s, filecomp, 0))) {
+		    e = s + sl;
 		    cc = 1;
+		}
 	    } else {
 		e = s + sl - fsl;
 		if ((test = !strncmp(s, fpre, fpl)))
@@ -4340,9 +4342,10 @@
 		  ((addwhat & CC_EXCMDS)  && !(hn->flags & DISABLED)))) ||
 		((addwhat & CC_BINDINGS) && !(hn->flags & DISABLED))))) {
 	if (sl >= rpl + rsl || mstack) {
-	    if (cp)
+	    if (cp) {
 		test = domatch(s, patcomp, 0);
-	    else {
+		e = s + sl;
+	    } else {
 		e = s + sl - rsl;
 		if ((test = !strncmp(s, rpre, rpl)))
 		    if ((test = !strcmp(e, rsuf))) {

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de