PATCH: dynamic named directories

PATCH: dynamic named directories

[Peter Stephenson]

Problem: six feet deep in WiFi and UWB bugs to fix.  Solution:  do
something else entirely.

This morning I had a brainwave/brainstorm that turned out to be easy to
implement and test without interfering with the rest of the shell, which
is the sort of feature I like.  I'm sure you'll tell me if it's a stupid
idea.

It's based on the way I use an Emacs function suite called "bufname"
that a colleague and I wrote a few years ago.  I have lots of parallel
directory hierarchies with slightly different names at different levels,
so normal static directory naming isn't really flexible enough.

Now you can use ~[<stuff>] and <stuff> gets passed down to a function
zsh_directory_name, which either parses it and picks an appropriate
directory or refuses.  (If the function isn't defined this syntax is
treated the way it always was, which typically wasn't all that useful.)
There's also a mechanism for going the other way.  See documentation and
examples.

Much of the patch is actually another simplification to doshfunc() which
was obvious when I worked out what the third argument was for.  It
should mean any PM_* flags applicable to functions (such as PM_TAGGED)
are handled more consistently.

Index: Doc/Zsh/expn.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/expn.yo,v
retrieving revision 1.91
diff -u -r1.91 expn.yo
--- Doc/Zsh/expn.yo	16 Jun 2008 16:39:18 -0000	1.91
+++ Doc/Zsh/expn.yo	25 Sep 2008 12:39:43 -0000
@@ -1308,8 +1308,73 @@
 option exchanges the effects of `tt(~PLUS())' and `tt(~-)' where they are
 followed by a number.
 
-cindex(directories, named)
-cindex(named directories)
+subsect(Dynamic named directories)
+cindex(directories, named, dynamic)
+cindex(named directories, dynamicic)
+cindex(dynamic named directories)
+
+The feature described here is only available if the shell function
+tt(zsh_directory_name) exists.
+
+A `tt(~)' followed by a string var(namstr) in unquoted square brackets is
+treated specially as a dynamic directory name.  Note that the first
+unquoted closing square bracket always terminates var(namstr).  The shell
+function is passed two arguments: the string tt(n) (for name) and
+var(namstr).  It should either set the array tt(reply) to a single element
+which is the directory corresponding to the name and return status zero
+(executing an assignment as the last statement is usually sufficient), or
+it should return status non-zero.  In the former case the element of reply
+is used as the directory; in the latter case the substitution is deemed to
+have failed and tt(NOMATCH) handling is applied if the option is set.
+
+The function tt(zsh_directory_name) is also used to see if a directory can
+be turned into a name, for example when printing the directory stack or
+when expanding tt(%~) in prompts.  In this case the function is passed two
+arguments: the string tt(d) (for directory) and the candidate for dynamic
+naming.  The function should either return non-zero status, if the
+directory cannot be named by the function, or it should set the array reply
+to consist of two elements: the first is the dynamic name for the directory
+(as would appear within `tt(~[)var(...)tt(])'), and the second is the
+prefix length of the directory to be replaced.  For example, if the trial
+directory is tt(/home/myname/src/zsh) and the dynamic name for
+tt(/home/myname/src) (which has 16 characters) is tt(s), then the function
+sets
+
+example(reply=(s 16))
+
+The directory name so returned is compared with possible static names for
+parts of the directory path, as described below; it is used if the prefix
+length matched (16 in the example) is longer than that matched by any
+static name.
+
+As a working example, here is a function that expands any dynamic names
+beginning with the string tt(p:) to directories below
+tt(/home/pws/perforce).  In this simple case a static name for the
+directory would be just as effective.
+
+example(zsh_directory_name() {
+  emulate -L zsh
+  setopt extendedglob
+  local -a match mbegin mend
+  if [[ $1 = d ]]; then
+    if [[ $2 = (#b)(/home/pws/perforce/)([^/]##)* ]]; then
+      typeset -ga reply
+      reply=(p:$match[2] $(( ${#match[1]} + ${#match[2]} )) )
+    else
+      return 1
+    fi
+  else
+    [[ $2 != (#b)p:(?*) ]] && return 1
+    typeset -ga reply
+    reply=(/home/pws/perforce/$match[1])
+  fi
+  return 0
+})
+
+subsect(Static named directories)
+cindex(directories, named, static)
+cindex(named directories, static)
+cindex(static named directories)
 A `tt(~)' followed by anything not already covered is looked up as a
 named directory, and replaced by the value of that named directory if found.
 Named directories are typically home directories for users on the system.
@@ -1329,6 +1394,8 @@
 except when the directory is tt(/) itself.  The parameters tt($PWD) and
 tt($OLDPWD) are never abbreviated in this fashion.
 
+subsect(`=' expansion)
+
 If a word begins with an unquoted `tt(=)'
 and the tt(EQUALS) option is set,
 the remainder of the word is taken as the
@@ -1336,6 +1403,8 @@
 exists by that name, the word is replaced
 by the full pathname of the command.
 
+subsect(Notes)
+
 Filename expansion is performed on the right hand side of a parameter
 assignment, including those appearing after commands of the
 tt(typeset) family.  In this case, the right hand side will be treated
@@ -1349,6 +1418,7 @@
 argument in the form `var(identifier)tt(=)var(expression)' becomes eligible
 for file expansion as described in the previous paragraph.  Quoting the
 first `tt(=)' also inhibits this.
+
 texinode(Filename Generation)()(Filename Expansion)(Expansion)
 sect(Filename Generation)
 cindex(filename generation)
Index: Src/exec.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/exec.c,v
retrieving revision 1.155
diff -u -r1.155 exec.c
--- Src/exec.c	22 Sep 2008 17:32:43 -0000	1.155
+++ Src/exec.c	25 Sep 2008 12:39:43 -0000
@@ -518,7 +518,7 @@
 	return 127;
 
     pushnode(args, arg0);
-    return doshfunc(shf, args, shf->node.flags, 1);
+    return doshfunc(shf, args, 1);
 }
 
 /* execute an external command */
@@ -4064,7 +4064,7 @@
     cmdsp = 0;
     if ((osfc = sfcontext) == SFC_NONE)
 	sfcontext = SFC_DIRECT;
-    doshfunc(shf, args, shf->node.flags, 0);
+    doshfunc(shf, args, 0);
     sfcontext = osfc;
     free(cmdstack);
     cmdstack = ocs;
@@ -4191,8 +4191,6 @@
  * in which the first element is the function name (even if
  * FUNCTIONARGZERO is set as this is handled inside this function).
  *
- * flags are a set of the PM_ flags associated with the function.
- *
  * If noreturnval is nonzero, then reset the current return
  * value (lastval) to its value before the shell function
  * was executed.  However, in any case return the status value
@@ -4202,13 +4200,14 @@
 
 /**/
 mod_export int
-doshfunc(Shfunc shfunc, LinkList doshargs, int flags, int noreturnval)
+doshfunc(Shfunc shfunc, LinkList doshargs, int noreturnval)
 {
     char **tab, **x, *oargv0;
     int oldzoptind, oldlastval, oldoptcind, oldnumpipestats, ret;
     int *oldpipestats = NULL;
     char saveopts[OPT_SIZE], *oldscriptname = scriptname;
     char *name = shfunc->node.nam;
+    int flags = shfunc->node.flags;
     char *fname = dupstring(name);
     int obreaks, saveemulation ;
     Eprog prog;
Index: Src/math.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/math.c,v
retrieving revision 1.34
diff -u -r1.34 math.c
--- Src/math.c	16 Sep 2008 15:07:12 -0000	1.34
+++ Src/math.c	25 Sep 2008 12:39:43 -0000
@@ -872,7 +872,7 @@
 			if (!shfunc)
 			    zerr("no such function: %s", shfnam);
 			else {
-			    doshfunc(shfunc, l, 0, 1);
+			    doshfunc(shfunc, l, 1);
 			    return lastmathval;
 			}
 		    } else {
Index: Src/signals.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/signals.c,v
retrieving revision 1.51
diff -u -r1.51 signals.c
--- Src/signals.c	16 Sep 2008 15:07:24 -0000	1.51
+++ Src/signals.c	25 Sep 2008 12:39:43 -0000
@@ -1160,7 +1160,7 @@
 	trapisfunc = isfunc = 1;
 
 	sfcontext = SFC_SIGNAL;
-	doshfunc((Shfunc)sigfn, args, 0, 1);
+	doshfunc((Shfunc)sigfn, args, 1);
 	sfcontext = osc;
 	freelinklist(args, (FreeFunc) NULL);
 	zsfree(name);
Index: Src/subst.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/subst.c,v
retrieving revision 1.86
diff -u -r1.86 subst.c
--- Src/subst.c	1 Sep 2008 20:18:48 -0000	1.86
+++ Src/subst.c	25 Sep 2008 12:39:45 -0000
@@ -528,7 +528,8 @@
     char *str = *namptr;
 
     if (*str == Tilde && str[1] != '=' && str[1] != Equals) {
-	char *ptr;
+	Shfunc dirfunc;
+	char *ptr, *tmp, *res;
 	int val;
 
 	val = zstrtol(str + 1, &ptr, 10);
@@ -539,9 +540,23 @@
 	    *namptr = dyncat(pwd, str + 2);
 	    return 1;
 	} else if (str[1] == '-' && isend(str[2])) {   /* ~- */
-	    char *tmp;
 	    *namptr = dyncat((tmp = oldpwd) ? tmp : pwd, str + 2);
 	    return 1;
+	} else if (str[1] == Inbrack &&
+		   (dirfunc = getshfunc("zsh_directory_name")) &&
+		   (ptr = strchr(str+2, Outbrack))) {
+	    char **arr;
+	    untokenize(tmp = dupstrpfx(str+2, ptr - (str+2)));
+	    remnulargs(tmp);
+	    arr = subst_string_by_func(dirfunc, "n", tmp);
+	    res = arr ? *arr : NULL;
+	    if (res) {
+		*namptr = dyncat(res, ptr+1);
+		return 1;
+	    }
+	    if (isset(NOMATCH))
+		zerr("no directory expansion: ~[%s]", tmp);
+	    return 0;
 	} else if (!inblank(str[1]) && isend(*ptr) &&
 		   (!idigit(str[1]) || (ptr - str < 4))) {
 	    char *ds;
Index: Src/utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
retrieving revision 1.200
diff -u -r1.200 utils.c
--- Src/utils.c	16 Sep 2008 15:07:28 -0000	1.200
+++ Src/utils.c	25 Sep 2008 12:39:45 -0000
@@ -826,6 +826,7 @@
 {
     static struct nameddir homenode = { {NULL, "", 0}, NULL, 0 };
     static int ffsz;
+    Shfunc func = getshfunc("zsh_directory_name");
 
     /* Invalidate directory cache if argument is NULL.  This is called *
      * whenever a node is added to or removed from the hash table, and *
@@ -841,7 +842,8 @@
 	return finddir_last = NULL;
     }
 
-    if(!strcmp(s, finddir_full) && *finddir_full)
+    /* It's not safe to use the cache while we have function transformations.*/
+    if(!func && !strcmp(s, finddir_full) && *finddir_full)
 	return finddir_last;
 
     if ((int)strlen(s) >= ffsz) {
@@ -853,6 +855,21 @@
     finddir_last=NULL;
     finddir_scan(&homenode.node, 0);
     scanhashtable(nameddirtab, 0, 0, 0, finddir_scan, 0);
+
+    if (func) {
+	char **ares = subst_string_by_func(func, "d", finddir_full);
+	int len;
+	if (ares && arrlen(ares) >= 2 &&
+	    (len = (int)zstrtol(ares[1], NULL, 10)) > finddir_best) {
+	    /* better duplicate this string since it's come from REPLY */
+	    finddir_last = (Nameddir)hcalloc(sizeof(struct nameddir));
+	    finddir_last->node.nam = tricat("[", dupstring(ares[0]), "]");
+	    finddir_last->dir = dupstrpfx(finddir_full, len);
+	    finddir_last->diff = len - strlen(finddir_last->node.nam);
+	    finddir_best = len;
+	}
+    }
+
     return finddir_last;
 }
 
@@ -1146,7 +1163,7 @@
     sfcontext = SFC_HOOK;
 
     if ((shfunc = getshfunc(name))) {
-	ret = doshfunc(shfunc, lnklst, 0, 1);
+	ret = doshfunc(shfunc, lnklst, 1);
 	stat = 0;
     }
 
@@ -1162,7 +1179,7 @@
 	if ((arrptr = getaparam(arrnam))) {
 	    for (; *arrptr; arrptr++) {
 		if ((shfunc = getshfunc(*arrptr))) {
-		    int newret = doshfunc(shfunc, lnklst, 0, 1);
+		    int newret = doshfunc(shfunc, lnklst, 1);
 		    if (!ret)
 			ret = newret;
 		    stat = 0;
@@ -2901,6 +2918,32 @@
     return (Shfunc) shfunctab->getnode(shfunctab, nam);
 }
 
+/*
+ * Call the function func to substitute string orig by setting
+ * the parameter reply.
+ * Return the array from reply, or NULL if the function returned
+ * non-zero status.
+ * The returned value comes directly from the parameter and
+ * so should be used before there is any chance of that
+ * being changed or unset.
+ * If arg1 is not NULL, it is used as an initial argument to
+ * the function, with the original string as the second argument.
+ */
+
+/**/
+char **
+subst_string_by_func(Shfunc func, char *arg1, char *orig)
+{
+    LinkList l = newlinklist();
+    addlinknode(l, func->node.nam);
+    if (arg1)
+	addlinknode(l, arg1);
+    addlinknode(l, orig);
+    if (doshfunc(func, l, 1))
+	return NULL;
+    return getaparam("reply");
+}
+
 /**/
 mod_export char **
 mkarray(char *s)
Index: Src/Modules/zftp.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Modules/zftp.c,v
retrieving revision 1.49
diff -u -r1.49 zftp.c
--- Src/Modules/zftp.c	16 Sep 2008 15:09:21 -0000	1.49
+++ Src/Modules/zftp.c	25 Sep 2008 12:39:46 -0000
@@ -1480,7 +1480,7 @@
 	int osc = sfcontext;
 
 	sfcontext = SFC_HOOK;
-	doshfunc(shfunc, NULL, 0, 1);
+	doshfunc(shfunc, NULL, 1);
 	sfcontext = osc;
 	/* Now add in the bit of the file we've got/sent already */
 	sofar = last_sofar = startat;
@@ -1613,7 +1613,7 @@
 
 	    zfsetparam("ZFTP_COUNT", &sofar, ZFPM_READONLY|ZFPM_INTEGER);
 	    sfcontext = SFC_HOOK;
-	    doshfunc(shfunc, NULL, 0, 1);
+	    doshfunc(shfunc, NULL, 1);
 	    sfcontext = osc;
 	    last_sofar = sofar;
 	}
@@ -2395,7 +2395,7 @@
 	int osc = sfcontext;
 
 	sfcontext = SFC_HOOK;
-	doshfunc(shfunc, NULL, 0, 1);
+	doshfunc(shfunc, NULL, 1);
 	sfcontext = osc;
     }
     return 0;
@@ -2615,7 +2615,7 @@
 	    zfsetparam("ZFTP_TRANSFER", ztrdup(recv ? "GF" : "PF"),
 		       ZFPM_READONLY);
 	    sfcontext = SFC_HOOK;
-	    doshfunc(shfunc, NULL, 0, 1);
+	    doshfunc(shfunc, NULL, 1);
 	    sfcontext = osc;
 	}
 	if (rest) {
@@ -2770,7 +2770,7 @@
 	    int osc = sfcontext;
 
 	    sfcontext = SFC_HOOK;
-	    doshfunc(shfunc, NULL, 0, 1);
+	    doshfunc(shfunc, NULL, 1);
 	    sfcontext = osc;
 	}
     }
Index: Src/Zle/compcore.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/compcore.c,v
retrieving revision 1.97
diff -u -r1.97 compcore.c
--- Src/Zle/compcore.c	16 Sep 2008 15:12:08 -0000	1.97
+++ Src/Zle/compcore.c	25 Sep 2008 12:39:46 -0000
@@ -814,7 +814,7 @@
 		while (*p)
 		    addlinknode(largs, dupstring(*p++));
 	    }
-	    doshfunc(shfunc, largs, 0, 0);
+	    doshfunc(shfunc, largs, 0);
 	    cfret = lastval;
 	    lastval = olv;
 	} OLDHEAPS;
Index: Src/Zle/compctl.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/compctl.c,v
retrieving revision 1.36
diff -u -r1.36 compctl.c
--- Src/Zle/compctl.c	16 Sep 2008 15:12:12 -0000	1.36
+++ Src/Zle/compctl.c	25 Sep 2008 12:39:47 -0000
@@ -3664,7 +3664,7 @@
 		incompctlfunc = 1;
 	    sfcontext = SFC_COMPLETE;
 	    /* Call the function. */
-	    doshfunc(shfunc, args, 0, 1);
+	    doshfunc(shfunc, args, 1);
 	    sfcontext = osc;
 	    incompctlfunc = 0;
 	    /* And get the result from the reply parameter. */
@@ -3839,7 +3839,7 @@
 	    if (incompfunc != 1)
 		incompctlfunc = 1;
 	    sfcontext = SFC_COMPLETE;
-	    doshfunc(shfunc, args, 0, 1);
+	    doshfunc(shfunc, args, 1);
 	    sfcontext = osc;
 	    incompctlfunc = 0;
 	    uv = "reply";
Index: Src/Zle/zle_main.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_main.c,v
retrieving revision 1.117
diff -u -r1.117 zle_main.c
--- Src/Zle/zle_main.c	22 Sep 2008 18:37:51 -0000	1.117
+++ Src/Zle/zle_main.c	25 Sep 2008 12:39:47 -0000
@@ -1330,7 +1330,7 @@
 	    makezleparams(0);
 	    sfcontext = SFC_WIDGET;
 	    opts[XTRACE] = 0;
-	    ret = doshfunc(shf, largs, shf->node.flags, 1);
+	    ret = doshfunc(shf, largs, 1);
 	    opts[XTRACE] = oxt;
 	    sfcontext = osc;
 	    endparamscope();
Index: Src/Zle/zle_misc.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_misc.c,v
retrieving revision 1.55
diff -u -r1.55 zle_misc.c
--- Src/Zle/zle_misc.c	16 Sep 2008 15:12:24 -0000	1.55
+++ Src/Zle/zle_misc.c	25 Sep 2008 12:39:47 -0000
@@ -1384,7 +1384,7 @@
 	    startparamscope();
 	    makezleparams(0);
 	    sfcontext = SFC_COMPLETE;
-	    doshfunc(shfunc, args, 0, 1);
+	    doshfunc(shfunc, args, 1);
 	    sfcontext = osc;
 	    endparamscope();
 
Index: Test/D01prompt.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/D01prompt.ztst,v
retrieving revision 1.3
diff -u -r1.3 D01prompt.ztst
--- Test/D01prompt.ztst	19 Oct 2005 08:39:03 -0000	1.3
+++ Test/D01prompt.ztst	25 Sep 2008 12:39:47 -0000
@@ -1,5 +1,7 @@
 %prep
 
+  mkdir prompt.tmp
+  cd prompt.tmp
   mydir=$PWD
   SHLVL=2
   setopt extendedglob
@@ -104,3 +106,44 @@
     print "Years do not agree in $date2, $date3"
   fi
 0:Dates produced by prompt escapes
+
+  mkdir foo
+  mkdir foo/bar
+  mkdir foo/bar/rod
+  (zsh_directory_name() {
+    emulate -L zsh
+    setopt extendedglob
+    local -a match mbegin mend
+    if [[ $1 = d ]]; then
+      if [[ $2 = (#b)(*bar)/rod ]]; then
+        reply=(barmy ${#match[1]})
+      else
+        return 1
+      fi
+    else
+      if [[ $2 = barmy ]]; then
+        reply=($mydir/foo/bar)
+      else
+        return 1
+      fi
+    fi
+  }
+  # success
+  print ~[barmy]/anything
+  cd foo/bar/rod
+  print -P %~
+  # failure
+  setopt nonomatch
+  print ~[scuzzy]/rubbish
+  cd ../..
+  print -P %~
+  # catastrophic failure
+  unsetopt nonomatch
+  print ~[scuzzy]/rubbish
+  )
+1q:Dynamic named directories
+>$mydir/foo/bar/anything
+>~[barmy]/rod
+>~[scuzzy]/rubbish
+>~mydir/foo
+?(eval):33: no directory expansion: ~[scuzzy]


-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: PATCH: dynamic named directories

[Mikael Magnusson]

I'll proof read the docs a bit (ie i'll read them, i can't read without
proof reading anyway) *does that*, okay, only one comment.

2008/9/25 Peter Stephenson <pws@csr.com>:
> Problem: six feet deep in WiFi and UWB bugs to fix.  Solution:  do
> something else entirely.
>
> This morning I had a brainwave/brainstorm that turned out to be easy to
> implement and test without interfering with the rest of the shell, which
> is the sort of feature I like.  I'm sure you'll tell me if it's a stupid
> idea.
>
> It's based on the way I use an Emacs function suite called "bufname"
> that a colleague and I wrote a few years ago.  I have lots of parallel
> directory hierarchies with slightly different names at different levels,
> so normal static directory naming isn't really flexible enough.
>
> Now you can use ~[<stuff>] and <stuff> gets passed down to a function
> zsh_directory_name, which either parses it and picks an appropriate
> directory or refuses.  (If the function isn't defined this syntax is
> treated the way it always was, which typically wasn't all that useful.)
> There's also a mechanism for going the other way.  See documentation and
> examples.
>
> Much of the patch is actually another simplification to doshfunc() which
> was obvious when I worked out what the third argument was for.  It
> should mean any PM_* flags applicable to functions (such as PM_TAGGED)
> are handled more consistently.
>
> Index: Doc/Zsh/expn.yo
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Doc/Zsh/expn.yo,v
> retrieving revision 1.91
> diff -u -r1.91 expn.yo
> --- Doc/Zsh/expn.yo     16 Jun 2008 16:39:18 -0000      1.91
> +++ Doc/Zsh/expn.yo     25 Sep 2008 12:39:43 -0000
> @@ -1308,8 +1308,73 @@
>  option exchanges the effects of `tt(~PLUS())' and `tt(~-)' where they are
>  followed by a number.
>
> -cindex(directories, named)
> -cindex(named directories)
> +subsect(Dynamic named directories)
> +cindex(directories, named, dynamic)
> +cindex(named directories, dynamicic)
> +cindex(dynamic named directories)
> +
> +The feature described here is only available if the shell function
> +tt(zsh_directory_name) exists.
> +
> +A `tt(~)' followed by a string var(namstr) in unquoted square brackets is
> +treated specially as a dynamic directory name.  Note that the first
> +unquoted closing square bracket always terminates var(namstr).  The shell
> +function is passed two arguments: the string tt(n) (for name) and
> +var(namstr).  It should either set the array tt(reply) to a single element
> +which is the directory corresponding to the name and return status zero
> +(executing an assignment as the last statement is usually sufficient), or
> +it should return status non-zero.  In the former case the element of reply

the what element of reply? i guess the first?

-- 
Mikael Magnusson

Re: PATCH: dynamic named directories

[Peter Stephenson]

"Mikael Magnusson" wrote:
> > +var(namstr).  It should either set the array tt(reply) to a single element
> > +which is the directory corresponding to the name and return status zero
> > +(executing an assignment as the last statement is usually sufficient), or
> > +it should return status non-zero.  In the former case the element of reply
> 
> the what element of reply? i guess the first?

As it's just said you should set the array to a single element, it can
be the first or the last, at your choice.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: PATCH: dynamic named directories

[Bart Schaefer]

On Sep 25,  3:32pm, Peter Stephenson wrote:
}
} Now you can use ~[<stuff>] and <stuff> gets passed down to a function
} zsh_directory_name, which either parses it and picks an appropriate
} directory or refuses.

Wow, it's the next best thing to symlinks with environment variable
references in them, like I had in 1990 on my dual-68020 Sony machine
running a Japanese version of 4.3 BSD.

Only maybe without the corresponding gaping security hole.

} -			    doshfunc(shfunc, l, 0, 1);
} +			    doshfunc(shfunc, l, 1);

Are you sure this is still equivalent to the old way?  Looks like here
[and in several similar places] doshfunc() was being explicitly told to
ignore shfunc->node.flags, whereas with your change it will always pay
attention to them.

Re: PATCH: dynamic named directories

[Peter Stephenson]

Bart Schaefer wrote:
> Only maybe without the corresponding gaping security hole.
> 
> } -			    doshfunc(shfunc, l, 0, 1);
> } +			    doshfunc(shfunc, l, 1);
> 
> Are you sure this is still equivalent to the old way?  Looks like here
> [and in several similar places] doshfunc() was being explicitly told to
> ignore shfunc->node.flags, whereas with your change it will always pay
> attention to them.

The flags in question are PM_UNDEFINED and PM_TAGGED.  The former is
just used to decide whether to set a script name (don't ask me why, but
it looks like an internal consistency matter rather than anything
user-visible); the latter to do tracing by setting XTRACE.  Hence I
don't think it's a security hole.  I suspect that 0 was being passed in
a lot of cases simply because the caller couldn't be bothered to keep
the flags around, which was what was happening even with the function
name in some cases before I changed the interface to pass in the shell
function rather than the code within it.

(By the way, I wonder if, for functions where tracing has been
requested, it would be a good idea if "emulate" didn't turn it off
again...)

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: PATCH: dynamic named directories

[Mikael Magnusson]

2008/9/25 Peter Stephenson <pws@csr.com>:
> Problem: six feet deep in WiFi and UWB bugs to fix.  Solution:  do
> something else entirely.
>
> This morning I had a brainwave/brainstorm that turned out to be easy to
> implement and test without interfering with the rest of the shell, which
> is the sort of feature I like.  I'm sure you'll tell me if it's a stupid
> idea.
>
> It's based on the way I use an Emacs function suite called "bufname"
> that a colleague and I wrote a few years ago.  I have lots of parallel
> directory hierarchies with slightly different names at different levels,
> so normal static directory naming isn't really flexible enough.
>
> Now you can use ~[<stuff>] and <stuff> gets passed down to a function
> zsh_directory_name, which either parses it and picks an appropriate
> directory or refuses.  (If the function isn't defined this syntax is
> treated the way it always was, which typically wasn't all that useful.)
> There's also a mechanism for going the other way.  See documentation and
> examples.
>
> Much of the patch is actually another simplification to doshfunc() which
> was obvious when I worked out what the third argument was for.  It
> should mean any PM_* flags applicable to functions (such as PM_TAGGED)
> are handled more consistently.

Here's line broken backtrace of the week

(gdb) run
Starting program: /usr/local/bin/zsh -f
fartmonstret% zsh_directory_name() {}
fartmonstret% cd ~[a b]

Program received signal SIGSEGV, Segmentation fault.
0x080c1a73 in filesubstr (namptr=0xafca0380, assign=0) at subst.c:560
560		} else if (!inblank(str[1]) && isend(*ptr) &&
(gdb) bt full
#0  0x080c1a73 in filesubstr (namptr=0xafca0380, assign=0) at subst.c:560
	dirfunc = (Shfunc) 0x81077d8
	tmp = 0xa7a9ff84 "\022\020"
	val = 0
	ptr = 0x0
	res = 0x1 <Address 0x1 out of bounds>
	str = 0xa7bbe640 "\225\217a"
#1  0x080c1614 in filesub (namptr=0xafca0380, assign=0) at subst.c:492
	eql = 0x0
	sub = 0x0
	str = 0x0
	ptr = 0xca01c4 <Address 0xca01c4 out of bounds>
	len = 100
#2  0x080c080a in prefork (list=0xa7bbe608, flags=0) at subst.c:113
	cptr = 0xa7bbe640 "\225\217a"
	node = (LinkNode) 0xa7bbe620
	stop = (LinkNode) 0xa7bbe62c
	keep = 0
	asssub = 0
#3  0x0806f2e7 in execcmd (state=0xafca09cc, input=0, output=0,
how=18, last1=2) at exec.c:2420
	hn = (HashNode) 0x80df4d4
	args = (LinkList) 0xa7bbe608
	node = (LinkNode) 0xafca0648
	fn = (Redir) 0x41b6cdb3
	mfds = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}
	text = 0xa7bbe600 "b\220"
	save = {-2, -2, -2, -2, -2, -2, -2, -2, -2, -2}
	fil = -1345714676
---Type <return> to continue, or q <return> to quit---
	dfil = 135038111
	is_cursh = -1480858112
	type = 6
	do_exec = 0
	i = 10
	htok = 1
	nullexec = 0
	assign = 0
	forked = 0
	is_shfunc = 0
	is_builtin = 1
	is_exec = 0
	use_defpath = 0
	cflags = 0
	checked = 1
	oautocont = -1
	redir = (LinkList) 0x0
	code = 102
	beg = (Wordcode) 0xa7bbe5ac
	varspc = (Wordcode) 0x0
	oxtrerr = (FILE *) 0x41c29560
	newxtrerr = (FILE *) 0x0
#4  0x0806d1f5 in execpline2 (state=0xafca09cc, pcode=195, how=18,
input=0, output=0, last1=0)
    at exec.c:1557
	pid = 134789985
	pipes = {0, -1345714360}
#5  0x0806c58b in execpline (state=0xafca09cc, slcode=5122, how=18,
last1=0) at exec.c:1343
	ipipe = {0, 0}
---Type <return> to continue, or q <return> to quit---
	opipe = {0, 0}
	pj = 0
	newjob = 1
	old_simple_pline = 0
	slflags = 0
	code = 195
	lastwj = 0
	lpforked = 0
#6  0x0806bdcc in execlist (state=0xafca09cc, dont_change_job=0,
exiting=0) at exec.c:1141
	donedebug = 0
	next = (Wordcode) 0xa7bbe5bc
	code = 5122
	ret = 0
	cj = 0
	csp = 0
	ltype = 18
	old_pline_level = 0
	old_list_pipe = 0
	oldlineno = 3
	oldnoerrexit = 0
	donetrap = 0
#7  0x0806b8ad in execode (p=0xa7bbe578, dont_change_job=0, exiting=0)
at exec.c:973
	s = {prog = 0xa7bbe578, pc = 0xa7bbe5bc, strs = 0xa7bbe5c0 "Ð廧à廧"}
#8  0x08084d11 in loop (toplevel=1, justonce=0) at init.c:181
	toksav = 1
	prog = (Eprog) 0xa7bbe578
#9  0x0808778e in zsh_main (argc=2, argv=0xafca0b04) at init.c:1406
	t = (char **) 0xafca0b0c
---Type <return> to continue, or q <return> to quit---
	t0 = 157
#10 0x08055236 in main (argc=Cannot access memory at address 0xffff9000
) at ./main.c:93

-- 
Mikael Magnusson

Re: PATCH: dynamic named directories

[Mikael Magnusson]

2008/9/26 Mikael Magnusson <mikachu@gmail.com>:
> 2008/9/25 Peter Stephenson <pws@csr.com>:
>
> Here's line broken backtrace of the week
>
> (gdb) run
> Starting program: /usr/local/bin/zsh -f
> fartmonstret% zsh_directory_name() {}
> fartmonstret% cd ~[a b]

Well, maybe the space is a bit of a red herring there. 'cd ~[a'
appears to give more or less the same result.

-- 
Mikael Magnusson

Re: PATCH: dynamic named directories

[Bart Schaefer]

On Sep 25,  5:05pm, Peter Stephenson wrote:
} Subject: Re: PATCH: dynamic named directories
}
} (By the way, I wonder if, for functions where tracing has been
} requested, it would be a good idea if "emulate" didn't turn it off
} again...)

Yes, that would be good.

Re: PATCH: dynamic named directories

[Peter Stephenson]

On Fri, 26 Sep 2008 14:56:32 +0200
"Mikael Magnusson" <mikachu@gmail.com> wrote:
> 2008/9/26 Mikael Magnusson <mikachu@gmail.com>:
> > 2008/9/25 Peter Stephenson <pws@csr.com>:
> >
> > Here's line broken backtrace of the week
> >
> > (gdb) run
> > Starting program: /usr/local/bin/zsh -f
> > fartmonstret% zsh_directory_name() {}
> > fartmonstret% cd ~[a b]
> 
> Well, maybe the space is a bit of a red herring there. 'cd ~[a'
> appears to give more or less the same result.

Oops.  Thanks.  The problem is the search for "]".  If that was
attempted but the name substitution failed, the pointer used in the next
test was messed up.  In the first case, the failure is after the call to
zsh_directory_name, in the second before since the search for "]"
itself failed.  (You'd have found the first case didn't crash if there
was already a value for $reply, which would have been used as the
directory---you do have to stick to the rules to get it to work
properly, although obviously it shouldn't crash.)

I've also spotted that the substitution code requires static named
directories to have the same set of chararacters as user names, whether
or not they are user names.  So (i) this should be documented (ii) hash
-d should check when defining static names for directories since it's
pointless and confusing if they can be defined but can't be used (there
isn't room to list the valid characters in the error message).

It's not obvious that this limitation is necessary, however.  It
obviously comes from when named directories could only be user names or
parameters; with "hash -d" we could in principle relax it.  However,
I don't think there's any good reason to use a wider name space and
there's every possibility of confusion with people wondering why tilde
squiggle squiggle squiggle slash doesn't do quite what they want.

Index: Doc/Zsh/expn.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/expn.yo,v
retrieving revision 1.92
diff -u -r1.92 expn.yo
--- Doc/Zsh/expn.yo	26 Sep 2008 09:11:28 -0000	1.92
+++ Doc/Zsh/expn.yo	27 Sep 2008 19:29:23 -0000
@@ -1375,13 +1375,16 @@
 cindex(directories, named, static)
 cindex(named directories, static)
 cindex(static named directories)
-A `tt(~)' followed by anything not already covered is looked up as a
+A `tt(~)' followed by anything not already covered consisting
+of any number of alphanumeric characters or underscore (`tt(_)'),
+hyphen (`tt(-)'), or dot (`tt(.)') is looked up as a
 named directory, and replaced by the value of that named directory if found.
 Named directories are typically home directories for users on the system.
 They may also be defined if the text after the `tt(~)' is the name
 of a string shell parameter whose value begins with a `tt(/)'.
 Note that trailing slashes will be removed from the path to the directory
 (though the original parameter is not modified).
+
 It is also possible to define directory names using the tt(-d) option to the
 tt(hash) builtin.
 
Index: Src/builtin.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/builtin.c,v
retrieving revision 1.208
diff -u -r1.208 builtin.c
--- Src/builtin.c	23 Sep 2008 08:42:01 -0000	1.208
+++ Src/builtin.c	27 Sep 2008 19:29:26 -0000
@@ -3262,9 +3262,19 @@
 		/* The argument is of the form foo=bar, *
 		 * so define an entry for the table.    */
 		if(OPT_ISSET(ops,'d')) {
-		    Nameddir nd = hn = zshcalloc(sizeof *nd);
-		    nd->node.flags = 0;
-		    nd->dir = ztrdup(asg->value);
+		    /* shouldn't return NULL if asg->name is not NULL */
+		    if (*itype_end(asg->name, IUSER, 0)) {
+			zwarnnam(name,
+				 "invalid character in directory name: %s",
+				 asg->name);
+			returnval = 1;
+			argv++;
+			continue;
+		    } else {
+			Nameddir nd = hn = zshcalloc(sizeof *nd);
+			nd->node.flags = 0;
+			nd->dir = ztrdup(asg->value);
+		    }
 		} else {
 		    Cmdnam cn = hn = zshcalloc(sizeof *cn);
 		    cn->node.flags = HASHED;
Index: Src/subst.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/subst.c,v
retrieving revision 1.87
diff -u -r1.87 subst.c
--- Src/subst.c	26 Sep 2008 09:11:30 -0000	1.87
+++ Src/subst.c	27 Sep 2008 19:29:27 -0000
@@ -529,7 +529,7 @@
 
     if (*str == Tilde && str[1] != '=' && str[1] != Equals) {
 	Shfunc dirfunc;
-	char *ptr, *tmp, *res;
+	char *ptr, *tmp, *res, *ptr2;
 	int val;
 
 	val = zstrtol(str + 1, &ptr, 10);
@@ -544,14 +544,14 @@
 	    return 1;
 	} else if (str[1] == Inbrack &&
 		   (dirfunc = getshfunc("zsh_directory_name")) &&
-		   (ptr = strchr(str+2, Outbrack))) {
+		   (ptr2 = strchr(str+2, Outbrack))) {
 	    char **arr;
-	    untokenize(tmp = dupstrpfx(str+2, ptr - (str+2)));
+	    untokenize(tmp = dupstrpfx(str+2, ptr2 - (str+2)));
 	    remnulargs(tmp);
 	    arr = subst_string_by_func(dirfunc, "n", tmp);
 	    res = arr ? *arr : NULL;
 	    if (res) {
-		*namptr = dyncat(res, ptr+1);
+		*namptr = dyncat(res, ptr2+1);
 		return 1;
 	    }
 	    if (isset(NOMATCH))


-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/