globbing in conditional expressions

globbing in conditional expressions

[Roman Neuhauser]

hello,

i was hoping i could combine conditional expressions with
some globbing, but i'm struggling to find the proper syntax.

: 0 > ls fubar
snafu
: 0 > f=fubar(/F)
: 0 > echo $f
fubar(/F)
: 0 > echo $~f
fubar
: 0 > (set -x; [[ -z $~f ]])
+zsh:183> [[ -z 'fubar(/F)' ]]
: 1 >

i'm sure this is possible and documented, and i've just failed
to put the pieces together.

-- 
roman

Re: globbing in conditional expressions

[Peter Stephenson]

On Wed, 07 May 2014 14:41:01 +0200
Roman Neuhauser <neuhauser@sigpipe.cz> wrote:
> i was hoping i could combine conditional expressions with
> some globbing, but i'm struggling to find the proper syntax.

You need to do the globbing first.  In conditional expressions you get
single-word expansion that doesn't do globbing.
 
> : 0 > ls fubar
> snafu
> : 0 > f=fubar(/F)
> : 0 > echo $f
> fubar(/F)
> : 0 > echo $~f
> fubar
> : 0 > (set -x; [[ -z $~f ]])

You'll need something like the following --- the (N) flag is needed if
you want the string to be empty if there's no match (unless you have the
NULL_GLOB option set, which isn't standard).

matches=(fubar(N/F))
[[ -z $matches ]]

pws

Re: globbing in conditional expressions

[Mikael Magnusson]

On 7 May 2014 14:41, Roman Neuhauser <neuhauser@sigpipe.cz> wrote:
> hello,
>
> i was hoping i could combine conditional expressions with
> some globbing, but i'm struggling to find the proper syntax.
>
> : 0 > ls fubar
> snafu
> : 0 > f=fubar(/F)
> : 0 > echo $f
> fubar(/F)
> : 0 > echo $~f
> fubar
> : 0 > (set -x; [[ -z $~f ]])
> +zsh:183> [[ -z 'fubar(/F)' ]]
> : 1 >
>
> i'm sure this is possible and documented, and i've just failed
> to put the pieces together.

a=( myglob*(/FN[1]) )
(( $#a )) && yay files

-- 
Mikael Magnusson

Re: globbing in conditional expressions

[Bart Schaefer]

On May 7,  2:41pm, Roman Neuhauser wrote:
}
} i was hoping i could combine conditional expressions with
} some globbing, but i'm struggling to find the proper syntax.

Conditional expressions only do pattern matching against strings, they
do not do globbing against file names.  So glob qualifiers like (/F)
have no special meaning in a conditional expression.

The way to handle this is to perform the glob first and then test the
result of globbing, e.g.

    if function { set - foobar(/FN) && [[ -z $1 ]] }
    then print not a directory or empty
    else print directory contains files
    fi

Use a subshell if the version of zsh is old enough not to support anonymous
functions.

Re: globbing in conditional expressions

[Daniel Shahaf]

Peter Stephenson wrote on Wed, May 07, 2014 at 15:44:07 +0100:
> matches=(fubar(N/F))
> [[ -z $matches ]]

All three solutions first set an array to the result of the glob.  Is
there a solution that short-circuits, i.e., that returns as soon as a
single matching filename is found?

This matters when the glob is matched against a large number of files.

Re: globbing in conditional expressions

[Peter Stephenson]

On Thu, 08 May 2014 10:55:22 +0000
Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> Peter Stephenson wrote on Wed, May 07, 2014 at 15:44:07 +0100:
> > matches=(fubar(N/F))
> > [[ -z $matches ]]
> 
> All three solutions first set an array to the result of the glob.  Is
> there a solution that short-circuits, i.e., that returns as soon as a
> single matching filename is found?
> 
> This matters when the glob is matched against a large number of files.

No, unless someone remembers I feature I don't know about, because
internally the glob proceeds until it's finished --- so even if you use
([1]) to select the first match, the glob has already generated all
matches at that point.

If you have multiple different patterns, you can do it by writing a
function to loop over them and return on the first match, but for a
single pattern I think you're stuck.

pws

Re: globbing in conditional expressions

[Bart Schaefer]

On May 8, 12:20pm, Peter Stephenson wrote:
}
} > All three solutions first set an array to the result of the glob.  Is
} > there a solution that short-circuits, i.e., that returns as soon as a
} > single matching filename is found?
} 
} No, unless someone remembers I feature I don't know about, because
} internally the glob proceeds until it's finished

Well ... you can do this horrible hack:

  found=
  # Stop after finding exactly one directory, for example:
  : **/*(e+'[[ -d $REPLY ]] && { found=$REPLY ; kill -INT $$ }'+)
  print $found

Of course to make this "useful" all the stuff that you'd test in the
glob qualifiers you instead need to test in the [[ ... ]] before you
assign found=$REPLY, probably by using the zsh/stat module; and you
can't avoid scanning an unknown number of potential NON-matches up
to the point where $found gets set, so in the worst case it's no
better (and probably much worse) than just letting the glob finish.

Re: globbing in conditional expressions

[Roman Neuhauser]

# schaefer@brasslantern.com / 2014-05-08 08:34:18 -0700:
> On May 8, 12:20pm, Peter Stephenson wrote:
> }
> } > All three solutions first set an array to the result of the glob.  Is
> } > there a solution that short-circuits, i.e., that returns as soon as a
> } > single matching filename is found?
> } 
> } No, unless someone remembers I feature I don't know about, because
> } internally the glob proceeds until it's finished
> 
> Well ... you can do this horrible hack:
> 
>   found=
>   # Stop after finding exactly one directory, for example:
>   : **/*(e+'[[ -d $REPLY ]] && { found=$REPLY ; kill -INT $$ }'+)
>   print $found
> 
> Of course to make this "useful" all the stuff that you'd test in the
> glob qualifiers you instead need to test in the [[ ... ]] before you
> assign found=$REPLY, probably by using the zsh/stat module; and you
> can't avoid scanning an unknown number of potential NON-matches up
> to the point where $found gets set, so in the worst case it's no
> better (and probably much worse) than just letting the glob finish.

maybe a crazy idea...  how about something like [[ -m pattern ]] which
would succeed iff pattern matched at least one path?  this could be
somewhat more amenable to shortcircuiting.

-- 
roman

Re: globbing in conditional expressions

[Bart Schaefer]

On May 8, 10:19pm, Roman Neuhauser wrote:
}
} maybe a crazy idea...  how about something like [[ -m pattern ]] which
} would succeed iff pattern matched at least one path?  this could be
} somewhat more amenable to shortcircuiting.

Returning to this after a bit of a detour through [[ ... ]] expression
parsing:

You could define (via zmodload) an operator that applies filename
generation to its argument, but changes to the internals of globbing
would be needed to make a short-circuit happen.  Then those changes
would have to be exposed somehow so that the operator could use them.

As I mentioned before, in the case of the match failing this would be
exactly as expensive as not short-circuiting.

Re: globbing in conditional expressions

[Daniel Shahaf]

[-- Attachment #1: Type: text/plain, Size: 1330 bytes --]

Bart Schaefer wrote on Tue, May 13, 2014 at 08:41:17 -0700:
> On May 8, 10:19pm, Roman Neuhauser wrote:
> }
> } maybe a crazy idea...  how about something like [[ -m pattern ]] which
> } would succeed iff pattern matched at least one path?  this could be
> } somewhat more amenable to shortcircuiting.
> 
> Returning to this after a bit of a detour through [[ ... ]] expression
> parsing:
> 
> You could define (via zmodload) an operator that applies filename
> generation to its argument, but changes to the internals of globbing
> would be needed to make a short-circuit happen.  Then those changes
> would have to be exposed somehow so that the operator could use them.
> 

I've taken a shot at making those changes, see attached.  This version
just adds a [[ -m $pattern ]] rather than a zmodload operator, but I'm
sure it could be converted to using zmodload.

(Aside: where is the documentation of internal functions in the C
source?  It isn't the *.c/*.h files where I looked for it.)

> As I mentioned before, in the case of the match failing this would be
> exactly as expensive as not short-circuiting.

Right, so short-circuiting would only be useful for callers that want to
know whether a pattern matches a large directory, but don't care about
having a list of matching filenames.  Does anyone have such a use-case?

[-- Attachment #2: 0003-Add-m-pattern-that-short-circuits.patch --]
[-- Type: text/x-patch, Size: 13275 bytes --]

>From 95dfc45753601eeb1d406a640c03434fcbc61704 Mon Sep 17 00:00:00 2001
From: Daniel Shahaf <d.s@daniel.shahaf.name>
Date: Tue, 13 May 2014 10:18:56 +0000
Subject: [PATCH 3/3] Add [[ -m pattern ]] that short-circuits.

---
 Doc/Zsh/cond.yo      |    3 ++
 Src/Zle/compctl.c    |    8 ++---
 Src/Zle/zle_tricky.c |    2 +-
 Src/cond.c           |   13 +++++++++
 Src/exec.c           |    8 ++---
 Src/glob.c           |   79 +++++++++++++++++++++++++++++++-------------------
 Src/parse.c          |    4 +--
 Src/subst.c          |    8 +++--
 Test/C02cond.ztst    |    3 ++
 9 files changed, 84 insertions(+), 44 deletions(-)

diff --git a/Doc/Zsh/cond.yo b/Doc/Zsh/cond.yo
index 9f8a7d8..ac07cf4 100644
--- a/Doc/Zsh/cond.yo
+++ b/Doc/Zsh/cond.yo
@@ -38,6 +38,9 @@ true if var(file) exists and is a symbolic link.
 item(tt(-k) var(file))(
 true if var(file) exists and has its sticky bit set.
 )
+item(tt(-m) var(pattern))(
+true if var(pattern) has any matches.  Short-circuits.
+)
 item(tt(-n) var(string))(
 true if length of var(string) is non-zero.
 )
diff --git a/Src/Zle/compctl.c b/Src/Zle/compctl.c
index 2563095..7565142 100644
--- a/Src/Zle/compctl.c
+++ b/Src/Zle/compctl.c
@@ -2228,7 +2228,7 @@ gen_matches_files(int dirs, int execs, int all)
 			    /* Do the globbing... */
 			    remnulargs(p);
 			    addlinknode(l, p);
-			    globlist(l, 0);
+			    globlist(l, 0, 0);
 			    /* And see if that produced a filename. */
 			    tt = nonempty(l);
 			    while (ugetnode(l));
@@ -3384,7 +3384,7 @@ makecomplistflags(Compctl cc, char *s, int incmd, int compadd)
 		tokenize(p);
 		remnulargs(p);
 		addlinknode(l, p);
-		globlist(l, 0);
+		globlist(l, 0, 0);
 
 		if (nonempty(l)) {
 		    /* And add the resulting words. */
@@ -3533,7 +3533,7 @@ makecomplistflags(Compctl cc, char *s, int incmd, int compadd)
 				/* Do the globbing. */
 				ng = opts[NULLGLOB];
 				opts[NULLGLOB] = 1;
-				globlist(l, 0);
+				globlist(l, 0, 0);
 				opts[NULLGLOB] = ng;
 				/* Get the results. */
 				if (nonempty(l) && peekfirst(l)) {
@@ -3730,7 +3730,7 @@ makecomplistflags(Compctl cc, char *s, int incmd, int compadd)
 	/* Fine, now do full expansion. */
 	prefork(foo, 0);
 	if (!errflag) {
-	    globlist(foo, 0);
+	    globlist(foo, 0, 0);
 	    if (!errflag)
 		/* And add the resulting words as matches. */
 		for (n = firstnode(foo); n; incnode(n))
diff --git a/Src/Zle/zle_tricky.c b/Src/Zle/zle_tricky.c
index 499c4ae..c4f97a8 100644
--- a/Src/Zle/zle_tricky.c
+++ b/Src/Zle/zle_tricky.c
@@ -2197,7 +2197,7 @@ doexpansion(char *s, int lst, int olst, int explincmd)
 	int ng = opts[NULLGLOB];
 
 	opts[NULLGLOB] = 1;
-	globlist(vl, 1);
+	globlist(vl, 0, 1);
 	opts[NULLGLOB] = ng;
     }
     if (errflag)
diff --git a/Src/cond.c b/Src/cond.c
index c673542..b5f162a 100644
--- a/Src/cond.c
+++ b/Src/cond.c
@@ -319,6 +319,19 @@ evalcond(Estate state, char *fromtest)
 	return (!(dostat(left) & S_ISGID));
     case 'k':
 	return (!(dostat(left) & S_ISVTX));
+    case 'm':
+	{
+	    LinkList globber = newsizedlist(2);
+	    tokenize(left);
+	    setsizednode(globber, 0, "");
+	    setsizednode(globber, 1, left);
+#if 1
+	    return (!globlist(globber, 1 /* short circuit */, 1 /* nountok */));
+#else
+	    globlist(globber, 0 /* short circuit */, 1 /* nountok */);
+	    return (!nextnode(firstnode(globber)));
+#endif
+	}
     case 'n':
 	return (!strlen(left));
     case 'o':
diff --git a/Src/exec.c b/Src/exec.c
index 8249def..aa381e9 100644
--- a/Src/exec.c
+++ b/Src/exec.c
@@ -2237,7 +2237,7 @@ addvars(Estate state, Wordcode pc, int addflags)
 		return;
 	    }
 	    if (isset(GLOBASSIGN) || !isstr)
-		globlist(vl, 0);
+		globlist(vl, 0, 0);
 	    if (errflag) {
 		state->pc = opc;
 		return;
@@ -2348,7 +2348,7 @@ execsubst(LinkList strs)
 	prefork(strs, esprefork);
 	if (esglob && !errflag) {
 	    LinkList ostrs = strs;
-	    globlist(strs, 0);
+	    globlist(strs, 0, 0);
 	    strs = ostrs;
 	}
     }
@@ -2621,7 +2621,7 @@ execcmd(Estate state, int input, int output, int how, int last1)
 	    if (!(cflags & BINF_NOGLOB))
 		while (!checked && !errflag && args && nonempty(args) &&
 		       has_token((char *) peekfirst(args)))
-		    zglob(args, firstnode(args), 0);
+		    zglob(args, firstnode(args), 0, 0);
 	    else if (!unglobbed) {
 		for (node = firstnode(args); node; incnode(node))
 		    untokenize((char *) getdata(node));
@@ -2942,7 +2942,7 @@ execcmd(Estate state, int input, int output, int how, int last1)
 
     if ((esglob = !(cflags & BINF_NOGLOB)) && args && htok) {
 	LinkList oargs = args;
-	globlist(args, 0);
+	globlist(args, 0, 0);
 	args = oargs;
     }
     if (errflag) {
diff --git a/Src/glob.c b/Src/glob.c
index 07dd7c2..1a521a5 100644
--- a/Src/glob.c
+++ b/Src/glob.c
@@ -452,8 +452,8 @@ insert(char *s, int checked)
  * tried all of it.                                */
 
 /**/
-static void
-scanner(Complist q)
+static int
+scanner(Complist q, int shortcircuit)
 {
     Patprog p;
     int closure;
@@ -463,14 +463,15 @@ scanner(Complist q)
 
     init_dirsav(&ds);
     if (!q)
-	return;
+	return -1;
 
     if ((closure = q->closure)) {
 	/* (foo/)# - match zero or more dirs */
 	if (q->closure == 2)	/* (foo/)## - match one or more dirs */
 	    q->closure = 1;
 	else
-	    scanner(q->next);
+	    if (scanner(q->next, shortcircuit) == 1)
+		return 1;
     }
     p = q->pat;
     /* Now the actual matching for the current path section. */
@@ -485,13 +486,13 @@ scanner(Complist q)
 	    int err;
 
 	    if (l >= PATH_MAX)
-		return;
+		return -1;
 	    err = lchdir(pathbuf + pathbufcwd, &ds, 0);
 	    if (err == -1)
-		return;
+		return -1;
 	    if (err) {
 		zerr("current directory lost during glob");
-		return;
+		return -1;
 	    }
 	    pathbufcwd = pathpos;
 	}
@@ -516,14 +517,18 @@ scanner(Complist q)
 		if (add) {
 		    addpath(str, l);
 		    if (!closure || !statfullpath("", NULL, 1))
-			scanner((q->closure) ? q : q->next);
+			if (scanner((q->closure) ? q : q->next, shortcircuit) == 1)
+			    return 1;
 		    pathbuf[pathpos = oppos] = '\0';
 		}
 	    }
 	} else {
 	    if (str[l])
 		str = dupstrpfx(str, l);
-	    insert(str, 0);
+	    if (!shortcircuit)
+		insert(str, 0);
+	    else
+		return 1;
 	}
     } else {
 	/* Do pattern matching on current path section. */
@@ -534,7 +539,7 @@ scanner(Complist q)
 	int subdirlen = 0;
 
 	if (lock == NULL)
-	    return;
+	    return -1;
 	while ((fn = zreaddir(lock, 1)) && !errflag) {
 	    /* prefix and suffix are zle trickery */
 	    if (!dirs && !colonmod &&
@@ -613,7 +618,10 @@ scanner(Complist q)
 		    subdirlen += sizeof(int);
 		} else
 		    /* if the last filename component, just add it */
-		    insert(fn, 1);
+		    if (!shortcircuit)
+			insert(fn, 1);
+		    else
+			return 1;
 	    }
 	}
 	closedir(lock);
@@ -626,7 +634,8 @@ scanner(Complist q)
 		fn += l + 1;
 		memcpy((char *)&errsfound, fn, sizeof(int));
 		fn += sizeof(int);
-		scanner((q->closure) ? q : q->next);  /* scan next level */
+		if (scanner((q->closure) ? q : q->next, shortcircuit) == 1) /* scan next level */
+		    return 1;
 		pathbuf[pathpos = oppos] = '\0';
 	    }
 	    hrealloc(subdirs, subdirlen, 0);
@@ -640,6 +649,7 @@ scanner(Complist q)
 	    close(ds.dirfd);
 	pathbufcwd = pbcwdsav;
     }
+    return 0;
 }
 
 /* This function tokenizes a zsh glob pattern */
@@ -1066,8 +1076,8 @@ insert_glob_match(LinkList list, LinkNode next, char *data)
  * into a series of nodes.                                      */
 
 /**/
-void
-zglob(LinkList list, LinkNode np, int nountok)
+int
+zglob(LinkList list, LinkNode np, int shortcircuit, int nountok)
 {
     struct qual *qo, *qn, *ql;
     LinkNode node = prevnode(np);
@@ -1084,7 +1094,7 @@ zglob(LinkList list, LinkNode np, int nountok)
     if (unset(GLOBOPT) || !haswilds(ostr) || unset(EXECOPT)) {
 	if (!nountok)
 	    untokenize(ostr);
-	return;
+	return -1;
     }
     save_globstate(saved);
 
@@ -1526,7 +1536,7 @@ zglob(LinkList list, LinkNode np, int nountok)
 		    if (gf_nsorts == MAX_SORTS) {
 			zerr("too many glob sort specifiers");
 			restore_globstate(saved);
-			return;
+			return -1;
 		    }
 
 		    /* usually just one character */
@@ -1548,14 +1558,14 @@ zglob(LinkList list, LinkNode np, int nountok)
 			     glob_exec_string(&send)) == NULL)
 			{
 			    restore_globstate(saved);
-			    return;
+			    return -1;
 			}
 			break;
 		    }
 		    default:
 			zerr("unknown sort specifier");
 			restore_globstate(saved);
-			return;
+			return -1;
 		    }
 		    if (t != GS_EXEC) {
 			if ((sense & 2) && !(t & (GS_NAME|GS_DEPTH)))
@@ -1563,7 +1573,7 @@ zglob(LinkList list, LinkNode np, int nountok)
 			if (gf_sorts & t) {
 			    zerr("doubled sort specifier");
 			    restore_globstate(saved);
-			    return;
+			    return -1;
 			}
 		    }
 		    gf_sorts |= t;
@@ -1600,7 +1610,7 @@ zglob(LinkList list, LinkNode np, int nountok)
 		    if (getindex(&s, &v, 0) || s == os) {
 			zerr("invalid subscript");
 			restore_globstate(saved);
-			return;
+			return -1;
 		    }
 		    first = v.start;
 		    end = v.end;
@@ -1623,7 +1633,7 @@ zglob(LinkList list, LinkNode np, int nountok)
 		    untokenize(--s);
 		    zerr("unknown file attribute: %c", *s);
 		    restore_globstate(saved);
-		    return;
+		    return -1;
 		}
 	    }
 	    if (func) {
@@ -1648,7 +1658,7 @@ zglob(LinkList list, LinkNode np, int nountok)
 	    }
 	    if (errflag) {
 		restore_globstate(saved);
-		return;
+		return -1;
 	    }
 	}
 
@@ -1710,11 +1720,11 @@ zglob(LinkList list, LinkNode np, int nountok)
 	    if (!nountok)
 		untokenize(ostr);
 	    insertlinknode(list, node, ostr);
-	    return;
+	    return -1;
 	}
 	errflag = 0;
 	zerr("bad pattern: %s", ostr);
-	return;
+	return -1;
     }
     if (!gf_nsorts) {
 	gf_sortlist[0].tp = gf_sorts = GS_NAME;
@@ -1722,14 +1732,22 @@ zglob(LinkList list, LinkNode np, int nountok)
     }
     /* Initialise receptacle for matched files, *
      * expanded by insert() where necessary.    */
-    matchptr = matchbuf = (Gmatch)zalloc((matchsz = 16) *
-					 sizeof(struct gmatch));
-    matchct = 0;
+    if (!shortcircuit) {
+	matchptr = matchbuf = (Gmatch)zalloc((matchsz = 16) *
+					     sizeof(struct gmatch));
+	matchct = 0;
+    }
     pattrystart();
 
     /* The actual processing takes place here: matches go into  *
      * matchbuf.  This is the only top-level call to scanner(). */
-    scanner(q);
+    if (scanner(q, shortcircuit) == 1)
+	return 1;
+    else if (shortcircuit) {
+	/* ### TODO: this returns 0 even if scanner() returned -1 */
+	restore_globstate(saved);
+	return 0;
+    }
 
     /* Deal with failures to match depending on options */
     if (matchct)
@@ -1741,7 +1759,7 @@ zglob(LinkList list, LinkNode np, int nountok)
 	    zerr("no matches found: %s", ostr);
 	    free(matchbuf);
 	    restore_globstate(saved);
-	    return;
+	    return -1;
 	} else {
 	    /* treat as an ordinary string */
 	    untokenize(matchptr->name = dupstring(ostr));
@@ -1844,6 +1862,7 @@ zglob(LinkList list, LinkNode np, int nountok)
     free(matchbuf);
 
     restore_globstate(saved);
+    return 0;
 }
 
 /* Return the trailing character for marking file types */
@@ -1995,7 +2014,7 @@ xpandredir(struct redir *fn, LinkList redirtab)
     prefork(&fake, isset(MULTIOS) ? 0 : PREFORK_SINGLE);
     /* Globbing is only done for multios. */
     if (!errflag && isset(MULTIOS))
-	globlist(&fake, 0);
+	globlist(&fake, 0, 0);
     if (errflag)
 	return 0;
     if (nonempty(&fake) && !nextnode(firstnode(&fake))) {
diff --git a/Src/parse.c b/Src/parse.c
index 530a070..52b3218 100644
--- a/Src/parse.c
+++ b/Src/parse.c
@@ -2124,7 +2124,7 @@ par_cond_2(void)
     }
     s1 = tokstr;
     if (condlex == testlex)
-	dble = (*s1 == '-' && strspn(s1+1, "abcdefghknoprstuwxzLONGS") == 1
+	dble = (*s1 == '-' && strspn(s1+1, "abcdefghkmnoprstuwxzLONGS") == 1
 		  && !s1[2]);
     condlex();
     if (tok == INANG || tok == OUTANG) {
@@ -2178,7 +2178,7 @@ par_cond_double(char *a, char *b)
 {
     if (a[0] != '-' || !a[1])
 	COND_ERROR("parse error: condition expected: %s", a);
-    else if (!a[2] && strspn(a+1, "abcdefgknoprstuwxzhLONGS") == 1) {
+    else if (!a[2] && strspn(a+1, "abcdefghkmnoprstuwxzLONGS") == 1) {
 	ecadd(WCB_COND(a[1], 0));
 	ecstr(b);
     } else {
diff --git a/Src/subst.c b/Src/subst.c
index 4713502..9ab04b3 100644
--- a/Src/subst.c
+++ b/Src/subst.c
@@ -370,18 +370,20 @@ quotesubst(char *str)
 }
 
 /**/
-mod_export void
-globlist(LinkList list, int nountok)
+mod_export int
+globlist(LinkList list, int shortcircuit, int nountok)
 {
     LinkNode node, next;
 
     badcshglob = 0;
     for (node = firstnode(list); !errflag && node; node = next) {
 	next = nextnode(node);
-	zglob(list, node, nountok);
+	if (zglob(list, node, shortcircuit, nountok) == 1)
+	    return 1;
     }
     if (badcshglob == 1)
 	zerr("no match");
+    return 0;
 }
 
 /* perform substitution on a single word */
diff --git a/Test/C02cond.ztst b/Test/C02cond.ztst
index 94fca8b..0c6cfbf 100644
--- a/Test/C02cond.ztst
+++ b/Test/C02cond.ztst
@@ -70,6 +70,9 @@
   [[ -k modish && ! -k zerolength ]]
 0:-k cond
 
+  [[ -m zero* ]] && [[ ! -m *nonexistent* ]]
+0:-m cond
+
   foo=foo
   bar=
   [[ -n $foo && ! -n $bar && ! -n '' ]]
-- 
1.7.10.4

Re: globbing in conditional expressions

[Bart Schaefer]

On May 14,  4:19am, Daniel Shahaf wrote:
}
} > You could define (via zmodload) an operator that applies filename
} > generation to its argument, but changes to the internals of globbing
} > would be needed to make a short-circuit happen.  Then those changes
} > would have to be exposed somehow so that the operator could use them.
} 
} I've taken a shot at making those changes, see attached.

Interesting!  A few thoughts just reading through the patch:

I'm very curious why you chose to insert a new parameter in the middle
of the globlist() and zglob() signatures, instead of appending it.

The way we'd have typically done this in the past would be to keep the
old call signature as it was and introduce a new entry point with the
new signature, to isolate the changes to as few files as possible.
This also makes it easier to match meaningful changes to the commits
that made them, given that nearly all of the changes in the calls are
not changes of semantics.

(The main body of the old function would become the body for the new
call signature, to which the old signature would then call through.
On the other hand, that does lead to some clutter, so I don't really
feel that strongly either way.)

The strspn(..., "abcdefghkmnoprstuwxzLONGS") probably should NOT have
the "m" added to them, because those are intended to be identifying
operators that are defined by POSIX "test".  It'd be a bit difficult
to use [ -m ... ] anyway because the glob pattern would expand before
the call to the test builtin.

Do I read correctly that "shortcircuit" also means "don't return any
file names, just return an indication of whether there is such a file"
??  (Since you don't allocate the matchbuf array when shortcircuit.)
If so, there might be a more descriptive name for it, "search_only"
or something.

The test [[ -m zero* ]] && [[ ! -m *nonexistent* ]] could be written
[[ -m zero* && ! -m *nonexistent* ]] -- if that doesn't parse (I see
no reason to think it wouldn't) then something is wrong.  Did you
check path searches like **/zero* and whether glob flags work right?

I may give this patch a try tomorrow sometime.

Re: globbing in conditional expressions

[Christoph (Stucki) von Stuckrad]

On Wed, 14 May 2014, Daniel Shahaf wrote:

> Bart Schaefer wrote on Tue, May 13, 2014 at 08:41:17 -0700:
...
> Right, so short-circuiting would only be useful for callers that want to
> know whether a pattern matches a large directory, but don't care about
> having a list of matching filenames.  Does anyone have such a use-case?

I often thought about 'this kind of check' while writing a cron script
to analyze logs.  This often starts with 'do a lot of things, but if
and only if there is a log present in /var/log/..some_pattern...

Of course setting null_glob and doing "for name in /var/log/...pattern..."
will do nearly the same, but enclosing all preliminaries for a set of
files AND their loop together in a clear question for the need to run,
seems preferable to me - so  I second creating '-m' ;-)

Stucki



-- 
Christoph von Stuckrad      * * |nickname |Mail <stucki@mi.fu-berlin.de> \
Freie Universitaet Berlin   |/_*|'stucki' |Tel(Mo.,Mi.):+49 30 838-75 459|
Mathematik & Informatik EDV |\ *|if online|  (Di,Do,Fr):+49 30 77 39 6600|
Takustr. 9 / 14195 Berlin   * * |on IRCnet|Fax(home):   +49 30 77 39 6601/

Re: globbing in conditional expressions

[Roman Neuhauser]

# d.s@daniel.shahaf.name / 2014-05-14 04:19:08 +0000:
> Bart Schaefer wrote on Tue, May 13, 2014 at 08:41:17 -0700:
> > On May 8, 10:19pm, Roman Neuhauser wrote:
> > }
> > } maybe a crazy idea...  how about something like [[ -m pattern ]] which
> > } would succeed iff pattern matched at least one path?  this could be
> > } somewhat more amenable to shortcircuiting.
> > 
> > Returning to this after a bit of a detour through [[ ... ]] expression
> > parsing:
> > 
> > You could define (via zmodload) an operator that applies filename
> > generation to its argument, but changes to the internals of globbing
> > would be needed to make a short-circuit happen.  Then those changes
> > would have to be exposed somehow so that the operator could use them.
> > 
> 
> I've taken a shot at making those changes, see attached.

wonderful, thank you!

> > As I mentioned before, in the case of the match failing this would be
> > exactly as expensive as not short-circuiting.
> 
> Right, so short-circuiting would only be useful for callers that want to
> know whether a pattern matches a large directory, but don't care about
> having a list of matching filenames.  Does anyone have such a use-case?

s/a large directory// and you have exactly my use case: i only care
about simple and succint code.  (lack of) improved efficiency was Bart's
concern.  my original use case was along these lines:

  if ! is-mount $dst; then
    if ! is-empty $dst; then
      printf "%s: garbage in %s\n" $0 $dst
      return 1
    fi
    run sudo mount --bind $src $dst
  fi

where is-empty is now forced to

  function is-empty # {{{
  {   
    local -a d
    d=(${1:?}(N/F))
    (( $#d == 0 ))
  } # }}}

whereas i expected to be able to write sth like

  if ! is-mount $dst; then
    if [[ -m $dst/*(D) ]]; then

or

  if ! is-mount $dst; then
    if [[ -m $dst(/F) ]]; then

this piece of code deals with directories typically under 10 entries
so i don't care about the performance impact of shortcircuting.

if i cared about the matched filenames, the code would be structured
differently and proably wouldn't call for a condexpr at all.

i'm trying to think of a situation where [[ -m $pat ]] && mangle $REPLY
would be useful for something other than foot-shooting... any ideas?

-- 
roman

Re: globbing in conditional expressions

[Bart Schaefer]

On May 14, 12:18am, Bart Schaefer wrote:
} Subject: Re: globbing in conditional expressions
}
} On May 14,  4:19am, Daniel Shahaf wrote:
} }
} } I've taken a shot at making those changes, see attached.
} 
} The strspn(..., "abcdefghkmnoprstuwxzLONGS") probably should NOT have
} the "m" added to them, because those are intended to be identifying
} operators that are defined by POSIX "test".

Upon testing, the strspn in par_cond_double() needs to have the "m".
It can be left out of strspn() in par_cond_2().

} to use [ -m ... ] anyway because the glob pattern would expand before
} the call to the test builtin.

Playing with this, I find that you can quote the pattern [ -m 'pat' ].
However, quoting the pattern in [[ -m 'pat' ]] does not have the
expected effect -- a glob is performed even though the pattern was
quoted, so there's no way to make ? * [ ] match literally.  Perhaps
not a show-stopper.
 
} check path searches like **/zero* and whether glob flags work right?

I tried this and it appears glob patterns confuse things:

torch% ls b*
zsh: no matches found: b*
torch% [[ -m b ]] && print OK 
torch% [[ -m b(/) ]] && print OK
OK

So, still a few issues to iron out.

Re: globbing in conditional expressions

[Daniel Shahaf]

Roman Neuhauser wrote on Wed, May 14, 2014 at 23:09:23 +0200:
> # d.s@daniel.shahaf.name / 2014-05-14 04:19:08 +0000:
> > Bart Schaefer wrote on Tue, May 13, 2014 at 08:41:17 -0700:
> > > On May 8, 10:19pm, Roman Neuhauser wrote:
> > > }
> > > } maybe a crazy idea...  how about something like [[ -m pattern ]] which
> > > } would succeed iff pattern matched at least one path?  this could be
> > > } somewhat more amenable to shortcircuiting.
> > > 
> > > Returning to this after a bit of a detour through [[ ... ]] expression
> > > parsing:
> > > 
> > > You could define (via zmodload) an operator that applies filename
> > > generation to its argument, but changes to the internals of globbing
> > > would be needed to make a short-circuit happen.  Then those changes
> > > would have to be exposed somehow so that the operator could use them.
> > > 
> > 
> > I've taken a shot at making those changes, see attached.
> 
> wonderful, thank you!
> 

You're welcome, but beware: the code doesn't work properly in all cases,
e.g., as Bart suspected, [[ -m **/*attr.yo(/) ]] (in a built zsh source
tree) false positives.

> i'm trying to think of a situation where [[ -m $pat ]] && mangle $REPLY
> would be useful for something other than foot-shooting... any ideas?

Well, there is:

    while [[ -m $pat ]] ;
      # Suppose -m implicitly sets $REPLY to a matching filename ...
      () { mangle $1 && rm $1 } $REPLY

Since the filename returned would be the first in readdir() order, this
could result in starvation (some files never reaching mangle()) unless
the underlying filesystem's readdir() provides more guarantees than the
POSIX readdir() does.

Re: globbing in conditional expressions

[Daniel Shahaf]

Bart Schaefer wrote on Wed, May 14, 2014 at 00:18:19 -0700:
> Do I read correctly that "shortcircuit" also means "don't return any
> file names, just return an indication of whether there is such a file"
> ??  (Since you don't allocate the matchbuf array when shortcircuit.)
> 

Of course.  As soon as you find a single matching filename you return to
the caller, so there are two options: either return just a boolean, or
return the first matching filename in readdir() order (via $REPLY, as in
Roman's post).  I'm not sure how idiomatic it would be for a [[ -x ]]
condition to set $REPLY.

> If so, there might be a more descriptive name for it, "search_only"
> or something.

I don't really how it's called.  Some more options are "list_matches"
(default TRUE) or "boolean_context" (default FALSE).  I'm fine with any
choice.

Re: globbing in conditional expressions

[Daniel Shahaf]

Bart Schaefer wrote on Wed, May 14, 2014 at 21:55:57 -0700:
> On May 14, 12:18am, Bart Schaefer wrote:
> } to use [ -m ... ] anyway because the glob pattern would expand before
> } the call to the test builtin.
> 
> Playing with this, I find that you can quote the pattern [ -m 'pat' ].
> However, quoting the pattern in [[ -m 'pat' ]] does not have the
> expected effect -- a glob is performed even though the pattern was
> quoted, so there's no way to make ? * [ ] match literally.  Perhaps
> not a show-stopper.
>  
> } check path searches like **/zero* and whether glob flags work right?
> 
> I tried this and it appears glob patterns confuse things:
...
> So, still a few issues to iron out.

Indeed.  Thanks for taking the time to test drive this.  I don't have
much time for this; if I make progress I'll post a new iteration to the
list, but in the meantime, perhaps someone else interested in the patch
could pick it up and make a new version.

Cheers,

Daniel

Re: globbing in conditional expressions

[Roman Neuhauser]

# d.s@daniel.shahaf.name / 2014-05-15 09:14:58 +0000:
> Roman Neuhauser wrote on Wed, May 14, 2014 at 23:09:23 +0200:
> > wonderful, thank you!
> 
> You're welcome, but beware: the code doesn't work properly in all cases,
> e.g., as Bart suspected, [[ -m **/*attr.yo(/) ]] (in a built zsh source
> tree) false positives.

understood, still, thanks a lot for kickstarting it.

> > i'm trying to think of a situation where [[ -m $pat ]] && mangle $REPLY
> > would be useful for something other than foot-shooting... any ideas?
> 
> Well, there is:
> 
>     while [[ -m $pat ]] ;
>       # Suppose -m implicitly sets $REPLY to a matching filename ...
>       () { mangle $1 && rm $1 } $REPLY
> 
> Since the filename returned would be the first in readdir() order, this
> could result in starvation (some files never reaching mangle()) unless
> the underlying filesystem's readdir() provides more guarantees than the
> POSIX readdir() does.

that's the foot-shooting part. :)  the above imo screams for change to

  for f in $~pat; do
    mangle $f && rm $f
  done

globbing can provide for ordering (like if you needed the oldest or
largest file).

if you need a random file, shortcircuiting readdir() is not the answer.

-- 
roman

Re: globbing in conditional expressions

[Bart Schaefer]

On May 15,  9:35am, Daniel Shahaf wrote:
}
} Bart Schaefer wrote on Wed, May 14, 2014 at 21:55:57 -0700:
} > 
} > I tried this and it appears glob patterns confuse things:
} 
} Indeed.  Thanks for taking the time to test drive this.

I'm 90% sure that the problem with the glob flags has to do with short-
circuiting at the wrong place, but I haven't tried very hard to find
which is wrong or where is right.

Re: globbing in conditional expressions

[Bart Schaefer]

On May 15,  9:29am, Daniel Shahaf wrote:
} Subject: Re: globbing in conditional expressions
}
} Bart Schaefer wrote on Wed, May 14, 2014 at 00:18:19 -0700:
} > Do I read correctly that "shortcircuit" also means "don't return any
} > file names, just return an indication of whether there is such a file"
} > ??  (Since you don't allocate the matchbuf array when shortcircuit.)
} 
} Of course.  As soon as you find a single matching filename you return to
} the caller, so there are two options: either return just a boolean, or
} return the first matching filename in readdir() order (via $REPLY, as in
} Roman's post).  I'm not sure how idiomatic it would be for a [[ -x ]]
} condition to set $REPLY.

I would not suggest that [[ -m ... ]] set $REPLY, but why does -m have
to be the only place where short-circuiting occurs?  I could imagine
adding a globbing flag (#X) [where I'm using X as a placeholder rather
than a suggestion for the actual flag] which means that the glob returns
only the first matching file it stumbles upon.

In fact if you had that flag you wouldn't need -m as a condition op, it
would suffice to do [ ! -z pat(NX) ].  The implementation also would not
need to change the call signatures of any of the C functions ...

Re: globbing in conditional expressions

[Daniel Shahaf]

[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]

Bart Schaefer wrote on Thu, May 15, 2014 at 07:50:03 -0700:
> On May 15,  9:29am, Daniel Shahaf wrote:
> } Subject: Re: globbing in conditional expressions
> }
> } Bart Schaefer wrote on Wed, May 14, 2014 at 00:18:19 -0700:
> } > Do I read correctly that "shortcircuit" also means "don't return any
> } > file names, just return an indication of whether there is such a file"
> } > ??  (Since you don't allocate the matchbuf array when shortcircuit.)
> } 
> } Of course.  As soon as you find a single matching filename you return to
> } the caller, so there are two options: either return just a boolean, or
> } return the first matching filename in readdir() order (via $REPLY, as in
> } Roman's post).  I'm not sure how idiomatic it would be for a [[ -x ]]
> } condition to set $REPLY.
> 
> I would not suggest that [[ -m ... ]] set $REPLY, but why does -m have
> to be the only place where short-circuiting occurs?  I could imagine
> adding a globbing flag (#X) [where I'm using X as a placeholder rather
> than a suggestion for the actual flag] which means that the glob returns
> only the first matching file it stumbles upon.
> 
> In fact if you had that flag you wouldn't need -m as a condition op, it
> would suffice to do [ ! -z pat(NX) ].  The implementation also would not
> need to change the call signatures of any of the C functions ...

That turned out to be much less code to implement.  It's attached, if
anyone finds it useful.

I couldn't get it to work via [ -z ], but this works:

% Src/zsh -fc '() { (( $#@ )) } *(NY); echo $?' 
0

% Src/zsh -fc '() { (( $#@ )) } *foobar(NY); echo $?'
1


[-- Attachment #2: 0001-Add-Y-glob-qualifier-that-short-circuits.patch --]
[-- Type: text/x-patch, Size: 5342 bytes --]

>From 23eafe9265691c0ea6d705100c1ee9e84850efa5 Mon Sep 17 00:00:00 2001
From: Daniel Shahaf <d.s@daniel.shahaf.name>
Date: Tue, 13 May 2014 10:18:56 +0000
Subject: [PATCH] Add (Y) glob qualifier that short-circuits.

---
 Doc/Zsh/expn.yo   |    4 ++++
 Src/glob.c        |   35 ++++++++++++++++++++++++-----------
 Test/D02glob.ztst |   11 +++++++++++
 3 files changed, 39 insertions(+), 11 deletions(-)

diff --git a/Doc/Zsh/expn.yo b/Doc/Zsh/expn.yo
index de0f454..65374e8 100644
--- a/Doc/Zsh/expn.yo
+++ b/Doc/Zsh/expn.yo
@@ -2560,6 +2560,10 @@ item(tt(n))(
 sets the tt(NUMERIC_GLOB_SORT) option for the current pattern
 pindex(NUMERIC_GLOB_SORT, setting in pattern)
 )
+item(tt(Y))(
+enables short-circuit mode: the pattern will expand to just the first
+matching filename, if any.
+)
 item(tt(o)var(c))(
 specifies how the names of the files should be sorted. If var(c) is
 tt(n) they are sorted by name (the default); if it is tt(L) they
diff --git a/Src/glob.c b/Src/glob.c
index 07dd7c2..eafb702 100644
--- a/Src/glob.c
+++ b/Src/glob.c
@@ -452,8 +452,8 @@ insert(char *s, int checked)
  * tried all of it.                                */
 
 /**/
-static void
-scanner(Complist q)
+static int
+scanner(Complist q, int shortcircuit)
 {
     Patprog p;
     int closure;
@@ -463,14 +463,15 @@ scanner(Complist q)
 
     init_dirsav(&ds);
     if (!q)
-	return;
+	return -1;
 
     if ((closure = q->closure)) {
 	/* (foo/)# - match zero or more dirs */
 	if (q->closure == 2)	/* (foo/)## - match one or more dirs */
 	    q->closure = 1;
 	else
-	    scanner(q->next);
+	    if (scanner(q->next, shortcircuit) == 1)
+		return 1;
     }
     p = q->pat;
     /* Now the actual matching for the current path section. */
@@ -485,13 +486,13 @@ scanner(Complist q)
 	    int err;
 
 	    if (l >= PATH_MAX)
-		return;
+		return -1;
 	    err = lchdir(pathbuf + pathbufcwd, &ds, 0);
 	    if (err == -1)
-		return;
+		return -1;
 	    if (err) {
 		zerr("current directory lost during glob");
-		return;
+		return -1;
 	    }
 	    pathbufcwd = pathpos;
 	}
@@ -516,7 +517,8 @@ scanner(Complist q)
 		if (add) {
 		    addpath(str, l);
 		    if (!closure || !statfullpath("", NULL, 1))
-			scanner((q->closure) ? q : q->next);
+			if (scanner((q->closure) ? q : q->next, shortcircuit) == 1)
+			    return 1;
 		    pathbuf[pathpos = oppos] = '\0';
 		}
 	    }
@@ -524,6 +526,8 @@ scanner(Complist q)
 	    if (str[l])
 		str = dupstrpfx(str, l);
 	    insert(str, 0);
+	    if (shortcircuit)
+		return 1;
 	}
     } else {
 	/* Do pattern matching on current path section. */
@@ -534,7 +538,7 @@ scanner(Complist q)
 	int subdirlen = 0;
 
 	if (lock == NULL)
-	    return;
+	    return -1;
 	while ((fn = zreaddir(lock, 1)) && !errflag) {
 	    /* prefix and suffix are zle trickery */
 	    if (!dirs && !colonmod &&
@@ -614,6 +618,8 @@ scanner(Complist q)
 		} else
 		    /* if the last filename component, just add it */
 		    insert(fn, 1);
+		    if (shortcircuit)
+			return 1;
 	    }
 	}
 	closedir(lock);
@@ -626,7 +632,8 @@ scanner(Complist q)
 		fn += l + 1;
 		memcpy((char *)&errsfound, fn, sizeof(int));
 		fn += sizeof(int);
-		scanner((q->closure) ? q : q->next);  /* scan next level */
+		if (scanner((q->closure) ? q : q->next, shortcircuit) == 1) /* scan next level */
+		    return 1;
 		pathbuf[pathpos = oppos] = '\0';
 	    }
 	    hrealloc(subdirs, subdirlen, 0);
@@ -640,6 +647,7 @@ scanner(Complist q)
 	    close(ds.dirfd);
 	pathbufcwd = pbcwdsav;
     }
+    return 0;
 }
 
 /* This function tokenizes a zsh glob pattern */
@@ -1080,6 +1088,7 @@ zglob(LinkList list, LinkNode np, int nountok)
 					/* and index+1 of the last match */
     struct globdata saved;		/* saved glob state              */
     int nobareglob = !isset(BAREGLOBQUAL);
+    int shortcircuit = 0;
 
     if (unset(GLOBOPT) || !haswilds(ostr) || unset(EXECOPT)) {
 	if (!nountok)
@@ -1461,6 +1470,10 @@ zglob(LinkList list, LinkNode np, int nountok)
 		    /* Numeric glob sort */
 		    gf_numsort = !(sense & 1);
 		    break;
+		case 'Y':
+		    /* Short circuit: just check if there are any matches */
+		    shortcircuit = !(sense & 1);
+		    break;
 		case 'a':
 		    /* Access time in given range */
 		    g_amc = 0;
@@ -1729,7 +1742,7 @@ zglob(LinkList list, LinkNode np, int nountok)
 
     /* The actual processing takes place here: matches go into  *
      * matchbuf.  This is the only top-level call to scanner(). */
-    scanner(q);
+    scanner(q, shortcircuit);
 
     /* Deal with failures to match depending on options */
     if (matchct)
diff --git a/Test/D02glob.ztst b/Test/D02glob.ztst
index 1f8f652..0ff088a 100644
--- a/Test/D02glob.ztst
+++ b/Test/D02glob.ztst
@@ -431,6 +431,7 @@
  mkdir glob.tmp/dir5
  touch glob.tmp/dir5/N123
  print glob.tmp/dir5/N<->(N)
+ rm -rf glob.tmp/dir5
 0:Numeric glob is not usurped by process substitution.
 >glob.tmp/dir5/N123
 
@@ -526,3 +527,13 @@
 >+bus+bus matches +(+bus|-car)
 >@sinhats matches @(@sinhats|wrensinfens)
 >!kerror matches !(!somethingelse)
+
+ (){ print $#@ } glob.tmp/dir*(Y)
+ (){ print $#@ } glob.tmp/file*(NY)
+ (){ [[ $1 = glob.tmp/dir? ]] && echo "(Y) returns a matching filename" } glob.tmp/dir*(Y)
+ (){ print $@:t } glob.tmp/dir*(Y^Y)
+0:short-circuit modifier
+>1
+>0
+>(Y) returns a matching filename
+>dir1 dir2 dir3 dir4
-- 
1.7.10.4

Re: globbing in conditional expressions

[Bart Schaefer]

On May 26, 11:52pm, Daniel Shahaf wrote:
}
} 
} Bart Schaefer wrote on Thu, May 15, 2014 at 07:50:03 -0700:
} > adding a globbing flag (#X) [where I'm using X as a placeholder rather
} > than a suggestion for the actual flag] which means that the glob returns
} > only the first matching file it stumbles upon.
} > 
} > In fact if you had that flag you wouldn't need -m as a condition op, it
} > would suffice to do [ ! -z pat(NX) ].
} 
} That turned out to be much less code to implement.  It's attached, if
} anyone finds it useful.

I like this one.  Is there a mnemonic for your choice of (Y) ?

} I couldn't get it to work via [ -z ]

Hmm, seems to work for me:

torch% print co*
config.h config.log config.modules config.modules.sh config.status
torch% print co*(Y)
config.h
torch% print ba*   
zsh: no matches found: ba*
torch% [ -z ba*(N) ] && echo OK 
OK
torch% [ ! -z ba*(NY) ] && echo OK
torch% [ ! -z co*(NY) ] && echo OK
OK


To zsh-workers:  Any reason not to commit/push this?

Re: globbing in conditional expressions

[Peter Stephenson]

On Thu, 29 May 2014 20:59:56 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> } I couldn't get it to work via [ -z ]
> 
> Hmm, seems to work for me:
>
> torch% [ -z ba*(N) ] && echo OK 
> OK
> torch% [ ! -z ba*(NY) ] && echo OK
> torch% [ ! -z co*(NY) ] && echo OK
> OK

Is this null glob safe with [ ?  I get true from both "[ -n ]" and
"[ -z ]" suggesting the case of an elided argument isn't properly
handled.

But this is specific to zsh --- why are we using [ rather than [[ anyway?

> To zsh-workers:  Any reason not to commit/push this?

Not that I know of.

pws

Re: globbing in conditional expressions

[Daniel Shahaf]

Bart Schaefer wrote on Thu, May 29, 2014 at 20:59:56 -0700:
> On May 26, 11:52pm, Daniel Shahaf wrote:
> }
> } 
> } Bart Schaefer wrote on Thu, May 15, 2014 at 07:50:03 -0700:
> } > adding a globbing flag (#X) [where I'm using X as a placeholder rather
> } > than a suggestion for the actual flag] which means that the glob returns
> } > only the first matching file it stumbles upon.
> } > 
> } > In fact if you had that flag you wouldn't need -m as a condition op, it
> } > would suffice to do [ ! -z pat(NX) ].
> } 
> } That turned out to be much less code to implement.  It's attached, if
> } anyone finds it useful.
> 
> I like this one.  Is there a mnemonic for your choice of (Y) ?

No.

That said, if it's to be commonly used with (N), then (NY) does serve as
a memorable syntax.

Re: globbing in conditional expressions

[Bart Schaefer]

On May 30,  9:47am, Peter Stephenson wrote:
}
} On Thu, 29 May 2014 20:59:56 -0700
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > } I couldn't get it to work via [ -z ]
} > 
} > Hmm, seems to work for me:
} >
} > torch% [ -z ba*(N) ] && echo OK 
} > OK
} > torch% [ ! -z ba*(NY) ] && echo OK
} > torch% [ ! -z co*(NY) ] && echo OK
} > OK
} 
} Is this null glob safe with [ ?  I get true from both "[ -n ]" and
} "[ -z ]" suggesting the case of an elided argument isn't properly
} handled.

Hm.  This is a side-effect of treating [ thing ] as [ -n thing ] :
[ -n ] becomes [ -n -n ] and [ -z ] becomes [ -n -z ].

Is there a non-convoluted way to have the special case of a glob with
BOTH of the (NY) qualifiers return empty string rather than nularg when
it finds no match?  Empty string is an illegal file name so for most
purposes it wouldn't otherwise affect usage.
 
} But this is specific to zsh --- why are we using [ rather than [[ anyway?

Because [[ ]] doesn't do globbing on its arguments, and this is a
filename generation rather than a pattern.  We messed around a few weeks
ago with trying to make [[ ]] do globbing and ran into tokenization
conflicts.

Re: globbing in conditional expressions

[Bart Schaefer]

On May 30,  8:55am, Bart Schaefer wrote:
}
} Is there a non-convoluted way to have the special case of a glob with
} BOTH of the (NY) qualifiers return empty string rather than nularg when
} it finds no match?  Empty string is an illegal file name so for most
} purposes it wouldn't otherwise affect usage.

This turns out to be pretty simple.  It'd additionally be nice if the
CSH_NULL_GLOB option caused any such empty strings to be removed when
some other pattern succeeds, but that seems rather far-reaching.  (The
following will require a tweak to Daniel's test in D02glob.)

diff --git a/Src/glob.c b/Src/glob.c
index 07dd7c2..5aa306b 100644
--- a/Src/glob.c
+++ b/Src/glob.c
@@ -1748,6 +1761,11 @@ zglob(LinkList list, LinkNode np, int nountok)
 	    matchptr++;
 	    matchct = 1;
 	}
+    } else if (shortcircuit) {
+      /* treat as an empty string */
+      matchptr->name = dupstring("");
+      matchptr++;
+      matchct = 1;
     }
 
     if (!(gf_sortlist[0].tp & GS_NONE)) {

Re: globbing in conditional expressions

[Daniel Shahaf]

Bart Schaefer wrote on Fri, May 30, 2014 at 21:43:20 -0700:
> On May 30,  8:55am, Bart Schaefer wrote:
> }
> } Is there a non-convoluted way to have the special case of a glob with
> } BOTH of the (NY) qualifiers return empty string rather than nularg when
> } it finds no match?  Empty string is an illegal file name so for most
> } purposes it wouldn't otherwise affect usage.

I beg to differ.  Making *(NY) return empty strings upon no match is
(by design) convenient for the [ -n *(NY) ] case, but it breaks other
use-cases.  For example,
    frob *.foo(NY) *.bar(NY)
would suddenly need to handle "" files in the argument list.

For testing whether the pattern expands to anything, this is already possible:
    if (){(($#@))} *(NY)
and after naming the anon func, it becomes:
    if has_any_matches *(NY)
which seems to me as readable as [ -n *(NY) ].  (For that matter, we could
define a unary test operator [ --is-not-null ] that acts like -n except
that it returns False rather than True in the nularg case, and write
[ --is-not-null *(NY) ].)

Perhaps an assignment-ish syntax would be useful?  Something like
    if [ -n ${(X)fname:=*(NY)} ]
as shorthand for:
    if _tmp=(*(NY)) && (( $#_tmp )) && fname=$_tmp[1] && [ -n $fname ]
?  That way also makes the matching filename available for the if's
body.  (The X stands for an expansion modifier that would make the RHS
be treated as a glob rather than as a literal string.)

Cheers,

Daniel

> This turns out to be pretty simple.  It'd additionally be nice if the
> CSH_NULL_GLOB option caused any such empty strings to be removed when
> some other pattern succeeds, but that seems rather far-reaching.  (The
> following will require a tweak to Daniel's test in D02glob.)
> 
> diff --git a/Src/glob.c b/Src/glob.c
> index 07dd7c2..5aa306b 100644
> --- a/Src/glob.c
> +++ b/Src/glob.c
> @@ -1748,6 +1761,11 @@ zglob(LinkList list, LinkNode np, int nountok)
>  	    matchptr++;
>  	    matchct = 1;
>  	}
> +    } else if (shortcircuit) {
> +      /* treat as an empty string */
> +      matchptr->name = dupstring("");
> +      matchptr++;
> +      matchct = 1;
>      }
>  
>      if (!(gf_sortlist[0].tp & GS_NONE)) {

Re: globbing in conditional expressions

[Bart Schaefer]

On May 31,  5:34pm, Daniel Shahaf wrote:
} Subject: Re: globbing in conditional expressions
}
} > } Is there a non-convoluted way to have the special case of a glob with
} > } BOTH of the (NY) qualifiers return empty string rather than nularg when
} > } it finds no match?  Empty string is an illegal file name so for most
} > } purposes it wouldn't otherwise affect usage.
} 
} I beg to differ.  Making *(NY) return empty strings upon no match is
} (by design) convenient for the [ -n *(NY) ] case, but it breaks other
} use-cases.

Yeah, and it's unnecessary given [[ -n *(#qNY) ]] works per PWS's patches
on the zsh-workers list, although that requires extendedglob.  I won't
commit users/18866.

Re: globbing in conditional expressions

[Peter Stephenson]

I think there's a problem with the Y qualifier with subdirectories.

I expect

print */*(NY/)

to find a subdirectory of a subdirectory.  Typically I get nothing.

My first guess was it was short-circuiting too early (on the first
segment), but I can't see any evidence that it works at all if there's a
glob before the last path segment.  For example


% print Completio?/*(NY/)

% print Completion/*(NY/)
Completion/Darwin


pws

Re: globbing in conditional expressions

[Daniel Shahaf]

[-- Attachment #1: Type: text/plain, Size: 245 bytes --]

Peter Stephenson wrote on Mon, Jun 02, 2014 at 10:40:50 +0100:
> % print Completio?/*(NY/)
> 
> % print Completion/*(NY/)
> Completion/Darwin

scanner() wrongly assumed that any call to insert() increases matchct.
Test and fix attached.

Daniel

[-- Attachment #2: 0001-After-10ae77c-Fix-users-18869.patch --]
[-- Type: text/x-patch, Size: 3326 bytes --]

>From 72770405cdd21bc75fc8266cd0fa39fb179b7118 Mon Sep 17 00:00:00 2001
From: Daniel Shahaf <d.s@daniel.shahaf.name>
Date: Mon, 2 Jun 2014 11:55:18 +0000
Subject: [PATCH] After 10ae77c: Fix users/18869

---
 Src/glob.c        |   21 +++++++++++----------
 Test/D02glob.ztst |    5 +++++
 2 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/Src/glob.c b/Src/glob.c
index 1420ac7..0ca63fc 100644
--- a/Src/glob.c
+++ b/Src/glob.c
@@ -297,15 +297,16 @@ statfullpath(const char *s, struct stat *st, int l)
 
 char **inserts;
 
-/* add a match to the list */
+/* add a match to the list.  Return 1 if it was inserted, 0 otherwise. */
 
 /**/
-static void
+static int
 insert(char *s, int checked)
 {
     struct stat buf, buf2, *bp;
     char *news = s;
     int statted = 0;
+    int inserted = 0;
 
     queue_signals();
     inserts = NULL;
@@ -316,7 +317,7 @@ insert(char *s, int checked)
 	checked = statted = 1;
 	if (statfullpath(s, &buf, 1)) {
 	    unqueue_signals();
-	    return;
+	    return inserted;
 	}
 	mode = buf.st_mode;
 	if (gf_follow) {
@@ -340,7 +341,7 @@ insert(char *s, int checked)
 
 	if (!statted && statfullpath(s, &buf, 1)) {
 	    unqueue_signals();
-	    return;
+	    return inserted;
 	}
 	news = dyncat(pathbuf, news);
 
@@ -365,7 +366,7 @@ insert(char *s, int checked)
 		/* Try next alternative, or return if there are no more */
 		if (!(qo = qo->or)) {
 		    unqueue_signals();
-		    return;
+		    return inserted;
 		}
 		qn = qo;
 		continue;
@@ -375,7 +376,7 @@ insert(char *s, int checked)
     } else if (!checked) {
 	if (statfullpath(s, NULL, 1)) {
 	    unqueue_signals();
-	    return;
+	    return inserted;
 	}
 	statted = 1;
 	news = dyncat(pathbuf, news);
@@ -435,6 +436,7 @@ insert(char *s, int checked)
 	}
 	matchptr++;
 
+	inserted = 1;
 	if (++matchct == matchsz) {
 	    matchbuf = (Gmatch )realloc((char *)matchbuf,
 					sizeof(struct gmatch) * (matchsz *= 2));
@@ -445,6 +447,7 @@ insert(char *s, int checked)
 	    break;
     }
     unqueue_signals();
+    return inserted;
 }
 
 /* Do the globbing:  scanner is called recursively *
@@ -525,8 +528,7 @@ scanner(Complist q, int shortcircuit)
 	} else {
 	    if (str[l])
 		str = dupstrpfx(str, l);
-	    insert(str, 0);
-	    if (shortcircuit)
+	    if (insert(str, 0) == 1 && shortcircuit)
 		return 1;
 	}
     } else {
@@ -617,8 +619,7 @@ scanner(Complist q, int shortcircuit)
 		    subdirlen += sizeof(int);
 		} else
 		    /* if the last filename component, just add it */
-		    insert(fn, 1);
-		    if (shortcircuit)
+		    if (insert(fn, 1) == 1 && shortcircuit)
 			return 1;
 	    }
 	}
diff --git a/Test/D02glob.ztst b/Test/D02glob.ztst
index d197098..9e29de2 100644
--- a/Test/D02glob.ztst
+++ b/Test/D02glob.ztst
@@ -546,9 +546,14 @@
  (){ print $#@ } glob.tmp/dir*(Y)
  (){ print $#@ } glob.tmp/file*(NY)
  (){ [[ $1 = glob.tmp/dir? ]] && echo "(Y) returns a matching filename" } glob.tmp/dir*(Y)
+ # Can be negated
  (){ print $@:t } glob.tmp/dir*(Y^Y)
+ (){ [[ $#@ -eq 1 ]] && print Globs before last path component } glob.tmp/dir?/subdir(NY)
+ (){ [[ $#@ -eq 0 ]] && print Respects qualifiers } glob.tmp/dir?/subdir(NY.)
 0:short-circuit modifier
 >1
 >0
 >(Y) returns a matching filename
 >dir1 dir2 dir3 dir4
+>Globs before last path component
+>Respects qualifiers
-- 
1.7.10.4

Re: globbing in conditional expressions

[Peter Stephenson]

On Mon, 02 Jun 2014 12:57:23 +0000
Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> Peter Stephenson wrote on Mon, Jun 02, 2014 at 10:40:50 +0100:
> > % print Completio?/*(NY/)
> > 
> > % print Completion/*(NY/)
> > Completion/Darwin
> 
> scanner() wrongly assumed that any call to insert() increases matchct.
> Test and fix attached.

Thanks, I've pushed that up.

pws