$(nooutput) problem

$(nooutput) problem

[Peter Stephenson]

It seems care is needed: (1) if a command is not a simple one no
arguments may be necessary, so that having no arguments in execcmd()
is perfectly legal; (2) single word substitution needs to keeps its
arguments, for example the variable assignment code expects something
back from prefork(); (3) there's another bug/incompatibility (nearer
the latter than the former) resulting from the failure to delete empty
arguments:

% ksh -c 'cmd=; $cmd print foo'
foo
% zsh -fc 'cmd=; $cmd print foo'
zsh: permission denied: print

(which is certainly not the right answer in any case).

For these reasons I've fixed the problem by getting prefork() to junk
empty nodes if and only if it is not doing single word substitution,
and execcmd() to return if and only if it is a `simple' command with
no arguments.

*** Src/exec.c.empty	Fri May 17 16:25:20 1996
--- Src/exec.c	Tue May 28 10:35:36 1996
***************
*** 1200,1205 ****
--- 1200,1208 ----
      /* Do prefork substitutions */
      prefork(args, (((type == CCASE) ? 4 : 0) | (assign ? 2 : isset(MAGICEQUALSUBST))));
  
+     if (type == SIMPLE && empty(args))
+ 	return;
+ 
      /* Set up special parameter $_ */
      zsfree(underscore);
      if (nonempty(args) && (underscore = ztrdup((char *) getdata(lastnode(args)))))
*** Src/subst.c.empty	Tue May 28 10:34:04 1996
--- Src/subst.c	Tue May 28 10:41:15 1996
***************
*** 51,57 ****
  void
  prefork(LinkList list, int flags)
  {
!     LinkNode node;
  
      for (node = firstnode(list); node; incnode(node)) {
  	char *str, *str3;
--- 51,57 ----
  void
  prefork(LinkList list, int flags)
  {
!     LinkNode node, next;
  
      for (node = firstnode(list); node; incnode(node)) {
  	char *str, *str3;
***************
*** 69,77 ****
  	else if (!(node = stringsubst(list, node, flags & 4)))
  	    return;
      }
!     for (node = firstnode(list); node; incnode(node)) {
  	if (*(char *)getdata(node))
  	    remnulargs(getdata(node));
  	if (unset(IGNOREBRACES) && !(flags & 4))
  	    while (hasbraces(getdata(node)))
  		xpandbraces(list, &node);
--- 69,82 ----
  	else if (!(node = stringsubst(list, node, flags & 4)))
  	    return;
      }
!     for (node = firstnode(list); node; node = next) {
! 	next = nextnode(node);
  	if (*(char *)getdata(node))
  	    remnulargs(getdata(node));
+ 	else if (!(flags & 4)) {
+ 	    uremnode(list, node);
+ 	    continue;
+ 	}
  	if (unset(IGNOREBRACES) && !(flags & 4))
  	    while (hasbraces(getdata(node)))
  		xpandbraces(list, &node);


-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77330
Deutches Electronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.

Re: $(nooutput) problem

[Zoltan Hidvegi]

> It seems care is needed: (1) if a command is not a simple one no
> arguments may be necessary, so that having no arguments in execcmd()
> is perfectly legal; (2) single word substitution needs to keeps its
> arguments, for example the variable assignment code expects something
> back from prefork(); (3) there's another bug/incompatibility (nearer
> the latter than the former) resulting from the failure to delete empty
> arguments:
> 
> % ksh -c 'cmd=; $cmd print foo'
> foo
> % zsh -fc 'cmd=; $cmd print foo'
> zsh: permission denied: print
> 
> (which is certainly not the right answer in any case).
> 
> For these reasons I've fixed the problem by getting prefork() to junk
> empty nodes if and only if it is not doing single word substitution,
> and execcmd() to return if and only if it is a `simple' command with
> no arguments.

I'm thinking about a different solution.  The problem is more complicated
since if one write foo* and there is no file beginning with foo and
nullglob is set globlist() will produce empty args.  I think globlist
should be move before fork().  Probably it should be executed right after
prefork.  fixcline may come right after that.  In execcmd there is a test
for empty(args).  This should be moved after prefork/globlist/fixcline.

Also I think that command, exec, noglob, nocorrect and - should be removed
from reswdtab and the related code should be removed from parse.c, and
these should be handled in execcmd.  This would enable to do things like

FOO=exec ; $FOO something

which whould improve sh compatibility.  In bash and ksh exec is a builtin
so it can be disabled or a function called exec can be defined.  This may
be implemented is zsh as well.  Unfortunately I do not fully understand
exec.c so it may take a while for me unless someone else wiser than me
makes these changes.

Zoltan

execcmd() reordering

[Peter Stephenson]

hzoli@cs.elte.hu wrote about problem with command lines expanding to nothing:
> The problem is more complicated
> since if one write foo* and there is no file beginning with foo and
> nullglob is set globlist() will produce empty args.  I think globlist
> should be move before fork().  Probably it should be executed right after
> prefork.  fixcline may come right after that.  In execcmd there is a test
> for empty(args).  This should be moved after prefork/globlist/fixcline.
> 
> Also I think that command, exec, noglob, nocorrect and - should be removed
> from reswdtab and the related code should be removed from parse.c, and
> these should be handled in execcmd.  This would enable to do things like
> 
> FOO=exec ; $FOO something
> 
> which whould improve sh compatibility.

Here's a biggish patch which does that.  I've listed the features of
the change below, in some detail since there are a few
incompatibilities which seem to be minor enough not to cause me any
loss of sleep.  (With a bit of luck, even Bart won't have to change
anything :-)).

WARNING: execcmd() is the center of the whole shooting match.  I
simply can't guarantee that changing things around in it won't have
some unexpected effect (read the following and you'll see what I
mean), though I think I've caught the obvious ones.  Please try this
out if you get a chance.  (It took me half an hour to realise I'd
broken nullexecs, it's that sort of piece of code.)

1) exec, noglob, - and command are treated more like commands; they can
appear from substitution, though not globbing (which would be stupid).
I've hijacked the last1 variable to tell execcmd() if it's the last
command in a pipeline: if it isn't, any exec flag gets ignored.  This
corresponds to the old behaviour implemented in parse.c (exactly, even
up to disallowing `exec >foo' in previous pipeline stages).

2) Now `nocorrect' must appear before any of the above.  Logical,
perhaps, considering that it is applied while the line is still being
parsed, but a change from before when you could say `noglob nocorrect'.

3) Also, variable assignment must appear before exec and friends.
Again, this is natural and is required in other Bourne shell clones.
Unfortunately there was a bug in old versions of the shell that you
couldn't use variable assignment *before* exec, and there was a
workaround that you could put it after.  This now won't work.  This is
probably the most noticeable incompatibility for people with legacy
software problems.

4) $(command-producing-no-output) bugs fixed, also
`blank=; $blank print foo' correctly calls the print builtin.
This patch therefore replaces the previous fix for this.

5) `noglob typeset foo=~/file' do not do tilde expansion without
    ^^^^^^ ^^^^^^
     etc.   etc.
magic_glob_subst set.  Other shells are more generous about when the
`=~' sequence is expanded anyway.  Maybe we should change.  Even
magic_equal_subst is fairly sparing about when to expand:  it insists
on the text before the = consisting only of characters which appear in
identifiers, something I wrote but now rather regret.

6) globbing is expanded before the fork.  Other than getting the
prompt back an iota later when running background commands, I don't
see this is a big deal.  One thing which seems to be a plus is that
failures to match are now handled synchronously for background
commands:

% unsetopt nonomatch
% ls Foo*Bar &
zsh: no matches found: Foo*Bar

instead of something like:

% ls Foo*Bar &
[1] 7621
% zsh: no matches found: Foo*Bar
% 

7) The command line is checked after expansion (but again before
globbing which didn't seem appropriate here) for things like
redirection with no command, implicit fg's and autoresume.  rm *
checks and autocd already behaved this way.

I suppose you might think the restriction that you can't autoresume a
command in the current directory with ./a*, or whatever, somewhat
arbitrary.  This can still be reordered if it's wanted.  The main
point is that the check for AUTORESUME comes most logically after the
one for %something, and that shouldn't be globbed.

Note, however, that globbing of redirs happens elsewhere, so you have
always been able to do `exec <INP*'.  What you can't do is something
like `exec *(N) <foo' where * expands to nothing and expect stdin to
change.

8)  fixcline() got moved up. This means things in the command line which
expand to nothing are removed (if completely blank), or untokenised
(if something like '') earlier.  The only consequent changes are that
for AUTOCD the word mustn't be completely blank, since
otherwise "$nonexistent" would have caused cd to home, which is a
little counterintuitive.  Also, `[' doesn't need specially
untokenising any more --- this must be a sign we're Doing the Right Thing.

9) (Scraping the bottom of the barrel here):
  addvar=varname $nonexistent
previously did an autocd to home, now does nothing at all:  that's
because the empty variable hasn't yet been removed from the command
line, so it doesn't know there is no command.  I think this is reasonable;
otherwise there would be what is essentially a syntactic change
depending on whether or not $nonexistent is set.

10) Technical, supposedly-user-transparent things
  (i)   I had to retain CFLAG_EXEC and hence the flags element of struct
        cmd since it appears once outside execcmd(), in execcursh()
        to determine if a current shell procedure is being exec'd
        and hence doesn't need to return.  It would be nicer to get rid
        of this altogether.
  (ii)  I wrapped a few overlong lines in the first half of exec.c.
  (iii) The test
  	 while (nonempty(args) && (s = (char *) peekfirst(args)) && !*s)
  	     ugetnode(args);
        after the fork is now unnecessary and has been removed.
  (iv)  The new tests for `command', `exec', etc., are just simple
        strcmp's.  It really didn't seem worth doing anything fancy
        for four short strings.
  (v)   In fact, globlist() doesn't appear *immediately* after prefork():
        I've noted above things for which this seems inappropriate.
  (vi)  I haven't looked at where the following unexpected (by me,
        that is) behaviour comes from:
          % NoSuchCom*(N)
          zsh: command not found: NoSuchCom*
        (I would have expected it just to disappear.)
  (vii) I get extra brownie points for doing this the week before the
        annual lattice field theory conference where I'm giving a
        talk.  Please don't tell my collaborators :-).

*** Src/exec.c.words	Thu May 30 15:45:12 1996
--- Src/exec.c	Thu May 30 17:49:09 1996
***************
*** 561,567 ****
   * that is about to exit, so we can exec instead of forking.  It gets *
   * passed all the way down to execcmd() which actually makes the      *
   * decision.  A 0 is always passed if the command is not the last in  *
!  * the pipeline.  This function assumes that the sublist is not NULL. */
  
  /**/
  int
--- 561,570 ----
   * that is about to exit, so we can exec instead of forking.  It gets *
   * passed all the way down to execcmd() which actually makes the      *
   * decision.  A 0 is always passed if the command is not the last in  *
!  * the pipeline.  This function assumes that the sublist is not NULL. *
!  * If last1 is zero but the command is at the end of a pipeline, we   *
!  * pass 2 down to execcmd().                                          *
!  */
  
  /**/
  int
***************
*** 759,765 ****
      if (pline_level == 1)
  	strcpy(list_pipe_text, getjobtext((void *) pline->left));
      if (pline->type == END) {
! 	execcmd(pline->left, input, output, how, last1);
  	pline->left = NULL;
      } else {
  	int old_list_pipe = list_pipe;
--- 762,768 ----
      if (pline_level == 1)
  	strcpy(list_pipe_text, getjobtext((void *) pline->left));
      if (pline->type == END) {
! 	execcmd(pline->left, input, output, how, last1 ? 1 : 2);
  	pline->left = NULL;
      } else {
  	int old_list_pipe = list_pipe;
***************
*** 1108,1113 ****
--- 1111,1118 ----
      int fil, is_cursh, type, i;
      int nullexec = 0, assign = 0, forked = 0;
      int is_shfunc = 0, is_builtin = 0;
+     /* Various flags to the command. */
+     int cf_command = 0, cf_noglob = 0, cf_dash = 0;
  
      doneps4 = 0;
      args = cmd->args;
***************
*** 1118,1123 ****
--- 1123,1178 ----
  	mfds[i] = NULL;
      }
  
+     /* Check if it's a builtin needing automatic MAGIC_EQUALS_SUBST      *
+      * handling.  Things like typeset need this.  We can't detect the    *
+      * command if it contains some tokens (e.g. x=ex; ${x}port), so this *
+      * only works in simple cases.  has_token() is called to make sure   *
+      * this really is a simple case.
+      *
+      * This is now even simpler, since the case `noglob typeset' is
+      * not handled.
+      */
+     if(type == SIMPLE && nonempty(args)) {
+ 	char *cmdarg = (char *) peekfirst(args);
+ 
+ 	if(!has_token(cmdarg) && !shfunctab->getnode(shfunctab, cmdarg)) {
+ 	    HashNode hn2 = NULL;
+ 	    LinkNode ln = args->first;
+ 
+ 	    while((hn2 = builtintab->getnode(builtintab, (char *) ln->dat)) &&
+ 	        ((Builtin) hn2)->funcid == BIN_BUILTIN &&
+ 		(ln = ln->next) && !has_token((char *) ln->dat));
+ 	    if(hn2)
+ 		assign = (hn2->flags & BINF_MAGICEQUALS);
+ 	}
+     }
+ 
+     /* Do prefork substitutions */
+     prefork(args, (((type == CCASE) ? 4 : 0)
+ 		   | (assign ? 2 : isset(MAGICEQUALSUBST))));
+ 
+     /* Look for pre-command strings: -, exec, command, noglob. */
+     while (nonempty(args)) {
+ 	char *farg = (char *)peekfirst(args);
+ 
+ 	if (*farg == '-' && farg[1] == '\0')
+ 	    cf_dash = 1;
+ 	else if (!strcmp(farg, "exec")) {
+ 	    /* Current shell should not fork unless the
+ 	     * exec occurs at the end of a pipeline.
+ 	     */
+ 	    if (last1 == 2)
+ 		cmd->flags |= CFLAG_EXEC;
+ 	} else if (!strcmp(farg, "command"))
+ 	    cf_command = 1;
+ 	else if (!strcmp(farg, "noglob"))
+ 	    cf_noglob = 1;
+ 	else
+ 	    break;
+ 
+ 	firstnode(args) = nextnode(firstnode(args));
+     }
+ 
      /* Empty command */
      if (type == SIMPLE && empty(args)) {
  	if (nonempty(cmd->redir)) {
***************
*** 1153,1159 ****
      /* If the command begins with `%', then assume it is a *
       * reference to a job in the job table.                */
      if (nonempty(args) && *(char *)peekfirst(args) == '%') {
! 	pushnode(args, dupstring((how & Z_DISOWN) ? "disown" : (how & Z_ASYNC) ? "bg" : "fg"));
  	how = Z_SYNC;
      }
  
--- 1208,1215 ----
      /* If the command begins with `%', then assume it is a *
       * reference to a job in the job table.                */
      if (nonempty(args) && *(char *)peekfirst(args) == '%') {
! 	pushnode(args, dupstring((how & Z_DISOWN)
! 				 ? "disown" : (how & Z_ASYNC) ? "bg" : "fg"));
  	how = Z_SYNC;
      }
  
***************
*** 1176,1216 ****
      else
  	text = NULL;
  
-     /* Check if it's a builtin needing automatic MAGIC_EQUALS_SUBST      *
-      * handling.  Things like typeset need this.  We can't detect the    *
-      * command if it contains some tokens (e.g. x=ex; ${x}port), so this *
-      * only works in simple cases.  has_token() is called to make sure   *
-      * this really is a simple case.                                     */
-     if(type == SIMPLE && nonempty(args)) {
- 	char *cmdarg = (char *) peekfirst(args);
- 
- 	if(!has_token(cmdarg) && !(cmd->flags & CFLAG_COMMAND)
- 	    && !shfunctab->getnode(shfunctab, cmdarg)) {
- 	    HashNode hn2 = NULL;
- 	    LinkNode ln = args->first;
- 
- 	    while((hn2 = builtintab->getnode(builtintab, (char *) ln->dat)) &&
- 	        ((Builtin) hn2)->funcid == BIN_BUILTIN &&
- 		(ln = ln->next) && !has_token((char *) ln->dat));
- 	    if(hn2)
- 		assign = (hn2->flags & BINF_MAGICEQUALS);
- 	}
-     }
- 
-     /* Do prefork substitutions */
-     prefork(args, (((type == CCASE) ? 4 : 0) | (assign ? 2 : isset(MAGICEQUALSUBST))));
- 
      /* Set up special parameter $_ */
      zsfree(underscore);
!     if (nonempty(args) && (underscore = ztrdup((char *) getdata(lastnode(args)))))
  	untokenize(underscore); 
      else
    	underscore = ztrdup("");
  
      /* Warn about "rm *" */
!     if (type == SIMPLE && interact && unset(RMSTARSILENT) && isset(SHINSTDIN) &&
! 	nonempty(args) && nextnode(firstnode(args)) && !strcmp(peekfirst(args), "rm") &&
! 	!(cmd->flags & CFLAG_NOGLOB)) {
  	LinkNode node, next;
  
  	for (node = nextnode(firstnode(args)); node && !errflag; node = next) {
--- 1232,1249 ----
      else
  	text = NULL;
  
      /* Set up special parameter $_ */
      zsfree(underscore);
!     if (nonempty(args)
! 	&& (underscore = ztrdup((char *) getdata(lastnode(args)))))
  	untokenize(underscore); 
      else
    	underscore = ztrdup("");
  
      /* Warn about "rm *" */
!     if (type == SIMPLE && interact && unset(RMSTARSILENT)
! 	&& isset(SHINSTDIN) && nonempty(args) && nextnode(firstnode(args))
! 	&& !strcmp(peekfirst(args), "rm") && !cf_noglob) {
  	LinkNode node, next;
  
  	for (node = nextnode(firstnode(args)); node && !errflag; node = next) {
***************
*** 1234,1282 ****
  	    errflag = 1;
      }
  
      if (errflag) {
  	lastval = 1;
  	return;
      }
  
      /* Resolve simple commands */
!     if (type == SIMPLE && nonempty(args)) {
! 	char *cmdarg, *s;
  
! 	/* Get argument in command position */
! 	cmdarg = (char *) peekfirst(args);
  
! 	/* If the command is `[' then we must untokenize it */
! 	if (cmdarg[0] == Inbrack && cmdarg[1] == '\0')
! 	    cmdarg[0] = '[';
! 
! 	/* Resolve shell functions and builtins */
! 	if (!(cmd->flags & CFLAG_COMMAND)) {
! 	    if ((hn = shfunctab->getnode(shfunctab, cmdarg))) {
! 		is_shfunc = 1;
! 	    } else if ((hn = builtintab->getnode(builtintab, cmdarg))) {
! 		is_builtin = 1;
  	    }
- 	}
  
! 	/* Resolve external commands */
! 	if (!hn) {
! 	    hn = cmdnamtab->getnode(cmdnamtab, cmdarg);
! 	    if (!hn && isset(HASHCMDS) && strcmp(cmdarg, "..")) {
! 		for (s = cmdarg; *s && *s != '/'; s++);
! 		if (!*s)
! 		    hn = (HashNode) hashcmd(cmdarg, pathchecked);
  	    }
- 	}
  
! 	/* If no command found yet, see if it is *
! 	 * a directory we should AUTOCD to.      */
! 	if (!hn && isset(AUTOCD) && isset(SHINSTDIN) && empty(cmd->redir)
! 	    && !nextnode(firstnode(args)) && (s = cancd(peekfirst(args)))) {
! 	    peekfirst(args) = (void *) s;
! 	    pushnode(args, dupstring("cd"));
! 	    if ((hn = builtintab->getnode(builtintab, "cd")))
! 		is_builtin = 1;
  	}
      }
  
--- 1267,1320 ----
  	    errflag = 1;
      }
  
+     if (!cf_noglob && !errflag)
+ 	globlist(args);
+ 
      if (errflag) {
  	lastval = 1;
  	return;
      }
+     fixcline(args);
  
      /* Resolve simple commands */
!     if (type == SIMPLE && !nullexec) {
! 	if (empty(args))
! 	    return;
! 	else {
! 	    char *cmdarg, *s;
  
! 	    /* Get argument in command position */
! 	    cmdarg = (char *) peekfirst(args);
  
! 	    /* Resolve shell functions and builtins */
! 	    if (!cf_command) {
! 		if ((hn = shfunctab->getnode(shfunctab, cmdarg))) {
! 		    is_shfunc = 1;
! 		} else if ((hn = builtintab->getnode(builtintab, cmdarg))) {
! 		    is_builtin = 1;
! 		}
  	    }
  
! 	    /* Resolve external commands */
! 	    if (!hn) {
! 		hn = cmdnamtab->getnode(cmdnamtab, cmdarg);
! 		if (!hn && isset(HASHCMDS) && strcmp(cmdarg, "..")) {
! 		    for (s = cmdarg; *s && *s != '/'; s++);
! 		    if (!*s)
! 			hn = (HashNode) hashcmd(cmdarg, pathchecked);
! 		}
  	    }
  
! 	    /* If no command found yet, see if it is *
! 	     * a directory we should AUTOCD to.      */
! 	    if (!hn && isset(AUTOCD) && isset(SHINSTDIN) && empty(cmd->redir)
! 		&& !nextnode(firstnode(args)) && *(char *)peekfirst(args)
! 		&& (s = cancd(peekfirst(args)))) {
! 		peekfirst(args) = (void *) s;
! 		pushnode(args, dupstring("cd"));
! 		if ((hn = builtintab->getnode(builtintab, "cd")))
! 		    is_builtin = 1;
! 	    }
  	}
      }
  
***************
*** 1291,1297 ****
       *    b) This is an external command and we can't do a `fake exec'.       *
       *                                                                        *
       * A `fake exec' is possible if we have all the following conditions:     *
!      * 1) last1 flag is set.  This indicates that the current shell will not  *
       *    be needed after the current command.  This is typically the case    *
       *    when when the command is the last stage in a subshell, or is the    *
       *    last command after the option `-c'.                                 *
--- 1329,1335 ----
       *    b) This is an external command and we can't do a `fake exec'.       *
       *                                                                        *
       * A `fake exec' is possible if we have all the following conditions:     *
!      * 1) last1 flag is 1.  This indicates that the current shell will not    *
       *    be needed after the current command.  This is typically the case    *
       *    when when the command is the last stage in a subshell, or is the    *
       *    last command after the option `-c'.                                 *
***************
*** 1306,1312 ****
  
      if ((how & Z_ASYNC) || (!(cmd->flags & CFLAG_EXEC) &&
         (((is_builtin || is_shfunc) && output) ||
!        (!is_cursh && (!last1 || sigtrapped[SIGZERR] ||
          sigtrapped[SIGEXIT] || havefiles()))))) {
  
  	pid_t pid;
--- 1344,1350 ----
  
      if ((how & Z_ASYNC) || (!(cmd->flags & CFLAG_EXEC) &&
         (((is_builtin || is_shfunc) && output) ||
!        (!is_cursh && (last1 != 1 || sigtrapped[SIGZERR] ||
          sigtrapped[SIGEXIT] || havefiles()))))) {
  
  	pid_t pid;
***************
*** 1363,1381 ****
  	entersubsh(how, 1, 1);
      }
  
-     if (!(cmd->flags & CFLAG_NOGLOB))
- 	globlist(args);
- 
-     if (errflag) {
- 	lastval = 1;
- 	goto err;
-     } else {
- 	char *s;
- 
- 	while (nonempty(args) && (s = (char *) peekfirst(args)) && !*s)
- 	    ugetnode(args);
-     }
- 
      /* Add pipeline input/output to mnodes */
      if (input)
  	addfd(forked, save, mfds, 0, input, 0);
--- 1401,1406 ----
***************
*** 1496,1502 ****
  		execcursh, exectime, execfuncdef, execfor, execwhile,
  		execrepeat, execif, execcase, execselect, execcond};
  
- 	    fixcline(args);
  	    lastval = (func[type - CURSH]) (cmd);
  	} else if (is_builtin || is_shfunc) {
  	    /* builtin or shell function */
--- 1521,1526 ----
***************
*** 1517,1523 ****
  		    return;
  		}
  	    }
- 	    fixcline(args);
  
  	    if (is_shfunc) {
  		/* It's a shell function */
--- 1541,1546 ----
***************
*** 1591,1609 ****
  	    }
  	    if (type == SIMPLE) {
  		closem(1);
! 		execute((Cmdnam) hn, cmd->flags & CFLAG_DASH);
  	    } else {		/* ( ... ) */
  		list_pipe = 0;
  		if (subsh_close >= 0)
  		    zclose(subsh_close);
                  subsh_close = -1;
  		/* If we're forked (and we should be), no need to return */
! 		execlist(cmd->u.list, 0, last1 || forked);
  	    }
  	}
      }
  
-   err:
      if (forked)
  	_exit(lastval);
      fixfds(save);
--- 1614,1631 ----
  	    }
  	    if (type == SIMPLE) {
  		closem(1);
! 		execute((Cmdnam) hn, cf_dash);
  	    } else {		/* ( ... ) */
  		list_pipe = 0;
  		if (subsh_close >= 0)
  		    zclose(subsh_close);
                  subsh_close = -1;
  		/* If we're forked (and we should be), no need to return */
! 		execlist(cmd->u.list, 0, (last1 == 1) || forked);
  	    }
  	}
      }
  
      if (forked)
  	_exit(lastval);
      fixfds(save);
*** Src/hashtable.h.words	Thu May 30 16:00:30 1996
--- Src/hashtable.h	Thu May 30 16:01:32 1996
***************
*** 34,42 ****
  #ifdef GLOBALS
  struct reswd reswds[] =
  {
-     {NULL, "-", 0, DASH},
      {NULL, "case", 0, CASE},
-     {NULL, "command", 0, COMMAND},
      {NULL, "coproc", 0, COPROC},
      {NULL, "do", 0, DO},
      {NULL, "done", 0, DONE},
--- 34,40 ----
***************
*** 44,57 ****
      {NULL, "else", 0, ELSE},
      {NULL, "end", 0, ZEND},
      {NULL, "esac", 0, ESAC},
-     {NULL, "exec", 0, EXEC},
      {NULL, "fi", 0, FI},
      {NULL, "for", 0, FOR},
      {NULL, "foreach", 0, FOREACH},
      {NULL, "function", 0, FUNC},
      {NULL, "if", 0, IF},
      {NULL, "nocorrect", 0, NOCORRECT},
-     {NULL, "noglob", 0, NOGLOB},
      {NULL, "repeat", 0, REPEAT},
      {NULL, "select", 0, SELECT},
      {NULL, "then", 0, THEN},
--- 42,53 ----
*** Src/parse.c.words	Thu May 30 15:57:06 1996
--- Src/parse.c	Thu May 30 16:04:48 1996
***************
*** 255,261 ****
      if (!(c = par_cmd()))
  	return NULL;
      if (tok == BAR) {
- 	c->flags &= ~CFLAG_EXEC;
  	cmdpush(CS_PIPE);
  	yylex();
  	while (tok == SEPER)
--- 255,260 ----
***************
*** 270,276 ****
      } else if (tok == BARAMP) {
  	struct redir *rdr = (struct redir *)allocnode(N_REDIR);
  
- 	c->flags &= ~CFLAG_EXEC;
  	rdr->type = MERGEOUT;
  	rdr->fd1 = 2;
  	rdr->fd2 = 1;
--- 269,274 ----
***************
*** 903,918 ****
  
      c->type = SIMPLE;
      for (;;) {
! 	if (tok == COMMAND)
! 	    c->flags |= CFLAG_COMMAND;
! 	else if (tok == EXEC)
! 	    c->flags |= CFLAG_EXEC;
! 	else if (tok == NOGLOB)
! 	    c->flags |= CFLAG_NOGLOB;
! 	else if (tok == NOCORRECT)
  	    nocorrect = 1;
- 	else if (tok == DASH)
- 	    c->flags |= CFLAG_DASH;
  	else if (tok == ENVSTRING) {
  	    struct varasg *v = (struct varasg *)make_varnode();
  
--- 901,908 ----
  
      c->type = SIMPLE;
      for (;;) {
! 	if (tok == NOCORRECT)
  	    nocorrect = 1;
  	else if (tok == ENVSTRING) {
  	    struct varasg *v = (struct varasg *)make_varnode();
  
*** Src/text.c.words	Mon May  6 16:08:59 1996
--- Src/text.c	Thu May 30 15:57:42 1996
***************
*** 198,211 ****
  	break;
      case N_CMD:
  	nn = _Cmd(n);
- 	if (nn->flags & CFLAG_EXEC)
- 	    taddstr("exec ");
- 	if (nn->flags & CFLAG_COMMAND)
- 	    taddstr("command ");
- 	if (nn->flags & CFLAG_NOGLOB)
- 	    taddstr("noglob ");
- 	if (nn->flags & CFLAG_DASH)
- 	    taddstr("- ");
  	switch (nn->type) {
  	case SIMPLE:
  	    getsimptext(nn);
--- 198,203 ----
*** Src/zsh.h.words	Thu May 30 16:00:33 1996
--- Src/zsh.h	Thu May 30 16:04:24 1996
***************
*** 129,158 ****
  #define AMPERBANG      38
  
  /* Tokens for reserved words */
- #define DASH           39	/* -         */
  #define CASE           40	/* case      */
! #define COMMAND        41	/* command   */
! #define COPROC         42	/* coproc    */
! #define DO             43	/* do        */
! #define DONE           44	/* done      */
! #define ELIF           45	/* elif      */
! #define ELSE           46	/* else      */
! #define ZEND           47	/* end       */
! #define ESAC           48	/* esac      */
! #define EXEC           49	/* exec      */
! #define FI             50	/* fi        */
! #define FOR            51	/* for       */
! #define FOREACH        52	/* foreach   */
! #define FUNC           53	/* function  */
! #define IF             54	/* if        */
! #define NOCORRECT      55	/* nocorrect */
! #define NOGLOB         56	/* noglob    */
! #define REPEAT         57	/* repeat    */
! #define SELECT         58	/* select    */
! #define THEN           59	/* then      */
! #define TIME           60	/* time      */
! #define UNTIL          61	/* until     */
! #define WHILE          62	/* while     */
  
  #define WRITE           0
  #define WRITENOW        1
--- 129,154 ----
  #define AMPERBANG      38
  
  /* Tokens for reserved words */
  #define CASE           40	/* case      */
! #define COPROC         41	/* coproc    */
! #define DO             42	/* do        */
! #define DONE           43	/* done      */
! #define ELIF           44	/* elif      */
! #define ELSE           45	/* else      */
! #define ZEND           46	/* end       */
! #define ESAC           47	/* esac      */
! #define FI             48	/* fi        */
! #define FOR            49	/* for       */
! #define FOREACH        50	/* foreach   */
! #define FUNC           51	/* function  */
! #define IF             52	/* if        */
! #define NOCORRECT      53	/* nocorrect */
! #define REPEAT         54	/* repeat    */
! #define SELECT         55	/* select    */
! #define THEN           56	/* then      */
! #define TIME           57	/* time      */
! #define UNTIL          58	/* until     */
! #define WHILE          59	/* while     */
  
  #define WRITE           0
  #define WRITENOW        1
***************
*** 396,404 ****
  
  /* flags for command modifiers */
  #define CFLAG_EXEC	(1<<0)	/* exec ...    */
- #define CFLAG_COMMAND	(1<<1)	/* command ... */
- #define CFLAG_NOGLOB	(1<<2)	/* noglob ...  */
- #define CFLAG_DASH	(1<<3)	/* - ...       */
  
  /* tree element for redirection lists */
  
--- 392,397 ----

-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77330
Deutches Electronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.

Re: execcmd() reordering

[Zoltan Hidvegi]

> 1) exec, noglob, - and command are treated more like commands; they can
> appear from substitution, though not globbing (which would be stupid).

Yes stupid, but that's how other shells behave.  But since noglob must be
detected before globlist() that's the best we can do.

> 6) globbing is expanded before the fork.  Other than getting the
> prompt back an iota later when running background commands, I don't
> see this is a big deal.  One thing which seems to be a plus is that
> failures to match are now handled synchronously for background
> commands:

Also it makes debugging much easier.  Now prefork is really a historic
name.  There used to be prefork and postfork and a lot of substitutions had
been done in the later.

> 8)  fixcline() got moved up. This means things in the command line which
> expand to nothing are removed (if completely blank), or untokenised
> (if something like '') earlier.  The only consequent changes are that
> for AUTOCD the word mustn't be completely blank, since
> otherwise "$nonexistent" would have caused cd to home, which is a
> little counterintuitive.  Also, `[' doesn't need specially
> untokenising any more --- this must be a sign we're Doing the Right Thing.

Also makecline can be simplified after that patch.

>   (iv)  The new tests for `command', `exec', etc., are just simple
>         strcmp's.  It really didn't seem worth doing anything fancy
>         for four short strings.

I was thinking about adding these into builtintab with some new BINF_
flags: e.g. all of these would have BINF_PREFIX and there would be one more
BINF_ flag for each.  This would improve sh compatibility.  Hopefully a
hash-table lookup is not slower than these strcmps.  If something is found
with BINF_PREFIX shfunctab may be searched which may override builtintab
similarily to ksh.

Zoltan

Re: execcmd() reordering

[Barton E. Schaefer]

On May 30,  8:00pm, Zoltan Hidvegi wrote:
} Subject: Re: execcmd() reordering
}
} > 6) globbing is expanded before the fork.  Other than getting the
} > prompt back an iota later when running background commands, I don't
} > see this is a big deal.  One thing which seems to be a plus is that
} > failures to match are now handled synchronously for background
} > commands:
} 
} Also it makes debugging much easier.  Now prefork is really a historic
} name.  There used to be prefork and postfork and a lot of substitutions had
} been done in the later.

Hmm; Is this going to cause a problem with stuff like:

zsh% (sleep 1 ; touch brandnewfile;) & (sleep 3; echo brand*;) &

-- 
Bart Schaefer                     Vice President, Technology, Z-Code Software
schaefer@z-code.com                  Division of NCD Software Corporation
http://www.well.com/www/barts           http://www.ncdsoft.com/ZMail/

Re: execcmd() reordering

[Zoltan Hidvegi]

[-- Attachment #1: Type: application/pgp, Size: 2632 bytes --]

Re: execcmd() reordering

[Zoltan Hidvegi]

> Hmm; Is this going to cause a problem with stuff like:
> 
> zsh% (sleep 1 ; touch brandnewfile;) & (sleep 3; echo brand*;) &

No, it isn't.

Zoltan

Re: execcmd() reordering

[Bart Schaefer]

On May 31, 12:55pm, Zoltan Hidvegi wrote:
} Subject: Re: execcmd() reordering
}
} > Oh, absolutely!  Does this change mean I can't interrupt long globs???
} > This would be terrible - esp. if the glob goes into a loop.
} 
} Do not worry.  Zsh never forked for builtins and you were able to interrupt
} things like echo **/*.  This is not changed.

Well, it's changed a little:

zagzig% echo **/*		(wait a couple of seconds, ^C)
zsh: bad pattern: **/*

It didn't used to complain about the pattern.  (Beta 19 without the
ececcmd patch has the problem; I haven't tried the execcmd patch yet.)

} ^Z will not work for
} suspending external commands but other than that there should be no
} noticable change

That's a very noticeable change, to me.

} unless something is backgrounded.

And then what?

} ^Z never worked for builtins and shell functions

I know, and that has always annoyed me.

} and the number of arguments that can be passed
} to external commands is always limited by the OS (although on modern
} systems this limit is very high).

What does that have to do with anything?  Zsh is still going to finish
the entire glob.  Even a very small glob is pretty slow on a filesystem
that's NFS mounted over a PPP link (as a pathological example).

On May 31,  1:58pm, Peter Stephenson wrote:
} Subject: Re: execcmd() reordering
}
} I didn't realise the problem with backgrounding recursive globs
} before, since I've never tried it.  I can't think of a way round other
} than using a subshell.  Bummer.  (Interruption is definitely OK,
} though.)

What if globbing was always treated as if it were a command sub?  E.g.
implement

	echo **/*

as

	echo $(echo **/*)

You could even make this behavior an option, SPAWN_GLOBS or some such,
and have an extendedglob metacharacter to use it for individual globs.

Now you've got a process handling the glob, so that can be stopped and
started without messing up the parent shell ... zsh would then have to
do some magic with the job table so that it could stash a partially-
parsed command string along with the job entry.

If the job later is foregrounded again, the current shell restarts the
globjob and picks up the parsing where it left off.

Backgrounding is tougher.  It *could* fork, resume the globjob, and then
the new subshell would pick up the parse; but then the subshell would be
a sibling of the globjob and wouldn't get the exit status notification.
I'd be satisfied with an error about not being able to background the
glob, as long as I could stop it, do something else briefly, and then
pick up again where I left off (rather than having to interrupt and then
start over from scratch).

(Maybe doing it as a command sub works around the problem of not being
able to stop non-builtins?  Then we'd at least be no worse off than we
were before.  I don't know what effect this reordering has on all the
other kinds of substitutions.)

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: execcmd() reordering

[Hrvoje Niksic]

In your mail, you said:
> Furthermore, it'd be really nice to be able to stop (^Z) an unexpectedly
> huge glob and then "bg" it, or to interrupt it with ^C if I decide it was
> a mistake.

It would be a great feature, especially for huge globbings. I've needed it
very often.

-- 
hniksic@srce.hr              |  Student of electrical engineering
hniksic@fly.cc.fer.hr        |  University of Zagreb, Croatia
------------------------------------------------------------------
`VI' - An editor used by those heretics that don't subscribe to
       the Emacs religion.

Re: execcmd() reordering

[Peter Stephenson]

One extra simplification:  the special handling for wild cards in
cancd(), which handles autocd, can now be removed, since this now
happens after globbing.

I didn't realise the problem with backgrounding recursive globs
before, since I've never tried it.  I can't think of a way round other
than using a subshell.  Bummer.  (Interruption is definitely OK,
though.)

*** Src/exec.c.cancd	Fri May 31 09:36:00 1996
--- Src/exec.c	Fri May 31 10:51:52 1996
***************
*** 2470,2488 ****
      (s[1] == '/' || !s[1] || (s[1] == '.' && (s[2] == '/' || !s[1])));
      char *t;
  
-     if (haswilds(s)) {
- 	LinkList l = newlinklist();
- 	int ne = noerrs;
- 
- 	noerrs = 1;
- 	addlinknode(l, dupstring(s));
- 	globlist(l);
- 	if (!errflag && nonempty(l) && !nextnode(firstnode(l)))
- 	    s = peekfirst(l);
- 	errflag = 0;
- 	noerrs = ne;
-     }
- 
      if (*s != '/') {
  	char sbuf[PATH_MAX], **cp;
  
--- 2470,2475 ----

-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77330
Deutches Electronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.

Re: execcmd() reordering

[Zoltan Hidvegi]

> Oh, absolutely!  Does this change mean I can't interrupt long globs???
> This would be terrible - esp. if the glob goes into a loop.

Do not worry.  Zsh never forked for builtins and you were able to interrupt
things like echo **/*.  This is not changed.  ^Z will not work for
suspending external commands but other than that there should be no
noticable change unless something is backgrounded.  ^Z never worked for
builtins and shell functions and the number of arguments that can be passed
to external commands is always limited by the OS (although on modern
systems this limit is very high).

Zoltan

Re: execcmd() reordering

[Duncan Sinclair]

"Barton E. Schaefer" writes:
>On May 30,  9:10pm, Zoltan Hidvegi wrote:
>} Subject: Re: execcmd() reordering
>}
>} > When you've got a *huge* expansion which is 50% of the run-time
>} > of the command, I'd like to be able to get on with other things.
>} 
>} You can use (foo **/*) &
>
>I tend to agree with Duncan here, I'm afraid.  I may not have any idea
>how large a list the glob is going to expand into; it's ridiculous to
>expect me to subshell every command that might do a glob.
>
>Furthermore, it'd be really nice to be able to stop (^Z) an unexpectedly
>huge glob and then "bg" it, or to interrupt it with ^C if I decide it was
>a mistake.

Oh, absolutely!  Does this change mean I can't interrupt long globs???
This would be terrible - esp. if the glob goes into a loop.

All the other things, I can live with.  This gives me nightmares.

Bye for now,


Duncan.

Re: execcmd() reordering

[Barton E. Schaefer]

On May 30,  9:10pm, Zoltan Hidvegi wrote:
} Subject: Re: execcmd() reordering
}
} > When you've got a *huge* expansion which is 50% of the run-time
} > of the command, I'd like to be able to get on with other things.
} 
} You can use (foo **/*) &

I tend to agree with Duncan here, I'm afraid.  I may not have any idea
how large a list the glob is going to expand into; it's ridiculous to
expect me to subshell every command that might do a glob.

Furthermore, it'd be really nice to be able to stop (^Z) an unexpectedly
huge glob and then "bg" it, or to interrupt it with ^C if I decide it was
a mistake.

-- 
Bart Schaefer                     Vice President, Technology, Z-Code Software
schaefer@z-code.com                  Division of NCD Software Corporation
http://www.well.com/www/barts           http://www.ncdsoft.com/ZMail/

Re: execcmd() reordering

[Zoltan Hidvegi]

> >2) Now `nocorrect' must appear before any of the above.
> 
> Why?

Because nocorrect must be handled in parse.c.  It cannot be moved into
exec.c since exec.c deals with parsed commands and nocorrect affects
parsing.

> There's a flag (-k, I think) which will allow the bourne shell to take
> variable assignments at any point of the command - does this work?

No.

> >magic_equal_subst is fairly sparing about when to expand:  it insists
> >on the text before the = consisting only of characters which appear in
> >identifiers, something I wrote but now rather regret.
> 
> Is this the stuff we argued about years ago?  I'm all for a change here.

I like the present behaviour of magic_equal_subst.  Could you tell me an
example when a less restrictive behaviour is desirable?

> >6) globbing is expanded before the fork.
> 
> I don't like this at all.
> 
> >Other than getting the
> >prompt back an iota later when running background commands, I don't
> >see this is a big deal.
> 
> When you've got a *huge* expansion which is 50% of the run-time
> of the command, I'd like to be able to get on with other things.
> 

You can use (foo **/*) &

Zoltan

Re: execcmd() reordering

[Duncan Sinclair]

Peter wrote about exec stuff:
>Here's a biggish patch which does that. ...

>2) Now `nocorrect' must appear before any of the above.

Why?

>3) Also, variable assignment must appear before exec and friends.
>Again, this is natural and is required in other Bourne shell clones.

There's a flag (-k, I think) which will allow the bourne shell to take
variable assignments at any point of the command - does this work?

>5) `noglob typeset foo=~/file' do not do tilde expansion without...

>Maybe we should change.  Even
>magic_equal_subst is fairly sparing about when to expand:  it insists
>on the text before the = consisting only of characters which appear in
>identifiers, something I wrote but now rather regret.

Is this the stuff we argued about years ago?  I'm all for a change here.

>6) globbing is expanded before the fork.

I don't like this at all.

>Other than getting the
>prompt back an iota later when running background commands, I don't
>see this is a big deal.

When you've got a *huge* expansion which is 50% of the run-time
of the command, I'd like to be able to get on with other things.

>  (vii) I get extra brownie points for doing this the week before the
>        annual lattice field theory conference where I'm giving a
>        talk.  Please don't tell my collaborators :-).

Have fun!



Duncan.