exec -a and parameter expansion

exec -a and parameter expansion

[Christian Neukirchen]

Hi,

trying to write a script that sets argv[0] for a child process, I tried
using exec -a and noticed this difference:

% zsh -c 'exec -a $OSTYPE ps ax' |grep ax
 3755 pts/19   RN+    0:00 ?OSTYPE ax

% bash -c 'exec -a $OSTYPE ps ax' |grep ax
 3876 pts/19   RN+    0:00 linux-gnu ax

I expected the latter output for zsh too.  (And I can't explain the "?".)
Is this a bug?

Thanks,
-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org

Re: exec -a and parameter expansion

[Phil Pennock]

On 2011-01-07 at 23:09 +0100, Christian Neukirchen wrote:
> trying to write a script that sets argv[0] for a child process, I tried
> using exec -a and noticed this difference:
> 
> % zsh -c 'exec -a $OSTYPE ps ax' |grep ax
>  3755 pts/19   RN+    0:00 ?OSTYPE ax
> 
> % bash -c 'exec -a $OSTYPE ps ax' |grep ax
>  3876 pts/19   RN+    0:00 linux-gnu ax
> 
> I expected the latter output for zsh too.  (And I can't explain the "?".)

In part it depends upon the OS, locale and what happens for unprintable
characters for you.

For me, that's hex [92  77  94  69], shown as [\   M   ^   E].

> Is this a bug?

Yes.  Looks like another one I introduced, I never tested the
interaction of "exec -a foo" with a variable as foo.

Unfortunately, my understanding of zsh's internal parse state is weak
enough that I don't know the fix.  Somewhere in the BINF_EXEC handling,
I *think* that some kind of unmeta() is needed, but my naive approaches
have not worked.

-Phil

Re: exec -a and parameter expansion

[Bart Schaefer]

On Jan 7,  5:56pm, Phil Pennock wrote:
} Subject: Re: exec -a and parameter expansion
}
} Unfortunately, my understanding of zsh's internal parse state is
} weak enough that I don't know the fix. Somewhere in the BINF_EXEC
} handling, I *think* that some kind of unmeta() is needed, but my naive
} approaches have not worked.

The problem here is that zsh does the following operations (this is all
in exec.c in the block before the comment "Do prefork substitutions"):

(1) Check that the command type is WC_SIMPLE;
(2) Check whether the first argument is a precommand modifier or other
special builtin ("exec" qualifies);
(3) Process options of those special builtins;
(4) Remove the precommand modifier or special builtin and its arguments
from the command line;
(5) "Do prefork substitutions"

Guess at which step variable expansion is performed?

Interestingly, if I simply move (5) up to before (1), I get EXACTLY
ONE failure from the "make check" test suite:

*** /tmp/zsh.ztst.out.31800     Fri Jan  7 18:08:59 2011
--- /tmp/zsh.ztst.tout.31800    Fri Jan  7 18:08:59 2011
***************
*** 1,4 ****
  1 1 0
! arg1 arg2
  noktarg1
  0 1
--- 1,4 ----
  1 1 0
! arg1
  noktarg1
  0 1
Test ../../zsh-4.0/Test/E01options.ztst failed: output differs from expected as
shown above for:
  setopt kshtypeset
  ktvars=(ktv1 ktv2)
  typeset ktfoo=`echo arg1 arg2` $ktvars
  print $+ktv1 $+ktv2 $+ktv3
  print $ktfoo
  unsetopt kshtypeset
  typeset noktfoo=`echo noktarg1 noktarg2`
  print $noktfoo
  print $+noktarg1 $+noktarg2
  unset ktfoo ktv1 ktv2 noktfoo noktarg2
Was testing: KSH_TYPESET option

So either there are some missing test cases for other things that
this would cause to break, or we should rearrange execcmd() so that
"typeset" is the only special-cased builtin ahead of prefork().

-- 

Re: exec -a and parameter expansion

[Bart Schaefer]

On Jan 7,  6:17pm, Bart Schaefer wrote:
} Subject: Re: exec -a and parameter expansion
}
} So either there are some missing test cases for other things that
} this would cause to break, or we should rearrange execcmd() so that
} "typeset" is the only special-cased builtin ahead of prefork().

The question, I guess, is whether/how this is supposed to work:

dash_p="-p"
command $dash_p echo "what path did I search?"

dash_a="-a"
exec $dash_a $OSTYPE echo "what did I exec?"

Currently options of special builtins must only appear literally, and
both of the above would report "command not found" (unless by some odd
chance you do have commands named "-p" and "-a").

If we follow bash's example, we should indeed be performing prefork()
a lot earlier in the handling of those special builtins, but I'm not
sure about the ones that are supposed to be true parser keywords.

Re: exec -a and parameter expansion

[Peter Stephenson]

On Fri, 07 Jan 2011 18:17:06 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> So either there are some missing test cases for other things that
> this would cause to break, or we should rearrange execcmd() so that
> "typeset" is the only special-cased builtin ahead of prefork().

I could be convinced the latter is correct.  Apart from KSH_TYPESET,
nothing stands out in my mind as needing special behaviour during the
expansion phase.  A careful search of the manual might be warranted.

There was some discussion of the KSH_TYPESET behaviour on the Austin
group list; I can't remember the exact upshot, but it's possible it
makes the proposed change a requirement.

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: exec -a and parameter expansion

[Peter Stephenson]

On Sat, 8 Jan 2011 21:52:39 +0000
Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> There was some discussion of the KSH_TYPESET behaviour on the Austin
> group list; I can't remember the exact upshot, but it's possible it
> makes the proposed change a requirement.

To litter the world with yet more emails: I think (but am too lazy to
check) it might have been worse and and you were expected to handle:

  t=typeset
  $t foo=`echo one two`

such that foo was assigned the value "one two".  In other words, you have
to expand the first word separately, remember which words you produced
(because it might have been more than one), check if the first word is
now typeset or a variant, then expand the words you haven't yet expanded
taking that into account.  This procedure is nicely covered by the
English slang word, 'ick'.

Neither bash nor ksh currently does this either, however, they
behave the same as zsh with KSH_TYPESET and SH_WORD_SPLIT.

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: exec -a and parameter expansion

[Bart Schaefer]

On Jan 8,  9:52pm, Peter Stephenson wrote:
} Subject: Re: exec -a and parameter expansion
}
} On Fri, 07 Jan 2011 18:17:06 -0800
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > So either there are some missing test cases for other things that
} > this would cause to break, or we should rearrange execcmd() so that
} > "typeset" is the only special-cased builtin ahead of prefork().
} 
} I could be convinced the latter is correct.  Apart from KSH_TYPESET,
} nothing stands out in my mind as needing special behaviour during the
} expansion phase.  A careful search of the manual might be warranted.

There is at least one complication, which is that "builtin typeset ..."
should work the same as "typeset".  And then there's

    noglob builtin typeset
    builtin noglob typeset
    exec builtin noglob typeset
    nocorrect exec noglob builtin typeset

and so on.

} There was some discussion of the KSH_TYPESET behaviour on the Austin
} group list; [...] it might have been worse and and you were expected
} to handle:
} 
}   t=typeset
}   $t foo=`echo one two`
} 
} such that foo was assigned the value "one two".

What this boils down to is that processing the entire command line
at once in prefork() can't be done.  You have to apply the prefork
substitutions to each word before deciding what to do with the next,
at least until you reach one of "command", "typeset", a non-special
builtin, or an external command.

Or prefork has to become clever enough to manage that part ...