Re: why is eval needed?

[Stephane Chazelas <stephane@chazelas.org>]

2022-11-19 14:21:32 -0800, Ray Andrews:
[...]
> Intuition is subjective.  Besides, usually such switches don't demand a
> space IIRC and it really is actually one argument, this is an exception.
> Even if the space is demanded, one might still think of a switch as a single
> semantic instruction even if it must be syntactically two words.  A filename
> with a space in it is NOT broken in half, it's still one entity, so I'm
> thinking the same way.  I myself was naively thinking of it as nothing more
> than a string of four characters to be dropped into another string of
> characters -- as simple as that.  As if it was the command line. But
> commands do have their need to group characters into arguments so the
> invisible rules must be followed.  ' -L 2 ' must be invisibly broken in
> half.  Which, interestingly 'eval' seems to do automatically.  It's a huge
> thing coming to understand that what to me might look like 'just' a string
> of characters, to a command, needs to be viewed differently.  Once I
> understand that, things get much simpler.  I guess internally zsh must have
> all sorts of meta data attached to arrays to keep track of what's
> joined/split to/from what else.  It would be cool to be able to somehow see
> this invisible stuff, some way to see the invisible structure of a command
> line.  But I'll be ready the next time something like this crops up.
[...]

Just to clarify, when you enter:

tree -L 1

or:

eval 'tree -L 1'

/usr/bin/tree ends up being executed but it doesn't receive any
space character in any of the 3 arguments it receives. Not
anymore that it receives coma characters when you enter the
equivalent

system "tree","-L","1";

perl code. Those spaces are part of the shell language syntax, just
like the comas are part of the perl language syntax.

It's the caller's role to pass tree, -L and 1 as separate
arguments.

What tree sees is argv[0] = "tree", argv[1] = "-L", argv[2] =
"1".

tree parses its options. It looks like it does not use the
standard getopt() API or the GNU getopt_long() API for that:

$ nm -D =tree | grep -i getopt
$ ltrace -e '*opt*@*' tree > /dev/null
+++ exited (status 0) +++

So it must be doing it by hand.

If it were using getopt(), upon encountering "-L" in argv[1], it
would look for the value for that -L option after the L in
argv[1], and since here there's nothing, it would look for it in
argv[2], if there was no argv[2], getopt() would fail and an
error would be reported.

tree is not using getopt() and it looks like it only accepts
values for options that take values in the next argument.

But even if it used getopt(), in:

tree '-L 1'
tree $'-L\x201'
tree -L\ 1
arg='-L 1'; tree $arg


(or system("tree", "-L 1") in perl)

"tree" would be able to find a value for that -L option but that
value would be " 1", not "1", same as if you had called it with
-L ' 1' (or '--level= 1' or --level=' 1' or --level ' 1' if it
were using GNU getopt_long()) and depending on how it decodes
strings into numbers, it could very well choke on that leading
space.

As it happens, here it uses strtoul() to do that and doesn't
even seem to check that the input string is fully decoded, so
leading whitespace are ignored and it's very liberal as to what
it accepts:

$ ltrace -e '*strto*[lif]@*' tree -L $'\x20\t\r\n123.5e+20 whatever' > /dev/null
tree->strtoul(" \t\r\n123.5e+20 whatever", 0, 0)                                                  = 123
+++ exited (status 0) +++

(it got 123 out of that
<space><tab><cr><lf>123.5e+20<space>whatever argument.

In any case zsh (or perl) has no saying on how tree may parse
its options. You have to invoke it the way its meant to be
invoked and in the case of tree, the option arguments must be
passed as separate arguments.

For most other commands, you would be able to pass them either
as one -L1 argument or two -L and 1 arguments and also be able
to combine more than one option in a single argument as in
-xyL1 (assuming -x and -y are options that don't take
arguments, are flags/boolean).

-- 
Stephane