grammar triviality with '&&'

grammar triviality with '&&'

[Ray Andrews]

I notice that zsh is quite tolerant as far as most
line wrap situations go, except for this:

     [ -e file1 ] &&
     [ -e file2 ] &&
     echo 'both files exist'

... good

     [ -e file1 ]
     && [ -e file2 ]
     && echo 'both files exist'

... syntax error, but we can fix it like this:

     [ -e file1 ]\
     && [ -e file2 ]\
     && echo 'both files exist'

I'm wondering why the line continuation is strictly necessary.
IOW, why doesn't the parser permit the '&&' on a new line?
I've been trying to come up with something that would
cause an ambiguity if it were permitted, but can't find anything,
but I doubt the rule would be there otherwise.

Re: grammar triviality with '&&'

[ZyX]

01.03.2015, 19:09, "Ray Andrews" <rayandrews@eastlink.ca>:
> I notice that zsh is quite tolerant as far as most
> line wrap situations go, except for this:
>
>      [ -e file1 ] &&
>      [ -e file2 ] &&
>      echo 'both files exist'
>
> ... good
>
>      [ -e file1 ]
>      && [ -e file2 ]
>      && echo 'both files exist'
>
> ... syntax error, but we can fix it like this:
>
>      [ -e file1 ]\
>      && [ -e file2 ]\
>      && echo 'both files exist'
>
> I'm wondering why the line continuation is strictly necessary.
> IOW, why doesn't the parser permit the '&&' on a new line?
> I've been trying to come up with something that would
> cause an ambiguity if it were permitted, but can't find anything,
> but I doubt the rule would be there otherwise.

Each of the lines is a sequence of `zle self-insert` followed by `zle accept-line`. In first and third cases it is possible to, based on parser state, determine that more input is required. In the second it is a complete command on the first line.

Note that zsh is a shell. Its main purpose is parsing user input, *not* scripts. What you ask will require special-casing script parsing (note: things like aliases or BANGHIST are processed prior to the time string is feed to the parser).

Note 2:

    ( echo "echo foo" ; sleep 5 ; echo "echo bar" ; sleep 5 ; echo "echo baz" ) | zsh -

outputs foo, bar, baz with 5 second intervals in between. Waiting for EOF or next line would not be very logical here, especially since shells may be and sometimes are used in non-interactive mode by the user and not by the script piping to stdin, and this is what special-casing is against (user input is already very special due to zle).

Re: grammar triviality with '&&'

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 714 bytes --]

On 03/01/2015 08:43 AM, ZyX wrote:
>
> Each of the lines is a sequence of `zle self-insert` followed by `zle accept-line`. In first and third cases it is possible to, based on parser state, determine that more input is required. In the second it is a complete command on the first line.
I see, so it's not a 'hard' syntactic issue, the limitation is in the 
rules for look ahead, namely that if the parser feels it has a complete 
sentence then it never looks at the next line.  OK, that's a crisp 
answer. Given how complex all that stuff is, I'd not even dare ask for 
that to be touched, something would break for sure. (Me, I've always 
preferred breaking lines before && and ||, but it's hardly important.)

Re: grammar triviality with '&&'

[Bart Schaefer]

On Mar 1,  9:01am, Ray Andrews wrote:
}
} I see, so it's not a 'hard' syntactic issue

Actually it IS a "hard" syntactic issue, in the sense that the grammar
for all *nix shells [1] both immemorial and standard, formally defines
newline as equivalent to semicolon.

You can demonstrate that this is the case [2] by:

torch% false && ; echo AND || ; echo OR
OR
torch% false ; && echo AND ; || echo OR
zsh: parse error near `&&'


[1] I don't consider here "shells" written to use the syntax of other
interpreted languages, such as shells that execute perl or php or
python statements.

[2] Actually bash throws a syntax error on that "&& ;", but both zsh
and ksh accept it.  Chet?

Re: grammar triviality with '&&'

[ZyX]

01.03.2015, 21:50, "Bart Schaefer" <schaefer@brasslantern.com>:
> On Mar 1,  9:01am, Ray Andrews wrote:
> }
> } I see, so it's not a 'hard' syntactic issue
>
> Actually it IS a "hard" syntactic issue, in the sense that the grammar
> for all *nix shells [1] both immemorial and standard, formally defines
> newline as equivalent to semicolon.
>
> You can demonstrate that this is the case [2] by:
>
> torch% false && ; echo AND || ; echo OR
> OR
> torch% false ; && echo AND ; || echo OR
> zsh: parse error near `&&'
>
> [1] I don't consider here "shells" written to use the syntax of other
> interpreted languages, such as shells that execute perl or php or
> python statements.
>
> [2] Actually bash throws a syntax error on that "&& ;", but both zsh
> and ksh accept it.  Chet?

(Almost) pure POSIX shells like dash, posh and busybox ash also throw syntax error here. This is a direct consequence of how grammar is defined in http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_10_02. Ksh and zsh are far from being POSIX shells.

Re: grammar triviality with '&&'

[Ray Andrews]

On 03/01/2015 10:48 AM, Bart Schaefer wrote:
> On Mar 1,  9:01am, Ray Andrews wrote:
> }
> } I see, so it's not a 'hard' syntactic issue
>
> Actually it IS a "hard" syntactic issue, in the sense that the grammar
> for all *nix shells [1] both immemorial and standard, formally defines
> newline as equivalent to semicolon.

I see.  If newline == semicolon, then the acceptable variations in line 
wrap that I've
noticed have simply been within that rule and there is no look ahead as 
I presumed.
This is a good thing to know.  As usual I refer in my head back to C 
where line wrap
issues virtually never exist, but that's obviously far more complicated 
to parse.
What that rule looses in formatting flexibility it gains in brevity and 
easier parsing
so it becomes clear why an interpreted language makes that choice.

Re: grammar triviality with '&&'

[ZyX]

01.03.2015, 22:48, "Ray Andrews" <rayandrews@eastlink.ca>:
> On 03/01/2015 10:48 AM, Bart Schaefer wrote:
>>  On Mar 1,  9:01am, Ray Andrews wrote:
>>  }
>>  } I see, so it's not a 'hard' syntactic issue
>>
>>  Actually it IS a "hard" syntactic issue, in the sense that the grammar
>>  for all *nix shells [1] both immemorial and standard, formally defines
>>  newline as equivalent to semicolon.
>
> I see.  If newline == semicolon, then the acceptable variations in line
> wrap that I've
> noticed have simply been within that rule and there is no look ahead as
> I presumed.
> This is a good thing to know.  As usual I refer in my head back to C
> where line wrap
> issues virtually never exist, but that's obviously far more complicated
> to parse.

C is far easier to parse. Two main complications are preprocessor and state: you cannot parse some constructs without knowing about typedefs and you can parse nothing without #defines. But C grammar is not as complicated as zsh one.

Handling newlines here is not a complication, not a tiny bit of this: after preprocessor you can simply translate newline to space (except when inside quotes). For preprocessor newline without `\` always ends the statement, `\\\n` is ignored.

Though it would not be so easy if you wanted to make C interpreted. Dunno how existing C interpreters handle this: never tried them, but it would be logical to run something when interpreter received newline and has a complete top-level statement.

> What that rule looses in formatting flexibility it gains in brevity and
> easier parsing
> so it becomes clear why an interpreted language makes that choice.

Re: grammar triviality with '&&'

[Lawrence Velázquez]

On Mar 1, 2015, at 12:01 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:

> I see, so it's not a 'hard' syntactic issue, the limitation is in the
> rules for look ahead, namely that if the parser feels it has
> a complete sentence then it never looks at the next line.

It's not just a parser implementation detail. Lists are *defined* to
terminate on newlines. From zshmisc(1):

    A list is a sequence of zero or more sublists, in which each sublist
    is terminated  by `;', `&', `&|', `&!', or a newline.

In your second example, `[ -e file1 ]` constitutes a list.

vq

Re: grammar triviality with '&&'

[Vincent Lefevre]

On 2015-03-01 13:49:38 -0500, Lawrence Velázquez wrote:
> On Mar 1, 2015, at 12:01 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:
> [...] From zshmisc(1):
> 
>     A list is a sequence of zero or more sublists, in which each sublist
>     is terminated  by `;', `&', `&|', `&!', or a newline.
> 
> In your second example, `[ -e file1 ]` constitutes a list.

However "&& [ -e file2 ]" could constitute a list too. There is
currently a parse error, but zsh could have an extension to accept
it as being equivalent to: "[[ $? -eq 0 ]] && [ -e file2 ]". Would
there be anything wrong with such an extension?

Similarly, "|| X" could be regarded as equivalent to
"[[ $? -ne 0 ]] || X" by zsh.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: grammar triviality with '&&'

[Ray Andrews]

On 03/01/2015 06:27 PM, Vincent Lefevre wrote:
> On 2015-03-01 13:49:38 -0500, Lawrence Velázquez wrote:
>> On Mar 1, 2015, at 12:01 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:
>> [...] From zshmisc(1):
>>
>>      A list is a sequence of zero or more sublists, in which each sublist
>>      is terminated  by `;', `&', `&|', `&!', or a newline.
>>
>> In your second example, `[ -e file1 ]` constitutes a list.
> However "&& [ -e file2 ]" could constitute a list too. There is
> currently a parse error, but zsh could have an extension to accept
> it as being equivalent to: "[[ $? -eq 0 ]] && [ -e file2 ]". Would
> there be anything wrong with such an extension?
>
> Similarly, "|| X" could be regarded as equivalent to
> "[[ $? -ne 0 ]] || X" by zsh.
>
I always thought of '&&' as logically equivalent to a keyword 'andif'
and '||' as 'orif' which is why I like to put them first on a line,
because we always put keywords first (mostly),
but it seems to me that even with Bart's definition of a 'hard'
syntax problem with the de-facto semicolon, if

[ -e file ]

... by itself on a line is legal, and if it returns success or failure
and if that success or failure is there for checking on the next
line, the '&&' could mean nothing else but 'check previous true/false',
so it should be unambiguous, so why wouldn't it be legal? It would be the
same as:

[ -e file ]; && ....

... which throws an error, but where is the ambiguity?  What else could it
mean?  Seems to me the semi doesn't/shouldn't change anything.

BTW, is there somewhere to read up on the 'list' idea, it's the first
I've ever heard of that concept.

Re: grammar triviality with '&&'

[Lawrence Velázquez]

On Mar 1, 2015, at 10:12 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:

> BTW, is there somewhere to read up on the 'list' idea, it's the first
> I've ever heard of that concept.

Yes, the "SIMPLE COMMANDS & PIPELINES" section of zshmisc(1).

vq

Re: grammar triviality with '&&'

[Kurtis Rader]

[-- Attachment #1: Type: text/plain, Size: 823 bytes --]

On Sun, Mar 1, 2015 at 6:27 PM, Vincent Lefevre <vincent@vinc17.net> wrote:

> However "&& [ -e file2 ]" could constitute a list too. There is
> currently a parse error, but zsh could have an extension to accept
> it as being equivalent to: "[[ $? -eq 0 ]] && [ -e file2 ]". Would
> there be anything wrong with such an extension?
>
> Similarly, "|| X" could be regarded as equivalent to
> "[[ $? -ne 0 ]] || X" by zsh.
>

How often would such a feature be useful? Very infrequently in my opinion.
Furthermore, it is almost guaranteed to cause anyone reading the statement
to wonder if  the author made a mistake as the idiom is unlike anything
else I can think of in Bourne style shells. I believe the benefits don't
justify the costs of the feature.

-- 
Kurtis Rader
Caretaker of the exceptional canines Junior and Hank

Re: grammar triviality with '&&'

[Ray Andrews]

On 03/01/2015 07:53 PM, Kurtis Rader wrote:
> How often would such a feature be useful? Very infrequently in my 
> opinion. Furthermore, it is almost guaranteed to cause anyone reading 
> the statement to wonder if the author made a mistake as the idiom is 
> unlike anything else I can think of in Bourne style shells. I believe 
> the benefits don't justify the costs of the feature. 

Well, what would the costs be?  It looks to me that the error is an 
arbitrary restriction and if it didn't break something I'd remove it 
even if for no  other reason that that it is arbitrary.  Of course if 
there were any complications, then it shouldn't be touched.  Oh, and of 
course no one has to use it if they think it would ruffle anyone's 
feathers as far as tradition, but I'd use it for  sure. Things should 
only be limited where it is necessary that they be limited.  I'd not be 
surprised if the code was actually simpler, '&&' would just grab the 
errorlevel of the previous command as always and there'd just be one 
less error to handle.

Re: grammar triviality with '&&'

[Kurtis Rader]

[-- Attachment #1: Type: text/plain, Size: 1958 bytes --]

On Sun, Mar 1, 2015 at 8:18 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:

> Well, what would the costs be?  It looks to me that the error is an
> arbitrary restriction and if it didn't break something I'd remove it even
> if for no  other reason that that it is arbitrary.  Of course if there were
> any complications, then it shouldn't be touched.  Oh, and of course no one
> has to use it if they think it would ruffle anyone's feathers as far as
> tradition, but I'd use it for  sure. Things should only be limited where it
> is necessary that they be limited.  I'd not be surprised if the code was
> actually simpler, '&&' would just grab the errorlevel of the previous
> command as always and there'd just be one less error to handle.
>

One obvious cost is that anyone reading a script that uses the feature will
naturally assume the author made a mistake and the script is invalid. The
reason this restriction exists is not arbitrary. It is a an obvious and
natural consequence of how the grammar was interpreted by the person(s) who
wrote the code for the Bourne-shell, Bash, Zsh, etcetera. Why is it simpler
for the code to "just grab the errorlevel of the previous command"  when it
sees a leading "&&" or "||"? Also, it is not just a matter of changing the
zsh implementation. Unit tests need to be added and existing tests likely
updated. Documentation needs to be changed. Whether the change might break
existing uses has to be carefully considered.

You say "I'd use it for sure". Please provide a non-contrived example,
preferably two or three, where this would be useful. How hard is it to be
explicit and include the "[[ $? -eq 0 ]]" in a script? If this feature were
truly useful for interactive use I might agree with you. But I see no
evidence and my 30+ years experience that it has no use outside of making a
tiny percentage of scripts a few characters shorter.

-- 
Kurtis Rader
Caretaker of the exceptional canines Junior and Hank

Re: grammar triviality with '&&'

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 2635 bytes --]

On 03/01/2015 09:22 PM, Kurtis Rader wrote:
>
> One obvious cost is that anyone reading a script that uses the feature 
> will naturally assume the author made a mistake and the script is 
> invalid.

So don't use it if, when or where tradition trumps practicality--which 
is almost everywhere.
> The reason this restriction exists is not arbitrary.

That may very well be the case, which is why I raised the question, but 
it seems that it might not be the case after all (Lawrence). I'm over 
fascinated by this sort of minutia.
> It is a an obvious and natural consequence of how the grammar was 
> interpreted by the person(s) who wrote the code for the Bourne-shell, 
> Bash, Zsh, etcetera. Why is it simpler for the code to "just grab the 
> errorlevel of the previous command"  when it sees a leading "&&" or "||"?

But is that not what it already does?  The question is whether a line 
break or a semi *necessarily* prevents that.  IOW, what ambiguity would 
this cause:

[ -e file1 ]
&& [ -e file2 ]
> Also, it is not just a matter of changing the zsh implementation. Unit 
> tests need to be added and existing tests likely updated. 
> Documentation needs to be changed. Whether the change might break 
> existing uses has to be carefully considered.

Absolutely, and that is probably reason enough why it won't happen. It 
would have to be nothing more than a relaxation of an arbitrary rule ( 
**if** it is arbitrary)--maybe there is a very good reason why 
errorlevel can't be passed beyond a newline or semi.  If there is such a 
good reason, I'd be educated to learn what it is.  But since "[[ $? -eq 
0 ]]" works past the newline, at this point I can't see what the 
linewrap would *necessarily* break.
>
> You say "I'd use it for sure". Please provide a non-contrived example, 
> preferably two or three, where this would be useful.

if [ -e 'shelly.txt' ]
&& [ -n "$ozymandias" ]
&& [ grep "I met a traveler" ]
then
   echo "Look on my works, ye mighty"
fi


... I have a trivial preference for that form, I find it neater. It is 
very unimportant.

> How hard is it to be explicit and include the "[[ $? -eq 0 ]]" in a 
> script? If this feature were truly useful for interactive use I might 
> agree with you. But I see no evidence and my 30+ years experience that 
> it has no use outside of making a tiny percentage of scripts a few 
> characters shorter.

Exactly so.  Whereas I'd like it for saving me the line continuation 
backslash, that is monumentally trivial, I'm interested in the theory of 
the thing.  I like Lawrence's thinking on the matter, however.

Don't take this more seriously than I take it.

Re: grammar triviality with '&&'

[Kurtis Rader]

[-- Attachment #1: Type: text/plain, Size: 1276 bytes --]

On Mon, Mar 2, 2015 at 8:17 AM, Ray Andrews <rayandrews@eastlink.ca> wrote:

>
> if [ -e 'shelly.txt' ]
> && [ -n "$ozymandias" ]
> && [ grep "I met a traveler" ]
> then
>   echo "Look on my works, ye mighty"
> fi
>

Why are you using ancient Bourne-shell syntax? If you need compatibility
with the Bourn-shell you can't use the feature you're asking for. If you
use the slightly less ancient Korn-shell syntax you can break the test
across lines the way you want:

#!/bin/zsh
ozymandia=yes

if [[ -e 'shelly.txt'
   && -n "$ozymandia"
   && -n $(grep "I met a traveler" shelly.txt) ]]
then
    echo "Look on my works, ye mighty"
fi

Note that I fixed your command as "grep" isn't a valid test; although, I
wouldn't write it that way in practice as it isn't efficient.

P.S., Are you aware that in the days of the Bourne-shell that the shell did
not interpret tests like "-e" it simply ran the "[" command and checked its
exit status. In fact that command should still exist on your system as a
hardlink to the "test" command. Run

$ ls -li '/bin/[' /bin/test
123034700 -rwxr-xr-x  2 root  wheel  18480 Sep  9 15:44 /bin/[*
123034700 -rwxr-xr-x  2 root  wheel  18480 Sep  9 15:44 /bin/test*

to see this.

-- 
Kurtis Rader
Caretaker of the exceptional canines Junior and Hank

Re: grammar triviality with '&&'

[ZyX]

02.03.2015, 21:54, "Kurtis Rader" <krader@skepticism.us>:
> On Mon, Mar 2, 2015 at 8:17 AM, Ray Andrews <rayandrews@eastlink.ca> wrote:
>>  if [ -e 'shelly.txt' ]
>>  && [ -n "$ozymandias" ]
>>  && [ grep "I met a traveler" ]
>>  then
>>    echo "Look on my works, ye mighty"
>>  fi
>
> Why are you using ancient Bourne-shell syntax? If you need compatibility
> with the Bourn-shell you can't use the feature you're asking for. If you
> use the slightly less ancient Korn-shell syntax you can break the test
> across lines the way you want:
>
> #!/bin/zsh
> ozymandia=yes
>
> if [[ -e 'shelly.txt'
>    && -n "$ozymandia"
>    && -n $(grep "I met a traveler" shelly.txt) ]]
> then
>     echo "Look on my works, ye mighty"
> fi
>
> Note that I fixed your command as "grep" isn't a valid test; although, I
> wouldn't write it that way in practice as it isn't efficient.
>
> P.S., Are you aware that in the days of the Bourne-shell that the shell did
> not interpret tests like "-e" it simply ran the "[" command and checked its
> exit status. In fact that command should still exist on your system as a
> hardlink to the "test" command. Run
>
> $ ls -li '/bin/[' /bin/test
> 123034700 -rwxr-xr-x  2 root  wheel  18480 Sep  9 15:44 /bin/[*
> 123034700 -rwxr-xr-x  2 root  wheel  18480 Sep  9 15:44 /bin/test*

Why hardlink and why in /bin?

% la /usr/bin/{test,\[}
-rwxr-xr-x 1 root 39K янв 31 04:03 /usr/bin/[*
-rwxr-xr-x 1 root 35K янв 31 04:03 /usr/bin/test*

: you see there is a 4K difference. Not sure why, but both belong to coreutils package.

>
> to see this.
>
> --
> Kurtis Rader
> Caretaker of the exceptional canines Junior and Hank

Re: grammar triviality with '&&'

[Kurtis Rader]

[-- Attachment #1: Type: text/plain, Size: 868 bytes --]

On Mar 2, 2015 11:03 AM, "ZyX" <kp-pav@yandex.ru> wrote:
>
> Why hardlink and why in /bin?
>
> % la /usr/bin/{test,\[}
> -rwxr-xr-x 1 root 39K янв 31 04:03 /usr/bin/[*
> -rwxr-xr-x 1 root 35K янв 31 04:03 /usr/bin/test*
>
> : you see there is a 4K difference. Not sure why, but both belong to
coreutils package.

A hardlink simply attaches a name to a file in UNIX like OSs. You can give
a file multiple names by creating multiple hardlinks. These commands are
normally in /bin because in the past the root and /usr file systems were
separated and those commands were needed before /usr was mounted.

It is strange that those two commands point to different programs on your
computer. It may be a simple mistake. Or maybe the maintainers of your
distro did it to maximize compatibility among the ancient and less ancient
parts of the distro.

Re: grammar triviality with '&&'

[ZyX]

02.03.2015, 23:18, "Kurtis Rader" <krader@skepticism.us>:
> On Mar 2, 2015 11:03 AM, "ZyX" <kp-pav@yandex.ru> wrote:
>>  Why hardlink and why in /bin?
>>
>>  % la /usr/bin/{test,\[}
>>  -rwxr-xr-x 1 root 39K янв 31 04:03 /usr/bin/[*
>>  -rwxr-xr-x 1 root 35K янв 31 04:03 /usr/bin/test*
>>
>>  : you see there is a 4K difference. Not sure why, but both belong to
>
> coreutils package.
>
> A hardlink simply attaches a name to a file in UNIX like OSs. You can give
> a file multiple names by creating multiple hardlinks. These commands are
> normally in /bin because in the past the root and /usr file systems were
> separated and those commands were needed before /usr was mounted.
>
> It is strange that those two commands point to different programs on your
> computer. It may be a simple mistake. Or maybe the maintainers of your
> distro did it to maximize compatibility among the ancient and less ancient
> parts of the distro.

I do not think this is a mistake. And I found at least two differences:

1. `command test --help` returns zero, `command \[ --help` displays help.
2. `[` requires `]` (unless it received `--help` or `--version`), no matter what name it is called with, `test` requires not.

Note that coreutils ebuild does *not* contain any references to `test` or `[`, patches as well do not (except when Makefile is patched not to regenerate man pages). This means that this difference is created by GNU developers, *not* by distro maintainers.

Re: grammar triviality with '&&'

[Kurtis Rader]

[-- Attachment #1: Type: text/plain, Size: 1963 bytes --]

On Mon, Mar 2, 2015 at 8:15 PM, ZyX <kp-pav@yandex.ru> wrote:
>
> I do not think this is a mistake. And I found at least two differences:
>
> 1. `command test --help` returns zero, `command \[ --help` displays help.
> 2. `[` requires `]` (unless it received `--help` or `--version`), no
> matter what name it is called with, `test` requires not.
>
> Note that coreutils ebuild does *not* contain any references to `test` or
> `[`, patches as well do not (except when Makefile is patched not to
> regenerate man pages). This means that this difference is created by GNU
> developers, *not* by distro maintainers.
>

Ugh! I can confirm your observations using my Ubuntu 14.04 server. Yet on
Mac OS X Yosemite the two names are linked to the same binary. Mac OS X
exhibits the traditional behavior. I have no idea why GNU based distros
have the two names linked to slightly differently behaving binaries.
Presumably the GNU team had a good reason for this departure from the
traditional behavior. Furthermore, the GNU team could have used a single
binary for both names since a program can easily use the name it was
invoked by to alter its behavior. But that difference doesn't affect my
point that traditionally /bin/test and /bin/[ referred to the same program
and /bin/sh didn't know how to do anything other than invoke an external
command with a sequence of "words" and evaluate its exit status. So "if [
some condition ]; then" was really just "if /bin/[ some condition \] ;
then".

Note that quoting is critical. Which is the main reason the Korn-shell
(ksh) introduced the "[[ ... ]]" syntax. Invoking /bin/[ (or /usr/bin/[)
requires a trailing "]". Invoking /bin/test (or /usr/bin/test) requires
that there be no trailing "]". How the arguments are passed to the two
commands is also important. The "/bin/[" form requires that the shell has
already tokenized the condition into words.

-- 
Kurtis Rader
Caretaker of the exceptional canines Junior and Hank

Re: grammar triviality with '&&'

[Vincent Lefevre]

On 2015-03-02 20:43:27 -0800, Kurtis Rader wrote:
> On Mon, Mar 2, 2015 at 8:15 PM, ZyX <kp-pav@yandex.ru> wrote:
> >
> > I do not think this is a mistake. And I found at least two differences:
> >
> > 1. `command test --help` returns zero, `command \[ --help` displays help.
> > 2. `[` requires `]` (unless it received `--help` or `--version`), no
> > matter what name it is called with, `test` requires not.
> >
> > Note that coreutils ebuild does *not* contain any references to `test` or
> > `[`, patches as well do not (except when Makefile is patched not to
> > regenerate man pages). This means that this difference is created by GNU
> > developers, *not* by distro maintainers.
> >
> 
> Ugh! I can confirm your observations using my Ubuntu 14.04 server. Yet on
> Mac OS X Yosemite the two names are linked to the same binary. Mac OS X
> exhibits the traditional behavior. I have no idea why GNU based distros
> have the two names linked to slightly differently behaving binaries.

Having different behaviors is required by POSIX:

  http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html

However you do not need different binaries to get different
behaviors: the binary could look at argv[0] to know how to
behave.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: grammar triviality with '&&'

[ZyX]

03.03.2015, 07:44, "Kurtis Rader" <krader@skepticism.us>:
> On Mon, Mar 2, 2015 at 8:15 PM, ZyX <kp-pav@yandex.ru> wrote:
>>  I do not think this is a mistake. And I found at least two differences:
>>
>>  1. `command test --help` returns zero, `command \[ --help` displays help.
>>  2. `[` requires `]` (unless it received `--help` or `--version`), no
>>  matter what name it is called with, `test` requires not.
>>
>>  Note that coreutils ebuild does *not* contain any references to `test` or
>>  `[`, patches as well do not (except when Makefile is patched not to
>>  regenerate man pages). This means that this difference is created by GNU
>>  developers, *not* by distro maintainers.
>
> Ugh! I can confirm your observations using my Ubuntu 14.04 server. Yet on
> Mac OS X Yosemite the two names are linked to the same binary. Mac OS X
> exhibits the traditional behavior. I have no idea why GNU based distros
> have the two names linked to slightly differently behaving binaries.
> Presumably the GNU team had a good reason for this departure from the
> traditional behavior. Furthermore, the GNU team could have used a single
> binary for both names since a program can easily use the name it was
> invoked by to alter its behavior. But that difference doesn't affect my
> point that traditionally /bin/test and /bin/[ referred to the same program
> and /bin/sh didn't know how to do anything other than invoke an external
> command with a sequence of "words" and evaluate its exit status. So "if [
> some condition ]; then" was really just "if /bin/[ some condition \] ;
> then".

If you want `test` and `[` point to the same executable on Gentoo you can compile coreutils with USE=multicall. According to the description this will make *all* programs from coreutils package be compiled into a single executable.

Other than that these programs have different behavior (required by the standard according to Vincent Lefevre). The only possible purpose for making them point to one executable with this remark is saving space*. But this is the case for USE=multicall (its purpose is explained directly in USE flag description).

* I would actually try to refactor this into 2 executables + library, but after reading ldd output for a few coreutils executables it now looks like coreutils developers thought that the least amount of linked libraries is the best option. As long as test and [ live in one repository this does not affect code reusage at all.

>
> Note that quoting is critical. Which is the main reason the Korn-shell
> (ksh) introduced the "[[ ... ]]" syntax. Invoking /bin/[ (or /usr/bin/[)
> requires a trailing "]". Invoking /bin/test (or /usr/bin/test) requires
> that there be no trailing "]". How the arguments are passed to the two
> commands is also important. The "/bin/[" form requires that the shell has
> already tokenized the condition into words.
>
> --
> Kurtis Rader
> Caretaker of the exceptional canines Junior and Hank

Re: grammar triviality with '&&'

[Ray Andrews]

On 03/02/2015 10:52 AM, Kurtis Rader wrote:
> Why are you using ancient Bourne-shell syntax? If you need compatibility
> with the Bourn-shell you can't use the feature you're asking for. If you
> use the slightly less ancient Korn-shell syntax you can break the test
> across lines the way you want:

So I see.  Thanks, I didn't know about using [[ that way:
>
> #!/bin/zsh
> ozymandia=yes
>
> if [[ -e 'shelly.txt'
>     && -n "$ozymandia"
>     && -n $(grep "I met a traveler" shelly.txt) ]]
> then
>      echo "Look on my works, ye mighty"
> fi
>
> Note that I fixed your command as "grep" isn't a valid test; although, I
> wouldn't write it that way in practice as it isn't efficient.

Doesn't matter, I just made that up.  Anyway, it was a theoretical issue 
gentlemen, I just
thought I'd ask, I don't really have any great difficulties using the 
line continuation.  It
seems this 'errexit' thing does create a hard syntactic issue so that 
answers my question.
>

Re: grammar triviality with '&&'

[Vincent Lefevre]

On 2015-03-01 19:53:28 -0800, Kurtis Rader wrote:
> On Sun, Mar 1, 2015 at 6:27 PM, Vincent Lefevre <vincent@vinc17.net> wrote:
> 
> > However "&& [ -e file2 ]" could constitute a list too. There is
> > currently a parse error, but zsh could have an extension to accept
> > it as being equivalent to: "[[ $? -eq 0 ]] && [ -e file2 ]". Would
> > there be anything wrong with such an extension?
> >
> > Similarly, "|| X" could be regarded as equivalent to
> > "[[ $? -ne 0 ]] || X" by zsh.
> >
> 
> How often would such a feature be useful? Very infrequently in my opinion.

I'm not so sure. It's probably useless in a script (except for
the style as in the original post, but this may not be a good
idea to use such a feature here). But I can see an interesting
use in an interactive shell when using the history. For instance,
one may want to run 3 (long) commands from time to time, by using
the history and accept-line-and-down-history:

cmd1
cmd2
cmd3

Now, assume that as soon as some command fails, you don't want
to execute the following ones, but you still want to have the
3 commands in the history when invoking the latest cmd1 by a
history search (something I often do). Currently, the only way
to do that is, AFAIK:

cmd1
[[ $? -eq 0 ]] && cmd2
[[ $? -eq 0 ]] && cmd3

But having

cmd1
&& cmd2
&& cmd3

could be nicer (I would also like the fact that one can easier see
that these 3 commands are related).

Of course, one could also do:

  cmd1 && cmd2 && cmd3

but if the commands produce output, one may prefer to run them
separately to analyze the output first (which is the main goal
of interactive use vs writing a script).

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: grammar triviality with '&&'

[Peter Stephenson]

On Mon, 2 Mar 2015 11:46:19 +0100
Vincent Lefevre <vincent@vinc17.net> wrote:
> Now, assume that as soon as some command fails, you don't want
> to execute the following ones, but you still want to have the
> 3 commands in the history when invoking the latest cmd1 by a
> history search (something I often do). Currently, the only way
> to do that is, AFAIK:
> 
> cmd1
> [[ $? -eq 0 ]] && cmd2
> [[ $? -eq 0 ]] && cmd3
> 
> But having
> 
> cmd1
> && cmd2
> && cmd3
> 
> could be nicer (I would also like the fact that one can easier see
> that these 3 commands are related).

% andfn() {
   integer stat=$?
   (( $stat > 0 )) && return $stat
   return 0
}
% alias and='andfn &&'

% print this worked
this worked
% and print so did this
so did this
% and { print but this failed; false; }
but this failed
% and print so this didn\'t get printed


-- 
Peter Stephenson | Principal Engineer Samsung Cambridge Solution Centre
Email: p.stephenson@samsung.com | Phone: +44 1223 434724 |
www.samsung.com

Re: grammar triviality with '&&'

[Bart Schaefer]

[-- Attachment #1: Type: text/plain, Size: 507 bytes --]

On Mar 2, 2015 3:17 AM, "Peter Stephenson" <p.stephenson@samsung.com> wrote:
>
>
> % andfn() {
>    integer stat=$?
>    (( $stat > 0 )) && return $stat
>    return 0
> }
> % alias and='andfn &&'

% alias and='(){ return $? } && '

(Trailing space thrown in to expand aliases again in the next word.)

Heck, for most purposes you don't even need $? preserved:

% alias and='(( $? )) || '

Heck #2, let's just invent some new operators:

alias -g @@=';(){ return $? } && '
alias -g ::=';(){ return $? } || '

Re: grammar triviality with '&&'

[Vincent Lefevre]

On 2015-03-02 11:19:25 -0800, Bart Schaefer wrote:
> % alias and='(){ return $? } && '
> 
> (Trailing space thrown in to expand aliases again in the next word.)

I've found a bug:

% alias '&&=(){ return $? } && '
% && echo OK
zsh: parse error near `&&'

Since && will be replaced by the alias, there's no reason to get
a parsing error. The zshexpn(1) man page says:

    Alias Expansion
        Aliases are expanded immediately before the command line is
        parsed as explained under Aliasing in zshmisc(1).

Note the "before the command line is parsed".

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: grammar triviality with '&&'

[Bart Schaefer]

On Mar 4,  3:47pm, Vincent Lefevre wrote:
}
} I've found a bug:
} 
} % alias '&&=(){ return $? } && '
} % && echo OK
} zsh: parse error near `&&'

Although I see PWS has already made a (broken?) stab at changing this,
I think that's a documentation omission rather than a code bug.  Some
things intentionally cannot be aliased.

Re: grammar triviality with '&&'

[Peter Stephenson]

On Wed, 4 Mar 2015 17:51:12 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Mar 4,  3:47pm, Vincent Lefevre wrote:
> }
> } I've found a bug:
> } 
> } % alias '&&=(){ return $? } && '
> } % && echo OK
> } zsh: parse error near `&&'
> 
> Although I see PWS has already made a (broken?) stab at changing this,
> I think that's a documentation omission rather than a code bug.  Some
> things intentionally cannot be aliased.

I don't think that makes sense: there's too much you already can alias.
You can alias reserved words and arbitrary magic sequences like \&, for
example, and consequently already have the power to do as much damage as
you like.  Forbidding it in this case would just be providing an
unmemorable list of special cases.

I suspect this dates back to very early days when the shell was much
weaker on translating lexical tokens back to to text.

pws

Re: grammar triviality with '&&'

[Bart Schaefer]

On Mar 2,  3:27am, Vincent Lefevre wrote:
}
} However "&& [ -e file2 ]" could constitute a list too. There is
} currently a parse error, but zsh could have an extension to accept
} it as being equivalent to: "[[ $? -eq 0 ]] && [ -e file2 ]". Would
} there be anything wrong with such an extension?

If nothing else, the errexit option would fail.  With errexit,

    false
    || anything

would never get beyond "false", whereas

    false || anything

proceeds at least through "anything".

You also get strange crap like

    while && this; do || if && that; then || thus; fi; done

which would mean what, exactly?

Finally if it's OK to have nothing before && / ||, then it would also be
syntactically OK to write

    && || && || && && ...

No.

Re: grammar triviality with '&&'

[Vincent Lefevre]

On 2015-03-02 00:54:40 -0800, Bart Schaefer wrote:
> On Mar 2,  3:27am, Vincent Lefevre wrote:
> }
> } However "&& [ -e file2 ]" could constitute a list too. There is
> } currently a parse error, but zsh could have an extension to accept
> } it as being equivalent to: "[[ $? -eq 0 ]] && [ -e file2 ]". Would
> } there be anything wrong with such an extension?
> 
> If nothing else, the errexit option would fail.  With errexit,
> 
>     false
>     || anything
> 
> would never get beyond "false", whereas
> 
>     false || anything
> 
> proceeds at least through "anything".

I agree that's not equivalent, but this would not be a problem
in a "if ... then" construct (though I think it's a bad idea
to use such a feature here).

> You also get strange crap like
> 
>     while && this; do || if && that; then || thus; fi; done
> 
> which would mean what, exactly?

while [[ $? -eq 0 ]] && this; do [[ $? -ne 0 ]] || if [[ $? -eq 0 ]] && that; then [[ $? -ne 0 ]] || thus; fi; done

> Finally if it's OK to have nothing before && / ||, then it would also be
> syntactically OK to write
> 
>     && || && || && && ...

I disagree. Only a && or || as the first word of a list would have
a special meaning.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: grammar triviality with '&&'

[Ray Andrews]

On 03/02/2015 02:31 AM, Vincent Lefevre wrote:
>
> Finally if it's OK to have nothing before && / ||, then it would also be
> syntactically OK to write
>
>      && || && || && && ...
> I disagree. Only a && or || as the first word of a list would have
> a special meaning.
>
Right, isn't the idea that newline or no newline, '&&' receives the 
value of the previous command?  All that would  change is that the 
newline would not break the pipe, no?

Re: grammar triviality with '&&'

[Bart Schaefer]

On Mar 2, 11:31am, Vincent Lefevre wrote:
} Subject: Re: grammar triviality with '&&'
}
} On 2015-03-02 00:54:40 -0800, Bart Schaefer wrote:
} > 
} > If nothing else, the errexit option would fail.  With errexit,
} 
} I agree that's not equivalent, but this would not be a problem
} in a "if ... then" construct (though I think it's a bad idea
} to use such a feature here).

Except that's exactly where Ray wants to use it!

} > You also get strange crap like
} > 
} >     while && this; do || if && that; then || thus; fi; done
} > 
} > which would mean what, exactly?
} 
} while [[ $? -eq 0 ]] && this; do [[ $? -ne 0 ]] || if [[ $? -eq 0 ]] && that; then [[ $? -ne 0 ]] || thus; fi; done

Yes obviously that's what you intend the literal interpretation to be,
but in what way is it sensibly meaningful in an actual program?

} > Finally if it's OK to have nothing before && / ||, then it would also be
} > syntactically OK to write
} > 
} >     && || && || && && ...
} 
} I disagree. Only a && or || as the first word of a list would have
} a special meaning.

You can't escape the lunacy that easily:

    && { || { && { || { && { && ... } } } } }

Re: grammar triviality with '&&'

[Ray Andrews]

On 03/02/2015 08:49 AM, Bart Schaefer wrote:
> On Mar 2, 11:31am, Vincent Lefevre wrote:
> } Subject: Re: grammar triviality with '&&'
> }
> } On 2015-03-02 00:54:40 -0800, Bart Schaefer wrote:
> } >
> } > If nothing else, the errexit option would fail.  With errexit,
> }
> } I agree that's not equivalent, but this would not be a problem
> } in a "if ... then" construct (though I think it's a bad idea
> } to use such a feature here).
>
> Except that's exactly where Ray wants to use it!
Prezactly.  My thinking is (surprise!) C-ish--the newline should be 
irrelevant.  Just as one can end a line with && so should one be able to 
begin a line the same way, same logic.  If there is a real monster 
hiding under the bed tho with this 'errexit' thing, then any tinkering 
could really break something.  Or maybe it could be a feature ... but 
it's in 'Kurtis' territory, i.e. we'd need to be very careful.  Nope, I 
was thinking of  " && [ -e file ] " being simply the logical equivalent 
of " [ -e file ] && " ... if in the latter case the syntax 'folds' 
itself around to the next line, then I'd expect that the opposite 
situation might also be legal.  A pipe must have commands/statements pre 
and post, no? We already permit the 'post' to be on the next line, so 
I'm just thinkin' that the 'pre' could be on the previous line in the 
same way.  No gotchas!

Re: grammar triviality with '&&'

[Bart Schaefer]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

On Mar 2, 2015 10:09 AM, "Ray Andrews" <rayandrews@eastlink.ca> wrote:
>
> My thinking is (surprise!) C-ish--the newline should be irrelevant.

(I should probably recommend at this point that you never try to take up
programming in Python.)

> Just as one can end a line with && so should one be able to begin a line
the same way, same logic.

I don't know if anyone has bothered to notice this, but as long as all of
your conditions are testable with [[ ]], the newlines are not significant
inside that construct.

Re: grammar triviality with '&&'

[Vincent Lefevre]

On 2015-03-02 08:49:58 -0800, Bart Schaefer wrote:
> On Mar 2, 11:31am, Vincent Lefevre wrote:
> } Subject: Re: grammar triviality with '&&'
> }
> } On 2015-03-02 00:54:40 -0800, Bart Schaefer wrote:
> } > 
> } > If nothing else, the errexit option would fail.  With errexit,
> } 
> } I agree that's not equivalent, but this would not be a problem
> } in a "if ... then" construct (though I think it's a bad idea
> } to use such a feature here).
> 
> Except that's exactly where Ray wants to use it!

The feature would be there. Then every programmer is free to do what
he likes.

> 
> } > You also get strange crap like
> } > 
> } >     while && this; do || if && that; then || thus; fi; done
> } > 
> } > which would mean what, exactly?
> } 
> } while [[ $? -eq 0 ]] && this; do [[ $? -ne 0 ]] || if [[ $? -eq 0 ]] && that; then [[ $? -ne 0 ]] || thus; fi; done
> 
> Yes obviously that's what you intend the literal interpretation to be,
> but in what way is it sensibly meaningful in an actual program?

A programmer is not forced to use this feature. Anyway even

  while [[ $? -eq 0 ]] && this; do [[ $? -ne 0 ]] || if [[ $? -eq 0 ]] && that; then [[ $? -ne 0 ]] || thus; fi; done

is not really meaningful while this can already be done in zsh.

> } > Finally if it's OK to have nothing before && / ||, then it would also be
> } > syntactically OK to write
> } > 
> } >     && || && || && && ...
> } 
> } I disagree. Only a && or || as the first word of a list would have
> } a special meaning.
> 
> You can't escape the lunacy that easily:
> 
>     && { || { && { || { && { && ... } } } } }

Same as above. It would not be the first shell feature that could
be misused (see IFS and heredoc, for instance).

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: grammar triviality with '&&'

[Ray Andrews]

On 03/04/2015 12:55 AM, Vincent Lefevre wrote:

> The feature would be there. Then every programmer is free to do what 
> he likes.

> A programmer is not forced to use this feature.

Exactly.  The traditionalist wouldn't touch it in any case.  All 
existing code would
remain (*must* remain) unaffected.  If an extra degree of freedom and 
capability
was possible, and IF it could be implemented with no gotchas, then why 
not? This
assuming of course that it could be implemented simply and efficiently (fat
chance).  I can hardly comment on the 'alias' method but it does seem 
contrived
and maybe full of gotchas.  It still seems to me that the " [[ $? -eq 0 
]] " test
is implicit before any '&&' anyway, and that since a line break doesn't 
make the
value disappear, then

&& ...

can mean nothing other than 'grab last '$?' and continue parsing'. The 
'errexit'
thing is a feature or a gotcha as you guys decide.

Re: grammar triviality with '&&'

[Bart Schaefer]

On Mar 4,  9:12am, Ray Andrews wrote:
}
} If an extra degree of freedom and capability was possible, and IF it
} could be implemented with no gotchas, then why not?

Because we don't need the grammar to become any more ad-hoc than it
already has?

Back in the depths of time, one of the reasons zsh was invented was
because csh's grammar was too irregular.  Csh scripts originally (I
can't speak for how tcsh &c. may be now) could in fact be interpreted
one line at a time, and literally were:  "while" loop was processed by
remembering the file position of the string "while" and then when the
"end" was encountered, seeking back and re-reading the lines again.
This made it possible to write some really entertaining scripts that
edited their own code on the fly, but made it impossible to syntax
check a multi-line control structure before beginning to execute it.

The Bourne-shell / POSIX shell grammar is at least regular enough to
be parsed before it is executed, even with all its other warts.
Introducing a case where the command to the left of && / || can be
empty if and only if the conjunction is the first token on a line, is
the kind of irregularity that should be avoided.

Further, in the standard definition of the grammar where pipelines
separated by && or || form a sublist, the terminator that follows is
syntactically significant to the whole sublist.  That is:

	one && two || three &

is parsed as

	{ one && two || three } &

which is not the same as

	one ; [[ $? = 0 ]] && two ; [[ $? = 0 ]] || three &

Now, as it turns out, zsh doesn't actually execute "one" and "two" in
the background when presented with "one && two || three &" but that is
a consequence of other implementation choices and not of the treatment
of "one && two || three" as a syntactic unit.

Re: grammar triviality with '&&'

[Ray Andrews]

On 03/05/2015 08:59 PM, Bart Schaefer wrote:
> On Mar 4,  9:12am, Ray Andrews wrote:
> }
> } If an extra degree of freedom and capability was possible, and IF it
> } could be implemented with no gotchas, then why not?
>
> Because we don't need the grammar to become any more ad-hoc than it
> already has?

A quite sufficient reason.  My very unlearned sense of it was that the 
errorlevel
test is implicit in any '&&' anyway, and since that test wraps 'down' 
legally
already, then it would be trivially simple to understand that that same 
value is
grabbed when '&&' starts a line.  I was liking Vincent's thinking.

Or not!  It seemed to me like a relaxation of
something arbitrary.  However if it really would be add-hoc then of 
course it
can't be touched.  Please recall that this is a *question*.
>
> Back in the depths of time, one of the reasons zsh was invented was
> because csh's grammar was too irregular.  Csh scripts originally (I
> can't speak for how tcsh &c. may be now) could in fact be interpreted
> one line at a time, and literally were:  "while" loop was processed by
> remembering the file position of the string "while" and then when the
> "end" was encountered, seeking back and re-reading the lines again.
> This made it possible to write some really entertaining scripts that
> edited their own code on the fly, but made it impossible to syntax
> check a multi-line control structure before beginning to execute it.

God forbid.
>
> The Bourne-shell / POSIX shell grammar is at least regular enough to
> be parsed before it is executed, even with all its other warts.
> Introducing a case where the command to the left of && / || can be
> empty if and only if the conjunction is the first token on a line, is
> the kind of irregularity that should be avoided.
>
> Further, in the standard definition of the grammar where pipelines
> separated by && or || form a sublist, the terminator that follows is
> syntactically significant to the whole sublist.  That is:
>
> 	one && two || three &
>
> is parsed as
>
> 	{ one && two || three } &
>
> which is not the same as
>
> 	one ; [[ $? = 0 ]] && two ; [[ $? = 0 ]] || three &

Well that kills my notion that it's implicit, I thought the latter was 
what was really going
on in any case.  Dare a mortal ask what the difference is between those?

The deeper
question is why shells were designed this way.  In C this sort of 
obscurity not only
doesn't exist, it couldn't exist, you can wrap code anyway you want 
(string printing
excepted).  But shells are written in C, yet the decision was made to 
make things
so convoluted that no one fully understands them.  This work was done by 
geniuses so
there musta been a good reason.

Re: grammar triviality with '&&'

[Lawrence Velázquez]

On Mar 6, 2015, at 11:10 AM, Ray Andrews <rayandrews@eastlink.ca> wrote:

> In C this sort of obscurity not only doesn't exist, it couldn't exist,
> you can wrap code anyway you want (string printing excepted).

Did you forget that every C syntactic construct has an explicit
terminating symbol (semicolon, brace, parenthesis, quotation mark,
etc.)? It's easy to wrap lines willy-nilly when you ignore newlines by
design. (Oops, except in preprocessor directives.)

This newlines-matter-except-when-they-don't game has been tried in other
languages, and this is the kind of thing that can happen when you're
sloppy about it:

http://jamesallardice.com/understanding-automatic-semi-colon-insertion-in-javascript/

> But shells are written in C

Irrelevant.

> yet the decision was made to make things so convoluted that no one
> fully understands them.

These "convoluted" bits would appear much less so if you did not insist
on forcing your understanding of other tools onto this one.

vq

Re: grammar triviality with '&&'

[Ray Andrews]

On 03/06/2015 12:23 PM, Lawrence Velázquez wrote:
> Did you forget that every C syntactic construct has an explicit
> terminating symbol (semicolon, brace, parenthesis, quotation mark,
> etc.)?

No, that's exactly my point.  But as Bart just explained, since newlines 
are syntactic in
interactive mode (how could it really be otherwise?) They must be 
syntactic in scripts,
and that is the heart of the matter.  It looks like what I've been 
'asking for' is just to
permit a sloppy parse of lines beginning with '&&'--a default value--but 
who really
needs that? Better to be explicit.  I repent of it.

 >These "convoluted" bits would appear much less so if you did not 
insist on forcing your understanding of other tools onto this one. vq
Not forcing, rather confessing why my internal parsing engine is so 
busted. Getting the C
out of me is turning out to be 10x harder than I expected.
Anyway, it's too much noise on the list.

Re: grammar triviality with '&&'

[Vincent Lefevre]

On 2015-03-05 20:59:51 -0800, Bart Schaefer wrote:
> On Mar 4,  9:12am, Ray Andrews wrote:
> }
> } If an extra degree of freedom and capability was possible, and IF it
> } could be implemented with no gotchas, then why not?
> 
> Because we don't need the grammar to become any more ad-hoc than it
> already has?

I don't think that the grammar would become more complex.
For instance, in the POSIX grammar, I suppose that it suffices
to change

and_or           :                         pipeline
                 | and_or AND_IF linebreak pipeline
                 | and_or OR_IF  linebreak pipeline
                 ;

to

and_or           :                         pipeline
                 |        AND_IF linebreak pipeline
                 |        OR_IF  linebreak pipeline
                 | and_or AND_IF linebreak pipeline
                 | and_or OR_IF  linebreak pipeline
                 ;

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: grammar triviality with '&&'

[Bart Schaefer]

On Mar 6,  8:10am, Ray Andrews wrote:
}
} The deeper question is why shells were designed this way.

On Mar 6,  5:32pm, Vincent Lefevre wrote:
}
} I don't think that the grammar would become more complex.

What you're both missing or at least glossing over is the interaction
between the grammar and the interactive interpreter.

There goals are:  (1) the grammar for scripts is identical to the grammer
for interactive use, (2) the execution order is identical both in scripts
and in interactive use, and (3) when used interactively, the input can be
interpreted [commands executed] as soon as a complete syntactic structure
has been recognized.

Under the current grammar, the interpreter always "knows," at the point
where a line break occurs, whether or not it has a complete syntactic
element that it can execute.  If you allow the and_or producion to put
a linebreak before AND_IF / OR_IF, then until it encounters an explicit
";" or "&" (or some other lookahead token that can't appar at that
position in the pipeline production) the interpreter doesn't "know"
whether it has actually reached the end of the "and_or" production.

Therefore it would either violate (2) to speculatively execute what has
been seen so far, or it would violate (3) to wait for the next token to
complete the lookahead.  Sure, we could either discard goal (1) to
make this an interactively-only feature [there's at least precedent for
that with setopts], or we can discard goal (2); we can do whatever
we want.  I assume you'd find it way too annoying to discard (3).

But this is in a sense analogous to suggesting that

    one | two | three

could wait to see what happens to "one" before deciding whether to send
the output along to "two".  The grammar would work just fine with

    one
    | two
    | three

but you'd never suggest that the shell should implicitly do

    one | tee temp1
    cat temp1 | two | tee temp2
    cat temp2 | three | tee temp3

just so that you can decide later whether you're interested in adding
yet another stage to the pipeline.  Or at least I hope you would not
suggest that, though if you look through the list archives people have
in fact asked how to capture output so they can re-use it later.  See
Functions/Misc/keeper for an example.

In any case, I'm done trying to explain this.  We've already put the lie
to the word "triviality" in the subject of this thread.

Re: grammar triviality with '&&'

[Ray Andrews]

On 03/06/2015 09:43 AM, Bart Schaefer wrote:
> } I don't think that the grammar would become more complex.
>
> What you're both missing or at least glossing over is the interaction
> between the grammar and the interactive interpreter.
>
> There goals are:  (1) the grammar for scripts is identical to the grammer
> for interactive use, (2) the execution order is identical both in scripts
> and in interactive use, and (3) when used interactively, the input can be
> interpreted [commands executed] as soon as a complete syntactic structure
> has been recognized.

Sounds sane.
> Under the current grammar, the interpreter always "knows," at the point
> where a line break occurs, whether or not it has a complete syntactic
> element that it can execute.  If you allow the and_or producion to put
> a linebreak before AND_IF / OR_IF, then until it encounters an explicit
> ";" or "&" (or some other lookahead token that can't appar at that
> position in the pipeline production) the interpreter doesn't "know"
> whether it has actually reached the end of the "and_or" production.
>
> Therefore it would either violate (2) to speculatively execute what has
> been seen so far, or it would violate (3) to wait for the next token to
> complete the lookahead.  Sure, we could either discard goal (1) to
> make this an interactively-only feature [there's at least precedent for
> that with setopts], or we can discard goal (2); we can do whatever
> we want.  I assume you'd find it way too annoying to discard (3).

Don't blame me!  I'm just riding along with Vincent's logic, he can 
argue this with
real knowledge, I'm just seeing what's at the bottom of this.

When I type at the prompt, it clearly knows when I'm not
finished, but If I type ' [ -e file1 ] ENTER', it goes to the next 
prompt, because it has no
way of knowing that I intended to add an '&&' after it on the next 
line.  Quite so.
But surely that's not the issue. The issue is whether

| two

can know to implicity grab the already existing return value from the 
last command,
whatever it may have been.  Yup, the previous line is executed and 
finished and gone,
but it has set the errorlevel, and that does persist, so it is 
legitimately there. Why
can't it be used? I see it as violating none of your three rules. It 
would be the same
interactively or in scripts, and it would execute at syntactic 
completion. All that would
change is that an obviously incomplete AND_IF would default to grabbing 
the previous
return value (which is still there).  Anyway, the parser would be making a
concession, and why should it?  No really good reason.  I'm satisfied, 
unless similar
concessions are made routinely elsewhere.  Nope, it ain't broken, don't 
fix it.
> But this is in a sense analogous to suggesting that
>
>      one | two | three
>
> could wait to see what happens to "one" before deciding whether to send
> the output along to "two".
>
>      one
>      | two
>      | three

Yes. Exactly.  The question was why that's not permitted, but I think I 
see why.
>
> just so that you can decide later whether you're interested in adding
> yet another stage to the pipeline.  Or at least I hope you would not
> suggest that, though if you look through the list archives people have
> in fact asked how to capture output so they can re-use it later.  See
> Functions/Misc/keeper for an example.

Nope.  I don't want to break things more broken than they already are.
> In any case, I'm done trying to explain this.  We've already put the lie
> to the word "triviality" in the subject of this thread.
>
Amen!  It was quite innocent at the time ;-)  But it has been deeply 
informative.

Re: grammar triviality with '&&'

[Vincent Lefevre]

On 2015-03-06 09:43:59 -0800, Bart Schaefer wrote:
> On Mar 6,  8:10am, Ray Andrews wrote:
> }
> } The deeper question is why shells were designed this way.
> 
> On Mar 6,  5:32pm, Vincent Lefevre wrote:
> }
> } I don't think that the grammar would become more complex.
> 
> What you're both missing or at least glossing over is the interaction
> between the grammar and the interactive interpreter.

I don't think I've introduced a difference.

> There goals are:  (1) the grammar for scripts is identical to the grammer
> for interactive use, (2) the execution order is identical both in scripts
> and in interactive use, and (3) when used interactively, the input can be
> interpreted [commands executed] as soon as a complete syntactic structure
> has been recognized.
> 
> Under the current grammar, the interpreter always "knows," at the point
> where a line break occurs, whether or not it has a complete syntactic
> element that it can execute.  If you allow the and_or producion to put
> a linebreak before AND_IF / OR_IF,
[...]

No, I did *not* allow that. Ray wanted that in his original post, but
my proposition is just a modification of the grammar that doesn't need
such a thing about the linebreak handling. There are some drawbacks
compared to what Ray wanted initially (e.g., don't do this with
"set -e"), but this is a compromise.

Note that since && and || can be used inside [[ ... ]], this change
is probably not needed in scripts. It could be useful interactively
without needing an alias.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)