bad math expression: illegal character: "

bad math expression: illegal character: "

[Artur Renault]

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

Hello zsh maintainers,

I ran into an issue with running a sourced script on zsh.

It’s a simple repro. Take the script repro.sh:

while (( "$#" )); do
echo $1
shift
done

Running this with source gives me the following error:
[~] source repro.sh                                                                                            18:15:53
repro.sh:1: bad math expression: illegal character: "

The same script works fine on bash:
[~] bash -c "source repro.sh hello"                                                                            18:17:23
hello

Anyone know what the issue could be here?

This is on Ubuntu, with oh-my-zsh on top.

Thanks,
Artur

Re: bad math expression: illegal character: "

[Lawrence Velázquez]

Hi Artur,


> On Jun 5, 2020, at 2:18 PM, Artur Renault <Artur.Renault@microsoft.com>
> wrote:
> 
> Hello zsh maintainers,
> 
> I ran into an issue with running a sourced script on zsh.
> 
> It’s a simple repro. Take the script repro.sh:
> 
> while (( "$#" )); do
> echo $1
> shift
> done
> 
> Running this with source gives me the following error:
> [~] source repro.sh                                                                                            18:15:53
> repro.sh:1: bad math expression: illegal character: "
> 
> The same script works fine on bash:
> [~] bash -c "source repro.sh hello"                                                                            18:17:23
> hello
> 
> Anyone know what the issue could be here?

For what it's worth, dash also rejects this. I assume that it and
zsh are just more strict about what constitutes a math expression.

I can't opine on which interpretation is "better", but you could
avoid the problem by removing the double quotes. Word splitting and
pathname generation are not performed inside ((...)) or $((...)).

vq

Re: bad math expression: illegal character: "

[Bart Schaefer]

On Fri, Jun 5, 2020 at 2:02 PM Lawrence Velázquez <vq@larryv.me> wrote:
>
> > On Jun 5, 2020, at 2:18 PM, Artur Renault <Artur.Renault@microsoft.com>
> >
> > while (( "$#" )); do
> >
> > repro.sh:1: bad math expression: illegal character: "
>
> For what it's worth, dash also rejects this. I assume that it and
> zsh are just more strict about what constitutes a math expression.

Does anybody know if this is something we should change for sh
emulation, or is it merely another unspecified POSIX corner-case?

Re: bad math expression: illegal character: "

[Mikael Magnusson]

On 6/18/20, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Fri, Jun 5, 2020 at 2:02 PM Lawrence Velázquez <vq@larryv.me> wrote:
>>
>> > On Jun 5, 2020, at 2:18 PM, Artur Renault <Artur.Renault@microsoft.com>
>> >
>> > while (( "$#" )); do
>> >
>> > repro.sh:1: bad math expression: illegal character: "
>>
>> For what it's worth, dash also rejects this. I assume that it and
>> zsh are just more strict about what constitutes a math expression.
>
> Does anybody know if this is something we should change for sh
> emulation, or is it merely another unspecified POSIX corner-case?

If this is a POSIX requirement, it must have been very specific about
when quotes should be allowed (bash):

$ (( "0" )); echo $?
1
$ echo $(( "0" ))
bash: "0" : syntax error: operand expected (error token is ""0" ")


-- 
Mikael Magnusson

Re: bad math expression: illegal character: "

[Lawrence Velázquez]

> On Jun 18, 2020, at 7:19 AM, Mikael Magnusson <mikachu@gmail.com> wrote:
> 
>> On 6/18/20, Bart Schaefer <schaefer@brasslantern.com> wrote:
>> 
>>> On Fri, Jun 5, 2020 at 2:02 PM Lawrence Velázquez <vq@larryv.me> wrote:
>>> 
>>>> On Jun 5, 2020, at 2:18 PM, Artur Renault <Artur.Renault@microsoft.com>
>>>> 
>>>> while (( "$#" )); do
>>>> 
>>>> repro.sh:1: bad math expression: illegal character: "
>>> 
>>> For what it's worth, dash also rejects this. I assume that it and
>>> zsh are just more strict about what constitutes a math expression.
>> 
>> Does anybody know if this is something we should change for sh
>> emulation, or is it merely another unspecified POSIX corner-case?
> 
> If this is a POSIX requirement, it must have been very specific about
> when quotes should be allowed (bash):
> 
> $ (( "0" )); echo $?
> 1
> $ echo $(( "0" ))
> bash: "0" : syntax error: operand expected (error token is ""0" ")

Looks like the bash behavior changed at some point?

    % /bin/bash --version | head -n 1
    GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin18)
    % /bin/bash -c '(( "0" )); echo $?'
    1
    % /bin/bash -c 'echo $(( "0" ))'
    /bin/bash: "0" : syntax error: operand expected (error token is ""0" ")

    % /opt/local/bin/bash --version | head -n 1
    GNU bash, version 5.0.17(1)-release (x86_64-apple-darwin18.7.0)
    % /opt/local/bin/bash -c '(( "0" )); echo $?'
    1
    % /opt/local/bin/bash -c 'echo $(( "0" ))'
    0

Anyway, unless I'm missing something, POSIX seems pretty clear about
this, as far as $((...)) goes:

    The expression shall be treated as if it were in double-quotes,
    except that a double-quote inside the expression is not treated
    specially. The shell shall expand all tokens in the expression
    for parameter expansion, command substitution, and **quote
    removal**. [Emphasis mine.]

    Next, the shell shall treat this as an arithmetic expression
    and substitute the value of the expression.

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04

vq

Re: bad math expression: illegal character: "

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 2662 bytes --]

Maybe a bash emulation could be added? Equal to:

    emulate zsh -o noshglob -o braceexpand -o kshglob

+ the quotes inside (( ))? emulate already accepts "bash" as the emulation
level.

On Fri, 19 Jun 2020 at 08:39, Lawrence Velázquez <vq@larryv.me> wrote:

> > On Jun 18, 2020, at 7:19 AM, Mikael Magnusson <mikachu@gmail.com> wrote:
> >
> >> On 6/18/20, Bart Schaefer <schaefer@brasslantern.com> wrote:
> >>
> >>> On Fri, Jun 5, 2020 at 2:02 PM Lawrence Velázquez <vq@larryv.me>
> wrote:
> >>>
> >>>> On Jun 5, 2020, at 2:18 PM, Artur Renault <
> Artur.Renault@microsoft.com>
> >>>>
> >>>> while (( "$#" )); do
> >>>>
> >>>> repro.sh:1: bad math expression: illegal character: "
> >>>
> >>> For what it's worth, dash also rejects this. I assume that it and
> >>> zsh are just more strict about what constitutes a math expression.
> >>
> >> Does anybody know if this is something we should change for sh
> >> emulation, or is it merely another unspecified POSIX corner-case?
> >
> > If this is a POSIX requirement, it must have been very specific about
> > when quotes should be allowed (bash):
> >
> > $ (( "0" )); echo $?
> > 1
> > $ echo $(( "0" ))
> > bash: "0" : syntax error: operand expected (error token is ""0" ")
>
> Looks like the bash behavior changed at some point?
>
>     % /bin/bash --version | head -n 1
>     GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin18)
>     % /bin/bash -c '(( "0" )); echo $?'
>     1
>     % /bin/bash -c 'echo $(( "0" ))'
>     /bin/bash: "0" : syntax error: operand expected (error token is ""0" ")
>
>     % /opt/local/bin/bash --version | head -n 1
>     GNU bash, version 5.0.17(1)-release (x86_64-apple-darwin18.7.0)
>     % /opt/local/bin/bash -c '(( "0" )); echo $?'
>     1
>     % /opt/local/bin/bash -c 'echo $(( "0" ))'
>     0
>
> Anyway, unless I'm missing something, POSIX seems pretty clear about
> this, as far as $((...)) goes:
>
>     The expression shall be treated as if it were in double-quotes,
>     except that a double-quote inside the expression is not treated
>     specially. The shell shall expand all tokens in the expression
>     for parameter expansion, command substitution, and **quote
>     removal**. [Emphasis mine.]
>
>     Next, the shell shall treat this as an arithmetic expression
>     and substitute the value of the expression.
>
>
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04
>
> vq



-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zinit
Blog: http://zdharma.org

Re: bad math expression: illegal character: "

[Peter Stephenson]

> On 19 June 2020 at 07:38 Lawrence Velázquez <vq@larryv.me> wrote:
> Anyway, unless I'm missing something, POSIX seems pretty clear about
> this, as far as $((...)) goes:
> 
>     The expression shall be treated as if it were in double-quotes,
>     except that a double-quote inside the expression is not treated
>     specially. The shell shall expand all tokens in the expression
>     for parameter expansion, command substitution, and **quote
>     removal**. [Emphasis mine.]
> 
>     Next, the shell shall treat this as an arithmetic expression
>     and substitute the value of the expression.
> 
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04

It's straightforward just to treat double quotes as if they're a
random spacing character.  Not sure the error is actually useful?
There was a test for it but that was simply to make sure we did
something with them.

pws

diff --git a/Src/math.c b/Src/math.c
index 905b910ec..b57ba42d4 100644
--- a/Src/math.c
+++ b/Src/math.c
@@ -831,6 +831,8 @@ zzlex(void)
 	case ' ': /* Fall through! */
 	case '\t':
 	case '\n':
+	case '"': /* POSIX says ignore these */
+	case Dnull:
 	    break;
 	default:
 	    if (idigit(*--ptr) || *ptr == '.')
diff --git a/Test/C01arith.ztst b/Test/C01arith.ztst
index 419f45292..d0092fefa 100644
--- a/Test/C01arith.ztst
+++ b/Test/C01arith.ztst
@@ -180,9 +180,10 @@
 1:bases beyond 36 don't work
 ?(eval):1: invalid base (must be 2 to 36 inclusive): 37
 
+  fail=39
   print $(( 3 + "fail" ))
-1:parse failure in arithmetic
-?(eval):1: bad math expression: operand expected at `"fail" '
+0:Double quotes are not treated specially in arithmetic
+>42
 
   alias 3=echo
   print $(( 3 + "OK"); echo "Worked")
@@ -487,3 +488,8 @@
   let noexist==0 )
 1:Arithmetic, NO_UNSET part 3
 ?(eval):2: noexist: parameter not set
+
+  print $(( "6+2" / "1+3" ))
+0:Double quotes are not treated specially in arithmetic (POSIX)
+# and do not do grouping!  this is 6 + (2/1) + 3
+>11

Re: bad math expression: illegal character: "

[Peter Stephenson]

> On 19 June 2020 at 16:55 Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> It's straightforward just to treat double quotes as if they're a
> random spacing character.  Not sure the error is actually useful?
> There was a test for it but that was simply to make sure we did
> something with them.

(In mathematical expressions.)

Doesn't sound like this is problematic, so I will commit it at some point.

pws

Re: bad math expression: illegal character: "

[Bart Schaefer]

Sorry for the very delayed response on this, given that it has already
been committed, but:

On Fri, Jun 19, 2020 at 8:55 AM Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
>
> It's straightforward just to treat double quotes as if they're a
> random spacing character.

Doesn't the POSIX requirement say "not treated specially"?  Treating
them as spaces rather than as themselves seems like "special"
treatment to me.  The original error might have been more in the
spirit of the directions.  On the other hand they're meaningless as
themselves, except to cause that very error, so I don't really have an
objection.