Math expression evaluation error?

Math expression evaluation error?

[jdh]

I believe I read that math expression evaluation will convert all
calculations in an expression if "any" value in that expression is a
real value.

That is not true if the word "any" is true or a bug has crawled into the
numeric expression evaluator.  Or I read wrong..

Example on my system, zsh 5.0.7 (i686-pc-linux-gnu), I get the following
results for the shown example:

$ echo $(( 6+3/8. + 10 + 5/8. )) $(( 6+3/8 + 10 + 5/8. ))
                               17. 16.625

In the above case only the second term which contains a real number is
evaluated as real , the previous term is taken as integer and decimal
values are truncated (non-existent.   This same behavior is demonstrated
in zcalc.  Probably uses the same logic.

Though I understand why this happens (or think I do).  It could cause
someone a major mistake if they are unaware of this and rely on the outcome.

Regards

  Darel

Re: Math expression evaluation error?

[Peter Stephenson]

On Fri, 09 Jan 2015 11:46:44 -0800
jdh <dhenman@gmail.com> wrote:
> I believe I read that math expression evaluation will convert all
> calculations in an expression if "any" value in that expression is a
> real value.

This isn't the case and wherever this came from needs correcting.
Please instead read the documentation under ARITHMETIC EVALUATION in the
zshmisc manual for what actually happens and let us if know if you think
that is inadequate.  The actually phrasing is "Promotion of integer to
floating point values is performed where necessary".  It also refers
above that to C --- there's an underlying assumption here that the user
has some idea of the operation of C-like languages, which might of
course not be the case.  "Where necessary" is in general where an
operator has a floating point number as one argument and an integer as
another; a C programmer would recognise the use of the word "promotion"
as implying this, a general user probably not.  So the wording could
probably be improved.  Brief and to the point suggestions on this from
anyone are welcome (long rambles on why anyone doesn't understand it
may be invigorating for the rambler but are less useful to us as
developers :-)).

See also the following shell option:

FORCE_FLOAT
       Constants in arithmetic evaluation will be treated  as  floating
       point  even without the use of a decimal point.  Integers in any
       base will be converted.

This is most useful in zcalc, which actually has an option for this.

       If the option `-f' is set, all numbers are treated  as  floating
       point,  hence for example the expression `3/4' evaluates to 0.75
       rather than 0.  Options must appear in separate words.

pws

Re: Math expression evaluation error?

[jdh]

Thanks for the detailed explanation.   I suggest that the an additional 
line be added to the manual, which would say something like:

"   Caution:  The evaluation of an expression is taken a term at a time 
and if there are mixed terms , say integer only and  float terms , the 
integer only ones will not be promoted to real values and the result of 
the whole expression would be an error to common standards. One can use 
the -f option or the  FORCE_FLOAT environment variable to change this 
behavior."

I apologize if this is in the manual already.

Regards.

On 2015年01月09日 12:15, Peter Stephenson wrote:
> On Fri, 09 Jan 2015 11:46:44 -0800
> jdh <dhenman@gmail.com> wrote:
>> I believe I read that math expression evaluation will convert all
>> calculations in an expression if "any" value in that expression is a
>> real value.
> This isn't the case and wherever this came from needs correcting.
> Please instead read the documentation under ARITHMETIC EVALUATION in the
> zshmisc manual for what actually happens and let us if know if you think
> that is inadequate.  The actually phrasing is "Promotion of integer to
> floating point values is performed where necessary".  It also refers
> above that to C --- there's an underlying assumption here that the user
> has some idea of the operation of C-like languages, which might of
> course not be the case.  "Where necessary" is in general where an
> operator has a floating point number as one argument and an integer as
> another; a C programmer would recognise the use of the word "promotion"
> as implying this, a general user probably not.  So the wording could
> probably be improved.  Brief and to the point suggestions on this from
> anyone are welcome (long rambles on why anyone doesn't understand it
> may be invigorating for the rambler but are less useful to us as
> developers :-)).
>
> See also the following shell option:
>
> FORCE_FLOAT
>         Constants in arithmetic evaluation will be treated  as  floating
>         point  even without the use of a decimal point.  Integers in any
>         base will be converted.
>
> This is most useful in zcalc, which actually has an option for this.
>
>         If the option `-f' is set, all numbers are treated  as  floating
>         point,  hence for example the expression `3/4' evaluates to 0.75
>         rather than 0.  Options must appear in separate words.
>
> pws

Re: Math expression evaluation error?

[Ray Andrews]

On 01/09/2015 01:40 PM, jdh wrote:
> Thanks for the detailed explanation.   I suggest that the an 
> additional line be added to the manual, which would say something like:
>
    {
    $ echo $(( 6+3/8. + 10 + 5/8  ))
    $ echo $(( 6+3/8  + 10 + 5/8. ))
    $ echo $(( 6+3/8. + 10 + 5/8. ))
    $ echo $(( 6+3/8  + 10 + 5/8  ))
    }

    16.375
    16.625
    17.
    16           << really?

    $ setopt force_float

    {
    $ echo $(( 6+3/8. + 10 + 5/8  ))
    $ echo $(( 6+3/8  + 10 + 5/8. ))
    $ echo $(( 6+3/8. + 10 + 5/8. ))
    $ echo $(( 6+3/8  + 10 + 5/8  ))
    }

    17.
    17.
    17.
    17.


Thanks for the heads up, I now know not to trust zsh math without the 
option.
I wonder tho why anyone would ever want the first set of results. 
Who/why/when
do we want bad math? Wouldn't we prefer accurate results as the default with
the option to have broken math for people who like broken math? Why not have
it work properly and that's that--nothing to explain, it just works as 
you'd expect it to work?

  6 + 3/8 + 10 + 5/8 ...
what sane calculator is going to give me '16' for that?

Re: Math expression evaluation error?

[Peter Stephenson]

On Fri, 09 Jan 2015 13:40:42 -0800
jdh <dhenman@gmail.com> wrote:
> Thanks for the detailed explanation.   I suggest that the an additional 
> line be added to the manual, which would say something like:
> 
> "   Caution:  The evaluation of an expression is taken a term at a time 
> and if there are mixed terms , say integer only and  float terms , the 
> integer only ones will not be promoted to real values and the result of 
> the whole expression would be an error to common standards. One can use 
> the -f option or the  FORCE_FLOAT environment variable to change this 
> behavior."

Thanks, that's just the sort of useful feedback on what people find
confusing that we need.  I've adapted it into the appropriate section of
the manual with minor corrections.

diff --git a/Doc/Zsh/arith.yo b/Doc/Zsh/arith.yo
index a620b73..5c334ce 100644
--- a/Doc/Zsh/arith.yo
+++ b/Doc/Zsh/arith.yo
@@ -233,6 +233,15 @@ necessary.  In addition, if any operator which requires an integer
 equivalents with assignment) is given a floating point argument, it will be
 silently rounded down to the next integer.
 
+Users should beware that, in common with many other programming
+languages but not software designed for calculation, the evaluation of
+an expression in zsh is taken a term at a time and promotion of integers
+to floating point does not occur in terms only containing integers.  A
+typical result of this is that a division such as tt(6/8) is truncated,
+in this being rounded down to 0.  The tt(FORCE_FLOAT) shell option can
+be used in scripts or functions where floating point evaluation is
+required throughout.
+
 Scalar variables can hold integer or floating point values at different
 times; there is no memory of the numeric type in this case.
 
 
pws

Re: Math expression evaluation error?

[Lawrence Velázquez]

On Jan 9, 2015, at 5:19 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:

> I wonder tho why anyone would ever want the first set of results. Who/why/when
> do we want bad math?

Mathematics restricted to the set of integers is not "bad math", no matter how much you assert that it is.

> Wouldn't we prefer accurate results as the default with the option to have broken math for people who like broken math?

Stop calling it "broken". It's the way C does math.

    % clang -Wno-format -o c-integer-div -x c - <<EOF
    heredoc> #include <stdio.h>
    heredoc> int main(void) {
    heredoc>    printf("integer division is not 'bad math': 3 / 8 = %f\n", 3 / 8);
    heredoc>    return 0;
    heredoc> }
    heredoc> EOF
    % ./c-integer-div
    integer division is not 'bad math': 3 / 8 = 0.000000

At this point, this is what a lot of programmers expect, especially since the rest of zsh's arithmetic is quite C-like.

> Why not have it work properly and that's that--nothing to explain, it just works as you'd expect it to work?

It'd work as YOU expect it to work. Plenty of other people expect it to work like C, and we'd have to explain to them why it doesn't. (We already have to do so for operator precedence and hexadecimal literals, among other things.)

> 6 + 3/8 + 10 + 5/8 ...
> what sane calculator is going to give me '16' for that?

zsh is not a calculator.

vq

Re: Math expression evaluation error?

[Ray Andrews]

On 01/09/2015 03:06 PM, Lawrence Velázquez wrote:
> On Jan 9, 2015, at 5:19 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>> I wonder tho why anyone would ever want the first set of results. Who/why/when
>> do we want bad math?
> Mathematics restricted to the set of integers is not "bad math", no matter how much you assert that it is.

I assert nothing, I ask a question.  I just 'automatically' presume that 
zsh calculations will be done  like any normal calculator would  do them.

$ echo $(( 3/8 + 5/8 ))
0

.. if that's what people want, then that's what they'll get, it just seems
a bit strange to me.  I'd say that the very fact that I enter a fraction 
tells
the shell that I'm expecting a real number answer.  Or maybe not.

But that's just my take on it, I'm not saying.  It seems to me that if the
shell accepts the fractions then it should return the correct answer.  I'd
make integer math  the option and have accurate results as the default
but that's just me.

Re: Math expression evaluation error?

[Bart Schaefer]

On Fri, Jan 9, 2015 at 3:52 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:
> It seems to me that if the
> shell accepts the fractions then it should return the correct answer.  I'd
> make integer math  the option and have accurate results as the default
> but that's just me.


3/8 is not a fraction.  Yes, in a text paragraph like this one I might
write 3/8 as a fraction because I can't write a superscript three over
a subscript eight, but in an arithmetic expression 0.375 is a fraction
and 3/8 is "integer 3 divided by integer 8", and the definition of
"divided by" includes "if both numerator and denominator are integers,
then the result must also be an integer".  As has been mentioned, this
is the way a LOT of programming languages work.

Re: Math expression evaluation error?

[Ray Andrews]

On 01/09/2015 04:25 PM, Bart Schaefer wrote:
> On Fri, Jan 9, 2015 at 3:52 PM, Ray Andrews <rayandrews@eastlink.ca> wrote:
>> It seems to me that if the
>> shell accepts the fractions then it should return the correct answer.  I'd
>> make integer math  the option and have accurate results as the default
>> but that's just me.
>
> 3/8 is not a fraction.  Yes, in a text paragraph like this one I might
> write 3/8 as a fraction because I can't write a superscript three over
> a subscript eight, but in an arithmetic expression 0.375 is a fraction
> and 3/8 is "integer 3 divided by integer 8", and the definition of
> "divided by" includes "if both numerator and denominator are integers,
> then the result must also be an integer".  As has been mentioned, this
> is the way a LOT of programming languages work.
Well, this is one of those things that won't change, but I must say that 
in my experience an overwhelming majority of people would say that 3/8 
is a fraction, and they would be surprised that in zsh, by default :

1/2 + 1/2 = 0

And I'll bet they'd wonder why even though zsh is quite capable of doing 
floating point math, it chooses to behave like an abacus by default, 
other programing languages notwithstanding.  Never mind, I'll just 
'setopt force_float' and have accuracy.   I'm glad I was forewarned 
about this, however.
>

Re: Math expression evaluation error?

[Bart Schaefer]

On Jan 9, 11:27pm, Ray Andrews wrote:
} Subject: Re: Math expression evaluation error?
}
} > 3/8 is not a fraction.
}
} Well, this is one of those things that won't change, but I must say that 
} in my experience an overwhelming majority of people would say that 3/8 
} is a fraction, and they would be surprised

OK, let's leave zsh out of this and try the standard "expr" utility,
which built-in arithmetic was designed to replace:

% expr 3 / 8
0
% expr 3.0 / 8.0
expr: non-numeric argument

Re: Math expression evaluation error?

[Ray Andrews]

On 01/10/2015 08:53 AM, Bart Schaefer wrote:
> OK, let's leave zsh out of this and try the standard "expr" utility,
> which built-in arithmetic was designed to replace:
>
> % expr 3 / 8
> 0
> % expr 3.0 / 8.0
> expr: non-numeric argument
>
>
Bart, I shouldn't argue because it's to the point of irritation, and 
that is never positive, but:  My stance is not to think about 'expr' or 
other programming languages, or what ksh did in '79 or about glorious 
traditions going back to the mainframe days, or any of that.  My stance 
is just to ask: What is most useful?  What would produce the maximum 
utility along with the least surprise, especially for a new user? In 
which direction is 'the future'?

I now know to just set the option, and I'm all good, problem averted. So 
it really is just about philosophy.  IMHO we have moved past the abacus 
and a new user, once s/he learns  that zsh does internal 
arithmetic--that it has an excellent internal calculator--will expect 
that: 3/8 + 5/8 = 1.  I think that 99% of humanity would expect that.  
I'm a math tutor, and I am positive that every one of  my students would 
expect it.  I'm not saying that integer arithmetic should not be 
available, but should it be the default?  How often does anyone want or 
expect:  1/2 + 1/2 = 0? Maybe someone really needs that, and far be it 
from me to deprive them of it, but I'm claiming that it doesn't happen 
very often.

    {
       echo $(( 1/2 + 1/2 ));
       echo $(( .5 + .5 ));
    }
    0
    1

... is, in my mind, a monstrosity.  The very act of typing: " 1/2 " is a 
request for floating point arithmetic in just about exactly the same way 
that typing a decimal point is.  I'm not sure that " 1/2 " means 
anything at all within the confines of integer/abacus arithmetic, I 
think it's an imaginary number and should return an error, not zero.  
IMHO the zsh manual for arithmetic would read something like this:

"
Zsh has a powerful built in calculator. Type expressions just as you 
would in any other calculator and get the results you expect with no 
strange surprises or gotchas. For example:

     $ echo $(( ((.5 + 1/2) * 2) **2
     4

Zsh also does integer arithmetic.  Users of the abacus will appreciate 
this ;-) To enter integer mode:

    $ setopt integer_arithmetic

In this mode, the above expression returns '1'. Have fun with that, 
however we   recommend you not use this mode for teaching arithmetic to 
children.
"

Just my personal opinion.

Re: Math expression evaluation error?

[Bart Schaefer]

On Jan 10, 10:48am, Ray Andrews wrote:
}
} is just to ask: What is most useful?  What would produce the maximum 
} utility along with the least surprise, especially for a new user? In 
} which direction is 'the future'?

Call me a cynic or a curmudgeon as you prefer, but the future of new
users who would give so little thought and study to the tools they
are using as to find this a problem more than once, is to not use
shells at all.

It *is* probably less than ideal that "zcalc -f" is not the default
for the actual calculator utility; but pretending that $(( ... )) could
ever be described "a calculator with no surprises or gotchas" regardless
of integer arithmetic, is misleading because it isn't a calculator and
trying to think of it that way is going to cause surprises and gotchas.

Re: Math expression evaluation error?

[Ray Andrews]

On 01/10/2015 11:14 AM, Bart Schaefer wrote:
> On Jan 10, 10:48am, Ray Andrews wrote:
> }
> } is just to ask: What is most useful?  What would produce the maximum
> } utility along with the least surprise, especially for a new user? In
> } which direction is 'the future'?
>
> Call me a cynic or a curmudgeon as you prefer, but the future of new
> users who would give so little thought and study to the tools they
> are using as to find this a problem more than once, is to not use
> shells at all.

That's quite legitimate.  The 'fix' is after all very simple.  You know, in
this sort of discussion pressures can build beyond what the issue
merits.  'setopt force_float' and there's nothing more to worry about.
But so long as you wish to discuss it, I'll just do what I do and advocate
for a friendlier and more intuitive shell where possible:  However simple
the fix may be, it remains my view that the intuitive behavior should
be the default.  It's just my paradigm.  I quite understand that it will
not prevail. Not worth the price of a beer, and certainly not worth any
disturbance.
>
> It *is* probably less than ideal that "zcalc -f" is not the default

And that's all I'm saying, (but, for "$ (( ))" too).

BTW, just Googling, I see that this issue has come up time and again.
Our original poster on this thread is not the first person to be got by
this gotcha.  Why not make the issue go away, once and for all?

One could have fun with some facetious advertising:
"Try zsh, the shell where 1/2 + 1/2 = 1 !!!"

> for the actual calculator utility; but pretending that $(( ... )) could
> ever be described "a calculator with no surprises or gotchas" regardless
> of integer arithmetic, is misleading because it isn't a calculator and
> trying to think of it that way is going to cause surprises and gotchas.

Well, it could come to a question of terminology or even of hard facts.  I
myself have (until this issue came up) never done anything beyond simple
integer addition and subtraction. From what I read, $ (( )) does quite
sophisticated calculations and it is ergo, ipso facto a calculator. And 
I also
presume that it accesses all the power of 'zcalc' (because why 
*wouldn't* it?)

However, if it does have gotchas in it (I mean beyond the current subject)
I can't say, I just presume it does what it advertises itself to do, and
my experiments with it bare that out-- and that's 'calculation' as I use
the word. But don't  take my pseudo manual as anything more than a
comment please it's just what I wish was the case.

Re: Math expression evaluation error?

[Bart Schaefer]

On Jan 10,  2:21pm, Ray Andrews wrote:
}
} BTW, just Googling, I see that this issue has come up time and again.
} Our original poster on this thread is not the first person to be got by
} this gotcha.  Why not make the issue go away, once and for all?

Because there is such a thing as standardized behavior across shells,
and while it is fine to add or extend features, egregiously changing
the behavior of the common subset is generally a bad idea.

} I also presume that it accesses all the power of 'zcalc' (because why 
} *wouldn't* it?)

It's actually the other way around.  zcalc accesses the power of $(( ))
but "looks" a bit more like a calculator.

-- 
Barton E. Schaefer

Re: Math expression evaluation error?

[Ray Andrews]

On 01/10/2015 05:51 PM, Bart Schaefer wrote:
> Because there is such a thing as standardized behavior across shells,
> and while it is fine to add or extend features, egregiously changing
> the behavior of the common subset is generally a bad idea.

Sure.  That's the best argument for leaving it alone and of course it is
perfectly sound.  I'd go the other way, liking friendliness more, but
I respect your decision. And I could be quite wrong about 1/2 + 1/2 = 0,
strange as it looks to me, rounding down could be vital to some code
somewhere.

Re: Math expression evaluation error?

[Vincent Lefevre]

On 2015-01-10 21:10:16 -0800, Ray Andrews wrote:
> Sure.  That's the best argument for leaving it alone and of course it is
> perfectly sound.  I'd go the other way, liking friendliness more, but
> I respect your decision. And I could be quite wrong about 1/2 + 1/2 = 0,
> strange as it looks to me, rounding down could be vital to some code
> somewhere.

It's only a notation problem. Unfortunately / was chosen (historically,
not by zsh) as the symbol for integer division. The shell expression
1/2 + 1/2 just means:

  integer_div(1,2) + integer_div(1,2)

where integer_div is the integer division function, which gives 0.
This is perfectly correct math.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Math expression evaluation error?

[Peter Stephenson]

On Fri, 9 Jan 2015 22:40:34 +0000
Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> +Users should beware that, in common with many other programming
> +languages but not software designed for calculation, the evaluation of
> +an expression in zsh is taken a term at a time and promotion of integers
> +to floating point does not occur in terms only containing integers.  A
> +typical result of this is that a division such as tt(6/8) is truncated,
> +in this being rounded down to 0.  The tt(FORCE_FLOAT) shell option can
> +be used in scripts or functions where floating point evaluation is
> +required throughout.

Before this drifts out of my mind, here's a similar warning for zcalc.

Assuming anybody actually bothers to read the documentation...

diff --git a/Doc/Zsh/contrib.yo b/Doc/Zsh/contrib.yo
index 5043243..00ede52 100644
--- a/Doc/Zsh/contrib.yo
+++ b/Doc/Zsh/contrib.yo
@@ -3227,8 +3227,18 @@ A reasonably powerful calculator based on zsh's arithmetic evaluation
 facility.  The syntax is similar to that of formulae in most programming
 languages; see
 ifzman(the section `Arithmetic Evaluation' in zmanref(zshmisc))\
-ifnzman(noderef(Arithmetic Evaluation)) for details.  The mathematical
-library tt(zsh/mathfunc) will be loaded if it is available; see
+ifnzman(noderef(Arithmetic Evaluation)) for details.
+
+Non-programmers should note that, as in many other programming
+languages, expressions involving only integers (whether constants
+without a `tt(.)', variables containing such constants as strings, or
+variables declared to be integers) are by default evaluated using
+integer arithmetic, which is not how an ordinary desk calculator
+operates.  To force floating point operation, pass the option tt(-f);
+see further notes below.
+
+The mathematical library tt(zsh/mathfunc) will be loaded if it is
+available; see
 ifzman(the section `The zsh/mathfunc Module' in zmanref(zshmodules))\
 ifnzman(noderef(The zsh/mathfunc Module)).  The mathematical functions
 correspond to the raw system libraries, so trigonometric functions are

pws

Re: Math expression evaluation error?

[ZyX]

12.01.2015, 12:18, "Vincent Lefevre" <vincent@vinc17.net>:
> On 2015-01-10 21:10:16 -0800, Ray Andrews wrote:
>>  Sure.  That's the best argument for leaving it alone and of course it is
>>  perfectly sound.  I'd go the other way, liking friendliness more, but
>>  I respect your decision. And I could be quite wrong about 1/2 + 1/2 = 0,
>>  strange as it looks to me, rounding down could be vital to some code
>>  somewhere.
>
> It's only a notation problem. Unfortunately / was chosen (historically,
> not by zsh) as the symbol for integer division. The shell expression
> 1/2 + 1/2 just means:
>
>   integer_div(1,2) + integer_div(1,2)
>
> where integer_div is the integer division function, which gives 0.
> This is perfectly correct math.

It is not integer division. 1.0/2 and 1/2.0 will give the same result: 0.5. If you want to see language with operator for integer division check out Python-3: it has / for always-floating-point-unless-overridden and // for always-integral-unless-overridden (note: *not* integer (i.e. int type), having float on either side of // will result in floating-point integral value). Though as usual you can do anything you like in your own class, including `rm -rf ~` when trying to use division.

I would rather call it “type-specific division”. And I think that Python-3 way is least surprising and most convenient variant. Though you should not use // for dividing non-integral floats: 1.5//0.1 == 14.0 (rounding+decimal-binary conversion issues I guess).

>
> --
> Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
> 100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
> Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Math expression evaluation error?

[Vincent Lefevre]

On 2015-01-12 19:18:22 +0300, ZyX wrote:
> 12.01.2015, 12:18, "Vincent Lefevre" <vincent@vinc17.net>:
> > It's only a notation problem. Unfortunately / was chosen (historically,
> > not by zsh) as the symbol for integer division. The shell expression
> > 1/2 + 1/2 just means:
> >
> >   integer_div(1,2) + integer_div(1,2)
> >
> > where integer_div is the integer division function, which gives 0.
> > This is perfectly correct math.
> 
> It is not integer division.

In POSIX, it is always an integer division.

> 1.0/2 and 1/2.0 will give the same result: 0.5.

because 1.0 and 2.0 don't have an integer type. Therefore the result is
unspecified by POSIX, and zsh chooses to do a floating-point division.

> If you want to see language with operator for integer division check
> out Python-3: it has / for always-floating-point-unless-overridden
> and // for always-integral-unless-overridden

For Python, but not for zsh, unfortunately. But this choice comes
from POSIX historical behavior (and perhaps earlier).

> I would rather call it “type-specific division”.

In zsh. In POSIX, it is *always* an integer division. There is no
floating-point there.

An alternative choice zsh could have done is to return an error
if the / operator is used in a floating-point context, and use a
different operator for the floating-point division. However the
use of / for integer division could still be surprising.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Math expression evaluation error?

[ZyX]

13.01.2015, 19:01, "Vincent Lefevre" <vincent@vinc17.net>:
> On 2015-01-12 19:18:22 +0300, ZyX wrote:
>>  12.01.2015, 12:18, "Vincent Lefevre" <vincent@vinc17.net>:
>>>  It's only a notation problem. Unfortunately / was chosen (historically,
>>>  not by zsh) as the symbol for integer division. The shell expression
>>>  1/2 + 1/2 just means:
>>>
>>>    integer_div(1,2) + integer_div(1,2)
>>>
>>>  where integer_div is the integer division function, which gives 0.
>>>  This is perfectly correct math.
>>  It is not integer division.
>
> In POSIX, it is always an integer division.

What?!

1. How POSIX is related? Zsh is not a POSIX shell and it is not emulation mode that is being discussed here.
2. If this standard is correct: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tagtcjh_15 then it references ISO C standard. The ISO C standard (C99) says the following for multiplicative operators:

   > 6.5.5 Multiplicative operators
   …
   > 3 The usual arithmetic conversions are performed on the operands.
   > 2 Each of the operands shall have arithmetic type. The operands of the % operator shall
   >   have integer type.
   …
   > 5 The result of the / operator is the quotient from the division of the first operand by the
   >   second; the result of the % operator is the remainder. In both operations, if the value of
   >   the second operand is zero, the behavior is undefined.
   > 6 When integers are divided …

   Where do you see a requirement for `/` to be integer division?
3. POSIX (http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04) says that “Only signed long integer arithmetic is required.” And the section referenced there talks about signed long or double. Signed long integer arithmetic is *required* to be in there, but signed long integer arithmetics is *not required* to be the *only* arithmetics present in POSIX shell.

So no, `/` has nothing to do with integer division, except that the referenced C standard expects well-defined integer division in case two integers are divided. If you have another POSIX standard or another interpretation please show it here.

>>  1.0/2 and 1/2.0 will give the same result: 0.5.
>
> because 1.0 and 2.0 don't have an integer type. Therefore the result is
> unspecified by POSIX, and zsh chooses to do a floating-point division.
>>  If you want to see language with operator for integer division check
>>  out Python-3: it has / for always-floating-point-unless-overridden
>>  and // for always-integral-unless-overridden
>
> For Python, but not for zsh, unfortunately. But this choice comes
> from POSIX historical behavior (and perhaps earlier).
>>  I would rather call it “type-specific division”.
>
> In zsh. In POSIX, it is *always* an integer division. There is no
> floating-point there.
>
> An alternative choice zsh could have done is to return an error
> if the / operator is used in a floating-point context, and use a
> different operator for the floating-point division. However the
> use of / for integer division could still be surprising.
>
> --
> Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
> 100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
> Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Math expression evaluation error?

[Vincent Lefevre]

On 2015-01-14 02:03:33 +0300, ZyX wrote:
> 13.01.2015, 19:01, "Vincent Lefevre" <vincent@vinc17.net>:
> > In POSIX, it is always an integer division.
> 
> What?!
> 
> 1. How POSIX is related? Zsh is not a POSIX shell and it is not emulation mode that is being discussed here.

Zsh is partly based on POSIX for compatibility. The big difference
is the lack of word splitting (unless SH_WORD_SPLIT is set). Otherwise
I think that one should expect similar behavior, unless there is a
good reason.

> 2. If this standard is correct: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tagtcjh_15 then it references ISO C standard.
[...]
>    Where do you see a requirement for `/` to be integer division?

The context is an integer arithmetic. Thus / is necessarily an
integer division, like in C with integer types.

> 3. POSIX (http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04) says that “Only signed long integer arithmetic is required.” And the section referenced there talks about signed long or double. Signed long integer arithmetic is *required* to be in there, but signed long integer arithmetics is *not required* to be the *only* arithmetics present in POSIX shell.

It says: "as long as it does not affect the results in cases where
there is no overflow"

If $((1/2)) gave 0.5 in some shells, I fear this would break various
scripts.

So, if a real-floating type is used, then for "/", I suppose that
the implementation should do a floor() after the division if the
context has operands recognized as integers, which is here equivalent
to the integer division despite the two roundings (proved in
http://www.vinc17.net/research/papers/rr_intdiv).

But POSIX doesn't specify the arithmetic evaluation on expressions
other than signed long integer arithmetic. An implementation that
decides that $((1.0/2)) gives 17 "as an extension" could still be
conforming.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Math expression evaluation error?

[Vincent Lefevre]

On 2015-01-09 22:40:34 +0000, Peter Stephenson wrote:
> diff --git a/Doc/Zsh/arith.yo b/Doc/Zsh/arith.yo
> index a620b73..5c334ce 100644
> --- a/Doc/Zsh/arith.yo
> +++ b/Doc/Zsh/arith.yo
> @@ -233,6 +233,15 @@ necessary.  In addition, if any operator which requires an integer
>  equivalents with assignment) is given a floating point argument, it will be
>  silently rounded down to the next integer.
>  
> +Users should beware that, in common with many other programming
> +languages but not software designed for calculation, the evaluation of
> +an expression in zsh is taken a term at a time and promotion of integers
> +to floating point does not occur in terms only containing integers.  A
> +typical result of this is that a division such as tt(6/8) is truncated,
> +in this being rounded down to 0.  The tt(FORCE_FLOAT) shell option can
> +be used in scripts or functions where floating point evaluation is
> +required throughout.

This explanation is incorrect (or very ambiguous). FORCE_FLOAT doesn't
force a floating-point evaluation; it just converts *constants* to
floating-point. See the differences:

ypig% setopt FORCE_FLOAT
ypig% integer a=1 b=2
ypig% echo $((1/2))
0.5
ypig% echo $((a/b))
0

I'm wondering whether this is the expected behavior, though.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Math expression evaluation error?

[Peter Stephenson]

On Wed, 14 Jan 2015 16:02:58 +0100
Vincent Lefevre <vincent@vinc17.net> wrote:
> FORCE_FLOAT doesn't
> force a floating-point evaluation; it just converts *constants* to
> floating-point. See the differences:
> 
> ypig% setopt FORCE_FLOAT
> ypig% integer a=1 b=2
> ypig% echo $((1/2))
> 0.5
> ypig% echo $((a/b))
> 0
> 
> I'm wondering whether this is the expected behavior, though.

I wouldn't have thought so --- I would guess when forcing floating
point calculations people would expect this to happen everywhere,
despite what the documentation for the option says.

This shouldn't be problematic if it's already working for constants
since variables appear in just the same contexts.

diff --git a/Doc/Zsh/options.yo b/Doc/Zsh/options.yo
index 076aaf4..8a0222c 100644
--- a/Doc/Zsh/options.yo
+++ b/Doc/Zsh/options.yo
@@ -496,9 +496,10 @@ pindex(NOFORCEFLOAT)
 cindex(floating point, forcing use of)
 cindex(forcing use of floating point)
 item(tt(FORCE_FLOAT))(
-Constants in arithmetic evaluation will be treated as floating point
-even without the use of a decimal point.  Integers in any base
-will be converted.
+Constants in arithmetic evaluation will be treated as
+floating point even without the use of a decimal point; the
+values of integer variables will be converted to floating point when
+used in arithmetic expressions.  Integers in any base will be converted.
 )
 pindex(GLOB)
 pindex(NO_GLOB)
diff --git a/Src/math.c b/Src/math.c
index db42d0f..c047725 100644
--- a/Src/math.c
+++ b/Src/math.c
@@ -336,16 +336,27 @@ enum prec_type {
 static mnumber
 getmathparam(struct mathvalue *mptr)
 {
+    mnumber result;
     if (!mptr->pval) {
 	char *s = mptr->lval;
 	mptr->pval = (Value)zhalloc(sizeof(struct value));
 	if (!getvalue(mptr->pval, &s, 1))
 	{
 	    mptr->pval = NULL;
+	    if (isset(FORCEFLOAT)) {
+		result.type = MN_FLOAT;
+		result.u.d = 0.0;
+		return result;
+	    }
 	    return zero_mnumber;
 	}
     }
-    return getnumvalue(mptr->pval);
+    result = getnumvalue(mptr->pval);
+    if (isset(FORCEFLOAT) && result.type == MN_INTEGER) {
+	result.type = MN_FLOAT;
+	result.u.d = (double)result.u.l;
+    }
+    return result;
 }
 
 static mnumber
diff --git a/Test/C01arith.ztst b/Test/C01arith.ztst
index 8e0730d..ea87af2 100644
--- a/Test/C01arith.ztst
+++ b/Test/C01arith.ztst
@@ -308,3 +308,13 @@
 >2
 >2
 # It's hard to test for integer to float.
+
+  integer ff1=3 ff2=4
+  print $(( ff1/ff2 ))
+  setopt force_float
+  print $(( ff1/ff2 ))
+  unsetopt force_float
+0:Variables are forced to floating point where necessary
+# 0.75 is exactly representable, don't expect rounding error.
+>0
+>0.75

Re: Math expression evaluation error?

[ZyX]

14.01.2015, 17:48, "Vincent Lefevre" <vincent@vinc17.net>:
> On 2015-01-14 02:03:33 +0300, ZyX wrote:
>>  13.01.2015, 19:01, "Vincent Lefevre" <vincent@vinc17.net>:
>>>  In POSIX, it is always an integer division.
>>  What?!
>>
>>  1. How POSIX is related? Zsh is not a POSIX shell and it is not emulation mode that is being discussed here.
>
> Zsh is partly based on POSIX for compatibility. The big difference
> is the lack of word splitting (unless SH_WORD_SPLIT is set). Otherwise
> I think that one should expect similar behavior, unless there is a
> good reason.
>>  2. If this standard is correct: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tagtcjh_15 then it references ISO C standard.
>
> [...]
>>     Where do you see a requirement for `/` to be integer division?
>
> The context is an integer arithmetic. Thus / is necessarily an
> integer division, like in C with integer types.

No. The context explicitly says that signed long integers or doubles should be used. Context of the ISO C standard obviously does not say anything like this.

>>  3. POSIX (http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04) says that “Only signed long integer arithmetic is required.” And the section referenced there talks about signed long or double. Signed long integer arithmetic is *required* to be in there, but signed long integer arithmetics is *not required* to be the *only* arithmetics present in POSIX shell.
>
> It says: "as long as it does not affect the results in cases where
> there is no overflow"
>
> If $((1/2)) gave 0.5 in some shells, I fear this would break various
> scripts.

This is as well against the standard because in C division of two integers yields integers and is performed according to integer division rules.

>
> So, if a real-floating type is used, then for "/", I suppose that
> the implementation should do a floor() after the division if the
> context has operands recognized as integers, which is here equivalent
> to the integer division despite the two roundings (proved in
> http://www.vinc17.net/research/papers/rr_intdiv).
>
> But POSIX doesn't specify the arithmetic evaluation on expressions
> other than signed long integer arithmetic. An implementation that
> decides that $((1.0/2)) gives 17 "as an extension" could still be
> conforming.

Nope, it cannot ever be conforming. POSIX does this by saying that semantic of operation is the same as in C, except that only signed long integers and doubles are allowed. In C 1.0/2 gives 0.5 because this is how conversion rules are defined.

If you want to say it does *show me where POSIX explicitly forbids doubles in definition of arithmetic expansion*. I only see it explicitly forbidding the internal shell type (which is allowed to be greater then long or even be floating-point) to affect the results of operation (unless there is an overflow which is in any case not defined for signed integers) which is the case if one tries to divide two integers (effectively requiring shell to keep track on types even if it wants to use double for everything) and also in case doubles are used, effectively meaning that if shell supports doubles it should yield doubles after division involving a double: “The evaluation of arithmetic expressions shall be equivalent to that described in Section 6.5, Expressions, of the ISO C standard.”.

>
> --
> Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
> 100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
> Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Math expression evaluation error?

[Vincent Lefevre]

On 2015-01-15 00:37:43 +0300, ZyX wrote:
> 14.01.2015, 17:48, "Vincent Lefevre" <vincent@vinc17.net>:
> > On 2015-01-14 02:03:33 +0300, ZyX wrote:
> >>  13.01.2015, 19:01, "Vincent Lefevre" <vincent@vinc17.net>:
> >>>  In POSIX, it is always an integer division.
> >>  What?!
> >>
> >>  1. How POSIX is related? Zsh is not a POSIX shell and it is not emulation mode that is being discussed here.
> >
> > Zsh is partly based on POSIX for compatibility. The big difference
> > is the lack of word splitting (unless SH_WORD_SPLIT is set). Otherwise
> > I think that one should expect similar behavior, unless there is a
> > good reason.
> >>  2. If this standard is correct: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tagtcjh_15 then it references ISO C standard.
> >
> > [...]
> >>     Where do you see a requirement for `/` to be integer division?
> >
> > The context is an integer arithmetic. Thus / is necessarily an
> > integer division, like in C with integer types.
> 
> No. The context explicitly says that signed long integers or doubles
> should be used. Context of the ISO C standard obviously does not say
> anything like this.

Wrong. The context of $((...)) is signed long arithmetic. If a shell
wants to use double's (such as 1.0), that's only an extension.

> > But POSIX doesn't specify the arithmetic evaluation on expressions
> > other than signed long integer arithmetic. An implementation that
> > decides that $((1.0/2)) gives 17 "as an extension" could still be
> > conforming.
> 
> Nope, it cannot ever be conforming. POSIX does this by saying that
> semantic of operation is the same as in C,

Only for what it specifies.

> except that only signed long integers and doubles are allowed.

No, POSIX says "As an extension, the shell may recognize arithmetic
expressions beyond those listed." and does not specify anything on
such expressions. $((1.0/2)) is such an arithmetic expression beyond
those listed.

In particular, it does *not* say that "only signed long integers and
doubles are allowed". A shell could implement other types (number
fields or whatever), and specific operators. And note that in such
an expression, "/" is not necessarily an operator, but could be part
of a constant.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Math expression evaluation error?

[Vincent Lefevre]

On 2015-01-15 14:34:21 +0100, Vincent Lefevre wrote:
> > > But POSIX doesn't specify the arithmetic evaluation on expressions
> > > other than signed long integer arithmetic. An implementation that
> > > decides that $((1.0/2)) gives 17 "as an extension" could still be
> > > conforming.
> > 
> > Nope, it cannot ever be conforming. POSIX does this by saying that
> > semantic of operation is the same as in C,
> 
> Only for what it specifies.

BTW, note also that the ISO C standard doesn't specify the accuracy,
unless Annex F (IEEE 754) is supported (which is not required by
POSIX). So, even if the shell claims to implement floating-point
arithmetic, any floating-point number as the result of $((1.0/2))
would be conforming.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Math expression evaluation error?

[Peter Stephenson]

On Wed, 14 Jan 2015 15:38:42 +0000
Peter Stephenson <p.stephenson@samsung.com> wrote:
> I wouldn't have thought so --- I would guess when forcing floating
> point calculations people would expect this to happen everywhere,
> despite what the documentation for the option says.

This notes the change of behaviour.

pws

diff --git a/README b/README
index 4d2b6c1..d2b8b69 100644
--- a/README
+++ b/README
@@ -38,10 +38,12 @@ details, see the documentation.
 Incompatibilites between 5.0.7 and 5.0.8
 ----------------------------------------
 
-A couple of arithmetic operations have changed: the new behaviour
-is intended to be more consistent, but is not compatible with the old.
+Various arithmetic operations have changed, in particular with respect
+to the choice of integer or floating point operations.  The new
+behaviour is intended to be more consistent, but is not compatible with
+the old.
 
-Previously, the modulus operation, `%', implicitly converted the
+1) Previously, the modulus operation, `%', implicitly converted the
 operation to integer and output an integer result, even if one
 or both of the arguments were floating point.  Now, the C math
 library fmod() operator is used to implement the operation where
@@ -57,7 +59,8 @@ New behaviour:
 % print $(( 5.5 % 2 ))
 1.5
 
-Previously, assignments to variables assigned the correct type to
+
+2) Previously, assignments to variables assigned the correct type to
 variables declared as floating point or integer, but this type was
 not propagated to the value of the expression, as a C programmer
 would naturally expect.  Now, the type of the variable is propagated
@@ -81,6 +84,44 @@ New behaviour:
 % print $var
 2
 
+
+3) Previously, the FORCE_FLOAT option only forced the use of floating
+point in arithmetic expressions for integers constants, i.e. numbers
+typed directly into the expression, but not for variables.  Hence
+an operation involving only integer variables (or string variables
+containing integers) was not forced to be performed with floating point
+arithmetic.  Now, operations involving variables are also forced to
+floating point.  For example:
+
+Old behaviour:
+
+% unsetopt FORCE_FLOAT
+% print $(( 1 / 2 ))
+0
+% integer i=1 j=2
+% print $(( i / j ))
+0
+% setopt FORCE_FLOAT
+% print $(( 1 / 2 ))
+0.5
+% print $(( i / j ))
+0
+
+New behaviour:
+
+% unsetopt FORCE_FLOAT
+% print $(( 1 / 2 ))
+0
+% integer i=1 j=2
+% print $(( i / j ))
+0
+% setopt FORCE_FLOAT
+% print $(( 1 / 2 ))
+0.5
+% print $(( i / j ))
+0.5
+
+
 Incompatibilities between 5.0.2 and 5.0.5
 -----------------------------------------
 

Re: Math expression evaluation error?

[ZyX]

15.01.2015, 16:44, "Vincent Lefevre" <vincent@vinc17.net>:
> On 2015-01-15 00:37:43 +0300, ZyX wrote:
>>  14.01.2015, 17:48, "Vincent Lefevre" <vincent@vinc17.net>:
>>>  On 2015-01-14 02:03:33 +0300, ZyX wrote:
>>>>   13.01.2015, 19:01, "Vincent Lefevre" <vincent@vinc17.net>:
>>>>>   In POSIX, it is always an integer division.
>>>>   What?!
>>>>
>>>>   1. How POSIX is related? Zsh is not a POSIX shell and it is not emulation mode that is being discussed here.
>>>  Zsh is partly based on POSIX for compatibility. The big difference
>>>  is the lack of word splitting (unless SH_WORD_SPLIT is set). Otherwise
>>>  I think that one should expect similar behavior, unless there is a
>>>  good reason.
>>>>   2. If this standard is correct: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html#tagtcjh_15 then it references ISO C standard.
>>>  [...]
>>>>      Where do you see a requirement for `/` to be integer division?
>>>  The context is an integer arithmetic. Thus / is necessarily an
>>>  integer division, like in C with integer types.
>>  No. The context explicitly says that signed long integers or doubles
>>  should be used. Context of the ISO C standard obviously does not say
>>  anything like this.
>
> Wrong. The context of $((...)) is signed long arithmetic. If a shell
> wants to use double's (such as 1.0), that's only an extension.

It is never written in the standard that context is signed long arithmetic. It is only written that signed long integers are the only required types, not the only possible.

>>>  But POSIX doesn't specify the arithmetic evaluation on expressions
>>>  other than signed long integer arithmetic. An implementation that
>>>  decides that $((1.0/2)) gives 17 "as an extension" could still be
>>>  conforming.
>>  Nope, it cannot ever be conforming. POSIX does this by saying that
>>  semantic of operation is the same as in C,
>
> Only for what it specifies.
>>  except that only signed long integers and doubles are allowed.
>
> No, POSIX says "As an extension, the shell may recognize arithmetic
> expressions beyond those listed." and does not specify anything on
> such expressions. $((1.0/2)) is such an arithmetic expression beyond
> those listed.
>
> In particular, it does *not* say that "only signed long integers and
> doubles are allowed". A shell could implement other types (number
> fields or whatever), and specific operators. And note that in such
> an expression, "/" is not necessarily an operator, but could be part
> of a constant.

Standard says that “The arithmetic expression shall be processed according to the rules given in Arithmetic Precision and Operations”. In the referenced section it talks about signed long integers and doubles, neither is forbidden by “the following exceptions”. It as well talks about C rules for arithmetic. So 1.0 is *listed*, but is not *required*. If you want to say POSIX does not list doubles show where it excludes double from the list. Exceptions bullet-list *does not* exclude anything, but selection, iteration and jump statements. All other points are given in a form “feature is not required” or “[of the similar features, I guess] only feature is required” which is not the same as “The arithmetic expression shall be processed according to the rules for signed integers given in …” (in which case doubles indeed were not falling into listed category).

If a shell wants to use double’s it falls into a listed category. But if shell wants to use `//` for always-floating-point division operator then it is what is “beyond those listed”. Zsh math functions are as well, even if they have names from math.h (requirements on such functions are put in the next sub…section (what should something that has lesser then third level called? This whatever is under 1.1.2 subsubsection) which was not referenced in shell standard). This also applies to $(( base#number )) syntax. But not to doubles because they are listed.

The standard *does* say that integer variables and constants shall be implemented as signed long integers and floating-point things should use double (or the equivalent, for both). It is also explicitly said that conversion rules between them are derived from C. So POSIX-compliant $(()) must either not use floating-point values or have double for floating-point values, but it must not have something strange for floating-point values: e.g. lower-precision float or Fraction with two big integers (I mean Fraction class from Python fractions module which basically is a numerator+denominator pair capable of representing any rational number (Python uses big integers for its only integer type)).

>
> --
> Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
> 100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
> Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Math expression evaluation error?

[Bart Schaefer]

Guys, this POSIX argument has gone on way too long and with way too much
context being quoted every time somebody replies.  I haven't seen any
suggestion emerge from this that we should be doing anything different
than we alreay decided to do, so unless you're taking this somewhere that
might end in a concrete proposal of some kind, please wrap it up or stop
copying the list.  Thanks.