Floating point calculus error...

Floating point calculus error...

[Patrick Aussems]

Hi all...

I was doing some shell scripts that were supposed to add CPU usages from
the ps output... But I got puzzled when I got an answer with a 17th
decimal while adding numbers with only 1 decimal... Strange isn't it?

This behaviour can be observed by doing the following command:

zyk@Amok:/home/zyk> echo $[0.4+0.4]

The value returned is: 0.80000000000000004 which is obviously not the
correct answer... Okay, floating points numbers are inaccurate, but
still...

Patrick

PS: I don't have the time to read the mailing lists, so I don't know if
this problem has already been reported...

PS2: Please CC me on replies, I'm not subscribed to this mailing list...

Re: Floating point calculus error...

[Vincent Lefevre]

On Sun, Aug 25, 2002 at 17:22:20 +0200, Patrick Aussems wrote:
> I was doing some shell scripts that were supposed to add CPU usages from
> the ps output... But I got puzzled when I got an answer with a 17th
> decimal while adding numbers with only 1 decimal... Strange isn't it?

The numbers are internally represented in binary and 0.4 isn't exactly
representable in binary... But here, this is a problem with zsh, which
gives an additional decimal digit. You can see with:

$ echo $[0.4]
0.40000000000000002

I think that it should give the minimum number of decimals in such a way
that when converting back to binary, one should get the same result.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/> - 100%
validated (X)HTML - Acorn Risc PC, Yellow Pig 17, Championnat International
des Jeux Mathématiques et Logiques, TETRHEX, etc.
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: Floating point calculus error...

[David Kågedal]

Vincent Lefevre <vincent@vinc17.org> writes:

> On Sun, Aug 25, 2002 at 17:22:20 +0200, Patrick Aussems wrote:
>> I was doing some shell scripts that were supposed to add CPU usages from
>> the ps output... But I got puzzled when I got an answer with a 17th
>> decimal while adding numbers with only 1 decimal... Strange isn't it?
>
> The numbers are internally represented in binary and 0.4 isn't exactly
> representable in binary... But here, this is a problem with zsh, which
> gives an additional decimal digit. You can see with:
>
> $ echo $[0.4]
> 0.40000000000000002
>
> I think that it should give the minimum number of decimals in such a way
> that when converting back to binary, one should get the same result.

But it does.

$ echo $[.40000000000000002]
0.40000000000000002

The point is, as you say, that 0.4 can't be represented as an exact
binary floating-point number, no matter how large the representation
is.  Just like 1/3 can't be represented as a decimal floating-point
number.

-- 
David Kågedal

Re: Floating point calculus error...

[Vincent Lefevre]

On Sun, Aug 25, 2002 at 18:32:38 +0200, David Kågedal wrote:
> Vincent Lefevre <vincent@vinc17.org> writes:
> > $ echo $[0.4]
> > 0.40000000000000002
> >
> > I think that it should give the minimum number of decimals in such a way
> > that when converting back to binary, one should get the same result.
> 
> But it does.

No, this is not the *minimum* number of decimal digits. 0.4 would
be sufficient in this case.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/> - 100%
validated (X)HTML - Acorn Risc PC, Yellow Pig 17, Championnat International
des Jeux Mathématiques et Logiques, TETRHEX, etc.
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: Floating point calculus error...

[Erik Trulsson]

On Sun, Aug 25, 2002 at 05:22:20PM +0200, Patrick Aussems wrote:
> Hi all...
> 
> I was doing some shell scripts that were supposed to add CPU usages from
> the ps output... But I got puzzled when I got an answer with a 17th
> decimal while adding numbers with only 1 decimal... Strange isn't it?

Not strange at all.  That sort of things is to be expected when using
floating point.

> 
> This behaviour can be observed by doing the following command:
> 
> zyk@Amok:/home/zyk> echo $[0.4+0.4]
> 
> The value returned is: 0.80000000000000004 which is obviously not the
> correct answer... Okay, floating points numbers are inaccurate, but
> still...

Since neither 0.4 or 0.8 cannot be represented exactly in a binary
floating point format one shouldn't expect the ouput to be quite
correct.  
The internal form that "0.4" gets converted to is actually some other
number which has almost but not exactly the value 0.4 When you then add
two such numbers you will get a number that is almost but not quite
0.8.

This is similar to the behaviour of most pocket calculators where
trying to calculate 3*(1/3) usually results in 0.9999999999 instead of
the mathematically correct 1.


> 
> Patrick
> 
> PS: I don't have the time to read the mailing lists, so I don't know if
> this problem has already been reported...

It is not really a problem. It is just the way floating point is.

> 
> PS2: Please CC me on replies, I'm not subscribed to this mailing list...
> 
> 

-- 
<Insert your favourite quote here.>
Erik Trulsson
ertr1013@student.uu.se

Re: Floating point calculus error...

[Bart Schaefer]

On Aug 25,  5:22pm, Patrick Aussems wrote:
}
} I was doing some shell scripts that were supposed to add CPU usages from
} the ps output... But I got puzzled when I got an answer with a 17th
} decimal while adding numbers with only 1 decimal... Strange isn't it?

Not really.

schaefer[507] print -l $[0.1] $[0.2] $[0.3] $[0.4]
0.10000000000000001
0.20000000000000001
0.29999999999999999
0.40000000000000002
schaefer[508] typeset -F 4 f
schaefer[509] for f in $[0.1] $[0.2] $[0.3] $[0.4]; print $f
0.1000
0.2000
0.3000
0.4000

} The value returned is: 0.80000000000000004 which is obviously not the
} correct answer... Okay, floating points numbers are inaccurate, but
} still...

Yes, floating point numbers are inaccurate.  Zsh simply doesn't round
when inside arithmetic expressions, only when outputting float-formatted
parameters where it knows how many decimal places to use (default 10, but
I forced 4 above).

For comparison, see what happens when the number is computed by floating
point addition rather than by explicit conversion from decimal.  In the
first example below (history number [510]) I've used $(($f)) to force zsh
to round down to 1 decimal place and then convert back to float again.
In the second case, $((f)) evaluates f entirely in math context, using as
much precision as possible, which also shows why the first case didn't stop
at 0.9.

schaefer[510] typeset -F 1 f ; for ((f=0; f < 1; f += 0.1)) print $f = $(($f))
0.0 = 0
0.1 = 0.10000000000000001
0.2 = 0.20000000000000001
0.3 = 0.29999999999999999
0.4 = 0.40000000000000002
0.5 = 0.5
0.6 = 0.59999999999999998
0.7 = 0.69999999999999996
0.8 = 0.80000000000000004
0.9 = 0.90000000000000002
1.0 = 1
schaefer[511] typeset -F 1 f ; for ((f=0; f < 1; f += 0.1)) print $f = $((f)) 
0.0 = 0
0.1 = 0.10000000000000001
0.2 = 0.20000000000000001
0.3 = 0.30000000000000004
0.4 = 0.40000000000000002
0.5 = 0.5
0.6 = 0.59999999999999998
0.7 = 0.69999999999999996
0.8 = 0.79999999999999993
0.9 = 0.89999999999999991
1.0 = 0.99999999999999989

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net