From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 10604 invoked from network); 25 Aug 2002 16:02:10 -0000 Received: from sunsite.dk (130.225.247.90) by ns1.primenet.com.au with SMTP; 25 Aug 2002 16:02:10 -0000 Received: (qmail 8220 invoked by alias); 25 Aug 2002 16:01:58 -0000 Mailing-List: contact zsh-users-help@sunsite.dk; run by ezmlm Precedence: bulk X-No-Archive: yes X-Seq: 5286 Received: (qmail 8205 invoked from network); 25 Aug 2002 16:01:58 -0000 X-Original-Recipient: Date: Sun, 25 Aug 2002 18:01:51 +0200 From: Erik Trulsson To: Patrick Aussems Cc: zsh-users@sunsite.auc.dk Subject: Re: Floating point calculus error... Message-ID: <20020825160151.GA5679@falcon.midgard.homeip.net> Mail-Followup-To: Patrick Aussems , zsh-users@sunsite.auc.dk References: <1030288941.517.8.camel@Amok> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1030288941.517.8.camel@Amok> User-Agent: Mutt/1.5.1i On Sun, Aug 25, 2002 at 05:22:20PM +0200, Patrick Aussems wrote: > Hi all... > > I was doing some shell scripts that were supposed to add CPU usages from > the ps output... But I got puzzled when I got an answer with a 17th > decimal while adding numbers with only 1 decimal... Strange isn't it? Not strange at all. That sort of things is to be expected when using floating point. > > This behaviour can be observed by doing the following command: > > zyk@Amok:/home/zyk> echo $[0.4+0.4] > > The value returned is: 0.80000000000000004 which is obviously not the > correct answer... Okay, floating points numbers are inaccurate, but > still... Since neither 0.4 or 0.8 cannot be represented exactly in a binary floating point format one shouldn't expect the ouput to be quite correct. The internal form that "0.4" gets converted to is actually some other number which has almost but not exactly the value 0.4 When you then add two such numbers you will get a number that is almost but not quite 0.8. This is similar to the behaviour of most pocket calculators where trying to calculate 3*(1/3) usually results in 0.9999999999 instead of the mathematically correct 1. > > Patrick > > PS: I don't have the time to read the mailing lists, so I don't know if > this problem has already been reported... It is not really a problem. It is just the way floating point is. > > PS2: Please CC me on replies, I'm not subscribed to this mailing list... > > -- Erik Trulsson ertr1013@student.uu.se