From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from primenet.com.au (ns1.primenet.com.au [203.24.36.2]) by inbox.vuxu.org (OpenSMTPD) with ESMTP id feca24ca for ; Sat, 21 Dec 2019 21:29:09 +0000 (UTC) Received: (qmail 25729 invoked by alias); 21 Dec 2019 21:29:04 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: List-Unsubscribe: X-Seq: 45125 Received: (qmail 29820 invoked by uid 1010); 21 Dec 2019 21:29:04 -0000 X-Qmail-Scanner-Diagnostics: from joooj.vinc17.net by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.102.1/25663. spamassassin: 3.4.2. Clear:RC:0(155.133.131.76):SA:0(-1.9/5.0):. Processed in 1.621618 secs); 21 Dec 2019 21:29:04 -0000 X-Envelope-From: vincent@vinc17.net X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | Received-SPF: none (ns1.primenet.com.au: domain at vinc17.net does not designate permitted sender hosts) Date: Sat, 21 Dec 2019 22:28:25 +0100 From: Vincent Lefevre To: zsh-workers@zsh.org Subject: Re: zsh converts a floating-point number to string with too much precision Message-ID: <20191221212825.GA849755@zira.vinc17.org> Mail-Followup-To: zsh-workers@zsh.org References: <20191220013711.GA708801@zira.vinc17.org> <20191220165824.ufvjtx37xt7dp2dt@chaz.gmail.com> <20191221005005.GB767822@zira.vinc17.org> <20191221084736.bokldw7tzxw3thn3@chaz.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20191221084736.bokldw7tzxw3thn3@chaz.gmail.com> X-Mailer-Info: https://www.vinc17.net/mutt/ User-Agent: Mutt/1.12.1+33 (6a74e24e) vl-117499 (2019-06-23) On 2019-12-21 08:47:36 +0000, Stephane Chazelas wrote: > OK, I think I see what you mean. > > So on a system (with a compiler) where C doubles are implemented > as IEEE 754 double precision, both 1.1 and 1.1000000000000001 > are represented as the same binary double (whose exact value is > 1.100000000000000088817841970012523233890533447265625). > > So you're saying echo $((1.1000000000000001)) and echo $((1.1)) > should output 1.1, because even though 1.1000000000000001 is > closer to that value than 1.1000000000000000, zsh should pick > the latter because people prefer to see shorter number > representations and in that case it doesn't matter which one we > pick as both lead to the same double. I now remember that our Handbook of Floating-Point Arithmetic covers this issue (Section 4.9.2.1 "Output conversion: from radix 2 to radix 10" in the 2nd edition). But it just gives references -------------------------------------------------------------------- "[...] Steele and White designed an algorithm for that. Their algorithm was later improved by Burger and Dybvig [86], and by Gay [211]. Gay's code is available for anyone to use, and is very robust.(17) Faster yet more complex algorithms have been introduced by Loitsch [394] and Andrysco et al. [16]. [...]" (17) At the time of writing this book, it can be obtained at http://www.netlib.org/fp/ (file dtoa.c). -------------------------------------------------------------------- and Burger and Dybvig's high-level algorithm. -- Vincent Lefèvre - Web: 100% accessible validated (X)HTML - Blog: Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)