zsh-workers
 help / color / mirror / code / Atom feed
From: Vincent Lefevre <vincent@vinc17.net>
To: zsh-workers@zsh.org
Subject: zsh converts a floating-point number to string with too much precision
Date: Fri, 20 Dec 2019 02:37:11 +0100	[thread overview]
Message-ID: <20191220013711.GA708801@zira.vinc17.org> (raw)

With zsh 5.7.1, I get:

zira% echo $((1.1))
1.1000000000000001

because zsh seems to first select the precision independently
from the value, i.e. 17 to be able to convert the string back
to floating point, preserving the original value, then it
outputs the closest number in this precision.

Instead, zsh should select the minimum precision so that the
inverse conversion can give the original value, i.e. it should
output 1.1 here.

FYI, GNU MPFR has the same issue with mpfr_printf and %Re (with
an empty precision field), and I regard this as a bug:
  https://sympa.inria.fr/sympa/arc/mpfr/2019-12/msg00000.html
  https://sympa.inria.fr/sympa/arc/mpfr/2019-12/msg00001.html

Note that Java does it right:

zira:~> cat tst.java
public class tst
{
  public static void main(String[] args)
  {
    double x;
    x = 0x1.1999999999999p+0;
    System.out.println(x);
    x = 0x1.199999999999ap+0;
    System.out.println(x);
    x = 0x1.199999999999bp+0;
    System.out.println(x);
  }
}
zira:~> javac tst.java
zira:~> java tst
1.0999999999999999
1.1
1.1000000000000003

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

             reply	other threads:[~2019-12-20  1:38 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-20  1:37 Vincent Lefevre [this message]
2019-12-20  3:38 ` Mikael Magnusson
2019-12-20 16:58 ` Stephane Chazelas
2019-12-20 17:12   ` Roman Perepelitsa
2019-12-21  0:50     ` Vincent Lefevre
2019-12-21  8:47       ` Stephane Chazelas
2019-12-21  9:43         ` Roman Perepelitsa
2019-12-21 17:56           ` Stephane Chazelas
2019-12-21 18:11             ` Stephane Chazelas
2019-12-21 18:20               ` Roman Perepelitsa
2019-12-21 21:28         ` Vincent Lefevre
2019-12-21  1:00   ` Vincent Lefevre

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191220013711.GA708801@zira.vinc17.org \
    --to=vincent@vinc17.net \
    --cc=zsh-workers@zsh.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).