zsh-workers
 help / color / mirror / code / Atom feed
* zsh converts a floating-point number to string with too much precision
@ 2019-12-20  1:37 Vincent Lefevre
  2019-12-20  3:38 ` Mikael Magnusson
  2019-12-20 16:58 ` Stephane Chazelas
  0 siblings, 2 replies; 12+ messages in thread
From: Vincent Lefevre @ 2019-12-20  1:37 UTC (permalink / raw)
  To: zsh-workers

With zsh 5.7.1, I get:

zira% echo $((1.1))
1.1000000000000001

because zsh seems to first select the precision independently
from the value, i.e. 17 to be able to convert the string back
to floating point, preserving the original value, then it
outputs the closest number in this precision.

Instead, zsh should select the minimum precision so that the
inverse conversion can give the original value, i.e. it should
output 1.1 here.

FYI, GNU MPFR has the same issue with mpfr_printf and %Re (with
an empty precision field), and I regard this as a bug:
  https://sympa.inria.fr/sympa/arc/mpfr/2019-12/msg00000.html
  https://sympa.inria.fr/sympa/arc/mpfr/2019-12/msg00001.html

Note that Java does it right:

zira:~> cat tst.java
public class tst
{
  public static void main(String[] args)
  {
    double x;
    x = 0x1.1999999999999p+0;
    System.out.println(x);
    x = 0x1.199999999999ap+0;
    System.out.println(x);
    x = 0x1.199999999999bp+0;
    System.out.println(x);
  }
}
zira:~> javac tst.java
zira:~> java tst
1.0999999999999999
1.1
1.1000000000000003

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2019-12-21 21:29 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-20  1:37 zsh converts a floating-point number to string with too much precision Vincent Lefevre
2019-12-20  3:38 ` Mikael Magnusson
2019-12-20 16:58 ` Stephane Chazelas
2019-12-20 17:12   ` Roman Perepelitsa
2019-12-21  0:50     ` Vincent Lefevre
2019-12-21  8:47       ` Stephane Chazelas
2019-12-21  9:43         ` Roman Perepelitsa
2019-12-21 17:56           ` Stephane Chazelas
2019-12-21 18:11             ` Stephane Chazelas
2019-12-21 18:20               ` Roman Perepelitsa
2019-12-21 21:28         ` Vincent Lefevre
2019-12-21  1:00   ` Vincent Lefevre

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).