zsh-users
 help / color / mirror / code / Atom feed
From: Roman Perepelitsa <roman.perepelitsa@gmail.com>
To: Ray Andrews <rayandrews@eastlink.ca>
Cc: Zsh Users <zsh-users@zsh.org>
Subject: Re: arithmetic anomaly
Date: Thu, 25 Mar 2021 23:54:04 +0100	[thread overview]
Message-ID: <CAN=4vMokvyhoRiF+y=YJg=fBzmwKj3kj0StKJCN+B_yLeg5fAw@mail.gmail.com> (raw)
In-Reply-To: <bf94da7b-b037-7264-0c7e-a39ee21f6fb6@eastlink.ca>

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]

On Thu, Mar 25, 2021 at 11:44 PM Ray Andrews <rayandrews@eastlink.ca> wrote:

> On 2021-03-25 1:40 p.m., Lawrence Velázquez wrote:
> >
> > The bad news is that *all* your figures -- your numerators, Brett's
> > numerators, and the denominators -- are actually incorrect for
> > level=15 and higher.
> Sheesh, what gives?  Seems no accuracy beyond 15 digits?
>

Indeed. 64-bit floating point numbers usually have 15.95 decimal digits of
precision. Depending on the compiler and how you write your code, you might
get higher precision but in practice many projects turn this off because
having borderline non-deterministic results of computation sucks.

This isn't specific to zsh. Floating numbers are nowadays the same
everywhere (IEEE 754).

Roman.

[-- Attachment #2: Type: text/html, Size: 1120 bytes --]

  reply	other threads:[~2021-03-25 22:54 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-25 15:52 Ray Andrews
2021-03-25 20:40 ` Lawrence Velázquez
2021-03-25 22:44   ` Ray Andrews
2021-03-25 22:54     ` Roman Perepelitsa [this message]
2021-03-25 23:57       ` Ray Andrews
2021-03-26  0:16 ` Oliver Kiddle
2021-03-26  0:37   ` Ray Andrews
2021-03-26 10:35     ` Roman Perepelitsa
2021-03-26 14:36       ` Ray Andrews
2021-03-26 14:49         ` Roman Perepelitsa
2021-03-26 15:35           ` Ray Andrews
2021-03-26 17:09             ` Lawrence Velázquez
2021-03-26 21:45               ` Ray Andrews
2021-03-26 21:59                 ` Lawrence Velázquez

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAN=4vMokvyhoRiF+y=YJg=fBzmwKj3kj0StKJCN+B_yLeg5fAw@mail.gmail.com' \
    --to=roman.perepelitsa@gmail.com \
    --cc=rayandrews@eastlink.ca \
    --cc=zsh-users@zsh.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).