help / color / mirror / code / Atom feed
From: Ray Andrews <rayandrews@eastlink.ca>
To: zsh-users@zsh.org
Subject: Re: arithmetic anomaly
Date: Fri, 26 Mar 2021 07:36:21 -0700	[thread overview]
Message-ID: <6d70bc8d-73d3-7866-6485-ee89dafdebc9@eastlink.ca> (raw)
In-Reply-To: <CAN=4vMpq2ECpWkFt5mAO1rwCGp0Q+UYCxnPr5kT6fReXhycSZQ@mail.gmail.com>

On 2021-03-26 3:35 a.m., Roman Perepelitsa wrote:
> Maybe this will help: 
> https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html 
> <https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html>

Good to know!

Seriously, I've often wondered what really goes on in a calculator. Ok, 
for now I know that zsh and probably everybody else will compute endless 
reams of garbage if you let it.  Here's my algorithm with the outer 
'for' loop increasing exponentially:

Ray's  Algorithm fractional:
Level: 1: win: 1 / 1
Level: 10: win: 6513215599 / 10000000000
Level: 100: win: 
483398068889452544 / 
Level: 1000: win: inf / inf
Level: 10000: win: inf / inf
Level: 100000: win: inf / inf
Level: 1000000: win: inf / inf

.... I'm asking why the 'Level: 100' result isn't just aborted after 15 
digits given that all the rest is garbage?

> Understanding how floating point numbers work is very useful. It's 
> virtually mandatory when writing numerical computation code. None of 
> this is specific to zsh.

Amen!  Seriously, I had no idea this sort of thing was permitted. Thanks 
Roman.  For now I'll use ' %d' for output and at least I'll know when 
things turn to mud, there must be limits to precision and there must be 
overflow at some point.  (But of course we hafta use floats if there's a 
division anywhere, just don't use them for display.)

  reply	other threads:[~2021-03-26 14:37 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-25 15:52 Ray Andrews
2021-03-25 20:40 ` Lawrence Velázquez
2021-03-25 22:44   ` Ray Andrews
2021-03-25 22:54     ` Roman Perepelitsa
2021-03-25 23:57       ` Ray Andrews
2021-03-26  0:16 ` Oliver Kiddle
2021-03-26  0:37   ` Ray Andrews
2021-03-26 10:35     ` Roman Perepelitsa
2021-03-26 14:36       ` Ray Andrews [this message]
2021-03-26 14:49         ` Roman Perepelitsa
2021-03-26 15:35           ` Ray Andrews
2021-03-26 17:09             ` Lawrence Velázquez
2021-03-26 21:45               ` Ray Andrews
2021-03-26 21:59                 ` Lawrence Velázquez

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6d70bc8d-73d3-7866-6485-ee89dafdebc9@eastlink.ca \
    --to=rayandrews@eastlink.ca \
    --cc=zsh-users@zsh.org \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).