```zsh-users
help / color / mirror / code / Atom feed```
```From: Ray Andrews <rayandrews@eastlink.ca>
To: zsh-users@zsh.org
Subject: Re: arithmetic anomaly
Date: Fri, 26 Mar 2021 07:36:21 -0700	[thread overview]

On 2021-03-26 3:35 a.m., Roman Perepelitsa wrote:
>
> Maybe this will help:
> https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
> <https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html>

Good to know!

Seriously, I've often wondered what really goes on in a calculator. Ok,
for now I know that zsh and probably everybody else will compute endless
reams of garbage if you let it.  Here's my algorithm with the outer
'for' loop increasing exponentially:

Ray's  Algorithm fractional:
Level: 1: win: 1 / 1
Level: 10: win: 6513215599 / 10000000000
Level: 100: win:
6339676587267708385630188881009247729557115329795338127068495312693456362833186743509112446
7602830066897486524892436064552875964380756727929631216242621046676674030227216977589219271
483398068889452544 /
9999999999999999697331222125103616594745032754550236264824175095034684843555407553419633840
4706251868027512415973882408182135734368278484639385041047239877871023591066789981811181813
306167128854888448
Level: 1000: win: inf / inf
Level: 10000: win: inf / inf
Level: 100000: win: inf / inf
Level: 1000000: win: inf / inf

.... I'm asking why the 'Level: 100' result isn't just aborted after 15
digits given that all the rest is garbage?

>
> Understanding how floating point numbers work is very useful. It's
> virtually mandatory when writing numerical computation code. None of
> this is specific to zsh.

Amen!  Seriously, I had no idea this sort of thing was permitted. Thanks
Roman.  For now I'll use ' %d' for output and at least I'll know when
things turn to mud, there must be limits to precision and there must be
overflow at some point.  (But of course we hafta use floats if there's a
division anywhere, just don't use them for display.)

```

```next prev parent reply	other threads:[~2021-03-26 14:37 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-25 15:52 Ray Andrews
2021-03-25 20:40 ` Lawrence Velázquez
2021-03-25 22:44   ` Ray Andrews
2021-03-25 22:54     ` Roman Perepelitsa
2021-03-25 23:57       ` Ray Andrews
2021-03-26  0:16 ` Oliver Kiddle
2021-03-26  0:37   ` Ray Andrews
2021-03-26 10:35     ` Roman Perepelitsa
2021-03-26 14:36       ` Ray Andrews [this message]
2021-03-26 14:49         ` Roman Perepelitsa
2021-03-26 15:35           ` Ray Andrews
2021-03-26 17:09             ` Lawrence Velázquez
2021-03-26 21:45               ` Ray Andrews
2021-03-26 21:59                 ` Lawrence Velázquez
```

```Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,

Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

switches of git-send-email(1):

git send-email \
--cc=zsh-users@zsh.org \

https://kernel.org/pub/software/scm/git/docs/git-send-email.html

```Code repositories for project(s) associated with this public inbox