Re: floating-point precision and zsh vs ksh93 and coreutils

[Vincent Lefevre <vincent@vinc17.net>]

On 2022-03-27 09:26:52 +0100, Stephane Chazelas wrote:
> 2022-03-23 17:36:46 +0100, Vincent Lefevre:
> [...]
> > Shouldn't zsh switch to long double?
> [...]
> 
> I'm not convinced it's a good idea. I had made a long write-up
> related to that some time ago at
> https://unix.stackexchange.com/questions/422122/why-does-0-1-expand-to-0-10000000000000001-in-zsh
> that we did discussed it here about the artefacts whereby echo
> $((0.1)) outputs 0.10000000000000001 for instance.
> 
> Several problems:
> - double is still the most commonly used float representation
>   used by default in other languages (perl, python, most if not
>   all awks...)
> - long double precision varies with the system/compiler/CPU
>   (64bit, 80bit, 128bit), whilst double is more consistently
>   64bit on Unix systems at least.
> - so to preserve the precision upon round-trip to/from decimal,
>   we'd need 17, 21 or 36 digit precision making for even uglier
>   artefacts (and very long numbers), and if we give up on
>   preserving precision, we get the same problems as affect
>   yash/ksh93 described at that link above.
> 
>   $ ksh -c 'if (( $((1. / 3)) == 1./3 )); then echo yes; else echo no; fi'
>   no
>   $ ksh -c 'echo $(( $((1. / 3)) - 1./3 ))'
>   -3.52365706057788941e-19

OK, I forgot that ksh was actually buggy. I thought that it output the
minimum number of digits that preserves the round trip back to binary.

Now, given that ksh doesn't even use a "long double compatible"
format for its output, I don't see any reason why coreutils uses
long double by default (IMHO, it should use a length modifier
for that, as in C).

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)