* inconsistent long number truncation
@ 2018-10-09 6:50 Stephane Chazelas
2018-10-09 13:30 ` Daniel Shahaf
0 siblings, 1 reply; 2+ messages in thread
From: Stephane Chazelas @ 2018-10-09 6:50 UTC (permalink / raw)
To: Zsh hackers list
From https://unix.stackexchange.com/a/474132
$ zsh -c 'echo $((12345678901234567890));echo $((12345678901234567890123))'
zsh:1: number truncated after 19 digits: 12345678901234567890
1234567890123456789
zsh:1: number truncated after 22 digits: 12345678901234567890123
-1363962815083169260
I think the:
newcalc = calc * base + *s - '0';
if (newcalc < calc)
Is an invalid way to check for numbers wrapping.
On GNU/Linux amd64
ulong 1234567890123456789*10 is
12345678901234567890 and:
(gdb) p calc
$7 = 12345678901234567890
(gdb) p calc*10
$8 = 12776324570088369204
(gdb) p calc*10*10
$9 = 17082781258626382344
(gdb) p calc*10*10*10
$10 = 4807115922877858896
Why do we truncate in the first place? That's hardly more useful
than letting the numbers wrap.
--
Stephane
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: inconsistent long number truncation
2018-10-09 6:50 inconsistent long number truncation Stephane Chazelas
@ 2018-10-09 13:30 ` Daniel Shahaf
0 siblings, 0 replies; 2+ messages in thread
From: Daniel Shahaf @ 2018-10-09 13:30 UTC (permalink / raw)
To: Zsh hackers list
Stephane Chazelas wrote on Tue, Oct 09, 2018 at 07:50:14 +0100:
> I think the:
>
>
> newcalc = calc * base + *s - '0';
> if (newcalc < calc)
>
> Is an invalid way to check for numbers wrapping.
>
Obviously right, as your examples show:
> (gdb) p calc
> $7 = 12345678901234567890
> (gdb) p calc*10
> $8 = 12776324570088369204
> Why do we truncate in the first place?
Judging by the comment at the top of the function, might that be for
compatibility with some strtol() implementation [that once was] out there?
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2018-10-09 13:30 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-10-09 6:50 inconsistent long number truncation Stephane Chazelas
2018-10-09 13:30 ` Daniel Shahaf
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/zsh/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).