* [PATCH][RFC] fix strtod int optimization in non-nearest rounding mode
@ 2016-09-04 2:51 Szabolcs Nagy
2016-09-09 23:55 ` Rich Felker
0 siblings, 1 reply; 2+ messages in thread
From: Szabolcs Nagy @ 2016-09-04 2:51 UTC (permalink / raw)
To: musl
the mid-sized integer optimization relies on lnz set up properly
to mark the last non-zero decimal digit, but this was not done
if the non-zero digit lied outside the KMAX digits of the base
10^9 number representation.
so if the fractional part was a very long list of zeros (>2048*9 on
x86) followed by non-zero digits then the integer optimization could
kick in discarding the tiny non-zero fraction which can mean wrong
result on non-nearest rounding mode.
strtof, strtod and strtold were all affected.
---
the dc counter is long long, but lnz is int, so this is
not correct on 64bit targets where dc can be >INT_MAX,
probably lnz should be just set to KMAX*9 or similar in
this case.
src/internal/floatscan.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/src/internal/floatscan.c b/src/internal/floatscan.c
index 66d01ef..682b80a 100644
--- a/src/internal/floatscan.c
+++ b/src/internal/floatscan.c
@@ -110,7 +110,10 @@ static long double decfloat(FILE *f, int c, int bits, int emin, int sign, int po
gotdig=1;
} else {
dc++;
- if (c!='0') x[KMAX-4] |= 1;
+ if (c!='0') {
+ lnz = dc;
+ x[KMAX-4] |= 1;
+ }
}
}
if (!gotrad) lrp=dc;
--
2.9.3
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [PATCH][RFC] fix strtod int optimization in non-nearest rounding mode
2016-09-04 2:51 [PATCH][RFC] fix strtod int optimization in non-nearest rounding mode Szabolcs Nagy
@ 2016-09-09 23:55 ` Rich Felker
0 siblings, 0 replies; 2+ messages in thread
From: Rich Felker @ 2016-09-09 23:55 UTC (permalink / raw)
To: musl
On Sun, Sep 04, 2016 at 04:51:03AM +0200, Szabolcs Nagy wrote:
> the mid-sized integer optimization relies on lnz set up properly
> to mark the last non-zero decimal digit, but this was not done
> if the non-zero digit lied outside the KMAX digits of the base
> 10^9 number representation.
>
> so if the fractional part was a very long list of zeros (>2048*9 on
> x86) followed by non-zero digits then the integer optimization could
> kick in discarding the tiny non-zero fraction which can mean wrong
> result on non-nearest rounding mode.
>
> strtof, strtod and strtold were all affected.
And *scanf. :-)
> ---
> the dc counter is long long, but lnz is int, so this is
> not correct on 64bit targets where dc can be >INT_MAX,
> probably lnz should be just set to KMAX*9 or similar in
> this case.
The issue is not exclusive to 64-bit targets. You can scanf a >2GB
decimal string on 32-bit too. We could just set lnz to match the
location where the |=1 takes place (on the order of KMAX*9), or we
could just make lnz long long (it's only used one other place). Your
approach is probably nicer though.
Rich
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2016-09-09 23:55 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-04 2:51 [PATCH][RFC] fix strtod int optimization in non-nearest rounding mode Szabolcs Nagy
2016-09-09 23:55 ` Rich Felker
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/musl/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).