From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/10432 Path: news.gmane.org!.POSTED!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: [PATCH][RFC] fix strtod int optimization in non-nearest rounding mode Date: Fri, 9 Sep 2016 19:55:14 -0400 Message-ID: <20160909235514.GU15995@brightrain.aerifal.cx> References: <20160904025102.GN1280@port70.net> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1473465339 3507 195.159.176.226 (9 Sep 2016 23:55:39 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 9 Sep 2016 23:55:39 +0000 (UTC) User-Agent: Mutt/1.5.21 (2010-09-15) To: musl@lists.openwall.com Original-X-From: musl-return-10445-gllmg-musl=m.gmane.org@lists.openwall.com Sat Sep 10 01:55:35 2016 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1biVdv-0000Ps-4F for gllmg-musl@m.gmane.org; Sat, 10 Sep 2016 01:55:35 +0200 Original-Received: (qmail 32367 invoked by uid 550); 9 Sep 2016 23:55:34 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 32343 invoked from network); 9 Sep 2016 23:55:33 -0000 Content-Disposition: inline In-Reply-To: <20160904025102.GN1280@port70.net> Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:10432 Archived-At: On Sun, Sep 04, 2016 at 04:51:03AM +0200, Szabolcs Nagy wrote: > the mid-sized integer optimization relies on lnz set up properly > to mark the last non-zero decimal digit, but this was not done > if the non-zero digit lied outside the KMAX digits of the base > 10^9 number representation. > > so if the fractional part was a very long list of zeros (>2048*9 on > x86) followed by non-zero digits then the integer optimization could > kick in discarding the tiny non-zero fraction which can mean wrong > result on non-nearest rounding mode. > > strtof, strtod and strtold were all affected. And *scanf. :-) > --- > the dc counter is long long, but lnz is int, so this is > not correct on 64bit targets where dc can be >INT_MAX, > probably lnz should be just set to KMAX*9 or similar in > this case. The issue is not exclusive to 64-bit targets. You can scanf a >2GB decimal string on 32-bit too. We could just set lnz to match the location where the |=1 takes place (on the order of KMAX*9), or we could just make lnz long long (it's only used one other place). Your approach is probably nicer though. Rich