From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/9077 Path: news.gmane.org!not-for-mail From: Szabolcs Nagy Newsgroups: gmane.linux.lib.musl.general Subject: Re: Possible infinite loop in qsort() Date: Sun, 10 Jan 2016 11:33:22 +0100 Message-ID: <20160110103321.GQ23362@port70.net> References: <20160109082139.GD2016@debian> <20160109090719.GA385@nyan> <20160110040516.GQ238@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1452422025 30951 80.91.229.3 (10 Jan 2016 10:33:45 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 10 Jan 2016 10:33:45 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-9090-gllmg-musl=m.gmane.org@lists.openwall.com Sun Jan 10 11:33:40 2016 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1aIDJa-0007E2-Fp for gllmg-musl@m.gmane.org; Sun, 10 Jan 2016 11:33:38 +0100 Original-Received: (qmail 6135 invoked by uid 550); 10 Jan 2016 10:33:34 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 6103 invoked from network); 10 Jan 2016 10:33:33 -0000 Mail-Followup-To: musl@lists.openwall.com Content-Disposition: inline In-Reply-To: <20160110040516.GQ238@brightrain.aerifal.cx> User-Agent: Mutt/1.5.24 (2015-08-30) Xref: news.gmane.org gmane.linux.lib.musl.general:9077 Archived-At: * Rich Felker [2016-01-09 23:05:16 -0500]: > On Sat, Jan 09, 2016 at 10:07:19AM +0100, Felix Janda wrote: > > Markus Wichmann wrote: > > > This is the Leonardo number precompute loop in qsort(): > > > > > > for(lp[0]=lp[1]=width, i=2; (lp[i]=lp[i-2]+lp[i-1]+width) < size; i++); > > > ... > > > > musl enforces that object sizes should not be greater than PTRDIFF_MAX. > > See for example the discussion at > > > > http://www.openwall.com/lists/musl/2013/06/27/7 > > > > So there will not be objects of size 3GB with musl on x32. Since the > > Leonardo numbers grow slower than 2^n in general no overflow should > > happen if "size" is valid. Otherwise, UB was invoked. > > Note also that if you do want to use this code on an implementation > without such a guarantee, only the case where the member size is 1 can > possibly have >SIZE_MAX/2 members. In that case, you can massively > optimize out the whole sort by just counting the number of times each > byte appears (in size_t[UCHAR_MAX+1] space which is tiny), sorting the > pairs (value,count) using the comparison function, then writing out > each value the appropriate number of times. one element array would have to be special cased too: qsort(p, 1, SIZE_MAX, cmp) would oob access lp in musl qsort(p, 1, SIZE_MAX/2, cmp) would loop for more than necessary before lp[i] reaches >=size but it's harmless because lp is large enough.