From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/10442 Path: news.gmane.org!.POSTED!not-for-mail From: u-uy74@aetey.se Newsgroups: gmane.linux.lib.musl.general Subject: Re: incompatibility between libtheora/mmx and musl ? Date: Wed, 14 Sep 2016 16:04:50 +0200 Message-ID: <20160914140450.GQ16436@example.net> References: <20160913180649.GN16436@example.net> <20160913204304.GY15995@brightrain.aerifal.cx> <20160914103253.GO16436@example.net> <20160914112400.GS1280@port70.net> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1473861935 5224 195.159.176.226 (14 Sep 2016 14:05:35 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 14 Sep 2016 14:05:35 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-10455-gllmg-musl=m.gmane.org@lists.openwall.com Wed Sep 14 16:05:18 2016 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1bkAoN-0008AH-V4 for gllmg-musl@m.gmane.org; Wed, 14 Sep 2016 16:05:16 +0200 Original-Received: (qmail 30066 invoked by uid 550); 14 Sep 2016 14:05:15 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 30045 invoked from network); 14 Sep 2016 14:05:14 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fripost.org; h= in-reply-to:content-disposition:content-type:content-type :mime-version:references:message-id:subject:subject:from:from :date:date; s=20140703; t=1473861903; x=1475676304; bh=VYyvCDJqC 08T4A8uc1P6tSA7kClZKUxFtXh4kmjRO8Q=; b=AbfArYjQPGkH7SsY5XepA4E6Q Eb/e74T50Xpo/SNYYvTi3ILrannxLP6Ut9batZOo9EHLxmkrgKVrq56KGvJn+h7s DxhI2O9JeMqMo5KKvXisG4/scmjLPP4VyJl8Lwf2je0B6IPTF6v/yBCHZ38ym+xe KMwnAsrmQkKku8zGvk= X-Virus-Scanned: Debian amavisd-new at fripost.org Content-Disposition: inline In-Reply-To: <20160914112400.GS1280@port70.net> Xref: news.gmane.org gmane.linux.lib.musl.general:10442 Archived-At: On Wed, Sep 14, 2016 at 01:24:00PM +0200, Szabolcs Nagy wrote: > > #define _ogg_malloc(x) malloc((x)+256) > > #define _ogg_calloc(x,y) calloc((x)+256,(y)) > > #define _ogg_realloc(y,x) realloc((y),(x)+256) > > #define _ogg_free free > > > > instead of the default > > > > #define _ogg_malloc malloc > > #define _ogg_calloc calloc > > #define _ogg_realloc realloc > > #define _ogg_free free > > > > did not make any difference. The crash on a test file occurs in the same > > way and the resulting partial output file is as long as otherwise. > > > > This may mean that this is not a simple overflowing but rather > > overwriting or reading distant "random" places (?) (register corruption?) > can be underflow (or the way they align the pointer returned by malloc) Pointer alignment yes they do in some cases but in a different layer, inside the malloc()-ed buffers, it is plain C and looks harmless to me. > you can increase/decrease alignment of musl's alloc by > changing SIZE_ALIGN in src/malloc/malloc.c Doubling the alignment did not apparently change the crashing. Reducing the alignment in half did not apparently change the crashing. (A single test file with a single quality setting tested crashed the same way, at the same place in the output stream) > (or you can try some hack in _ogg_malloc/free if you are > sure that's what they are using) Yes it is present/used for this very purpose, to enable easy "hijacking". OTOH when I checked the arguments in gdb they looked always sane, up to the last and crashing realloc() call. That's why I do not expect seeing anything unusual there. Valgrind did not see any bad free()s either. > there can be some call abi issue (register clobbering, > stack alignment,..) because of the asm, but that's hard > to check. Is musl in any way special compared to glibc/uclibc in its register usage? > you may try tracing malloc calls (i don't know an easy > way other than instrumenting musl, you can try python > scripting gdb, the default gdb command language is not > enough for reporting malloc args and return values). This is something I wished to avoid. It does not promise much either, but I may possibly try this if nothing else helps. Thanks everyone for the help! Rune