From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/10443 Path: news.gmane.org!.POSTED!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: incompatibility between libtheora/mmx and musl ? Date: Wed, 14 Sep 2016 10:28:42 -0400 Message-ID: <20160914142842.GZ15995@brightrain.aerifal.cx> References: <20160913180649.GN16436@example.net> <20160913204304.GY15995@brightrain.aerifal.cx> <20160914103253.GO16436@example.net> <20160914112400.GS1280@port70.net> <20160914140450.GQ16436@example.net> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1473863352 16104 195.159.176.226 (14 Sep 2016 14:29:12 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 14 Sep 2016 14:29:12 +0000 (UTC) User-Agent: Mutt/1.5.21 (2010-09-15) To: musl@lists.openwall.com Original-X-From: musl-return-10456-gllmg-musl=m.gmane.org@lists.openwall.com Wed Sep 14 16:29:09 2016 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1bkBBI-0002gu-0M for gllmg-musl@m.gmane.org; Wed, 14 Sep 2016 16:28:56 +0200 Original-Received: (qmail 11988 invoked by uid 550); 14 Sep 2016 14:28:55 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 11970 invoked from network); 14 Sep 2016 14:28:55 -0000 Content-Disposition: inline In-Reply-To: <20160914140450.GQ16436@example.net> Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:10443 Archived-At: On Wed, Sep 14, 2016 at 04:04:50PM +0200, u-uy74@aetey.se wrote: > On Wed, Sep 14, 2016 at 01:24:00PM +0200, Szabolcs Nagy wrote: > > > #define _ogg_malloc(x) malloc((x)+256) > > > #define _ogg_calloc(x,y) calloc((x)+256,(y)) > > > #define _ogg_realloc(y,x) realloc((y),(x)+256) > > > #define _ogg_free free > > > > > > instead of the default > > > > > > #define _ogg_malloc malloc > > > #define _ogg_calloc calloc > > > #define _ogg_realloc realloc > > > #define _ogg_free free > > > > > > did not make any difference. The crash on a test file occurs in the same > > > way and the resulting partial output file is as long as otherwise. > > > > > > This may mean that this is not a simple overflowing but rather > > > overwriting or reading distant "random" places (?) (register corruption?) > > > can be underflow (or the way they align the pointer returned by malloc) > > Pointer alignment yes they do in some cases but in a different layer, > inside the malloc()-ed buffers, it is plain C and looks harmless to me. > > > you can increase/decrease alignment of musl's alloc by > > changing SIZE_ALIGN in src/malloc/malloc.c > > Doubling the alignment did not apparently change the crashing. > > Reducing the alignment in half did not apparently change the crashing. > > (A single test file with a single quality setting tested > crashed the same way, at the same place in the output stream) I think this was a bad recommendation to test. musl's SIZE_ALIGN is a macro to avoid magic numbers and make the code (very minimally) self-documenting, but that doesn't mean it's a parameter. It's tied into assumptions about the actual metadata structure and changing it is almost surely going to introduce internal inconsistency/corruption in the malloc implementation. > > (or you can try some hack in _ogg_malloc/free if you are > > sure that's what they are using) > > Yes it is present/used for this very purpose, to enable easy "hijacking". > > OTOH when I checked the arguments in gdb they looked always sane, up to > the last and crashing realloc() call. That's why I do not expect seeing > anything unusual there. > > Valgrind did not see any bad free()s either. > > > there can be some call abi issue (register clobbering, > > stack alignment,..) because of the asm, but that's hard > > to check. > > Is musl in any way special compared to glibc/uclibc in its register usage? Not in principle; this is mandated by the ABI. But it's possible that their violation of ABI contracts is visible with some implementations but not others. For example if they're calling malloc from code that's using asm it's possible that they assume the floating point registers (or mmx state) are call-saved rather than call-clobbered. This is an invalid assumption that might happen to actively break on musl but not glibc. IIRC you need some special instructions to switch between x87 and (original) mmx usage; perhaps they're missing this somewhere. Rich