From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_LOW,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 18672 invoked from network); 26 Jun 2023 21:06:46 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 26 Jun 2023 21:06:46 -0000 Received: (qmail 17982 invoked by uid 550); 26 Jun 2023 21:06:41 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 17939 invoked from network); 26 Jun 2023 21:06:40 -0000 Date: Mon, 26 Jun 2023 17:06:28 -0400 From: Rich Felker To: Immolo Cc: musl@lists.openwall.com Message-ID: <20230626210628.GS4163@brightrain.aerifal.cx> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [musl] m68k - malloc causing 'out of memory: my_alloc caller' in rsync On Mon, Jun 26, 2023 at 08:34:04PM +0100, Immolo wrote: > Hi, > > I've been testing using Gentoo on m68k with musl -1.2.4 over the last few > days and hit an interesting issue when running `emerge --sync` to update > the portage tree would cause the following error: > > [receiver] out of memory: my_alloc caller (file=flist.c, line=311) > rsync error: error allocating core memory buffers (code 22) at util2.c(123) > [receiver=3.2.7] > [generator] out of memory: my_alloc caller (file=flist.c, line=311) > rsync error: error allocating core memory buffers (code 22) at util2.c(123) > [generator=3.2.7] > rsync: [receiver] write error: Broken pipe (32) > Full output here: https://bpa.st/3VDNM > > Looking at the source code of the files it highlights it seems to be an > issue in the malloc: > https://github.com/WayneD/rsync/blob/master/util2.c#L123 > https://github.com/WayneD/rsync/blob/master/flist.c#L311 > https://github.com/WayneD/rsync/blob/master/fileio.c#L159 > > I've tested to see if a local rsync mirror will cause the same error with > random files and found it happens at around 62 files being mirrored but > size of the file does not matter. > I run musll on multiple architectures and this is the first time I've run > across it and have confirmed the Gentoo glibc m68k install does not run > into this. I guess you should try putting a breakpoint on that line in flist.c and see what values are being passed to realloc_array. Looking at the definition of realloc_array (at first I mistook this for the new reallocarray function, but it's a custom thing in terms of their my_alloc), this code does not look good. Unless max_alloc has been set, there is no overflow checking, so aside from whatever is going on with m68k, there is probably an exploitable integer overflow bug: https://github.com/WayneD/rsync/blob/6f3c5eccee6cf4dead68b9f3fda8fc2ff90dc311/util2.c#L87 I'm guessing the underlying problem is either some mismatched function call signature that only happens to mismatch the call ABI on m68k, or perhaps some weird effect of structs having almost no alignment on m68k. But I without actually seeing the values involved at the point of failure, it's hard to narrow it down. Rich