From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/13652 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Szabolcs Nagy Newsgroups: gmane.linux.lib.musl.general Subject: Re: Infinite loop in malloc Date: Sat, 26 Jan 2019 00:11:37 +0100 Message-ID: <20190125231136.GX21289@port70.net> References: <20190125151350.GB20330@r> <20190125222832.GW21289@port70.net> Reply-To: musl@lists.openwall.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="152520"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mutt/1.10.1 (2018-07-13) To: r yang , musl@lists.openwall.com Original-X-From: musl-return-13668-gllmg-musl=m.gmane.org@lists.openwall.com Sat Jan 26 00:11:51 2019 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.89) (envelope-from ) id 1gnAdb-000dbM-MB for gllmg-musl@m.gmane.org; Sat, 26 Jan 2019 00:11:51 +0100 Original-Received: (qmail 13389 invoked by uid 550); 25 Jan 2019 23:11:49 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 13371 invoked from network); 25 Jan 2019 23:11:48 -0000 Mail-Followup-To: r yang , musl@lists.openwall.com Content-Disposition: inline In-Reply-To: <20190125222832.GW21289@port70.net> Xref: news.gmane.org gmane.linux.lib.musl.general:13652 Archived-At: * Szabolcs Nagy [2019-01-25 23:28:32 +0100]: > * r yang [2019-01-25 10:13:50 -0500]: > > pmbootstrap is a development environment to build/install postmarketOS > > (based on Alpine Linux) for Android devices. One of the things it does > > is use qemu static to emulate an ARM based Alpine Linux chroot > > environment. > > > > There is a bug while compiling certain packages in the qemu ARM chroot. > > The qemu process can get stuck in an infinite loop when calling malloc. > > > > pmbootstrap uses Alpine Linux edge repositories. It's using the current > > musl package version 1.1.20. > > > > Here is a gdb backtrace. > > #0 malloc (n=, n@entry=9) at src/malloc/malloc.c:320 > > #1 0x0000000060184ad3 in g_malloc (n_bytes=n_bytes@entry=9) at gmem.c:99 > > #2 0x000000006018bcab in g_strdup (str=, str@entry=0x60200abf "call_rcu") at gstrfuncs.c:363 > > #3 0x000000006016e31d in qemu_thread_create (thread=thread@entry=0x7ffe89fb1a10, name=name@entry=0x60200abf "call_rcu", > > start_routine=start_routine@entry=0x60174c00 , arg=arg@entry=0x0, mode=mode@entry=1) at /home/pmos/build/src/qemu-3.1.0/util/qemu-thread-posix.c:526 > > #4 0x0000000060174b99 in rcu_init_complete () at /home/pmos/build/src/qemu-3.1.0/util/rcu.c:327 > > #5 0x00000000601c4fac in __fork_handler (who=1) at src/thread/pthread_atfork.c:26 > > #6 0x00000000601be8db in fork () at src/process/fork.c:33 it seems the issue is simply that qemu-arm-static is a multi-threaded process and here it forks and calls malloc in the fork handler of the child process. it's easy to imagine that if fork runs concurrently with a free the malloc state remains corrupted in the child hence the malloc fails there. i'm not sure if musl can detect or fix this up easily. > > #7 0x000000006009d191 in do_fork (env=0x62ef0ed0, flags=flags@entry=17, newsp=newsp@entry=0, parent_tidptr=parent_tidptr@entry=0, newtls=newtls@entry=0, > > child_tidptr=child_tidptr@entry=0) at /home/pmos/build/src/qemu-3.1.0/linux-user/syscall.c:5528 > > #8 0x00000000600af894 in do_syscall1 (cpu_env=cpu_env@entry=0x62ef0ed0, num=num@entry=2, arg1=arg1@entry=0, arg2=arg2@entry=-8700192, arg3=, arg4=8, > > arg5=1015744, arg6=-75664, arg7=0, arg8=0) at /home/pmos/build/src/qemu-3.1.0/linux-user/syscall.c:7042 > > #9 0x00000000600a835c in do_syscall (cpu_env=cpu_env@entry=0x62ef0ed0, num=2, arg1=0, arg2=-8700192, arg3=, arg4=, arg5=1015744, arg6=-75664, > > arg7=0, arg8=0) at /home/pmos/build/src/qemu-3.1.0/linux-user/syscall.c:11533 > > #10 0x00000000600c265f in cpu_loop (env=env@entry=0x62ef0ed0) at /home/pmos/build/src/qemu-3.1.0/linux-user/arm/cpu_loop.c:360 > > #11 0x00000000600417a2 in main (argc=, argv=0x7ffe89fb5958, envp=) at /home/pmos/build/src/qemu-3.1.0/linux-user/main.c:819 > > > > > > It is taking the malloc code path where n <= MMAP_THRESHOLD. None of > > the conditions which break from the for loop are met. > > > > In the first condition the mask value is never zero: > > mask = mal.binmap & -(1ULL< > if (!mask) { ... } > > > > Examining the value in gdb: > > (gdb) printf "%X\n", mask > > 204701 > > > > The bin head points to the bin itself so this condition is never met: > > c = mal.bins[j].head; > > if (c != BIN_TO_CHUNK(j)) { ... } > > > > Examining the values in gdb: > > (gdb) printf "%X\n", mal.bins[j].head > > 62337FC0 > > (gdb) printf "%X\n", (struct chunk *)((char *)(&mal.bins[j].head) - (2*sizeof(size_t))) > > 62337FC0 > > > > > > Reproducing this issue: > > It is not always 100% reproducible. On occasion it will not get stuck > > in an infinite loop. With my testing on 2 computers, will happen on > > most attempts to compile. > > thanks i managed to reproduce this on my laptop with the commands below. > i'll try to look into it. > > > > > $ git clone https://gitlab.com/postmarketOS/pmbootstrap.git > > $ cd pmboostrap > > > > Configure pmbootstrap > > $ ./pmbootstrap.py init > > > > Enter an Android device when prompted. > > Use device: samsung-i9100 > > Leave other settings as the default. > > > > Check out the pmaports repository that will reproduce this issue. > > $ cd /path/to/pmboostrap/aports > > $ git remote add ryang2678 https://gitlab.com/ryang2678/pmaports.git > > $ git fetch ryang2678 debug-musl-malloc > > $ git checkout debug-musl-malloc > > > > Compile qemu static with debug symbols. > > Alpine Linux doesn't provide a qemu package with debug symbols. > > The debug-musl-malloc branch contains a qemu APKBUILD with debugging > > enabled. > > $ cd /path/to/pmboostrap > > $ ./pmbootstrap.py build qemu > > > > Try to compile networkmanager and wait for build to get stuck. > > $ ./pmbootstrap.py build networkmanager --arch=armhf --force > > > > > > To observe the stuck qemu process: > > > > Enter chroot shell: > > $ ./pmbootstrap.py chroot > > > > Install musl debug symbols. > > $ apk add musl-dbg > > > > Get musl source code > > $ cd /home/pmos > > $ git clone git://git.musl-libc.org/musl > > $ cd /home/pmos/musl > > $ git checkout v1.1.20 > > > > Attach gdb to stuck process > > $ gdb -tui /usr/bin/qemu-arm > > directory /home/pmos/musl > > attach