From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.2 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by inbox.vuxu.org (OpenSMTPD) with SMTP id 23dfd1a8 for ; Tue, 28 Jan 2020 13:29:16 +0000 (UTC) Received: (qmail 16059 invoked by uid 550); 28 Jan 2020 13:29:14 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 16041 invoked from network); 28 Jan 2020 13:29:14 -0000 Date: Tue, 28 Jan 2020 08:29:00 -0500 From: Rich Felker To: Leesoo Ahn Cc: musl@lists.openwall.com Message-ID: <20200128132900.GE30412@brightrain.aerifal.cx> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: Rich Felker Subject: Re: [musl] Memory leak issue in multi-threaded program On Tue, Jan 28, 2020 at 02:44:07PM +0900, Leesoo Ahn wrote: > Dear musl developers, > > Hello!, it seems that musl currently has a memory leak issue in > multi-threaded program. It occurs in the below situation of latest > (v1.1.24) source. Also, not only in 32-bits[1], but also 64-bits[2] > as well. > > When a program create and run, at least, two threads or more with > pthread APIs, VSZ of the program by ps command keeps increasing. But > here is a weird thing that it is fine 'IF ONLY ONE' pthread is > created and run. > > To confirm the issue in your host machine, please follow the instructions, > > 0. Clone the musl git and get inside. > 1. Build with these options for static build, ./configure > --prefix=$(pwd)/_build_dir --disable-shared > 2. Download the test code[3], then build with the command, > ../_build_dir/bin/musl-gcc ./test.c > 3. Run this script, ./a.out &; while [ 1 ]; do { ps aux | grep > [a].out | grep -v grep; sleep 1; } done > > You may figure out that VSZ keeps increasing. > > BUT, when I make it to try to allocate memory all the time by kernel > mmap with this diff[4] as workaround, although it creates more > pthreads than 2, the issue never happens. > > It would be really thankful if you guys could confirm it and find > out the way to fix the bug. This is a known issue described in: https://www.openwall.com/lists/musl/2018/10/30/2 and likely several times before that, though it was not realized that people were hitting it in practice (vs it just being theoretical) until around that time. I posted an experimental mitigation patch last spring: https://www.openwall.com/lists/musl/2019/04/12/4 but it's not heavily tested and its impact on performance is significant. I think it should be ok if you need an immediate fix, but you should do some testing to make sure. If you go this route, reports of any problems (or success) would be nice to hear about. Further work in that direction was not done because it was already planned that musl's malloc implementation will be replaced, and that the replacement will solve this and other problems in much better ways. This is work in progress and is intended for merge in the next release cycle: https://www.openwall.com/lists/musl/2019/10/22/3 https://github.com/richfelker/mallocng-draft Hope this information helps. Rich