From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/5720 Path: news.gmane.org!not-for-mail From: Szabolcs Nagy Newsgroups: gmane.linux.lib.musl.general Subject: Re: openmp/pthreads and fork... Date: Tue, 5 Aug 2014 19:47:21 +0200 Message-ID: <20140805174720.GE22308@port70.net> References: <20140805055650.GA2108@newbook> <20140805165137.GF1674@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1407260861 21250 80.91.229.3 (5 Aug 2014 17:47:41 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 5 Aug 2014 17:47:41 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-5725-gllmg-musl=m.gmane.org@lists.openwall.com Tue Aug 05 19:47:37 2014 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1XEiph-0007eR-Ll for gllmg-musl@plane.gmane.org; Tue, 05 Aug 2014 19:47:33 +0200 Original-Received: (qmail 7816 invoked by uid 550); 5 Aug 2014 17:47:33 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 7808 invoked from network); 5 Aug 2014 17:47:32 -0000 Mail-Followup-To: musl@lists.openwall.com Content-Disposition: inline In-Reply-To: <20140805165137.GF1674@brightrain.aerifal.cx> User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:5720 Archived-At: * Rich Felker [2014-08-05 12:51:37 -0400]: > On Mon, Aug 04, 2014 at 10:56:51PM -0700, Isaac Dunham wrote: > > Some programs that use BLAS will fork(); python is one of these. > > With OpenBLAS, this had caused "random" segfaults due to use of threads > > by the library both before and after the fork. > > The OpenBLAS issue is here: > > https://github.com/xianyi/OpenBLAS/issues/294 > > > > They worked around this using pthread_atfork() to cleanup before the fork(). > > This is unlikely to work. pthread_atfork is basically unable to do > what it was designed for, since, after fork, the new thread in the > child is not the owner of any mutexes that were obtained by the parent > before forking, and thus cannot unlock them. It also cannot destroy > and reinitialize them since they're locked. > > -How correct, and how likely to work with musl, is the fix for libgomp > > mentioned here: > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60035 > > Very unlikely since it's calling free which could run with malloc > locks permanently deadlocked. > the gomp patch says that it only fixes the case when fork is called when no gomp threads are running (by default the threads are kept around holding locks, they should be shut down before fork) in that case it is enough to check a single global flag before the new thread pool is initialized (and the flag can be set from pthread_atfork child handler reasonably safely) now i see that the openblas issue has all sorts of comments with invalid solutions and in the end their fix is pthread_atfork(BLASFUNC(blas_thread_shutdown), NULL, NULL); which should work unless fork was called from a sig handler otherwise i dont see any call to free in AS-safe context