From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/1729 Path: news.gmane.org!not-for-mail From: musl Newsgroups: gmane.linux.lib.musl.general Subject: Re: ldso: dlclose. Date: Fri, 24 Aug 2012 15:54:25 +0200 Message-ID: <50378791.4090901@gmail.com> References: <503113C5.5010206@gmail.com> <20120820004803.GA27715@brightrain.aerifal.cx> <5603ddad712718518eed1430f5d00450@exys.org> <20120823124816.GP27715@brightrain.aerifal.cx> <20120824000209.74ab2a3b@sibserver.ru> <20120823180138.GR27715@brightrain.aerifal.cx> <503732BC.1030507@gmail.com> <20120824122708.GY27715@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1345816490 19956 80.91.229.3 (24 Aug 2012 13:54:50 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 24 Aug 2012 13:54:50 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-1730-gllmg-musl=m.gmane.org@lists.openwall.com Fri Aug 24 15:54:47 2012 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1T4uLR-0006NS-0e for gllmg-musl@plane.gmane.org; Fri, 24 Aug 2012 15:54:41 +0200 Original-Received: (qmail 11721 invoked by uid 550); 24 Aug 2012 13:54:39 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 11711 invoked from network); 24 Aug 2012 13:54:39 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=tQO/6gV5JW9Cy8uZdwieuJExZPsg5ey3kWM2balVadE=; b=q0Df5Dl7b3LOqaqfyPl5jXGtVQgQ8gd9q54mmqjbliE0OKvEz5i1g2L30bk1h826gj a20t0fg0QasvuldzvUqjimHUAYufl91Wpo1iV6W/dx9fNsR3c4tb3C9t2oaPPZYYufIW 5dWeMWOWpbpWZ2EQADic5DYguzNOakvaEkZzANNGcGN3VMaWNOaf6fOxeC0YeA2YM7fi 7fdV1zcYZerwL1HBYQwWmob6/YlGZgntnRPYvPJvgcMI8VTGseobaVmaQs5lUkiPPqnc HiIyVkSXr7fauHPCFO5nwy/rAGU/MoW6YguLDSfvy9pf9FR80mirw+cp5OKOQEFlRED/ LkQQ== User-Agent: Mozilla/5.0 (X11; Linux i686; rv:14.0) Gecko/20120714 Thunderbird/14.0 In-Reply-To: <20120824122708.GY27715@brightrain.aerifal.cx> Xref: news.gmane.org gmane.linux.lib.musl.general:1729 Archived-At: On 24/08/2012 14:27, Rich Felker wrote: > On Fri, Aug 24, 2012 at 09:52:28AM +0200, musl wrote: >> On 23/08/2012 20:01, Rich Felker wrote: >>> On Fri, Aug 24, 2012 at 12:02:09AM +0800, orc wrote: >>>> On Thu, 23 Aug 2012 08:48:16 -0400 >>>> Rich Felker wrote: >>>> >>>>> Anyway, unless the issue is fixed in binutils so that the vast >>>>> majority of libraries are marked non-unloadable, I don't see anything >>>>> we can do in musl. "glibc does it that way too" is not an excuse for >>>>> adding unsafe/non-robust behavior to musl. >>>>> >>>>> Rich >>>> The whole dlopen/dlclose/dlsym functions family are 'harmful': even if >>>> we want static linking, application will still rely on them and fail >>>> invisibly, creating more headaches. >>>> I think better leave dlclose() in it's current state now. It will always >>>> 'success', nobody will care. >>> In my view, there are only two downsides to the current behavior: >>> >>> 1. Some buggy plugin-based applications may expect dlclose(plugin) to >>> call the destructors in the plugin. This is of course an invalid >>> expectation per POSIX, but it may be the reality for some apps. >> Indeed, many plugins implem rely on constructors/destructors to >> allocate/free memory or intialize/cleanup context. >> This may lead to memory leaks or other issues if the plugin is >> loaded/unloaded multiple times. > A plugin cannot be loaded more than once. Subsequent calls to dlopen > use the existing loaded image. The only way it could be loaded again > is if the file were replaced by a new version. > > I think maybe you're not realizing that the "leak" can only happen if > a new version of the .so file is put in place of the old one... I was talking about this specific case : 1) unloding a plugin 2) updating the plugin (new plugin.so) 3) reloading the plugin During the whole sequence the application is up and running. Here is how I should do it if dlclose is implemented per posix : 1) stop the application 2) update the plugin 3) restart the application The application is not available during this sequence. > >>> 2. In an extremely long-lived app that loads and unloads plugins which >>> may be upgraded multiple times during the application's lifetime, each >>> new version of the plugin will consume additional virtual memory space >>> and commit charge, i.e. you have a memory leak. In the real world the >>> leak should be very slow, but it could become significant if the >>> plugins are very large and get reinstalled many times, perhaps if >>> someone is experimenting and running "make install" each time... >> It might be worst for long-lived apps running in a memory >> constrained environment (embedded systems). > Yes, but in this kind of system, ANY use of dynamic memory allocation > is frowned upon. Dynamic module loading even moreso. And of course I > don't think you'll be constantly replacing .so files on such a system > with new versions. > >>> In my view #2 is a very low-priority problem that's not worth caring >>> about on its own, but #1 may be relevant. If does become an important >>> issue that we can't get fixed at the application level, I think the >>> solution would be to add unloading, but have it only take effect for >>> the actual argument to dlopen/dlclose, never any libraries implicitly >>> loaded as dependencies (and of course to honor the flag that prevents >>> unloading). >> Does this mean you want to call plugin destructors in dlclose >> function and keep the plugin memory mapping ? > No. Calling dtors and unloading always come in a pair. You cannot call > dtors but keep and reuse the mapping because the static-storage > objects would retain their old values from the prior load, but a new > load would be visible to the code in the plugin. > > The potential design I'm talking about would have only the dlopen'd > library itself ever unloaded/unmapped. For example, if myplugin.so > depends on libfoo.so and libbar.so, libfoo.so and libbar.so, which > were implicitly loaded when loading myplugin.sh, will never be > unmappable. Only myplugin.so itself would be unmappable. On > unloading/unmapping dtors would be called as usual, and then the > reference would be removed entirely from the DSO chain, causing it to > be searched-out and loaded new next time dlopen is called. > > I do not want to do this except as a last resort, since as I've > already mentioned it's highly error-prone (see glibc) and fragile. I understand your concern and I'll modify my code to get rid of the dlclose function. I hope there's no other apps or libs relying on gnu dlclose specific implem. It should not if they've read carrefully the dlclose man page :-). BTW, thanks for taking the time to explain the dlclose implications. > > Rich