From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/2430 Path: news.gmane.org!not-for-mail From: Paul Schutte Newsgroups: gmane.linux.lib.musl.general Subject: Re: static linking and dlopen Date: Sun, 9 Dec 2012 02:04:43 +0200 Message-ID: References: <20121208225237.GV20323@brightrain.aerifal.cx> <50C3CA75.8000504@comcast.net> <20121208232301.GW20323@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=0016e6db2da423dd6604d0603201 X-Trace: ger.gmane.org 1355011498 10110 80.91.229.3 (9 Dec 2012 00:04:58 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 9 Dec 2012 00:04:58 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-2431-gllmg-musl=m.gmane.org@lists.openwall.com Sun Dec 09 01:05:12 2012 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1ThUOJ-0000um-85 for gllmg-musl@plane.gmane.org; Sun, 09 Dec 2012 01:05:07 +0100 Original-Received: (qmail 28166 invoked by uid 550); 9 Dec 2012 00:04:54 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 28158 invoked from network); 9 Dec 2012 00:04:54 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=I8JiCm082whZGME5D+Wl+lKaL+z+cWOIOBYadHKq9Sk=; b=GIeIPYh00VozZ0JtESJa3gP/eqWHPsq+zdKy/5YlVtc6ulfcztp78pekMcTfNeF8OY w2gtupQTHu180tZviJzWIX25qFZSi0zU3kK7M0T61jgidPlfn9Jwq5gEiaKBEW1xsvg1 MNq6Nu5ie2keYDb36Zczm/5WVXsyzz1l0D5JSQVS7JCBnNYoAm86fQZIzc2CB1Pvz9a4 HFY+TLPpwYDO6DzC1/OJNBMxdChNCEESRJr/NYIy386H182subkkvh6SifT2XBGJKlI3 nkOYD4dduxtuRkmTE9SDzFsFi8oTnwkURmPAV+SFHT6N2by7SWVg0ngM8Si1bJ1z2MAI ZalA== In-Reply-To: <20121208232301.GW20323@brightrain.aerifal.cx> Xref: news.gmane.org gmane.linux.lib.musl.general:2430 Archived-At: --0016e6db2da423dd6604d0603201 Content-Type: text/plain; charset=ISO-8859-1 On Sun, Dec 9, 2012 at 1:23 AM, Rich Felker wrote: > I think it depends a lot on what you're using dlopen for. A lot of > programs these days use it as a ridiculous part of the development > model rather than for any real purpose; in my book, this is among the > highest levels of Considered Harmful. There's really no reason for any > code that's internal to an application (developed in the same tree, by > the same authors, and built at the same time) to be dynamically linked > at all, much less dynamically loaded. All this accomplishes is making > the program a lot slower and more bloated. > > I can not agree with you more on this. > On the flip side, the main legitimate uses for dynamic linking and > loading are (1) sharing code that's used by a wide range of > applications and allowing it to be upgraded system-wide all at once, > and (2) facilitating the extension of an application with third-party > code. Usage 1 applies mostly to dynamic linking; 2 mostly to dynamic > loading (dlopen). > Point 1 is probably the reason why most libraries end up as dynamic libraries. I was wondering about distributing all libraries as static libraries and then have the package manager link the application statically as the final step of the installation. This way the package manager can keep track of dependencies and re-link them if a library change. Distributions like Gentoo who install from source is actually in a very good position to take advantage of static linking. But I can see a lot of compiling/linking happening with this approach. Another idea would be to just install a stub where the binary would be. First time you run this stub, it will link the binary and store it on the disk in some sort of cache. Then just do an exec of that binary. Second time that you run this stub, it will check in this cache, link it again if it is not there or just exec it if found. This way only the stuff that gets used will be re-linked. You can force a re-link by clearing the cache. This what made me wonder about programs that use dlopen. I also wonder if the gain would be worth the trouble. I have seen a reduction of up to 50% RSS usage on programs that has a lot of shared libraries. It should improve responsiveness as there will be less paging. Thanks for all the answers. Regards Paul --0016e6db2da423dd6604d0603201 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Sun, Dec 9, 2012 at 1:23 AM, Rich Fel= ker <dalias@aerifal.cx> wrote:
I think it depends a lot on what you're using dlopen f= or. A lot of
programs these days use it as a ridiculous part of the development
model rather than for any real purpose; in my book, this is among the
highest levels of Considered Harmful. There's really no reason for any<= br> code that's internal to an application (developed in the same tree, by<= br> the same authors, and built at the same time) to be dynamically linked
at all, much less dynamically loaded. All this accomplishes is making
the program a lot slower and more bloated.

I can not agree with you more on this.
=A0
On the flip side, the main legitimate uses for dynamic linking and
loading are (1) sharing code that's used by a wide range of
applications and allowing it to be upgraded system-wide all at once,
and (2) facilitating the extension of an application with third-party
code. Usage 1 applies mostly to dynamic linking; 2 mostly to dynamic
loading (dlopen).

Point 1 is probably the = reason why most libraries end up as dynamic libraries.

=
I was wondering about distributing all libraries as static libraries a= nd then have the package manager link the application statically as the fin= al step of the installation. This way the package manager can keep track of= =A0dependencies and re-link them if a library change.

Distributions like Gentoo who install from source is ac= tually in a very good position to take advantage of static linking.

But I can see a lot of compiling/linking happening with t= his approach.

Another idea would be to just install a stub where the = binary would be. First time you run this stub, it will link the binary and = store it on the disk in some sort of cache. Then just do an exec of that bi= nary. Second time that you run this stub, it will check in this cache, link= it again if it is not there or just exec it if found. This way only the st= uff that gets used will be re-linked. You can force a re-link by clearing t= he cache. This what made me wonder about programs that use dlopen.

I also wonder if the gain would be worth the trouble. I= have seen a reduction of up to 50% RSS usage on programs that has a lot of= shared libraries. It should improve responsiveness as there will be less p= aging.

Thanks for all the answers.

Re= gards
Paul

--0016e6db2da423dd6604d0603201--