From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/10320 Path: news.gmane.org!not-for-mail From: Jason Ramapuram Newsgroups: gmane.linux.lib.musl.general Subject: Re: Regarding whole-archive linking libc.a into a shared lib Date: Thu, 14 Jul 2016 12:49:31 +0200 Message-ID: References: Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=94eb2c07724acb56f5053796415b X-Trace: ger.gmane.org 1468493406 2934 80.91.229.3 (14 Jul 2016 10:50:06 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 14 Jul 2016 10:50:06 +0000 (UTC) Cc: musl@lists.openwall.com To: Aidan Hobson Sayers Original-X-From: musl-return-10333-gllmg-musl=m.gmane.org@lists.openwall.com Thu Jul 14 12:49:52 2016 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1bNeDC-0001r7-Sv for gllmg-musl@m.gmane.org; Thu, 14 Jul 2016 12:49:47 +0200 Original-Received: (qmail 32656 invoked by uid 550); 14 Jul 2016 10:49:44 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 32634 invoked from network); 14 Jul 2016 10:49:43 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=11cwrIRon9uIjpTqfOiB0U+vuCXSQzNLlgo71MEs3sY=; b=MEdGQmZ6zMUSr2exwUMV37yYovIJC/fDZlya8zuznR38HGrmRZ1OZNBB+R+tmi7U/A 9iTJKutwHq0jlW2Mtwlmw8zHO6T3iolC3LRXEToEC1eUwgui6BSGswemjvk13k4v/y66 HSnyCUroZdW6nlcgSLx1fDyq2Lkm5NkwNEzI9GBxBZXO3Gu0S6bAOZLWUpJrqExoFFyq +c8Vs16AhzSWG8AozVA36qzKOjBsmnZW9xr6irBCpTnimqrcx/iwc45SyObTRiE30DcT Cki1uwhAaQQgSg0wuCyWRXDHXe2VpYYp0oDL8eVJCY+C2caotGwjySMN8oY+iHI8aw+J Gpbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=11cwrIRon9uIjpTqfOiB0U+vuCXSQzNLlgo71MEs3sY=; b=JebZNBROVveTjvuZE9CH9s7wvoIpiVJ2daNGSb2rmKudm3V8sU0Tky7aIs3nBtnY66 JN/A3UWRBCebRiXATp9nVk9Dquy+i49ooA4LCAu2zt8SQSyL6OsXvHfUIZfLHniZu1dU XyMMLvyam1EyOgEVBtD7jFVmh7nWUiNgvdpWdn9gOmo4FDT4R9591tx1as3q22W1lbU1 y6j7KcvLhQqTmSH/yqWkTHJcIuIU0Fd+P1He7GFb/CaDUfutk0O9Hz2FV1SyOLoZ7w2j EGQypKJWUegBLKKo6OETqVvkIvl9fxWVBkMZMmLh67N2x0LnKlgNzfzSjs927SIvfBR/ T8Hg== X-Gm-Message-State: ALyK8tLdGJ+oA8DYHO07N0hlaL+5QRFBG7/Q3CXqvyVJBGnsLGHIn87T8aq/SyBqC0+zajnc2awnyH5u46HtXA== X-Received: by 10.107.188.131 with SMTP id m125mr14648601iof.1.1468493372019; Thu, 14 Jul 2016 03:49:32 -0700 (PDT) In-Reply-To: Xref: news.gmane.org gmane.linux.lib.musl.general:10320 Archived-At: --94eb2c07724acb56f5053796415b Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks for the link. Just went through the entire thread. I think we can solve the problems as such though: 1. Have some form of linker that mangles all the symbols internally in the newly linked shared library, eg: FILE* --> FILE_mangled* so that internally it has it's own libc calls that are independent from anything that a binary might require. 2. Block allocate a chunk of memory internally in the library so that allocations are now using that internal block and not malloc-ing. However, I'm not quite sure how 1) could be solved at this time. On Thu, Jul 14, 2016 at 12:17 PM, Aidan Hobson Sayers wrote: > There's a brief coverage of some of the problems with this idea in > http://www.openwall.com/lists/musl/2012/12/08/4 > > The relevant part is this bit: "But now you're potentially using two > different versions of libc in the same program; if implementation-interna= l > data structures like FILE or the pthread structure are not identical > between the 2 versions, you'll hit an ABI incompatibility, despite the fa= ct > that these data structures were intended to be implementation-internal an= d > never affect ABI. Even without that issue, you have issues like potential= ly > 2 copies of malloc trying to manage the heap without being aware of one > another, and thus clobbering it." > > It's worth reading the whole thread as there a couple of other directly > relevant bits. I've wanted to do the same as you in the past, but the > number of possible (/probable) issues was quite discouraging. > > On 14 Jul 2016 11:04 am, "Jason Ramapuram" > wrote: > >> Hello there, >> >> Is it at all possible to link musl's libc.a statically into a shared lib= ? >> After a lot of trying I either get relocation errors (when using musl-gc= c >> built on say centos7) as such ( >> http://stackoverflow.com/questions/38251344/musl-fails-to-link-libc-a-in= to-shared-library) >> or hidden symbols not being defined like below (when using Alpine linux)= : >> >> /home/buildozer/aports/main/musll/src/musl-1.1.12/src/ldso/x86_64/tlsdes= c.s:36: >> undefined reference to `__tls_get_new' >> /usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-li= nux-musl/bin/ld: >> tgt/Linux-x86_64/mylib/lib/mylib.so: hidden symbol `__tls_get_new=E2=80= =99 isn't >> defined >> /usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-li= nux-musl/bin/ld: >> final link failed: Bad value >> collect2: error: ld returned 1 exit status >> Makefile:58: recipe for target =E2=80=98tgt/Linux-x86_64/mulib/lib/mylib= .so=E2=80=99 >> failed >> make: *** [tgt/Linux-x86_64/mylib/lib/mylib.so] Error 1 >> >> I understand this might conflict when someone tries to link against this >> and libc.so from a binary, however would it be possible somehow to mangl= e >> all the libc calls internally inside the shared lib? Is there a tool tha= t >> would be able to do this? It would be really nice to have a portable sha= red >> lib that has libc built in. >> >> >> --94eb2c07724acb56f5053796415b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks for the link. Just went through the entire thread. = I think we can solve the problems as such though:
  1. Have some for= m of linker that mangles all the symbols internally in the newly linked sha= red library, eg: FILE* --> FILE_mangled* so that internally it has it= 9;s own libc calls that are independent from anything that a binary might r= equire.
  2. Block allocate a chunk of memory internally in the libr= ary so that allocations are now using that internal block and not malloc-in= g.
However, I'm not quite sure how 1) could be solved at = this time.


<= div class=3D"gmail_quote">On Thu, Jul 14, 2016 at 12:17 PM, Aidan Hobson Sa= yers <aidanphs@gmail.com> wrote:

There's a brief coverage of some of the problems= with this idea in http://www.openwall.com/lists/musl/2012/12/08/4

The relevant part is this bit: "But now you're pote= ntially using two different versions of libc in the same program; if implem= entation-internal data structures like FILE or the pthread structure are no= t identical between the 2 versions, you'll hit an ABI incompatibility, = despite the fact that these data structures were intended to be implementat= ion-internal and never affect ABI. Even without that issue, you have issues= like potentially 2 copies of malloc trying to manage the heap without bein= g aware of one another, and thus clobbering it."

It's worth reading the whole thread as there a couple of= other directly relevant bits. I've wanted to do the same as you in the= past, but the number of possible (/probable) issues was quite discouraging= .


On 14 Jul 2016 11= :04 am, "Jason Ramapuram" <jason.ramapuram@gmail.com> wrote:

Hello there,

Is it at all possible to link musl's libc.a statically into a shared= lib? After a lot of trying I either get relocation errors (when using musl= -gcc built on say centos7) as such (http://stackoverflow.com/questions/38251344/musl-fails-to-link-libc-a= -into-shared-library) or hidden symbols not being defined like below (w= hen using Alpine linux) :

/home/buildozer/aports/main/musll/src/musl-1.1.12/src/lds= o/x86_64/tlsdesc.s:36: undefined reference to `__tls_get_new'
/usr/l= ib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/= bin/ld: tgt/Linux-x86_64/mylib/lib/mylib.so: hidden symbol `__tls_get_new= =E2=80=99 isn't defined
/usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/= ../../../../x86_64-alpine-linux-musl/bin/ld: final link failed: Bad valuecollect2: error: ld returned 1 exit status
Makefile:58: recipe for tar= get =E2=80=98tgt/Linux-x86_64/mulib/lib/mylib.so=E2=80=99 failed
make: *= ** [tgt/Linux-x86_64/mylib/lib/mylib.so] Error 1

I understand this might conflict when someone tries t= o link against this and libc.so from a binary, however would it be possible= somehow to mangle all the libc calls internally inside the shared lib? Is = there a tool that would be able to do this? It would be really nice to have= a portable shared lib that has libc built in.



--94eb2c07724acb56f5053796415b--