Regarding whole-archive linking libc.a into a shared lib

Regarding whole-archive linking libc.a into a shared lib

[Jason Ramapuram]

[-- Attachment #1: Type: text/plain, Size: 1299 bytes --]

Hello there,

Is it at all possible to link musl's libc.a statically into a shared lib?
After a lot of trying I either get relocation errors (when using musl-gcc
built on say centos7) as such (
http://stackoverflow.com/questions/38251344/musl-fails-to-link-libc-a-into-shared-library)
or hidden symbols not being defined like below (when using Alpine linux) :

/home/buildozer/aports/main/musll/src/musl-1.1.12/src/ldso/x86_64/tlsdesc.s:36:
undefined reference to `__tls_get_new'
/usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld:
tgt/Linux-x86_64/mylib/lib/mylib.so: hidden symbol `__tls_get_new’ isn't
defined
/usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld:
final link failed: Bad value
collect2: error: ld returned 1 exit status
Makefile:58: recipe for target ‘tgt/Linux-x86_64/mulib/lib/mylib.so’ failed
make: *** [tgt/Linux-x86_64/mylib/lib/mylib.so] Error 1

I understand this might conflict when someone tries to link against this
and libc.so from a binary, however would it be possible somehow to mangle
all the libc calls internally inside the shared lib? Is there a tool that
would be able to do this? It would be really nice to have a portable shared
lib that has libc built in.

[-- Attachment #2: Type: text/html, Size: 1628 bytes --]

Re: Regarding whole-archive linking libc.a into a shared lib

[Aidan Hobson Sayers]

[-- Attachment #1: Type: text/plain, Size: 2319 bytes --]

There's a brief coverage of some of the problems with this idea in
http://www.openwall.com/lists/musl/2012/12/08/4

The relevant part is this bit: "But now you're potentially using two
different versions of libc in the same program; if implementation-internal
data structures like FILE or the pthread structure are not identical
between the 2 versions, you'll hit an ABI incompatibility, despite the fact
that these data structures were intended to be implementation-internal and
never affect ABI. Even without that issue, you have issues like potentially
2 copies of malloc trying to manage the heap without being aware of one
another, and thus clobbering it."

It's worth reading the whole thread as there a couple of other directly
relevant bits. I've wanted to do the same as you in the past, but the
number of possible (/probable) issues was quite discouraging.

On 14 Jul 2016 11:04 am, "Jason Ramapuram" <jason.ramapuram@gmail.com>
wrote:

> Hello there,
>
> Is it at all possible to link musl's libc.a statically into a shared lib?
> After a lot of trying I either get relocation errors (when using musl-gcc
> built on say centos7) as such (
> http://stackoverflow.com/questions/38251344/musl-fails-to-link-libc-a-into-shared-library)
> or hidden symbols not being defined like below (when using Alpine linux) :
>
> /home/buildozer/aports/main/musll/src/musl-1.1.12/src/ldso/x86_64/tlsdesc.s:36:
> undefined reference to `__tls_get_new'
> /usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld:
> tgt/Linux-x86_64/mylib/lib/mylib.so: hidden symbol `__tls_get_new’ isn't
> defined
> /usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld:
> final link failed: Bad value
> collect2: error: ld returned 1 exit status
> Makefile:58: recipe for target ‘tgt/Linux-x86_64/mulib/lib/mylib.so’ failed
> make: *** [tgt/Linux-x86_64/mylib/lib/mylib.so] Error 1
>
> I understand this might conflict when someone tries to link against this
> and libc.so from a binary, however would it be possible somehow to mangle
> all the libc calls internally inside the shared lib? Is there a tool that
> would be able to do this? It would be really nice to have a portable shared
> lib that has libc built in.
>
>
>

[-- Attachment #2: Type: text/html, Size: 2947 bytes --]

Re: Regarding whole-archive linking libc.a into a shared lib

[Jason Ramapuram]

[-- Attachment #1: Type: text/plain, Size: 3056 bytes --]

Thanks for the link. Just went through the entire thread. I think we can
solve the problems as such though:

   1. Have some form of linker that mangles all the symbols internally in
   the newly linked shared library, eg: FILE* --> FILE_mangled* so that
   internally it has it's own libc calls that are independent from anything
   that a binary might require.
   2. Block allocate a chunk of memory internally in the library so that
   allocations are now using that internal block and not malloc-ing.

However, I'm not quite sure how 1) could be solved at this time.


On Thu, Jul 14, 2016 at 12:17 PM, Aidan Hobson Sayers <aidanphs@gmail.com>
wrote:

> There's a brief coverage of some of the problems with this idea in
> http://www.openwall.com/lists/musl/2012/12/08/4
>
> The relevant part is this bit: "But now you're potentially using two
> different versions of libc in the same program; if implementation-internal
> data structures like FILE or the pthread structure are not identical
> between the 2 versions, you'll hit an ABI incompatibility, despite the fact
> that these data structures were intended to be implementation-internal and
> never affect ABI. Even without that issue, you have issues like potentially
> 2 copies of malloc trying to manage the heap without being aware of one
> another, and thus clobbering it."
>
> It's worth reading the whole thread as there a couple of other directly
> relevant bits. I've wanted to do the same as you in the past, but the
> number of possible (/probable) issues was quite discouraging.
>
> On 14 Jul 2016 11:04 am, "Jason Ramapuram" <jason.ramapuram@gmail.com>
> wrote:
>
>> Hello there,
>>
>> Is it at all possible to link musl's libc.a statically into a shared lib?
>> After a lot of trying I either get relocation errors (when using musl-gcc
>> built on say centos7) as such (
>> http://stackoverflow.com/questions/38251344/musl-fails-to-link-libc-a-into-shared-library)
>> or hidden symbols not being defined like below (when using Alpine linux) :
>>
>> /home/buildozer/aports/main/musll/src/musl-1.1.12/src/ldso/x86_64/tlsdesc.s:36:
>> undefined reference to `__tls_get_new'
>> /usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld:
>> tgt/Linux-x86_64/mylib/lib/mylib.so: hidden symbol `__tls_get_new’ isn't
>> defined
>> /usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld:
>> final link failed: Bad value
>> collect2: error: ld returned 1 exit status
>> Makefile:58: recipe for target ‘tgt/Linux-x86_64/mulib/lib/mylib.so’
>> failed
>> make: *** [tgt/Linux-x86_64/mylib/lib/mylib.so] Error 1
>>
>> I understand this might conflict when someone tries to link against this
>> and libc.so from a binary, however would it be possible somehow to mangle
>> all the libc calls internally inside the shared lib? Is there a tool that
>> would be able to do this? It would be really nice to have a portable shared
>> lib that has libc built in.
>>
>>
>>

[-- Attachment #2: Type: text/html, Size: 4025 bytes --]

Re: Regarding whole-archive linking libc.a into a shared lib

[Szabolcs Nagy]

* Jason Ramapuram <jason.ramapuram@gmail.com> [2016-07-14 12:49:31 +0200]:
> Thanks for the link. Just went through the entire thread. I think we can
> solve the problems as such though:
> 
>    1. Have some form of linker that mangles all the symbols internally in
>    the newly linked shared library, eg: FILE* --> FILE_mangled* so that
>    internally it has it's own libc calls that are independent from anything
>    that a binary might require.
>    2. Block allocate a chunk of memory internally in the library so that
>    allocations are now using that internal block and not malloc-ing.
> 
> However, I'm not quite sure how 1) could be solved at this time.
> 

1) libc maintains global state which will be duplicated in your
shared lib (e.g. envvars, stdio buffers, timezone, locales)
name mangling does not fix their behaviour (they will behave
inconsistently with other parts of the application).

2) brk is not the only global state that can only have one owner,
the thread pointer is another example: if your shared lib does
any tls access (e.g. errno, stack canary) then it has to match
the internals of the libc in the main application (won't work
across libc versions).

static linking libc into a shared lib simply does not make sense.

Re: Regarding whole-archive linking libc.a into a shared lib

[Rich Felker]

On Thu, Jul 14, 2016 at 12:49:31PM +0200, Jason Ramapuram wrote:
> Thanks for the link. Just went through the entire thread. I think we can
> solve the problems as such though:
> 
>    1. Have some form of linker that mangles all the symbols internally in
>    the newly linked shared library, eg: FILE* --> FILE_mangled* so that
>    internally it has it's own libc calls that are independent from anything
>    that a binary might require.

This is already trivially possible via the --exclude-libs ld option,
which is confusingly named but converts all symbols in particular (or
all) static libs being linked into hidden symbols. That is sufficient
for safely linking some dependency libs statically into a shared
library without the risk of breaking things in the program that loads
the shared library (I suspect it would work well for something like
zlib), but it depends on there being no additional hidden interface
surfaces that interact with the rest of the program. For libc those
would include (some as noted by others):

- the thread structure pointed to by the thread pointer
- signal dispositions used internally
- assumptions about what actions are excluded by locks
- possible leaks of pointers to allocated memory from one libc's
  malloc to the other one's free.

In addition, libc initialization (including the thread
structure/pointer for the main thread) is performed as part of the
call to __libc_start_main from the crt1 entry point. This will never
happen for a copy embedded inside your shared library, and thus you
would be running with a libc in uninitialized state. In practice the
thread pointer, which is the main thing that needs initialization,
would already have been initialized by the "real" libc, so things
might "happen to work" with an exact-same-version musl libc.so
matching the libc.a you linked with, but on the opposite end of the
spectrum things would horribly blow up when your linked-in musl finds
a glibc-format thread structure at that address.

This is probably only the tip of the iceberg as to what can go wrong
with your idea. On the flip side, I don't see what advantages you
could get by static linking libc.a into a shared library. If your
library is shared, then it necessarily already has a shared libc.so
available at runtime (simply because it's being used in a
dynamic-linked program). If your intent is to produce a shared library
that works with both musl and glibc, that's a nice idea, but a
different approach is probably needed, and it would be an interesting
discussion to spin off of this thread.

Rich