Re: Problems with pthreads from a shared object?

[Barry Flartus <barry.flartus@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4128 bytes --]

On Thu, Nov 8, 2018 at 12:37 PM Rich Felker <dalias@libc.org> wrote:

> On Thu, Nov 08, 2018 at 12:14:39PM -0500, Barry Flartus wrote:
> > >
> > > I think it would be helpful to understand what you're trying to do.
> > >
> >
> > Thanks for your help so far. Let me try and explain what I'm trying to
> > accomplish. I have a program that runs as an executable and uses
> pthreads.
> > I've compiled this program with musl statically with the end goal of it
> > being portable across older and newer systems. I want to also be able to
> > compile this program as a shared object so it may be loaded via dlopen()
> > inside of a glibc program. As mentioned previously, if I compile my
> shared
> > object with glibc, it loads via dlopen().
> >
> > My lack of understanding is this: if I directly compile in musl's libc.a
> > (which contains its implementation of pthreads) into my shared object,
> > shouldn't it have the relevant pthreads functions compiled in, without
> > having runtime issues? That's what seems to work for my musl-compiled
>
> It has the right code for being able to run in a process where the
> (part of) libc inside your .so is the only libc in the process. It
> does not have the right code for interoperating with a different
> arbitrary libc in the same process where both of them rightly believe
> themselves the sole owner of various process-global or thread-global
> state.
>
> > static executable, so I'm trying to wrap my head around why it wouldn't
> > work for a shared object. In my case, I'm considering the shared object
> > just a different form-factor for the same program.
>
> Conceptually, the reasons it won't work are unbounded; any particular
> list could become incomplete with newer versions of musl and/or glibc.
>
> In practice, issues you would definitely face include but are not
> limited to:
>
> - Anything that depends on initialization at startup time will fail,
>   as the musl __libc_start_main was never called.
>
> - Aside from a few fields which are compiler ABI, the content/layout
>   of the thread structure is an implementation detail subject to
>   change even between versions, and will not match between musl and
>   glibc. This matters even if the glibc code is not starting threads
>   since the main thread will have a glibc-format thread structure
>   pointed to by its thread pointer (%fs:0 on x86_64) and any glibc
>   functions called from your library will expect glibc-format thread
>   structures (even if the thread was started by the musl
>   pthread_create).
>
> - Assuming you didn't link with -Bsymbolic[-functions], calls to libc
>   functions that were linked in your library may (will) get interposed
>   by glibc functions already present, and the musl functions won't
>   actually get called, except for the ones that don't exist in glibc
>   (because they're in its librt or libpthread or something). And when
>   some functions from musl which do end up getting called execute,
>   they'll call a mix of glibc functions (when they reference something
>   bu its public symbol) and musl functions (when they call
>   musl-internal interfaces). These will be expecting/operaring on
>   different and incompatible data structures.
>
> - If you did link with -Bsymbolic[-functions], you'd end up calling
>   musl's malloc in the same process as glibc's malloc and they'd be
>   fighting over ownership of the brk. Things would also blow up badly
>   if a pointer returned by one's malloc was passed to the other's
>   realloc/free at some point.
>
> - Many more things...
>
> Ultimately, it's *always* unsafe to have two libraries with
> conflicting interfaces linked into the same program, and even moreso
> when they're something as central (with ownership of central global
> state) as libc.
>
> Rich
>

Rich, thank you so much for taking the time to explain this to me!
Ultimately
my problems stem from my lack knowledge in this area, so you really have
taught me so much and I am very appreciative of that.

Additionally, thank you for your work on musl, it really is fantastic! I
will
continue to use it in the correct manner.

Best,
Barry

[-- Attachment #2: Type: text/html, Size: 4842 bytes --]