STB_GNU_UNIQUE not handled as original spec intended

STB_GNU_UNIQUE not handled as original spec intended

[Alexander Monakov]

Hello,

At present the dynamic linker in musl handles STB_GNU_UNIQUE not in the way
the original spec intended.  Which is not necessarily a bad thing, but the
confusion about what that symbol binding type is supposed to convey probably
is.  Given that the commit that introduced handling into dynlink.c said:

commit e152ee9778846c1f233641b2d3562ccdb081c6a9
Author: Rich Felker <dalias@aerifal.cx>
Date:   Wed Jul 24 11:53:23 2013 -0400

    support STB_GNU_UNIQUE symbol bindings in dynamic linker
    
    these are needed for some C++ library binaries including most builds
    of libstdc++. I'm not entirely clear on the rationale. this patch does
    not implement any special semantics for them, but as far as I can
    tell, no special treatment is needed in correctly-linked programs;
    this binding seems to exist only for catching incorrectly-linked
    programs.

...it seems that either reasons for STB_GNU_UNIQUE were unclear at that time,
or I'm missing what "correctly-linked programs" was supposed to mean.  :)

So, to reiterate, my goal here is to show what STB_GNU_UNIQUE is supposed to
achieve, and make sure that choice made in musl is clear.

In the end of this email I'm pasting a minimal testcase that fails with musl.

STB_GNU_UNIQUE is marking a data symbol that should be unique in a running
program, *even when DSOs defining that symbol are all loaded with RTLD_LOCAL*.
Apart from behavior under dlopen(..., ... | RTLD_LOCAL), I don't see any way
it's different from a normal binding.

The original cause for the new binding type was a desire to support dlopen'ed
plugins implemented in C++ that reference data expected to be unified in
normal link (via what C++ calls "vague linkage").  These are emails from when
the binding was introduced:

https://gcc.gnu.org/ml/gcc-patches/2009-07/msg01240.html
https://www.redhat.com/archives/posix-c++-wg/2009-August/msg00002.html

At the moment, my personal view is that STB_GNU_UNIQUE made things messier.
The way it overrides RTLD_LOCAL sometimes makes it harder to reason about
program behavior, and the way it's opt-out rather than opt-in makes it easier
to accidentally write code that works on Linux with modern toolchain, but
fails with old toolchain, or other OSes without a similar binding type.  Here
are some emails from people dissatisfied with the development:

https://sourceware.org/ml/binutils/2011-10/msg00276.html
https://sourceware.org/ml/libc-alpha/2011-10/msg00066.html

Hope that clears things up.
Alexander

cat <<'EOF' >Makefile
test: main libfoo.so libbar.so
  ./main

main: LDLIBS=-ldl

lib%.so: %.cc
  $(CC) -fPIC -shared -o $@ $<
EOF
cat <<'EOF' >bar.cc
#include "singleton.h"

extern "C" int bar()
{
  return Singleton<int>::getInstance()++;
}
EOF
cat <<'EOF' >foo.cc
#include "singleton.h"

extern "C" int foo()
{
  return Singleton<int>::getInstance();
}
EOF
cat <<'EOF' >main.c
#include <dlfcn.h>

int main()
{
  void *libfoo = dlopen("./libfoo.so", RTLD_NOW);
  void *libbar = dlopen("./libbar.so", RTLD_NOW);

  int (*foo)(void) = dlsym(libfoo, "foo");
  int (*bar)(void) = dlsym(libbar, "bar");

  bar();
  return !foo();
}
EOF
cat <<'EOF' >singleton.h
template<class T>
struct Singleton {
  static T& getInstance()
  {
    static T instance;
    return instance;
  }
};
EOF

Re: STB_GNU_UNIQUE not handled as original spec intended

[Rich Felker]

On Mon, Aug 31, 2015 at 03:51:16PM +0300, Alexander Monakov wrote:
> Hello,
> 
> At present the dynamic linker in musl handles STB_GNU_UNIQUE not in the way
> the original spec intended.  Which is not necessarily a bad thing, but the
> confusion about what that symbol binding type is supposed to convey probably
> is.  Given that the commit that introduced handling into dynlink.c said:
> 
> commit e152ee9778846c1f233641b2d3562ccdb081c6a9
> Author: Rich Felker <dalias@aerifal.cx>
> Date:   Wed Jul 24 11:53:23 2013 -0400
> 
>     support STB_GNU_UNIQUE symbol bindings in dynamic linker
>     
>     these are needed for some C++ library binaries including most builds
>     of libstdc++. I'm not entirely clear on the rationale. this patch does
>     not implement any special semantics for them, but as far as I can
>     tell, no special treatment is needed in correctly-linked programs;
>     this binding seems to exist only for catching incorrectly-linked
>     programs.
> 
> ....it seems that either reasons for STB_GNU_UNIQUE were unclear at that time,
> or I'm missing what "correctly-linked programs" was supposed to mean.  :)

I roughly agree. I think it's probably a mix -- partly a lack of full
understanding of how the feature works, and partly a very broad
concept of what "incorrectly linked" might mean.

> STB_GNU_UNIQUE is marking a data symbol that should be unique in a running
> program, *even when DSOs defining that symbol are all loaded with RTLD_LOCAL*.
> Apart from behavior under dlopen(..., ... | RTLD_LOCAL), I don't see any way
> it's different from a normal binding.

It should be noted that supporting this precludes a simple version of
the proposed dynamic linker optimization to have a separate
global-namespace linked list -- even non-global libs would have to be
searched if they contain STB_GNU_UNIQUE symbols, at least in some
cases. I don't actually want to think too much about how this would
need to work unless we are actually interested in supporting it.

> The original cause for the new binding type was a desire to support dlopen'ed
> plugins implemented in C++ that reference data expected to be unified in
> normal link (via what C++ calls "vague linkage").  These are emails from when
> the binding was introduced:

I understand that failure to support this violates the C++ semantics,
but RTLD_LOCAL _inherently_ violates C global semantics anyway. The
whole point of RTLD_LOCAL is that you want to, at least to some
extent, ignore the language's concept of a global symbol namespace and
instead isolate your module/plugin/whatever so that it can't step on
others' symbol names.

> https://gcc.gnu.org/ml/gcc-patches/2009-07/msg01240.html
> https://www.redhat.com/archives/posix-c++-wg/2009-August/msg00002.html
> 
> At the moment, my personal view is that STB_GNU_UNIQUE made things messier.
> The way it overrides RTLD_LOCAL sometimes makes it harder to reason about
> program behavior, and the way it's opt-out rather than opt-in makes it easier
> to accidentally write code that works on Linux with modern toolchain, but
> fails with old toolchain, or other OSes without a similar binding type.  Here
> are some emails from people dissatisfied with the development:
> 
> https://sourceware.org/ml/binutils/2011-10/msg00276.html
> https://sourceware.org/ml/libc-alpha/2011-10/msg00066.html

I agree completely. So I'm fairly strongly in favor of leaving musl's
(non-)handling of this feature the way it is.

Rich