[PATCH 1/2] add musl-clang, a wrapper for system clang installs

[PATCH 1/2] add musl-clang, a wrapper for system clang installs

[Shiz]

musl-clang allows the user to compile musl-powered programs using their
already existent clang install, without needing a special cross compiler.
it does this by wrapping around both the system clang install and the linker
and passing them special flags to re-target musl at runtime. it does only
affect invocations done through the special musl-clang wrapper script,
so that the user setup remains fully intact otherwise.

the clang wrapper consists of the compiler frontend wrapper script,
musl-clang, and the linker wrapper script, ld.musl-clang.
musl-clang makes sure clang invokes ld.musl-clang to link objects;
neither script needs to be in PATH for the wrapper to work.
---
 .gitignore             |  1 +
 Makefile               |  6 +++++-
 tools/ld.musl-clang.in | 45 +++++++++++++++++++++++++++++++++++++++++++++
 tools/musl-clang.in    | 35 +++++++++++++++++++++++++++++++++++
 4 files changed, 86 insertions(+), 1 deletion(-)
 create mode 100644 tools/ld.musl-clang.in
 create mode 100644 tools/musl-clang.in

diff --git a/.gitignore b/.gitignore
index 070f549..00e4d0a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,5 +7,6 @@ arch/*/bits/alltypes.h
 config.mak
 include/bits
 tools/musl-gcc
+tools/*musl-clang
 lib/musl-gcc.specs
 src/internal/version.h
diff --git a/Makefile b/Makefile
index 3bd7b4d..0537c6a 100644
--- a/Makefile
+++ b/Makefile
@@ -49,7 +49,7 @@ STATIC_LIBS = lib/libc.a
 SHARED_LIBS = lib/libc.so
 TOOL_LIBS = lib/musl-gcc.specs
 ALL_LIBS = $(CRT_LIBS) $(STATIC_LIBS) $(SHARED_LIBS) $(EMPTY_LIBS) $(TOOL_LIBS)
-ALL_TOOLS = tools/musl-gcc
+ALL_TOOLS = tools/musl-gcc tools/musl-clang tools/ld.musl-clang
 
 LDSO_PATHNAME = $(syslibdir)/ld-musl-$(ARCH)$(SUBARCH).so.1
 
@@ -156,6 +156,10 @@ tools/musl-gcc: config.mak
 	printf '#!/bin/sh\nexec "$${REALGCC:-gcc}" "$$@" -specs "%s/musl-gcc.specs"\n' "$(libdir)" > $@
 	chmod +x $@
 
+tools/%-clang: tools/%-clang.in config.mak
+	sed -e 's!@INCDIR@!$(includedir)!g' -e 's!@LIBDIR@!$(libdir)!g' -e 's!@LDSO@!$(LDSO_PATHNAME)!g' $< > $@
+	chmod +x $@
+
 $(DESTDIR)$(bindir)/%: tools/%
 	$(INSTALL) -D $< $@
 
diff --git a/tools/ld.musl-clang.in b/tools/ld.musl-clang.in
new file mode 100644
index 0000000..bb77bb1
--- /dev/null
+++ b/tools/ld.musl-clang.in
@@ -0,0 +1,45 @@
+#!/bin/sh
+cc="@CC@"
+libc_lib="@LIBDIR@"
+ldso="@LDSO@"
+cleared=
+shared=
+crt=
+userlinkdir=
+userlink=
+
+for x ; do
+    test "$cleared" || set -- ; cleared=1
+
+    case "$x" in
+        -L-user-start)
+            userlinkdir=1
+            ;;
+        -L-user-end)
+            userlinkdir=
+            ;;
+        -L*)
+            test "$userlinkdir" && set -- "$@" "$x"
+            ;;
+        -l-user-start)
+            test "$shared" || set -- "$@" "$libc_lib/Scrt1.o" "$libc_lib/crti.o"
+            userlink=1
+            ;;
+        -l-user-end)
+            userlink=
+            ;;
+        -l*)
+            test "$crt" -o "$shared" || set -- "$@" "$libc_lib/crtn.o" ; crt=1
+            test "$userlink" && set -- "$@" "$x"
+            ;;
+        -shared)
+            shared=1
+            set -- "$@" -shared
+            ;;
+        *)
+            set -- "$@" "$x"
+            ;;
+    esac
+done
+
+exec $($cc -print-prog-name=ld) -nostdlib "$@" -dynamic-linker "$ldso"
diff --git a/tools/musl-clang.in b/tools/musl-clang.in
new file mode 100644
index 0000000..51e0196
--- /dev/null
+++ b/tools/musl-clang.in
@@ -0,0 +1,35 @@
+#!/bin/sh
+cc="@CC@"
+libc_inc="@INCDIR@"
+libc_lib="@LIBDIR@"
+thisdir="`cd "$(dirname "$0")"; pwd`"
+
+# prevent clang from running the linker (and erroring) on no input.
+sflags=
+eflags=
+for x ; do
+    case "$x" in
+        -l*) input=1 ;;
+        -*) input= ;;
+        *) input=1 ;;
+    esac
+    if test "$input" ; then
+        sflags="-l-user-start"
+        eflags="-lc -l-user-end"
+        break
+    fi
+done
+
+exec $cc \
+    -B"$thisdir" \
+    -fuse-ld=musl-clang \
+    -rtlib=compiler-rt \
+    -nostdinc \
+    -nostartfiles \
+    -isystem "$libc_inc" \
+    -L-user-start \
+    $sflags \
+    "$@" \
+    $eflags \
+    -L"$libc_lib" \
+    -L-user-end
-- 
2.3.6

[PATCH 2/2] build: overhaul wrapper script system for multiple wrapper support

[Shiz]

this overhauls part of the build system in order to support multiple
toolchain wrapper scripts, as opposed to solely the musl-gcc wrapper as
before. it thereby replaces --enable-gcc-wrapper with --enable-wrapper=...,
which has the options 'auto' (the default, detect whether to use wrappers),
'all' (build and install all wrappers), 'no' (don't build any) and finally
the options named after the individual compiler scripts (currently 'clang'
and 'gcc' are available) to build and install only that wrapper.
the old --enable-gcc-wrapper is removed from --help, but still available.

it also modifies the wrappers to use the C compiler specified to the build
system as 'inner' compiler, when applicable. as wrapper detection works by
probing this compiler, it may not work with any other.
---
 Makefile  |  7 +++++--
 configure | 67 +++++++++++++++++++++++++++++++++++++++++++--------------------
 2 files changed, 51 insertions(+), 23 deletions(-)

diff --git a/Makefile b/Makefile
index 0537c6a..a5c25a6 100644
--- a/Makefile
+++ b/Makefile
@@ -51,6 +51,9 @@ TOOL_LIBS = lib/musl-gcc.specs
 ALL_LIBS = $(CRT_LIBS) $(STATIC_LIBS) $(SHARED_LIBS) $(EMPTY_LIBS) $(TOOL_LIBS)
 ALL_TOOLS = tools/musl-gcc tools/musl-clang tools/ld.musl-clang
 
+WRAPCC_GCC = gcc
+WRAPCC_CLANG = clang
+
 LDSO_PATHNAME = $(syslibdir)/ld-musl-$(ARCH)$(SUBARCH).so.1
 
 -include config.mak
@@ -153,11 +156,11 @@ lib/musl-gcc.specs: tools/musl-gcc.specs.sh config.mak
 	sh $< "$(includedir)" "$(libdir)" "$(LDSO_PATHNAME)" > $@
 
 tools/musl-gcc: config.mak
-	printf '#!/bin/sh\nexec "$${REALGCC:-gcc}" "$$@" -specs "%s/musl-gcc.specs"\n' "$(libdir)" > $@
+	printf '#!/bin/sh\nexec "$${REALGCC:-$(WRAPCC_GCC)}" "$$@" -specs "%s/musl-gcc.specs"\n' "$(libdir)" > $@
 	chmod +x $@
 
 tools/%-clang: tools/%-clang.in config.mak
-	sed -e 's!@INCDIR@!$(includedir)!g' -e 's!@LIBDIR@!$(libdir)!g' -e 's!@LDSO@!$(LDSO_PATHNAME)!g' $< > $@
+	sed -e 's!@CC@!$(WRAPCC_CLANG)!g' -e 's!@INCDIR@!$(includedir)!g' -e 's!@LIBDIR@!$(libdir)!g' -e 's!@LDSO@!$(LDSO_PATHNAME)!g' $< > $@
 	chmod +x $@
 
 $(DESTDIR)$(bindir)/%: tools/%
diff --git a/configure b/configure
index 7b29ae4..a5296e6 100755
--- a/configure
+++ b/configure
@@ -28,7 +28,7 @@ Optional features:
   --enable-debug          build with debugging information [disabled]
   --enable-warnings       build with recommended warnings flags [disabled]
   --enable-visibility     use global visibility options to optimize PIC [auto]
-  --enable-gcc-wrapper    build musl-gcc toolchain wrapper [auto]
+  --enable-wrapper=...    build given musl toolchain wrapper [auto]
   --disable-shared        inhibit building shared library [enabled]
   --disable-static        inhibit building static library [enabled]
 
@@ -130,7 +130,8 @@ warnings=no
 visibility=auto
 shared=auto
 static=yes
-wrapper=auto
+gcc_wrapper=auto
+clang_wrapper=auto
 
 for arg ; do
 case "$arg" in
@@ -154,8 +155,13 @@ case "$arg" in
 --disable-warnings|--enable-warnings=no) warnings=no ;;
 --enable-visibility|--enable-visibility=yes) visibility=yes ;;
 --disable-visibility|--enable-visibility=no) visibility=no ;;
---enable-gcc-wrapper|--enable-gcc-wrapper=yes) wrapper=yes ;;
---disable-gcc-wrapper|--enable-gcc-wrapper=no) wrapper=no ;;
+--enable-wrapper|--enable-wrapper=yes) gcc_wrapper=auto ; clang_wrapper=auto ;;
+--enable-wrapper=all) gcc_wrapper=yes ; clang_wrapper=yes ;;
+--enable-wrapper=clang) gcc_wrapper=no ; clang_wrapper=yes ;;
+--enable-wrapper=gcc) gcc_wrapper=yes ; clang_wrapper=no ;;
+--disable-wrapper|--enable-wrapper=no) gcc_wrapper=no ; clang_wrapper=no ;;
+--enable-gcc-wrapper|--enable-gcc-wrapper=yes) gcc_wrapper=yes ;;
+--disable-gcc-wrapper|--enable-gcc-wrapper=no) gcc_wrapper=no ;;
 --enable-*|--disable-*|--with-*|--without-*|--*dir=*|--build=*) ;;
 --host=*|--target=*) target=${arg#*=} ;;
 -* ) echo "$0: unknown option $arg" ;;
@@ -215,35 +221,47 @@ tryldflag LDFLAGS_TRY -Werror=unknown-warning-option
 tryldflag LDFLAGS_TRY -Werror=unused-command-line-argument
 
 #
-# Need to know if the compiler is gcc to decide whether to build the
-# musl-gcc wrapper, and for critical bug detection in some gcc versions.
+# Need to know if the compiler is gcc or clang to decide which toolchain
+# wrappers to build.
 #
-printf "checking whether compiler is gcc... "
-if fnmatch '*gcc\ version*' "$(LC_ALL=C $CC -v 2>&1)" ; then
-cc_is_gcc=yes
-else
-cc_is_gcc=no
+printf "checking for C compiler family... "
+cc_ver="$(LC_ALL=C $CC -v 2>&1)"
+cc_family=unknown
+if fnmatch '*gcc\ version*' "$cc_ver" ; then
+cc_family=gcc
+fi
+if fnmatch '*clang\ version*' "$cc_ver" ; then
+cc_family=clang
 fi
-echo "$cc_is_gcc"
+echo "$cc_family"
 
 #
-# Only build musl-gcc wrapper if toolchain does not already target musl
+# Only build wrapper if toolchain does not already target musl
 #
-if test "$wrapper" = auto ; then
+if test "$gcc_wrapper" = auto ; then
 printf "checking whether to build musl-gcc wrapper... "
-if test "$cc_is_gcc" = yes ; then
-wrapper=yes
+if test "$cc_family" = gcc ; then
+gcc_wrapper=yes
 while read line ; do
-case "$line" in */ld-musl-*) wrapper=no ;; esac
+case "$line" in */ld-musl-*) gcc_wrapper=no ;; esac
 done <<EOF
 $($CC -dumpspecs)
 EOF
 else
-wrapper=no
+gcc_wrapper=no
 fi
-echo "$wrapper"
+echo "$gcc_wrapper"
 fi
 
+if test "$clang_wrapper" = auto ; then
+printf "checking whether to build musl-clang wrapper... "
+if test "$cc_family" = clang ; then
+clang_wrapper=yes
+else
+clang_wrapper=no
+fi
+echo "$clang_wrapper"
+fi
 
 
 #
@@ -583,8 +601,15 @@ OPTIMIZE_GLOBS = $OPTIMIZE_GLOBS
 EOF
 test "x$static" = xno && echo "STATIC_LIBS ="
 test "x$shared" = xno && echo "SHARED_LIBS ="
-test "x$wrapper" = xno && echo "ALL_TOOLS ="
-test "x$wrapper" = xno && echo "TOOL_LIBS ="
+test "x$gcc_wrapper" = xyes -a "x$cc_family" = xgcc && echo 'WRAPCC_GCC = $(CC)'
+test "x$clang_wrapper" = xyes -a "x$cc_family" = xclang && echo 'WRAPCC_CLANG = $(CC)'
+printf "ALL_TOOLS ="
+test "x$gcc_wrapper" = xyes && printf " tools/musl-gcc"
+test "x$clang_wrapper" = xyes && printf " tools/musl-clang tools/ld.musl-clang"
+echo
+printf "TOOL_LIBS ="
+test "x$gcc_wrapper" = xyes && printf " lib/musl-gcc.specs"
+echo
 exec 1>&3 3>&-
 
 printf "done\n"
-- 
2.3.6

Re: [PATCH 1/2] add musl-clang, a wrapper for system clang installs

[Rich Felker]

On Fri, May 29, 2015 at 06:48:45PM +0200, Shiz wrote:
> +# prevent clang from running the linker (and erroring) on no input.
> +sflags=
> +eflags=
> +for x ; do
> +    case "$x" in
> +        -l*) input=1 ;;
> +        -*) input= ;;
> +        *) input=1 ;;
> +    esac
> +    if test "$input" ; then
> +        sflags="-l-user-start"
> +        eflags="-lc -l-user-end"
> +        break
> +    fi
> +done

This logic looks wrong. For example, "-L foo" would result in input=1,
no? And same for any other options with arguments.

Rich

Re: [PATCH 1/2] add musl-clang, a wrapper for system clang installs

[Shiz]

[-- Attachment #1: Type: text/plain, Size: 599 bytes --]

> On 29 May 2015, at 19:03, Rich Felker <dalias@libc.org> wrote:
> 
> This logic looks wrong. For example, "-L foo" would result in input=1,
> no? And same for any other options with arguments.
> 
> Rich

Hmm, I’m afraid you’re right. I’ll need to rethink this approach...
I was slightly annoyed by it being needed in the first place, but there’s no
other way from what I could see than these markers to figure out what comes
from a user and what doesn’t, since clang re-orders -l and -L arguments.
If they weren’t needed I could just move -lc to the linker wrapper...

-S

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH 1/2] add musl-clang, a wrapper for system clang installs

[Rich Felker]

On Fri, May 29, 2015 at 07:11:18PM +0200, Shiz wrote:
> > On 29 May 2015, at 19:03, Rich Felker <dalias@libc.org> wrote:
> > 
> > This logic looks wrong. For example, "-L foo" would result in input=1,
> > no? And same for any other options with arguments.
> > 
> > Rich
> 
> Hmm, I’m afraid you’re right. I’ll need to rethink this approach...
> I was slightly annoyed by it being needed in the first place, but there’s no
> other way from what I could see than these markers to figure out what comes
> from a user and what doesn’t, since clang re-orders -l and -L arguments.
> If they weren’t needed I could just move -lc to the linker wrapper...

What if you add a bogus prefix to all -l and -L options provided by
the user? Then the wrapper can remove any -l or -L options without the
prefix, then remove the prefix from the ones that remain.

Rich

Re: [PATCH 1/2] add musl-clang, a wrapper for system clang installs

[Shiz]

[-- Attachment #1: Type: text/plain, Size: 795 bytes --]

> On 29 May 2015, at 19:13, Rich Felker <dalias@libc.org> wrote:
> 
> On Fri, May 29, 2015 at 07:11:18PM +0200, Shiz wrote:
> 
> What if you add a bogus prefix to all -l and -L options provided by
> the user? Then the wrapper can remove any -l or -L options without the
> prefix, then remove the prefix from the ones that remain.
> 
> Rich

This isn’t a bad idea, but it modifies the behaviour in the same way
musl-clang does, except the other way around: it will not have clang
invoke ld if only -l is supplied by the user and no object paths are,
which is what normally should happen.

I’m pretty sure I’ve devised a fix that doesn’t modify behaviour and
makes ld.musl-clang a bit cleaner as a side-effect - I’ll post an updated
patch after a bit of testing.

-S

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

[PATCH v2] add musl-clang, a wrapper for system clang installs

[Shiz]

musl-clang allows the user to compile musl-powered programs using their
already existent clang install, without needing a special cross compiler.
it does this by wrapping around both the system clang install and the linker
and passing them special flags to re-target musl at runtime. it does only
affect invocations done through the special musl-clang wrapper script,
so that the user setup remains fully intact otherwise.

the clang wrapper consists of the compiler frontend wrapper script,
musl-clang, and the linker wrapper script, ld.musl-clang.
musl-clang makes sure clang invokes ld.musl-clang to link objects;
neither script needs to be in PATH for the wrapper to work.
---
 .gitignore             |  1 +
 Makefile               |  6 +++++-
 tools/ld.musl-clang.in | 44 +++++++++++++++++++++++++++++++++++++++++++++
 tools/musl-clang.in    | 29 +++++++++++++++++++++++++++++++++++
 4 files changed, 79 insertions(+), 1 deletion(-)
 create mode 100644 tools/ld.musl-clang.in
 create mode 100644 tools/musl-clang.in

diff --git a/.gitignore b/.gitignore
index 070f549..00e4d0a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,5 +7,6 @@ arch/*/bits/alltypes.h
 config.mak
 include/bits
 tools/musl-gcc
+tools/*musl-clang
 lib/musl-gcc.specs
 src/internal/version.h
diff --git a/Makefile b/Makefile
index 3bd7b4d..0537c6a 100644
--- a/Makefile
+++ b/Makefile
@@ -49,7 +49,7 @@ STATIC_LIBS = lib/libc.a
 SHARED_LIBS = lib/libc.so
 TOOL_LIBS = lib/musl-gcc.specs
 ALL_LIBS = $(CRT_LIBS) $(STATIC_LIBS) $(SHARED_LIBS) $(EMPTY_LIBS) $(TOOL_LIBS)
-ALL_TOOLS = tools/musl-gcc
+ALL_TOOLS = tools/musl-gcc tools/musl-clang tools/ld.musl-clang
 
 LDSO_PATHNAME = $(syslibdir)/ld-musl-$(ARCH)$(SUBARCH).so.1
 
@@ -156,6 +156,10 @@ tools/musl-gcc: config.mak
 	printf '#!/bin/sh\nexec "$${REALGCC:-gcc}" "$$@" -specs "%s/musl-gcc.specs"\n' "$(libdir)" > $@
 	chmod +x $@
 
+tools/%-clang: tools/%-clang.in config.mak
+	sed -e 's!@INCDIR@!$(includedir)!g' -e 's!@LIBDIR@!$(libdir)!g' -e 's!@LDSO@!$(LDSO_PATHNAME)!g' $< > $@
+	chmod +x $@
+
 $(DESTDIR)$(bindir)/%: tools/%
 	$(INSTALL) -D $< $@
 
diff --git a/tools/ld.musl-clang.in b/tools/ld.musl-clang.in
new file mode 100644
index 0000000..34f5054
--- /dev/null
+++ b/tools/ld.musl-clang.in
@@ -0,0 +1,44 @@
+#!/bin/sh
+cc=clang
+libc_lib="@LIBDIR@"
+ldso="@LDSO@"
+cleared=
+shared=
+userlinkdir=
+userlink=
+
+for x ; do
+    test "$cleared" || set -- ; cleared=1
+
+    case "$x" in
+        -L-user-start)
+            userlinkdir=1
+            ;;
+        -L-user-end)
+            userlinkdir=
+            ;;
+        -L*)
+            test "$userlinkdir" && set -- "$@" "$x"
+            ;;
+        -l-user-start)
+            userlink=1
+            ;;
+        -l-user-end)
+            userlink=
+            ;;
+        -l*)
+            test "$userlink" && set -- "$@" "$x"
+            ;;
+        -shared)
+            shared=1
+            set -- "$@" -shared
+            ;;
+        *)
+            set -- "$@" "$x"
+            ;;
+    esac
+done
+
+set -- "$@" -lc -dynamic-linker "$ldso"
+test "$shared" || set -- "$libc_lib/Scrt1.o" "$libc_lib/crti.o" "$@" "$libc_lib/crtn.o"
+exec $($cc -print-prog-name=ld) -nostdlib "$@"
diff --git a/tools/musl-clang.in b/tools/musl-clang.in
new file mode 100644
index 0000000..13ef5bb
--- /dev/null
+++ b/tools/musl-clang.in
@@ -0,0 +1,29 @@
+#!/bin/sh
+cc=clang
+libc_inc="@INCDIR@"
+libc_lib="@LIBDIR@"
+thisdir="`cd "$(dirname "$0")"; pwd`"
+
+# prevent clang from running the linker (and erroring) on no input.
+for x ; do
+    case "$x" in
+        -l*) input=1 ;;
+        *) input= ;;
+    esac
+    if test "$input" ; then
+        set -- -l-user-start "$@" -l-user-end
+        break
+    fi
+done
+
+exec $cc \
+    -B"$thisdir" \
+    -fuse-ld=musl-clang \
+    -rtlib=compiler-rt \
+    -nostdinc \
+    -nostartfiles \
+    -isystem "$libc_inc" \
+    -L-user-start \
+    "$@" \
+    -L"$libc_lib" \
+    -L-user-end

Re: [PATCH 2/2] build: overhaul wrapper script system for multiple wrapper support

[Rich Felker]

On Fri, May 29, 2015 at 06:55:12PM +0200, Shiz wrote:
> this overhauls part of the build system in order to support multiple
> toolchain wrapper scripts, as opposed to solely the musl-gcc wrapper as
> before. it thereby replaces --enable-gcc-wrapper with --enable-wrapper=...,
> which has the options 'auto' (the default, detect whether to use wrappers),
> 'all' (build and install all wrappers), 'no' (don't build any) and finally
> the options named after the individual compiler scripts (currently 'clang'
> and 'gcc' are available) to build and install only that wrapper.
> the old --enable-gcc-wrapper is removed from --help, but still available.
> 
> it also modifies the wrappers to use the C compiler specified to the build
> system as 'inner' compiler, when applicable. as wrapper detection works by
> probing this compiler, it may not work with any other.

I believe there's still some change of behavior here that also
violates the principle of least surprise:

For the old option:

yes -- forcibly build wrapper regardless of
compiler or existing-musl-target detection

auto -- build depends on 2 conditions: $CC being gcc and not being
musl-targeted

no -- never build

For the new option:

yes or auto (same) -- build of gcc wrapper depends on 2 conditions:
$CC being gcc and not being musl-targeted. build of clang wrapper
depends only on one condition: $CC being clang.

gcc - unconditionally build gcc wrapper, never build clang wrapper

clang - unconditionally build clang wrapper, never build gcc wrapper

no -- never build

The most surprising part, which I think really should be fixed before
this is committed, is that the clang wrapper is built if you compile
musl with clang on a musl-native system. (Note that the gcc test right
now for musl-native is also wrong if the gcc toolchain omits dynamic
linking support entirely, but that's an existing bug, not a
regression.)

Even if that's fixed, it's also a problem, I think, that yes/auto are
equated. Presumably someone using --enable-wrapper with an existing
musl-targeted toolchain would want to bypass the musl-native detection
and force the wrapper (whichever one is appropriate for their
compiler) to be built. This is especially important to have if the
musl-native test has false positives, which I think it will if we take
the following approach I'd like to take:

Instead of testing for musl-native, test whether the toolchain is
targetting another known non-musl target, which is basically a matter
of #ifdef __GLIBC__. This ensures that the wrapper is never auto-built
for a musl-native system (which could happen before if the musl-native
test failed) and avoids compiler-specific hacks; we can simply have a
general test for known-non-native-toolchain.

If this is acceptable, it would probably make sense to do this change
before adding the clang wrapper infrastructure so that the clang
wrapper tests can use the result of it.

Rich

Re: [PATCH 2/2] build: overhaul wrapper script system for multiple wrapper support

[Shiz]

[-- Attachment #1: Type: text/plain, Size: 1824 bytes --]

> On 01 Jun 2015, at 05:18, Rich Felker <dalias@libc.org> wrote:
> 
> Even if that's fixed, it's also a problem, I think, that yes/auto are
> equated. Presumably someone using --enable-wrapper with an existing
> musl-targeted toolchain would want to bypass the musl-native detection
> and force the wrapper (whichever one is appropriate for their
> compiler) to be built.

You’re right, this is something that should be addressed. I’ll fix that.

> This is especially important to have if the
> musl-native test has false positives, which I think it will if we take
> the following approach I'd like to take:
> 
> Instead of testing for musl-native, test whether the toolchain is
> targetting another known non-musl target, which is basically a matter
> of #ifdef __GLIBC__. This ensures that the wrapper is never auto-built
> for a musl-native system (which could happen before if the musl-native
> test failed) and avoids compiler-specific hacks; we can simply have a
> general test for known-non-native-toolchain.

I’m not sure I’m a big fan of this approach. It’s perfectly reasonable for
targets to exist which are both not musl and don’t define __GLIBC__.
I think a much more reasonable approach would be to check the target triple
($CC -dumpmachine) for *musl* - I believe any compiler which would target
musl systems would have this in its triple right now. The reason why I
omitted the detection in the initial patch was because I wanted some more
time to think the approach over, I should’ve mentioned that in the message.
clang, gcc and cparser all support -dumpmachine, and we already presume a
gcc-ish command line interface for a lot of things, so I wouldn’t see any
harm in taking this approach. pcc doesn’t support this sadly, however.

Any thoughts?

-S

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH 2/2] build: overhaul wrapper script system for multiple wrapper support

[Rich Felker]

On Mon, Jun 01, 2015 at 04:15:36PM +0200, Shiz wrote:
> > On 01 Jun 2015, at 05:18, Rich Felker <dalias@libc.org> wrote:
> > 
> > Even if that's fixed, it's also a problem, I think, that yes/auto are
> > equated. Presumably someone using --enable-wrapper with an existing
> > musl-targeted toolchain would want to bypass the musl-native detection
> > and force the wrapper (whichever one is appropriate for their
> > compiler) to be built.
> 
> You’re right, this is something that should be addressed. I’ll fix that.
> 
> > This is especially important to have if the
> > musl-native test has false positives, which I think it will if we take
> > the following approach I'd like to take:
> > 
> > Instead of testing for musl-native, test whether the toolchain is
> > targetting another known non-musl target, which is basically a matter
> > of #ifdef __GLIBC__. This ensures that the wrapper is never auto-built
> > for a musl-native system (which could happen before if the musl-native
> > test failed) and avoids compiler-specific hacks; we can simply have a
> > general test for known-non-native-toolchain.
> 
> I’m not sure I’m a big fan of this approach. It’s perfectly reasonable for
> targets to exist which are both not musl and don’t define __GLIBC__.
> I think a much more reasonable approach would be to check the target triple
> ($CC -dumpmachine) for *musl* - I believe any compiler which would target
> musl systems would have this in its triple right now. The reason why I
> omitted the detection in the initial patch was because I wanted some more
> time to think the approach over, I should’ve mentioned that in the message.
> clang, gcc and cparser all support -dumpmachine, and we already presume a
> gcc-ish command line interface for a lot of things, so I wouldn’t see any
> harm in taking this approach. pcc doesn’t support this sadly, however.
> 
> Any thoughts?

There are two reasons I prefer the approach I described:

1. It's better not to auto-enable wrappers unless we're pretty
confident they work. The wrappers are not magically universal; they're
a way to use musl with a preexisting non-musl-targeted toolchain that
meets a fairly large set of internal assumptions, and they won't
necessarily work with arbitrary toolchains. In particular I'm pretty
sure musl-gcc does not work with Rob's toolchains from Aboriginal
Linux that are using their own wrapper (named gcc) around an internal
gcc elsewhere, and presumably (being uclibc based) these would even be
detected as "ok for wrapper".

2, A false positive for enabling the wrapper (installing it on a
musl-native system) is worse than a false-negative (omitting it) since
native systems/cross-compilers are intended as the first-class usage
case and wrappers are more of a demo/minor usage case. I can't safe
for sure, but I suspect there might be minor breakage (versus a
full-featured native toolchain) in using the wrappers on top of an
already-native toolchain, and I don't want to ྀmislead users to do this
by installing a script that looks like it's meant to be used.

I don't believe the dumpmachine approach is at all robust for gcc
versions less than gcc 6 where musl support is upstream, and it's
worse than the current dynamic linker check. There are no musl-cross
patches at all for gcc versions prior to 4.0.3, whereas gcc3 works
well and some dists (e.g. Sabotage) are even using it to bootstrap.
This isn't terribly difficult, since the main essential thing that
needs to be patched is the libstdc++ glibcism. Most of the rest of
musl-cross is "re-teaching" gcc that some important features it
enables only for glibc are not glibc-only (but this may not be needed
if gcc already thinks musl is a *-gnu target). In any case I'm pretty
sure there are users of musl using it with toolchains that output
???-linux-gnu as their -dumpmachine, and I don't expect to see this
phased out quickly.

Rich

Re: [PATCH 2/2] build: overhaul wrapper script system for multiple wrapper support

[Shiz]

[-- Attachment #1: Type: text/plain, Size: 2633 bytes --]

> On 01 Jun 2015, at 16:47, Rich Felker <dalias@libc.org> wrote:
> 
> There are two reasons I prefer the approach I described:
> 
> 1. It's better not to auto-enable wrappers unless we're pretty
> confident they work. The wrappers are not magically universal; they're
> a way to use musl with a preexisting non-musl-targeted toolchain that
> meets a fairly large set of internal assumptions, and they won't
> necessarily work with arbitrary toolchains. In particular I'm pretty
> sure musl-gcc does not work with Rob's toolchains from Aboriginal
> Linux that are using their own wrapper (named gcc) around an internal
> gcc elsewhere, and presumably (being uclibc based) these would even be
> detected as "ok for wrapper”.

I’d like to note that a __GLIBC__ check would not help here either
presumably, as uclibc defines __GLIBC__ as well. :)

> 2, A false positive for enabling the wrapper (installing it on a
> musl-native system) is worse than a false-negative (omitting it) since
> native systems/cross-compilers are intended as the first-class usage
> case and wrappers are more of a demo/minor usage case. I can't safe
> for sure, but I suspect there might be minor breakage (versus a
> full-featured native toolchain) in using the wrappers on top of an
> already-native toolchain, and I don't want to ྀmislead users to do this
> by installing a script that looks like it's meant to be used.
> 
> I don't believe the dumpmachine approach is at all robust for gcc
> versions less than gcc 6 where musl support is upstream, and it's
> worse than the current dynamic linker check. There are no musl-cross
> patches at all for gcc versions prior to 4.0.3, whereas gcc3 works
> well and some dists (e.g. Sabotage) are even using it to bootstrap.

Understood, I wasn’t aware of this. On Gentoo’s musl profile, the gcc
CHOST is reconfigured to $arch-gentoo-linux-musl, so -dumpmachine works
as expected on every gcc it builds:

# gcc -dumpmachine
x86_64-gentoo-linux-musl

I assumed other musl distros would do something similar.

So I see your use case for testing in C code itself as opposed to testing
compiler features. That being said, I’m still not at all a fan of __GLIBC__,
for reasons mentioned in the previous post. While a false-positive is worse
than a false-negative, I feel just solely testing for this provides a large
opportunity for false-negatives, and even some false positives as you yourself
mentioned in point one.

It’s a tough thing to check thoroughly and accurately, but I do not think
checking __GLIBC__ is at all the solution either.

-S


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH 2/2] build: overhaul wrapper script system for multiple wrapper support

[Rich Felker]

On Mon, Jun 01, 2015 at 05:39:46PM +0200, Shiz wrote:
> > On 01 Jun 2015, at 16:47, Rich Felker <dalias@libc.org> wrote:
> > 
> > There are two reasons I prefer the approach I described:
> > 
> > 1. It's better not to auto-enable wrappers unless we're pretty
> > confident they work. The wrappers are not magically universal; they're
> > a way to use musl with a preexisting non-musl-targeted toolchain that
> > meets a fairly large set of internal assumptions, and they won't
> > necessarily work with arbitrary toolchains. In particular I'm pretty
> > sure musl-gcc does not work with Rob's toolchains from Aboriginal
> > Linux that are using their own wrapper (named gcc) around an internal
> > gcc elsewhere, and presumably (being uclibc based) these would even be
> > detected as "ok for wrapper”.
> 
> I’d like to note that a __GLIBC__ check would not help here either
> presumably, as uclibc defines __GLIBC__ as well. :)

Yes, that's what I was trying to say.

> So I see your use case for testing in C code itself as opposed to testing
> compiler features. That being said, I’m still not at all a fan of __GLIBC__,
> for reasons mentioned in the previous post. While a false-positive is worse
> than a false-negative, I feel just solely testing for this provides a large
> opportunity for false-negatives, and even some false positives as you yourself
> mentioned in point one.

I'm not aware of any real-world false-negatives. A Bionic-based system
would probably be one, if any such system with a compiler toolchain
exists, but depending on the properties of the toolchain that might
even be a proper negative rather than a false-negative, so I'd
actually prefer to research whether the wrappers work for such
Bionic-based toolchains and then enable them explicitly if they do,
rather than just assuming they work.

> It’s a tough thing to check thoroughly and accurately, but I do not think
> checking __GLIBC__ is at all the solution either.

Still open to more options.

Rich