Unable to build with --enable-shared

Unable to build with --enable-shared

[Samuel Sadok]

Hi!

I am unable to build musl with --enable-shared. Multiple issues in
unrelated projects (e.g. here: https://bugs.swift.org/browse/SR-1023)
suggest that the culprit is a change in behaviour between binutils
2.25 and 2.26.


This is what I did:

$ make clean
$ ./configure --prefix=/usr/packages/musl --target=x86_64-pc-linux-gnu
CFLAGS='-B/usr/local/bin/x86_64-pc-linux-gnu-' --enable-static
--enable-shared --disable-gcc-wrapper
$ make
[...]
x86_64-pc-linux-gnu-gcc  -D_XOPEN_SOURCE=700 -I./arch/x86_64
-I./arch/generic -Iobj/src/internal -I./src/internal -Iobj/include
-I./include  -include vis.h -B/usr/local/bin/x86_64-pc-linux-gnu-
-fPIC -c -o obj/src/process/posix_spawn.lo src/process/posix_spawn.c
[...]
x86_64-pc-linux-gnu-gcc  -D_XOPEN_SOURCE=700 -I./arch/x86_64
-I./arch/generic -Iobj/src/internal -I./src/internal -Iobj/include
-I./include  -include vis.h -B/usr/local/bin/x86_64-pc-linux-gnu-
-nostdlib -shared -Wl,-e,_dlstart -o lib/libc.so obj/src/aio/aio.lo [a
ton of objects] obj/ldso/dynlink.lo
/usr/local/bin/x86_64-pc-linux-gnu-ld: obj/src/process/posix_spawn.lo:
relocation R_X86_64_PC32 against protected symbol `execve' can not be
used when making a shared object
/usr/local/bin/x86_64-pc-linux-gnu-ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make: *** [lib/libc.so] Error 1


This is the toolchain I am using:

$ /usr/local/bin/x86_64-pc-linux-gnu-ld -v
GNU ld (GNU Binutils) 2.27

$ x86_64-pc-linux-gnu-gcc -v
Using built-in specs.
COLLECT_GCC=/usr/local/bin/x86_64-pc-linux-gnu-gcc-original
COLLECT_LTO_WRAPPER=/usr/packages/gcc-x86_64-pc-linux-gnu/libexec/gcc/x86_64-pc-linux-gnu/6.1.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /Data/Projects/Linux/gcc/fetch/gcc-6.1.0/configure
--enable-languages=all --without-headers --disable-multilib
--disable-shared --enable-static --target=x86_64-pc-linux-gnu
--with-gmp=/usr/packages/gmp --with-mpfr=/usr/packages/mpfr
--with-mpc=/usr/packages/mpc
--prefix=/usr/packages/gcc-x86_64-pc-linux-gnu : (reconfigured)
/Data/Projects/Linux/gcc/fetch/gcc-6.1.0/configure
--enable-languages=all --without-headers --disable-multilib
--disable-shared --enable-static --target=x86_64-pc-linux-gnu
--with-gmp=/usr/packages/gmp --with-mpfr=/usr/packages/mpfr
--with-mpc=/usr/packages/mpc
--prefix=/usr/packages/gcc-x86_64-pc-linux-gnu
Thread model: posix
gcc version 6.1.0 (GCC)


Can anyone confirm this issue with binutils >= 2.26? It seems strange
to me that this would not have come up yet, so maybe someone has an
idea what else might be the problem?


Samuel

Re: Unable to build with --enable-shared

[Szabolcs Nagy]

* Samuel Sadok <innovation-labs@appinstall.ch> [2016-10-22 22:37:46 +0200]:
> I am unable to build musl with --enable-shared. Multiple issues in
> unrelated projects (e.g. here: https://bugs.swift.org/browse/SR-1023)
> suggest that the culprit is a change in behaviour between binutils
> 2.25 and 2.26.
> 

that bug is different, that's about object symbols

(swift seems to misuse protected visibility objects,
and instead of fixing the problem they switched to the
gold linker which does not yet have the bfd linker fix,
such incompetence is frustrating..  however the issue
does not affect musl: we don't mark objects protected
to avoid issues with broken toolchains.)

> $ make
> [...]
> x86_64-pc-linux-gnu-gcc  -D_XOPEN_SOURCE=700 -I./arch/x86_64
> -I./arch/generic -Iobj/src/internal -I./src/internal -Iobj/include
> -I./include  -include vis.h -B/usr/local/bin/x86_64-pc-linux-gnu-
> -fPIC -c -o obj/src/process/posix_spawn.lo src/process/posix_spawn.c

some flags are missing here.. e.g. freestanding flags

> [...]
> x86_64-pc-linux-gnu-gcc  -D_XOPEN_SOURCE=700 -I./arch/x86_64
> -I./arch/generic -Iobj/src/internal -I./src/internal -Iobj/include
> -I./include  -include vis.h -B/usr/local/bin/x86_64-pc-linux-gnu-
> -nostdlib -shared -Wl,-e,_dlstart -o lib/libc.so obj/src/aio/aio.lo [a
> ton of objects] obj/ldso/dynlink.lo
> /usr/local/bin/x86_64-pc-linux-gnu-ld: obj/src/process/posix_spawn.lo:
> relocation R_X86_64_PC32 against protected symbol `execve' can not be
> used when making a shared object
> /usr/local/bin/x86_64-pc-linux-gnu-ld: final link failed: Bad value
> collect2: error: ld returned 1 exit status
> make: *** [lib/libc.so] Error 1
> 

execve is a protected function symbol, so it should be bound locally,
i think R_X86_64_PC32 is ok for that, so it's not clear to me what
goes wrong..

Re: Unable to build with --enable-shared

[Samuel Sadok]

2016-10-22 23:58 GMT+02:00 Szabolcs Nagy <nsz@port70.net>:
> * Samuel Sadok <innovation-labs@appinstall.ch> [2016-10-22 22:37:46 +0200]:
>> I am unable to build musl with --enable-shared. Multiple issues in
>> unrelated projects (e.g. here: https://bugs.swift.org/browse/SR-1023)
>> suggest that the culprit is a change in behaviour between binutils
>> 2.25 and 2.26.
>>
>
> that bug is different, that's about object symbols
>
> (swift seems to misuse protected visibility objects,
> and instead of fixing the problem they switched to the
> gold linker which does not yet have the bfd linker fix,
> such incompetence is frustrating..  however the issue
> does not affect musl: we don't mark objects protected
> to avoid issues with broken toolchains.)

That's precisely why I'm trying to get away from Apple.

>
>> $ make
>> [...]
>> x86_64-pc-linux-gnu-gcc  -D_XOPEN_SOURCE=700 -I./arch/x86_64
>> -I./arch/generic -Iobj/src/internal -I./src/internal -Iobj/include
>> -I./include  -include vis.h -B/usr/local/bin/x86_64-pc-linux-gnu-
>> -fPIC -c -o obj/src/process/posix_spawn.lo src/process/posix_spawn.c
>
> some flags are missing here.. e.g. freestanding flags

Thanks for the pointer, that turned out to be the problem.

I should mention that I am cross-compiling from macOS. Since I had a
Linux VM flying around, I tried to build musl there (success) and
compared the logs.
Looking at config.mak, there apparently was a serious misconfiguration:

config.mak on the cross-build system (macOS):

CFLAGS_AUTO = -include vis.h
CFLAGS_C99FSE =
CFLAGS_MEMOPS =
CFLAGS_NOSSP =
LDFLAGS_AUTO =
LIBCC =

config.mak on Ubuntu:

CFLAGS_AUTO = -Os -pipe -fomit-frame-pointer -fno-unwind-tables
-fno-asynchronous-unwind-tables -ffunction-sections -fdata-sections
-Werror=implicit-function-declaration -Werror=implicit-int
-Werror=pointer-sign -Werror=pointer-arith -include vis.h
CFLAGS_C99FSE = -std=c99 -nostdinc -ffreestanding
-fexcess-precision=standard -frounding-math -Wa,--noexecstack
CFLAGS_MEMOPS = -fno-tree-loop-distribute-patterns
CFLAGS_NOSSP = -fno-stack-protector
LDFLAGS_AUTO = -Wl,--sort-section,alignment -Wl,--sort-common
-Wl,--gc-sections -Wl,--hash-style=both -Wl,--no-undefined
-Wl,--exclude-libs=ALL -Wl,-Bsymbolic-functions
LIBCC = -lgcc -lgcc_eh

After adding the missing flags (except the seemingly unused -lgcc_eh),
the build succeeded on macOS.

I'm not sure if this can be fixed cleanly by passing appropriate
arguments to configure or if the script should be patched. My
experience with configure scripts is limited, but I will try to look
further into what went wrong exactly.

Re: Unable to build with --enable-shared

[Rich Felker]

On Sun, Oct 23, 2016 at 02:22:28AM +0200, Samuel Sadok wrote:
> 2016-10-22 23:58 GMT+02:00 Szabolcs Nagy <nsz@port70.net>:
> > * Samuel Sadok <innovation-labs@appinstall.ch> [2016-10-22 22:37:46 +0200]:
> >> I am unable to build musl with --enable-shared. Multiple issues in
> >> unrelated projects (e.g. here: https://bugs.swift.org/browse/SR-1023)
> >> suggest that the culprit is a change in behaviour between binutils
> >> 2.25 and 2.26.
> >>
> >
> > that bug is different, that's about object symbols
> >
> > (swift seems to misuse protected visibility objects,
> > and instead of fixing the problem they switched to the
> > gold linker which does not yet have the bfd linker fix,
> > such incompetence is frustrating..  however the issue
> > does not affect musl: we don't mark objects protected
> > to avoid issues with broken toolchains.)
> 
> That's precisely why I'm trying to get away from Apple.
> 
> >
> >> $ make
> >> [...]
> >> x86_64-pc-linux-gnu-gcc  -D_XOPEN_SOURCE=700 -I./arch/x86_64
> >> -I./arch/generic -Iobj/src/internal -I./src/internal -Iobj/include
> >> -I./include  -include vis.h -B/usr/local/bin/x86_64-pc-linux-gnu-
> >> -fPIC -c -o obj/src/process/posix_spawn.lo src/process/posix_spawn.c
> >
> > some flags are missing here.. e.g. freestanding flags
> 
> Thanks for the pointer, that turned out to be the problem.
> 
> I should mention that I am cross-compiling from macOS. Since I had a
> Linux VM flying around, I tried to build musl there (success) and
> compared the logs.
> Looking at config.mak, there apparently was a serious misconfiguration:
> 
> config.mak on the cross-build system (macOS):
> 
> CFLAGS_AUTO = -include vis.h
> CFLAGS_C99FSE =
> CFLAGS_MEMOPS =
> CFLAGS_NOSSP =
> LDFLAGS_AUTO =
> LIBCC =

This probably indicates something is wrong with your cross toolchain;
even if you manually fix config.mak, I would be concerned that
something might have been built wrong. What cross toolchain are you
using and how was it setup? How did you invoke configure?

Rich

Re: Unable to build with --enable-shared

[Samuel Sadok]

2016-10-23 18:17 GMT+02:00 Rich Felker <dalias@libc.org>:
> On Sun, Oct 23, 2016 at 02:22:28AM +0200, Samuel Sadok wrote:
>> 2016-10-22 23:58 GMT+02:00 Szabolcs Nagy <nsz@port70.net>:
>> > * Samuel Sadok <innovation-labs@appinstall.ch> [2016-10-22 22:37:46 +0200]:
>> >> I am unable to build musl with --enable-shared. Multiple issues in
>> >> unrelated projects (e.g. here: https://bugs.swift.org/browse/SR-1023)
>> >> suggest that the culprit is a change in behaviour between binutils
>> >> 2.25 and 2.26.
>> >>
>> >
>> > that bug is different, that's about object symbols
>> >
>> > (swift seems to misuse protected visibility objects,
>> > and instead of fixing the problem they switched to the
>> > gold linker which does not yet have the bfd linker fix,
>> > such incompetence is frustrating..  however the issue
>> > does not affect musl: we don't mark objects protected
>> > to avoid issues with broken toolchains.)
>>
>> That's precisely why I'm trying to get away from Apple.
>>
>> >
>> >> $ make
>> >> [...]
>> >> x86_64-pc-linux-gnu-gcc  -D_XOPEN_SOURCE=700 -I./arch/x86_64
>> >> -I./arch/generic -Iobj/src/internal -I./src/internal -Iobj/include
>> >> -I./include  -include vis.h -B/usr/local/bin/x86_64-pc-linux-gnu-
>> >> -fPIC -c -o obj/src/process/posix_spawn.lo src/process/posix_spawn.c
>> >
>> > some flags are missing here.. e.g. freestanding flags
>>
>> Thanks for the pointer, that turned out to be the problem.
>>
>> I should mention that I am cross-compiling from macOS. Since I had a
>> Linux VM flying around, I tried to build musl there (success) and
>> compared the logs.
>> Looking at config.mak, there apparently was a serious misconfiguration:
>>
>> config.mak on the cross-build system (macOS):
>>
>> CFLAGS_AUTO = -include vis.h
>> CFLAGS_C99FSE =
>> CFLAGS_MEMOPS =
>> CFLAGS_NOSSP =
>> LDFLAGS_AUTO =
>> LIBCC =
>
> This probably indicates something is wrong with your cross toolchain;
> even if you manually fix config.mak, I would be concerned that
> something might have been built wrong. What cross toolchain are you
> using and how was it setup? How did you invoke configure?
>
> Rich

I managed to track down the problem.

I'm using a cross-toolchain I built from source with a slight
misconfiguration. Specifically, I must tell it where it is
(-B/usr/local/bin/x86_64-pc-linux-gnu-), otherwise it uses the
(incompatible) default linker.

Now when the configure script checks whether some flag is working, it
forgets to pass the user-defined $CFLAGS to the compiler, which then
fails:

tryflag () {
printf "checking whether compiler accepts %s... " "$2"
echo "typedef int x;" > "$tmpc"
if $CC $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
[...]

Thus, we end up with no flags at all.

Even though I'm aware that my setup is quite non-standard, I would
consider this a bug in the configure script. I appended a patch that
fixes the problem in all places where I spotted it. Using this patch I
am able to configure and build musl as expected.
Does this seem reasonable to you?



From 084678f6c93ed0bf305ea0fbb35a33810c4c9ccc Mon Sep 17 00:00:00 2001
From: Samuel Sadok <innovation-labs@appinstall.ch>
Date: Mon, 24 Oct 2016 00:10:04 +0200
Subject: [PATCH] Pass CFLAGS to compiler under all circumstances in configure
 script

---
 configure | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/configure b/configure
index 707eb12..c1466ce 100755
--- a/configure
+++ b/configure
@@ -72,7 +72,7 @@ echo "typedef int x;" > "$tmpc"
 echo "#if $1" >> "$tmpc"
 echo "#error yes" >> "$tmpc"
 echo "#endif" >> "$tmpc"
-if $CC $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
+if $CC $CFLAGS $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
 printf "false\n"
 return 1
 else
@@ -84,7 +84,7 @@ fi
 tryflag () {
 printf "checking whether compiler accepts %s... " "$2"
 echo "typedef int x;" > "$tmpc"
-if $CC $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
+if $CC $CFLAGS $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
 printf "yes\n"
 eval "$1=\"\${$1} \$2\""
 eval "$1=\${$1# }"
@@ -98,7 +98,7 @@ fi
 tryldflag () {
 printf "checking whether linker accepts %s... " "$2"
 echo "typedef int x;" > "$tmpc"
-if $CC $LDFLAGS_TRY -nostdlib -shared "$2" -o /dev/null "$tmpc"
>/dev/null 2>&1 ; then
+if $CC $CFLAGS $LDFLAGS_TRY -nostdlib -shared "$2" -o /dev/null
"$tmpc" >/dev/null 2>&1 ; then
 printf "yes\n"
 eval "$1=\"\${$1} \$2\""
 eval "$1=\${$1# }"
@@ -275,7 +275,7 @@ echo "#if ! __GLIBC__" >> "$tmpc"
 echo "#error no" >> "$tmpc"
 echo "#endif" >> "$tmpc"
 printf "checking for toolchain wrapper to build... "
-if test "$wrapper" = auto && ! $CC -c -o /dev/null "$tmpc" >/dev/null
2>&1 ; then
+if test "$wrapper" = auto && ! $CC $CFLAGS -c -o /dev/null "$tmpc"
>/dev/null 2>&1 ; then
 echo "none"
 elif test "$cc_family" = gcc ; then
 gcc_wrapper=yes
@@ -303,7 +303,7 @@ fi
 # Find the target architecture
 #
 printf "checking target system type... "
-test -n "$target" || target=$($CC -dumpmachine 2>/dev/null) || target=unknown
+test -n "$target" || target=$($CC $CFLAGS -dumpmachine 2>/dev/null)
|| target=unknown
 printf "%s\n" "$target"

 #
@@ -397,7 +397,7 @@ test "$debug" = yes && CFLAGS_AUTO=-g
 printf "checking whether we should preprocess assembly to add
debugging information... "
 if fnmatch '-g*|*\ -g*' "$CFLAGS_AUTO $CFLAGS" &&
    test -f "tools/add-cfi.$ARCH.awk" &&
-   printf ".file 1 \"srcfile.s\"\n.line
1\n.cfi_startproc\n.cfi_endproc" | $CC -g -x assembler -c -o /dev/null
2>/dev/null -
+   printf ".file 1 \"srcfile.s\"\n.line
1\n.cfi_startproc\n.cfi_endproc" | $CC $CFLAGS -g -x assembler -c -o
/dev/null 2>/dev/null -
 then
   ADD_CFI=yes
 else
@@ -588,7 +588,7 @@ tryldflag LDFLAGS_AUTO -Wl,-Bsymbolic-functions
 # Find compiler runtime library
 test -z "$LIBCC" && tryldflag LIBCC -lgcc && tryldflag LIBCC -lgcc_eh
 test -z "$LIBCC" && tryldflag LIBCC -lcompiler_rt
-test -z "$LIBCC" && try_libcc=`$CC -print-file-name=libpcc.a 2>/dev/null` \
+test -z "$LIBCC" && try_libcc=`$CC $CFLAGS -print-file-name=libpcc.a
2>/dev/null` \
                  && tryldflag LIBCC "$try_libcc"
 printf "using compiler runtime libraries: %s\n" "$LIBCC"

@@ -719,6 +719,7 @@ printf "creating config.mak... "
 cmdline=$(quote "$0")
 for i ; do cmdline="$cmdline $(quote "$i")" ; done

+
 exec 3>&1 1>config.mak


-- 
2.8.1

Re: Unable to build with --enable-shared

[Laine Gholson]

I suggest that you put '-B/usr/local/bin/x86_64-pc-linux-gnu', in $CC, because it is needed for the C compiler to run correctly, e.g CC="x86_64-pc-linux-gnu-gcc -B/usr/local/bin/x86_64-pc-linux-gnu", and that will work fine with the current configure script.

On 10/23/16 17:20, Samuel Sadok wrote:
> 2016-10-23 18:17 GMT+02:00 Rich Felker <dalias@libc.org>:
>> On Sun, Oct 23, 2016 at 02:22:28AM +0200, Samuel Sadok wrote:
>>> 2016-10-22 23:58 GMT+02:00 Szabolcs Nagy <nsz@port70.net>:
>>>> * Samuel Sadok <innovation-labs@appinstall.ch> [2016-10-22 22:37:46 +0200]:
>>>>> I am unable to build musl with --enable-shared. Multiple issues in
>>>>> unrelated projects (e.g. here: https://bugs.swift.org/browse/SR-1023)
>>>>> suggest that the culprit is a change in behaviour between binutils
>>>>> 2.25 and 2.26.
>>>>>
>>>>
>>>> that bug is different, that's about object symbols
>>>>
>>>> (swift seems to misuse protected visibility objects,
>>>> and instead of fixing the problem they switched to the
>>>> gold linker which does not yet have the bfd linker fix,
>>>> such incompetence is frustrating..  however the issue
>>>> does not affect musl: we don't mark objects protected
>>>> to avoid issues with broken toolchains.)
>>>
>>> That's precisely why I'm trying to get away from Apple.
>>>
>>>>
>>>>> $ make
>>>>> [...]
>>>>> x86_64-pc-linux-gnu-gcc  -D_XOPEN_SOURCE=700 -I./arch/x86_64
>>>>> -I./arch/generic -Iobj/src/internal -I./src/internal -Iobj/include
>>>>> -I./include  -include vis.h -B/usr/local/bin/x86_64-pc-linux-gnu-
>>>>> -fPIC -c -o obj/src/process/posix_spawn.lo src/process/posix_spawn.c
>>>>
>>>> some flags are missing here.. e.g. freestanding flags
>>>
>>> Thanks for the pointer, that turned out to be the problem.
>>>
>>> I should mention that I am cross-compiling from macOS. Since I had a
>>> Linux VM flying around, I tried to build musl there (success) and
>>> compared the logs.
>>> Looking at config.mak, there apparently was a serious misconfiguration:
>>>
>>> config.mak on the cross-build system (macOS):
>>>
>>> CFLAGS_AUTO = -include vis.h
>>> CFLAGS_C99FSE =
>>> CFLAGS_MEMOPS =
>>> CFLAGS_NOSSP =
>>> LDFLAGS_AUTO =
>>> LIBCC =
>>
>> This probably indicates something is wrong with your cross toolchain;
>> even if you manually fix config.mak, I would be concerned that
>> something might have been built wrong. What cross toolchain are you
>> using and how was it setup? How did you invoke configure?
>>
>> Rich
>
> I managed to track down the problem.
>
> I'm using a cross-toolchain I built from source with a slight
> misconfiguration. Specifically, I must tell it where it is
> (-B/usr/local/bin/x86_64-pc-linux-gnu-), otherwise it uses the
> (incompatible) default linker.
>
> Now when the configure script checks whether some flag is working, it
> forgets to pass the user-defined $CFLAGS to the compiler, which then
> fails:
>
> tryflag () {
> printf "checking whether compiler accepts %s... " "$2"
> echo "typedef int x;" > "$tmpc"
> if $CC $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
> [...]
>
> Thus, we end up with no flags at all.
>
> Even though I'm aware that my setup is quite non-standard, I would
> consider this a bug in the configure script. I appended a patch that
> fixes the problem in all places where I spotted it. Using this patch I
> am able to configure and build musl as expected.
> Does this seem reasonable to you?
>
>
>
> From 084678f6c93ed0bf305ea0fbb35a33810c4c9ccc Mon Sep 17 00:00:00 2001
> From: Samuel Sadok <innovation-labs@appinstall.ch>
> Date: Mon, 24 Oct 2016 00:10:04 +0200
> Subject: [PATCH] Pass CFLAGS to compiler under all circumstances in configure
>  script
>
> ---
>  configure | 15 ++++++++-------
>  1 file changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/configure b/configure
> index 707eb12..c1466ce 100755
> --- a/configure
> +++ b/configure
> @@ -72,7 +72,7 @@ echo "typedef int x;" > "$tmpc"
>  echo "#if $1" >> "$tmpc"
>  echo "#error yes" >> "$tmpc"
>  echo "#endif" >> "$tmpc"
> -if $CC $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
> +if $CC $CFLAGS $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
>  printf "false\n"
>  return 1
>  else
> @@ -84,7 +84,7 @@ fi
>  tryflag () {
>  printf "checking whether compiler accepts %s... " "$2"
>  echo "typedef int x;" > "$tmpc"
> -if $CC $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
> +if $CC $CFLAGS $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
>  printf "yes\n"
>  eval "$1=\"\${$1} \$2\""
>  eval "$1=\${$1# }"
> @@ -98,7 +98,7 @@ fi
>  tryldflag () {
>  printf "checking whether linker accepts %s... " "$2"
>  echo "typedef int x;" > "$tmpc"
> -if $CC $LDFLAGS_TRY -nostdlib -shared "$2" -o /dev/null "$tmpc"
>> /dev/null 2>&1 ; then
> +if $CC $CFLAGS $LDFLAGS_TRY -nostdlib -shared "$2" -o /dev/null
> "$tmpc" >/dev/null 2>&1 ; then
>  printf "yes\n"
>  eval "$1=\"\${$1} \$2\""
>  eval "$1=\${$1# }"
> @@ -275,7 +275,7 @@ echo "#if ! __GLIBC__" >> "$tmpc"
>  echo "#error no" >> "$tmpc"
>  echo "#endif" >> "$tmpc"
>  printf "checking for toolchain wrapper to build... "
> -if test "$wrapper" = auto && ! $CC -c -o /dev/null "$tmpc" >/dev/null
> 2>&1 ; then
> +if test "$wrapper" = auto && ! $CC $CFLAGS -c -o /dev/null "$tmpc"
>> /dev/null 2>&1 ; then
>  echo "none"
>  elif test "$cc_family" = gcc ; then
>  gcc_wrapper=yes
> @@ -303,7 +303,7 @@ fi
>  # Find the target architecture
>  #
>  printf "checking target system type... "
> -test -n "$target" || target=$($CC -dumpmachine 2>/dev/null) || target=unknown
> +test -n "$target" || target=$($CC $CFLAGS -dumpmachine 2>/dev/null)
> || target=unknown
>  printf "%s\n" "$target"
>
>  #
> @@ -397,7 +397,7 @@ test "$debug" = yes && CFLAGS_AUTO=-g
>  printf "checking whether we should preprocess assembly to add
> debugging information... "
>  if fnmatch '-g*|*\ -g*' "$CFLAGS_AUTO $CFLAGS" &&
>     test -f "tools/add-cfi.$ARCH.awk" &&
> -   printf ".file 1 \"srcfile.s\"\n.line
> 1\n.cfi_startproc\n.cfi_endproc" | $CC -g -x assembler -c -o /dev/null
> 2>/dev/null -
> +   printf ".file 1 \"srcfile.s\"\n.line
> 1\n.cfi_startproc\n.cfi_endproc" | $CC $CFLAGS -g -x assembler -c -o
> /dev/null 2>/dev/null -
>  then
>    ADD_CFI=yes
>  else
> @@ -588,7 +588,7 @@ tryldflag LDFLAGS_AUTO -Wl,-Bsymbolic-functions
>  # Find compiler runtime library
>  test -z "$LIBCC" && tryldflag LIBCC -lgcc && tryldflag LIBCC -lgcc_eh
>  test -z "$LIBCC" && tryldflag LIBCC -lcompiler_rt
> -test -z "$LIBCC" && try_libcc=`$CC -print-file-name=libpcc.a 2>/dev/null` \
> +test -z "$LIBCC" && try_libcc=`$CC $CFLAGS -print-file-name=libpcc.a
> 2>/dev/null` \
>                   && tryldflag LIBCC "$try_libcc"
>  printf "using compiler runtime libraries: %s\n" "$LIBCC"
>
> @@ -719,6 +719,7 @@ printf "creating config.mak... "
>  cmdline=$(quote "$0")
>  for i ; do cmdline="$cmdline $(quote "$i")" ; done
>
> +
>  exec 3>&1 1>config.mak
>
>

Re: Unable to build with --enable-shared

[Rich Felker]

On Sun, Oct 23, 2016 at 07:19:02PM -0500, Laine Gholson wrote:
> I suggest that you put '-B/usr/local/bin/x86_64-pc-linux-gnu', in
> $CC, because it is needed for the C compiler to run correctly, e.g
> CC="x86_64-pc-linux-gnu-gcc -B/usr/local/bin/x86_64-pc-linux-gnu",
> and that will work fine with the current configure script.

Indeed. Options which are of the class "the compiler driver does not
work at all, or does not have the right target/ABI, without this
option" belong in $CC. This also applies to things like -m32.

In Samuel's case, though, I think the cross compiler was just built
incorrectly if it's not searching the right tool path by default.
Maybe it doesn't realize it's a cross compiler or something.

Rich


> On 10/23/16 17:20, Samuel Sadok wrote:
> >2016-10-23 18:17 GMT+02:00 Rich Felker <dalias@libc.org>:
> >>On Sun, Oct 23, 2016 at 02:22:28AM +0200, Samuel Sadok wrote:
> >>>2016-10-22 23:58 GMT+02:00 Szabolcs Nagy <nsz@port70.net>:
> >>>>* Samuel Sadok <innovation-labs@appinstall.ch> [2016-10-22 22:37:46 +0200]:
> >>>>>I am unable to build musl with --enable-shared. Multiple issues in
> >>>>>unrelated projects (e.g. here: https://bugs.swift.org/browse/SR-1023)
> >>>>>suggest that the culprit is a change in behaviour between binutils
> >>>>>2.25 and 2.26.
> >>>>>
> >>>>
> >>>>that bug is different, that's about object symbols
> >>>>
> >>>>(swift seems to misuse protected visibility objects,
> >>>>and instead of fixing the problem they switched to the
> >>>>gold linker which does not yet have the bfd linker fix,
> >>>>such incompetence is frustrating..  however the issue
> >>>>does not affect musl: we don't mark objects protected
> >>>>to avoid issues with broken toolchains.)
> >>>
> >>>That's precisely why I'm trying to get away from Apple.
> >>>
> >>>>
> >>>>>$ make
> >>>>>[...]
> >>>>>x86_64-pc-linux-gnu-gcc  -D_XOPEN_SOURCE=700 -I./arch/x86_64
> >>>>>-I./arch/generic -Iobj/src/internal -I./src/internal -Iobj/include
> >>>>>-I./include  -include vis.h -B/usr/local/bin/x86_64-pc-linux-gnu-
> >>>>>-fPIC -c -o obj/src/process/posix_spawn.lo src/process/posix_spawn.c
> >>>>
> >>>>some flags are missing here.. e.g. freestanding flags
> >>>
> >>>Thanks for the pointer, that turned out to be the problem.
> >>>
> >>>I should mention that I am cross-compiling from macOS. Since I had a
> >>>Linux VM flying around, I tried to build musl there (success) and
> >>>compared the logs.
> >>>Looking at config.mak, there apparently was a serious misconfiguration:
> >>>
> >>>config.mak on the cross-build system (macOS):
> >>>
> >>>CFLAGS_AUTO = -include vis.h
> >>>CFLAGS_C99FSE =
> >>>CFLAGS_MEMOPS =
> >>>CFLAGS_NOSSP =
> >>>LDFLAGS_AUTO =
> >>>LIBCC =
> >>
> >>This probably indicates something is wrong with your cross toolchain;
> >>even if you manually fix config.mak, I would be concerned that
> >>something might have been built wrong. What cross toolchain are you
> >>using and how was it setup? How did you invoke configure?
> >>
> >>Rich
> >
> >I managed to track down the problem.
> >
> >I'm using a cross-toolchain I built from source with a slight
> >misconfiguration. Specifically, I must tell it where it is
> >(-B/usr/local/bin/x86_64-pc-linux-gnu-), otherwise it uses the
> >(incompatible) default linker.
> >
> >Now when the configure script checks whether some flag is working, it
> >forgets to pass the user-defined $CFLAGS to the compiler, which then
> >fails:
> >
> >tryflag () {
> >printf "checking whether compiler accepts %s... " "$2"
> >echo "typedef int x;" > "$tmpc"
> >if $CC $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
> >[...]
> >
> >Thus, we end up with no flags at all.
> >
> >Even though I'm aware that my setup is quite non-standard, I would
> >consider this a bug in the configure script. I appended a patch that
> >fixes the problem in all places where I spotted it. Using this patch I
> >am able to configure and build musl as expected.
> >Does this seem reasonable to you?
> >
> >
> >
> >From 084678f6c93ed0bf305ea0fbb35a33810c4c9ccc Mon Sep 17 00:00:00 2001
> >From: Samuel Sadok <innovation-labs@appinstall.ch>
> >Date: Mon, 24 Oct 2016 00:10:04 +0200
> >Subject: [PATCH] Pass CFLAGS to compiler under all circumstances in configure
> > script
> >
> >---
> > configure | 15 ++++++++-------
> > 1 file changed, 8 insertions(+), 7 deletions(-)
> >
> >diff --git a/configure b/configure
> >index 707eb12..c1466ce 100755
> >--- a/configure
> >+++ b/configure
> >@@ -72,7 +72,7 @@ echo "typedef int x;" > "$tmpc"
> > echo "#if $1" >> "$tmpc"
> > echo "#error yes" >> "$tmpc"
> > echo "#endif" >> "$tmpc"
> >-if $CC $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
> >+if $CC $CFLAGS $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
> > printf "false\n"
> > return 1
> > else
> >@@ -84,7 +84,7 @@ fi
> > tryflag () {
> > printf "checking whether compiler accepts %s... " "$2"
> > echo "typedef int x;" > "$tmpc"
> >-if $CC $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
> >+if $CC $CFLAGS $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
> > printf "yes\n"
> > eval "$1=\"\${$1} \$2\""
> > eval "$1=\${$1# }"
> >@@ -98,7 +98,7 @@ fi
> > tryldflag () {
> > printf "checking whether linker accepts %s... " "$2"
> > echo "typedef int x;" > "$tmpc"
> >-if $CC $LDFLAGS_TRY -nostdlib -shared "$2" -o /dev/null "$tmpc"
> >>/dev/null 2>&1 ; then
> >+if $CC $CFLAGS $LDFLAGS_TRY -nostdlib -shared "$2" -o /dev/null
> >"$tmpc" >/dev/null 2>&1 ; then
> > printf "yes\n"
> > eval "$1=\"\${$1} \$2\""
> > eval "$1=\${$1# }"
> >@@ -275,7 +275,7 @@ echo "#if ! __GLIBC__" >> "$tmpc"
> > echo "#error no" >> "$tmpc"
> > echo "#endif" >> "$tmpc"
> > printf "checking for toolchain wrapper to build... "
> >-if test "$wrapper" = auto && ! $CC -c -o /dev/null "$tmpc" >/dev/null
> >2>&1 ; then
> >+if test "$wrapper" = auto && ! $CC $CFLAGS -c -o /dev/null "$tmpc"
> >>/dev/null 2>&1 ; then
> > echo "none"
> > elif test "$cc_family" = gcc ; then
> > gcc_wrapper=yes
> >@@ -303,7 +303,7 @@ fi
> > # Find the target architecture
> > #
> > printf "checking target system type... "
> >-test -n "$target" || target=$($CC -dumpmachine 2>/dev/null) || target=unknown
> >+test -n "$target" || target=$($CC $CFLAGS -dumpmachine 2>/dev/null)
> >|| target=unknown
> > printf "%s\n" "$target"
> >
> > #
> >@@ -397,7 +397,7 @@ test "$debug" = yes && CFLAGS_AUTO=-g
> > printf "checking whether we should preprocess assembly to add
> >debugging information... "
> > if fnmatch '-g*|*\ -g*' "$CFLAGS_AUTO $CFLAGS" &&
> >    test -f "tools/add-cfi.$ARCH.awk" &&
> >-   printf ".file 1 \"srcfile.s\"\n.line
> >1\n.cfi_startproc\n.cfi_endproc" | $CC -g -x assembler -c -o /dev/null
> >2>/dev/null -
> >+   printf ".file 1 \"srcfile.s\"\n.line
> >1\n.cfi_startproc\n.cfi_endproc" | $CC $CFLAGS -g -x assembler -c -o
> >/dev/null 2>/dev/null -
> > then
> >   ADD_CFI=yes
> > else
> >@@ -588,7 +588,7 @@ tryldflag LDFLAGS_AUTO -Wl,-Bsymbolic-functions
> > # Find compiler runtime library
> > test -z "$LIBCC" && tryldflag LIBCC -lgcc && tryldflag LIBCC -lgcc_eh
> > test -z "$LIBCC" && tryldflag LIBCC -lcompiler_rt
> >-test -z "$LIBCC" && try_libcc=`$CC -print-file-name=libpcc.a 2>/dev/null` \
> >+test -z "$LIBCC" && try_libcc=`$CC $CFLAGS -print-file-name=libpcc.a
> >2>/dev/null` \
> >                  && tryldflag LIBCC "$try_libcc"
> > printf "using compiler runtime libraries: %s\n" "$LIBCC"
> >
> >@@ -719,6 +719,7 @@ printf "creating config.mak... "
> > cmdline=$(quote "$0")
> > for i ; do cmdline="$cmdline $(quote "$i")" ; done
> >
> >+
> > exec 3>&1 1>config.mak
> >
> >

Re: Unable to build with --enable-shared

[Samuel Sadok]

2016-10-24 2:28 GMT+02:00 Rich Felker <dalias@libc.org>:
> On Sun, Oct 23, 2016 at 07:19:02PM -0500, Laine Gholson wrote:
>> I suggest that you put '-B/usr/local/bin/x86_64-pc-linux-gnu', in
>> $CC, because it is needed for the C compiler to run correctly, e.g
>> CC="x86_64-pc-linux-gnu-gcc -B/usr/local/bin/x86_64-pc-linux-gnu",
>> and that will work fine with the current configure script.
>
> Indeed. Options which are of the class "the compiler driver does not
> work at all, or does not have the right target/ABI, without this
> option" belong in $CC. This also applies to things like -m32.

You're right, that seems a lot cleaner. However I still think that the
configure script should report that something is wrong. For instance,
you could call tryflag and tryldflag with an empty flag argument and
abort if it fails, since further checks are no longer valid. I think
that would be nicely in line with the idea of a configure script,
which is in part to test the waters and fail gracefully if it's not
happy with its environment.

>
> In Samuel's case, though, I think the cross compiler was just built
> incorrectly if it's not searching the right tool path by default.
> Maybe it doesn't realize it's a cross compiler or something.
>
> Rich

Yes, I messed up the prefixes when building the toolchain. In
particular, the binutils and gcc are in different locations, which
confuses them. It works though when I set up symlinks and use the -B
flag.

>
>
>> On 10/23/16 17:20, Samuel Sadok wrote:
>> >2016-10-23 18:17 GMT+02:00 Rich Felker <dalias@libc.org>:
>> >>On Sun, Oct 23, 2016 at 02:22:28AM +0200, Samuel Sadok wrote:
>> >>>2016-10-22 23:58 GMT+02:00 Szabolcs Nagy <nsz@port70.net>:
>> >>>>* Samuel Sadok <innovation-labs@appinstall.ch> [2016-10-22 22:37:46 +0200]:
>> >>>>>I am unable to build musl with --enable-shared. Multiple issues in
>> >>>>>unrelated projects (e.g. here: https://bugs.swift.org/browse/SR-1023)
>> >>>>>suggest that the culprit is a change in behaviour between binutils
>> >>>>>2.25 and 2.26.
>> >>>>>
>> >>>>
>> >>>>that bug is different, that's about object symbols
>> >>>>
>> >>>>(swift seems to misuse protected visibility objects,
>> >>>>and instead of fixing the problem they switched to the
>> >>>>gold linker which does not yet have the bfd linker fix,
>> >>>>such incompetence is frustrating..  however the issue
>> >>>>does not affect musl: we don't mark objects protected
>> >>>>to avoid issues with broken toolchains.)
>> >>>
>> >>>That's precisely why I'm trying to get away from Apple.
>> >>>
>> >>>>
>> >>>>>$ make
>> >>>>>[...]
>> >>>>>x86_64-pc-linux-gnu-gcc  -D_XOPEN_SOURCE=700 -I./arch/x86_64
>> >>>>>-I./arch/generic -Iobj/src/internal -I./src/internal -Iobj/include
>> >>>>>-I./include  -include vis.h -B/usr/local/bin/x86_64-pc-linux-gnu-
>> >>>>>-fPIC -c -o obj/src/process/posix_spawn.lo src/process/posix_spawn.c
>> >>>>
>> >>>>some flags are missing here.. e.g. freestanding flags
>> >>>
>> >>>Thanks for the pointer, that turned out to be the problem.
>> >>>
>> >>>I should mention that I am cross-compiling from macOS. Since I had a
>> >>>Linux VM flying around, I tried to build musl there (success) and
>> >>>compared the logs.
>> >>>Looking at config.mak, there apparently was a serious misconfiguration:
>> >>>
>> >>>config.mak on the cross-build system (macOS):
>> >>>
>> >>>CFLAGS_AUTO = -include vis.h
>> >>>CFLAGS_C99FSE =
>> >>>CFLAGS_MEMOPS =
>> >>>CFLAGS_NOSSP =
>> >>>LDFLAGS_AUTO =
>> >>>LIBCC =
>> >>
>> >>This probably indicates something is wrong with your cross toolchain;
>> >>even if you manually fix config.mak, I would be concerned that
>> >>something might have been built wrong. What cross toolchain are you
>> >>using and how was it setup? How did you invoke configure?
>> >>
>> >>Rich
>> >
>> >I managed to track down the problem.
>> >
>> >I'm using a cross-toolchain I built from source with a slight
>> >misconfiguration. Specifically, I must tell it where it is
>> >(-B/usr/local/bin/x86_64-pc-linux-gnu-), otherwise it uses the
>> >(incompatible) default linker.
>> >
>> >Now when the configure script checks whether some flag is working, it
>> >forgets to pass the user-defined $CFLAGS to the compiler, which then
>> >fails:
>> >
>> >tryflag () {
>> >printf "checking whether compiler accepts %s... " "$2"
>> >echo "typedef int x;" > "$tmpc"
>> >if $CC $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
>> >[...]
>> >
>> >Thus, we end up with no flags at all.
>> >
>> >Even though I'm aware that my setup is quite non-standard, I would
>> >consider this a bug in the configure script. I appended a patch that
>> >fixes the problem in all places where I spotted it. Using this patch I
>> >am able to configure and build musl as expected.
>> >Does this seem reasonable to you?
>> >
>> >
>> >
>> >From 084678f6c93ed0bf305ea0fbb35a33810c4c9ccc Mon Sep 17 00:00:00 2001
>> >From: Samuel Sadok <innovation-labs@appinstall.ch>
>> >Date: Mon, 24 Oct 2016 00:10:04 +0200
>> >Subject: [PATCH] Pass CFLAGS to compiler under all circumstances in configure
>> > script
>> >
>> >---
>> > configure | 15 ++++++++-------
>> > 1 file changed, 8 insertions(+), 7 deletions(-)
>> >
>> >diff --git a/configure b/configure
>> >index 707eb12..c1466ce 100755
>> >--- a/configure
>> >+++ b/configure
>> >@@ -72,7 +72,7 @@ echo "typedef int x;" > "$tmpc"
>> > echo "#if $1" >> "$tmpc"
>> > echo "#error yes" >> "$tmpc"
>> > echo "#endif" >> "$tmpc"
>> >-if $CC $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
>> >+if $CC $CFLAGS $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
>> > printf "false\n"
>> > return 1
>> > else
>> >@@ -84,7 +84,7 @@ fi
>> > tryflag () {
>> > printf "checking whether compiler accepts %s... " "$2"
>> > echo "typedef int x;" > "$tmpc"
>> >-if $CC $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
>> >+if $CC $CFLAGS $CFLAGS_TRY $2 -c -o /dev/null "$tmpc" >/dev/null 2>&1 ; then
>> > printf "yes\n"
>> > eval "$1=\"\${$1} \$2\""
>> > eval "$1=\${$1# }"
>> >@@ -98,7 +98,7 @@ fi
>> > tryldflag () {
>> > printf "checking whether linker accepts %s... " "$2"
>> > echo "typedef int x;" > "$tmpc"
>> >-if $CC $LDFLAGS_TRY -nostdlib -shared "$2" -o /dev/null "$tmpc"
>> >>/dev/null 2>&1 ; then
>> >+if $CC $CFLAGS $LDFLAGS_TRY -nostdlib -shared "$2" -o /dev/null
>> >"$tmpc" >/dev/null 2>&1 ; then
>> > printf "yes\n"
>> > eval "$1=\"\${$1} \$2\""
>> > eval "$1=\${$1# }"
>> >@@ -275,7 +275,7 @@ echo "#if ! __GLIBC__" >> "$tmpc"
>> > echo "#error no" >> "$tmpc"
>> > echo "#endif" >> "$tmpc"
>> > printf "checking for toolchain wrapper to build... "
>> >-if test "$wrapper" = auto && ! $CC -c -o /dev/null "$tmpc" >/dev/null
>> >2>&1 ; then
>> >+if test "$wrapper" = auto && ! $CC $CFLAGS -c -o /dev/null "$tmpc"
>> >>/dev/null 2>&1 ; then
>> > echo "none"
>> > elif test "$cc_family" = gcc ; then
>> > gcc_wrapper=yes
>> >@@ -303,7 +303,7 @@ fi
>> > # Find the target architecture
>> > #
>> > printf "checking target system type... "
>> >-test -n "$target" || target=$($CC -dumpmachine 2>/dev/null) || target=unknown
>> >+test -n "$target" || target=$($CC $CFLAGS -dumpmachine 2>/dev/null)
>> >|| target=unknown
>> > printf "%s\n" "$target"
>> >
>> > #
>> >@@ -397,7 +397,7 @@ test "$debug" = yes && CFLAGS_AUTO=-g
>> > printf "checking whether we should preprocess assembly to add
>> >debugging information... "
>> > if fnmatch '-g*|*\ -g*' "$CFLAGS_AUTO $CFLAGS" &&
>> >    test -f "tools/add-cfi.$ARCH.awk" &&
>> >-   printf ".file 1 \"srcfile.s\"\n.line
>> >1\n.cfi_startproc\n.cfi_endproc" | $CC -g -x assembler -c -o /dev/null
>> >2>/dev/null -
>> >+   printf ".file 1 \"srcfile.s\"\n.line
>> >1\n.cfi_startproc\n.cfi_endproc" | $CC $CFLAGS -g -x assembler -c -o
>> >/dev/null 2>/dev/null -
>> > then
>> >   ADD_CFI=yes
>> > else
>> >@@ -588,7 +588,7 @@ tryldflag LDFLAGS_AUTO -Wl,-Bsymbolic-functions
>> > # Find compiler runtime library
>> > test -z "$LIBCC" && tryldflag LIBCC -lgcc && tryldflag LIBCC -lgcc_eh
>> > test -z "$LIBCC" && tryldflag LIBCC -lcompiler_rt
>> >-test -z "$LIBCC" && try_libcc=`$CC -print-file-name=libpcc.a 2>/dev/null` \
>> >+test -z "$LIBCC" && try_libcc=`$CC $CFLAGS -print-file-name=libpcc.a
>> >2>/dev/null` \
>> >                  && tryldflag LIBCC "$try_libcc"
>> > printf "using compiler runtime libraries: %s\n" "$LIBCC"
>> >
>> >@@ -719,6 +719,7 @@ printf "creating config.mak... "
>> > cmdline=$(quote "$0")
>> > for i ; do cmdline="$cmdline $(quote "$i")" ; done
>> >
>> >+
>> > exec 3>&1 1>config.mak
>> >
>> >