Having hard time adding to CFLAGS

Having hard time adding to CFLAGS

[Denys Vlasenko]

Let's say I need to add a gcc option to my musl build.

configure says:
...
Some influential environment variables:
  CC                      C compiler command [detected]
  CFLAGS                  C compiler flags [-Os -pipe ...]
  CROSS_COMPILE           prefix for cross compiler and tools [none]
  LIBCC                   compiler runtime library [detected

So I try this, combining all possible ways of passing CFLAGS
(past experience is that different projects do it differently).

CFLAGS is in environment, and on both configure and make
command lines:

export CFLAGS="-falign-functions=1"    # for example
./configure CFLAGS="$CFLAGS"
make CFLAGS="$CFLAGS"

It does work, but resulting libc.so is twice as big:
   text       data        bss        dec        hex    filename
 564099       1944      11768     577811      8d113    musl.1/lib/libc.so
 917805       2130      11736     931671      e3757    musl.2/lib/libc.so

The cause is that gcc invocation for each .c file in both cases start normally:

gcc -std=c99 -nostdinc -ffreestanding -fexcess-precision=standard
-frounding-math -D_XOPEN_SOURCE=700 -I./arch/x86_64 -I./src/internal
-I./include...

but then, build without explicit CFLAGS use this:

... -Os -pipe -fomit-frame-pointer -fno-unwind-tables
-fno-asynchronous-unwind-tables -Wa,--noexecstack
-Werror=implicit-function-declaration -Werror=implicit-int
-Werror=pointer-sign -Werror=pointer-arith -include vis.h  -fPIC
-DSHARED -c -o src/aio/aio.lo src/aio/aio.c

and one with CFLAGS loses these flags, in particular, it has no -Os
and no -fPIC:

... -falign-functions=1 -c -o src/aio/aio.o src/aio/aio.c

Evidently, my CFLAGS replaced needed flags instead of being added at the end.

Can this be fixed? If user needs to use e.g. EXTRA_CFLAGS instead,
please fix configure --help.

Re: Having hard time adding to CFLAGS

[Josiah Worcester]

[-- Attachment #1: Type: text/plain, Size: 2689 bytes --]

On Thu, Oct 22, 2015 at 3:31 PM Denys Vlasenko <vda.linux@googlemail.com>
wrote:

> Let's say I need to add a gcc option to my musl build.
>
> configure says:
> ...
> Some influential environment variables:
>   CC                      C compiler command [detected]
>   CFLAGS                  C compiler flags [-Os -pipe ...]
>   CROSS_COMPILE           prefix for cross compiler and tools [none]
>   LIBCC                   compiler runtime library [detected
>
> So I try this, combining all possible ways of passing CFLAGS
> (past experience is that different projects do it differently).
>
> CFLAGS is in environment, and on both configure and make
> command lines:
>
> export CFLAGS="-falign-functions=1"    # for example
> ./configure CFLAGS="$CFLAGS"
> make CFLAGS="$CFLAGS"
>
> It does work, but resulting libc.so is twice as big:
>    text       data        bss        dec        hex    filename
>  564099       1944      11768     577811      8d113    musl.1/lib/libc.so
>  917805       2130      11736     931671      e3757    musl.2/lib/libc.so
>
> The cause is that gcc invocation for each .c file in both cases start
> normally:
>
> gcc -std=c99 -nostdinc -ffreestanding -fexcess-precision=standard
> -frounding-math -D_XOPEN_SOURCE=700 -I./arch/x86_64 -I./src/internal
> -I./include...
>
> but then, build without explicit CFLAGS use this:
>
> ... -Os -pipe -fomit-frame-pointer -fno-unwind-tables
> -fno-asynchronous-unwind-tables -Wa,--noexecstack
> -Werror=implicit-function-declaration -Werror=implicit-int
> -Werror=pointer-sign -Werror=pointer-arith -include vis.h  -fPIC
> -DSHARED -c -o src/aio/aio.lo src/aio/aio.c
>
> and one with CFLAGS loses these flags, in particular, it has no -Os
> and no -fPIC:
>
> ... -falign-functions=1 -c -o src/aio/aio.o src/aio/aio.c
>
> Evidently, my CFLAGS replaced needed flags instead of being added at the
> end.
>
> Can this be fixed? If user needs to use e.g. EXTRA_CFLAGS instead,
> please fix configure --help.
>

This appears to be an issue with how you're expecting CFLAGS to work. As
with almost anything with configure, all the optimization flags you want
need to be in CFLAGS. Normally, a program would have nothing in there
otherwise, so you'd be just getting the "-falign-functions=1" flag going to
GCC. If you want to optimize it, you would actually need CFLAGS="-Os ...".

As for the "no -fPIC", I believe you're looking at the build for two
different objects. musl's build system builds both a static and shared
library, and the line you pasted there is the one that it uses for the
static library's objects. You could tell if it was intended to go into a
shared object because it would have a ".lo" suffix instead.

[-- Attachment #2: Type: text/html, Size: 3319 bytes --]

Re: Having hard time adding to CFLAGS

[Szabolcs Nagy]

[-- Attachment #1: Type: text/plain, Size: 1204 bytes --]

* Denys Vlasenko <vda.linux@googlemail.com> [2015-10-23 00:31:09 +0200]:
> Let's say I need to add a gcc option to my musl build.
> 
> configure says:
> ...
> Some influential environment variables:
>   CC                      C compiler command [detected]
>   CFLAGS                  C compiler flags [-Os -pipe ...]
>   CROSS_COMPILE           prefix for cross compiler and tools [none]
>   LIBCC                   compiler runtime library [detected
> 
> So I try this, combining all possible ways of passing CFLAGS
> (past experience is that different projects do it differently).
> 
> CFLAGS is in environment, and on both configure and make
> command lines:
> 
> export CFLAGS="-falign-functions=1"    # for example
> ./configure CFLAGS="$CFLAGS"
> make CFLAGS="$CFLAGS"

this is not what configure said...

> Evidently, my CFLAGS replaced needed flags instead of being added at the end.
> 
> Can this be fixed? If user needs to use e.g. EXTRA_CFLAGS instead,
> please fix configure --help.

it can be fixed, but i think 'needed flag' is not
always clear and overriding CFLAGS on the make
commandline is not polite.

the attached patch makes this work, but i
consider -Os to be not part of 'needed'

[-- Attachment #2: a.diff --]
[-- Type: text/x-diff, Size: 1393 bytes --]

diff --git a/Makefile b/Makefile
index 844a017..f713286 100644
--- a/Makefile
+++ b/Makefile
@@ -94,22 +94,22 @@ crt/crt1.o crt/Scrt1.o crt/rcrt1.o src/ldso/dlstart.lo: $(wildcard arch/$(ARCH)/
 
 crt/rcrt1.o: src/ldso/dlstart.c
 
-crt/Scrt1.o crt/rcrt1.o: CFLAGS += -fPIC
+crt/Scrt1.o crt/rcrt1.o: CFLAGS_ALL += -fPIC
 
 OPTIMIZE_SRCS = $(wildcard $(OPTIMIZE_GLOBS:%=src/%))
-$(OPTIMIZE_SRCS:%.c=%.o) $(OPTIMIZE_SRCS:%.c=%.lo): CFLAGS += -O3
+$(OPTIMIZE_SRCS:%.c=%.o) $(OPTIMIZE_SRCS:%.c=%.lo): CFLAGS_ALL += -O3
 
 MEMOPS_SRCS = src/string/memcpy.c src/string/memmove.c src/string/memcmp.c src/string/memset.c
-$(MEMOPS_SRCS:%.c=%.o) $(MEMOPS_SRCS:%.c=%.lo): CFLAGS += $(CFLAGS_MEMOPS)
+$(MEMOPS_SRCS:%.c=%.o) $(MEMOPS_SRCS:%.c=%.lo): CFLAGS_ALL += $(CFLAGS_MEMOPS)
 
 NOSSP_SRCS = $(wildcard crt/*.c) \
 	src/env/__libc_start_main.c src/env/__init_tls.c \
 	src/thread/__set_thread_area.c src/env/__stack_chk_fail.c \
 	src/string/memset.c src/string/memcpy.c \
 	src/ldso/dlstart.c src/ldso/dynlink.c
-$(NOSSP_SRCS:%.c=%.o) $(NOSSP_SRCS:%.c=%.lo): CFLAGS += $(CFLAGS_NOSSP)
+$(NOSSP_SRCS:%.c=%.o) $(NOSSP_SRCS:%.c=%.lo): CFLAGS_ALL += $(CFLAGS_NOSSP)
 
-$(CRT_LIBS:lib/%=crt/%): CFLAGS += -DCRT
+$(CRT_LIBS:lib/%=crt/%): CFLAGS_ALL += -DCRT
 
 # This incantation ensures that changes to any subarch asm files will
 # force the corresponding object file to be rebuilt, even if the implicit

Re: Having hard time adding to CFLAGS

[Denys Vlasenko]

On Fri, Oct 23, 2015 at 1:04 AM, Josiah Worcester <josiahw@gmail.com> wrote:
> On Thu, Oct 22, 2015 at 3:31 PM Denys Vlasenko <vda.linux@googlemail.com>
> wrote:
>> Evidently, my CFLAGS replaced needed flags instead of being added at the
>> end.
>>
>> Can this be fixed? If user needs to use e.g. EXTRA_CFLAGS instead,
>> please fix configure --help.
>
>
> This appears to be an issue with how you're expecting CFLAGS to work.

I don't expect CFLAGS to work in any particular way. I would use whichever
variable musl build system's documentation tells me to
in order to add my own flags to gcc.

My point is, documentation does not tell me how to achieve that.

> As
> with almost anything with configure, all the optimization flags you want
> need to be in CFLAGS. Normally, a program would have nothing in there
> otherwise, so you'd be just getting the "-falign-functions=1" flag going to
> GCC. If you want to optimize it, you would actually need CFLAGS="-Os ...".

And I would need to know to add -fPIC etc, right?

Now, how would I make -fPIC appear *only where it is needed*?
The project may build some *.c files as part of a *library*
and therefore need -fPIC, and some other *.c files as a part
of a *binary*, where -fPIC is better not be used.

> As for the "no -fPIC", I believe you're looking at the build for two
> different objects.

No, I don't.

Re: Having hard time adding to CFLAGS

[Rich Felker]

On Fri, Oct 23, 2015 at 01:23:30AM +0200, Szabolcs Nagy wrote:
> * Denys Vlasenko <vda.linux@googlemail.com> [2015-10-23 00:31:09 +0200]:
> > Let's say I need to add a gcc option to my musl build.
> > 
> > configure says:
> > ...
> > Some influential environment variables:
> >   CC                      C compiler command [detected]
> >   CFLAGS                  C compiler flags [-Os -pipe ...]
> >   CROSS_COMPILE           prefix for cross compiler and tools [none]
> >   LIBCC                   compiler runtime library [detected
> > 
> > So I try this, combining all possible ways of passing CFLAGS
> > (past experience is that different projects do it differently).
> > 
> > CFLAGS is in environment, and on both configure and make
> > command lines:
> > 
> > export CFLAGS="-falign-functions=1"    # for example
> > ./configure CFLAGS="$CFLAGS"
> > make CFLAGS="$CFLAGS"
> 
> this is not what configure said...
> 
> > Evidently, my CFLAGS replaced needed flags instead of being added at the end.
> > 
> > Can this be fixed? If user needs to use e.g. EXTRA_CFLAGS instead,
> > please fix configure --help.
> 
> it can be fixed, but i think 'needed flag' is not
> always clear and overriding CFLAGS on the make
> commandline is not polite.

Agreed. If you pass CFLAGS to configure, there's no reason to override
it later. But it also shouldn't break.

> the attached patch makes this work, but i
> consider -Os to be not part of 'needed'

> diff --git a/Makefile b/Makefile
> index 844a017..f713286 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -94,22 +94,22 @@ crt/crt1.o crt/Scrt1.o crt/rcrt1.o src/ldso/dlstart.lo: $(wildcard arch/$(ARCH)/
>  
>  crt/rcrt1.o: src/ldso/dlstart.c
>  
> -crt/Scrt1.o crt/rcrt1.o: CFLAGS += -fPIC
> +crt/Scrt1.o crt/rcrt1.o: CFLAGS_ALL += -fPIC

This is the correct fix. I was not aware that make variables set from
the make command line would take precedence over the target-specific
+= concatenations. The intent has always been that editing CFLAGS
should not break the build (unless you put really inapproriate stuff
there, of course).

Rich

Re: Having hard time adding to CFLAGS

[Rich Felker]

On Thu, Oct 22, 2015 at 11:02:36PM -0400, Rich Felker wrote:
> > the attached patch makes this work, but i
> > consider -Os to be not part of 'needed'
> 
> > diff --git a/Makefile b/Makefile
> > index 844a017..f713286 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -94,22 +94,22 @@ crt/crt1.o crt/Scrt1.o crt/rcrt1.o src/ldso/dlstart.lo: $(wildcard arch/$(ARCH)/
> >  
> >  crt/rcrt1.o: src/ldso/dlstart.c
> >  
> > -crt/Scrt1.o crt/rcrt1.o: CFLAGS += -fPIC
> > +crt/Scrt1.o crt/rcrt1.o: CFLAGS_ALL += -fPIC
> 
> This is the correct fix. I was not aware that make variables set from
> the make command line would take precedence over the target-specific
> += concatenations. The intent has always been that editing CFLAGS
> should not break the build (unless you put really inapproriate stuff
> there, of course).

I've committed this with one change (omitting the -O3 thing that's
really optional) and another related fix.

Note however that overriding CFLAGS at make time is still a bad idea.
It will suppress all the warning options configure detected and other
useful but non-essential things like -fno-unwind-tables and
-fno-asynchronous-unwind-tables. We should probably discuss whether
this behavior is desirable. We could factor out all of the stuff
configure detects into a CFLAGS_AUTO and leave CFLAGS just containing
the user-provided options. Opinions?

Rich

Re: Having hard time adding to CFLAGS

[Denys Vlasenko]

On Fri, Oct 23, 2015 at 6:09 AM, Rich Felker <dalias@libc.org> wrote:
> On Thu, Oct 22, 2015 at 11:02:36PM -0400, Rich Felker wrote:
>> > the attached patch makes this work, but i
>> > consider -Os to be not part of 'needed'
>>
>> > diff --git a/Makefile b/Makefile
>> > index 844a017..f713286 100644
>> > --- a/Makefile
>> > +++ b/Makefile
>> > @@ -94,22 +94,22 @@ crt/crt1.o crt/Scrt1.o crt/rcrt1.o src/ldso/dlstart.lo: $(wildcard arch/$(ARCH)/
>> >
>> >  crt/rcrt1.o: src/ldso/dlstart.c
>> >
>> > -crt/Scrt1.o crt/rcrt1.o: CFLAGS += -fPIC
>> > +crt/Scrt1.o crt/rcrt1.o: CFLAGS_ALL += -fPIC
>>
>> This is the correct fix. I was not aware that make variables set from
>> the make command line would take precedence over the target-specific
>> += concatenations. The intent has always been that editing CFLAGS
>> should not break the build (unless you put really inapproriate stuff
>> there, of course).
>
> I've committed this with one change (omitting the -O3 thing that's
> really optional) and another related fix.
>
> Note however that overriding CFLAGS at make time is still a bad idea.
> It will suppress all the warning options configure detected and other
> useful but non-essential things like -fno-unwind-tables and
> -fno-asynchronous-unwind-tables. We should probably discuss whether
> this behavior is desirable. We could factor out all of the stuff
> configure detects into a CFLAGS_AUTO and leave CFLAGS just containing
> the user-provided options. Opinions?

Make configure --help warn/explain what would happen if
make CFLAGS=foo is run.

Currently, it is confusing. See for yourself:


Usage: $0 [OPTION]... [VAR=VALUE]... [TARGET]

To assign environment variables (e.g., CC, CFLAGS...), specify them as
VAR=VALUE.  See below for descriptions of some of the useful variables.
...
("ok, I got it. CFLAGS should go to configure's command line!")
...
Some influential environment variables:
  CC                      C compiler command [detected]
  CFLAGS                  C compiler flags [-Os -pipe ...]
  CROSS_COMPILE           prefix for cross compiler and tools [none]
  LIBCC                   compiler runtime library [detected]

Use these variables to override the choices made by configure.

("What? You just said that CFLAGS should be on command line!
Now you are saying it should be in the environment!
What it is?")

Re: Having hard time adding to CFLAGS

[Rich Felker]

On Fri, Oct 23, 2015 at 06:53:29AM +0200, Denys Vlasenko wrote:
> On Fri, Oct 23, 2015 at 6:09 AM, Rich Felker <dalias@libc.org> wrote:
> > On Thu, Oct 22, 2015 at 11:02:36PM -0400, Rich Felker wrote:
> >> > the attached patch makes this work, but i
> >> > consider -Os to be not part of 'needed'
> >>
> >> > diff --git a/Makefile b/Makefile
> >> > index 844a017..f713286 100644
> >> > --- a/Makefile
> >> > +++ b/Makefile
> >> > @@ -94,22 +94,22 @@ crt/crt1.o crt/Scrt1.o crt/rcrt1.o src/ldso/dlstart.lo: $(wildcard arch/$(ARCH)/
> >> >
> >> >  crt/rcrt1.o: src/ldso/dlstart.c
> >> >
> >> > -crt/Scrt1.o crt/rcrt1.o: CFLAGS += -fPIC
> >> > +crt/Scrt1.o crt/rcrt1.o: CFLAGS_ALL += -fPIC
> >>
> >> This is the correct fix. I was not aware that make variables set from
> >> the make command line would take precedence over the target-specific
> >> += concatenations. The intent has always been that editing CFLAGS
> >> should not break the build (unless you put really inapproriate stuff
> >> there, of course).
> >
> > I've committed this with one change (omitting the -O3 thing that's
> > really optional) and another related fix.
> >
> > Note however that overriding CFLAGS at make time is still a bad idea.
> > It will suppress all the warning options configure detected and other
> > useful but non-essential things like -fno-unwind-tables and
> > -fno-asynchronous-unwind-tables. We should probably discuss whether
> > this behavior is desirable. We could factor out all of the stuff
> > configure detects into a CFLAGS_AUTO and leave CFLAGS just containing
> > the user-provided options. Opinions?
> 
> Make configure --help warn/explain what would happen if
> make CFLAGS=foo is run.
> 
> Currently, it is confusing. See for yourself:
> 
> 
> Usage: $0 [OPTION]... [VAR=VALUE]... [TARGET]
> 
> To assign environment variables (e.g., CC, CFLAGS...), specify them as
> VAR=VALUE.  See below for descriptions of some of the useful variables.
> ....
> ("ok, I got it. CFLAGS should go to configure's command line!")
> ....
> Some influential environment variables:
>   CC                      C compiler command [detected]
>   CFLAGS                  C compiler flags [-Os -pipe ...]
>   CROSS_COMPILE           prefix for cross compiler and tools [none]
>   LIBCC                   compiler runtime library [detected]
> 
> Use these variables to override the choices made by configure.
> 
> ("What? You just said that CFLAGS should be on command line!
> Now you are saying it should be in the environment!
> What it is?")

The behavior is the same as autoconf: they're accepted either as
environment variables or on the configure command line. Some users do:

	CFLAGS=... ./configure ...

and others do:

	./configure CFLAGS=... ...

Following a principle of least surprise, I've tried to match the
behavior of autoconf-generated configure scripts, which actually has
something resembling a specification outside of the autoconf
implementation:

https://www.gnu.org/prep/standards/html_node/Configuration.html

I can add some language to the help text to make it explicit that
either is accepted.

Rich

Re: Having hard time adding to CFLAGS

[Denys Vlasenko]

On Fri, Oct 23, 2015 at 7:16 AM, Rich Felker <dalias@libc.org> wrote:
>> Currently, it is confusing. See for yourself:
>>
>>
>> Usage: $0 [OPTION]... [VAR=VALUE]... [TARGET]
>>
>> To assign environment variables (e.g., CC, CFLAGS...), specify them as
>> VAR=VALUE.  See below for descriptions of some of the useful variables.
>> ....
>> ("ok, I got it. CFLAGS should go to configure's command line!")
>> ....
>> Some influential environment variables:
>>   CC                      C compiler command [detected]
>>   CFLAGS                  C compiler flags [-Os -pipe ...]
>>   CROSS_COMPILE           prefix for cross compiler and tools [none]
>>   LIBCC                   compiler runtime library [detected]
>>
>> Use these variables to override the choices made by configure.
>>
>> ("What? You just said that CFLAGS should be on command line!
>> Now you are saying it should be in the environment!
>> What it is?")
>
> The behavior is the same as autoconf: they're accepted either as
> environment variables or on the configure command line. Some users do:
>
>         CFLAGS=... ./configure ...
>
> and others do:
>
>         ./configure CFLAGS=... ...

I tested both and they indeed work identically.

> I can add some language to the help text to make it explicit that
> either is accepted.

I'm sending you a patch with amended --help text.

Re: Having hard time adding to CFLAGS

[Rich Felker]

On Fri, Oct 23, 2015 at 08:47:18AM +0200, Denys Vlasenko wrote:
> On Fri, Oct 23, 2015 at 7:16 AM, Rich Felker <dalias@libc.org> wrote:
> >> Currently, it is confusing. See for yourself:
> >>
> >>
> >> Usage: $0 [OPTION]... [VAR=VALUE]... [TARGET]
> >>
> >> To assign environment variables (e.g., CC, CFLAGS...), specify them as
> >> VAR=VALUE.  See below for descriptions of some of the useful variables.
> >> ....
> >> ("ok, I got it. CFLAGS should go to configure's command line!")
> >> ....
> >> Some influential environment variables:
> >>   CC                      C compiler command [detected]
> >>   CFLAGS                  C compiler flags [-Os -pipe ...]
> >>   CROSS_COMPILE           prefix for cross compiler and tools [none]
> >>   LIBCC                   compiler runtime library [detected]
> >>
> >> Use these variables to override the choices made by configure.
> >>
> >> ("What? You just said that CFLAGS should be on command line!
> >> Now you are saying it should be in the environment!
> >> What it is?")
> >
> > The behavior is the same as autoconf: they're accepted either as
> > environment variables or on the configure command line. Some users do:
> >
> >         CFLAGS=... ./configure ...
> >
> > and others do:
> >
> >         ./configure CFLAGS=... ...
> 
> I tested both and they indeed work identically.
> 
> > I can add some language to the help text to make it explicit that
> > either is accepted.
> 
> I'm sending you a patch with amended --help text.

I'd like to continue discussion of whether the current behavior is
actually desirable. It's quite possible that we should move the
configure-generated CFLAGS to a separate variable where they're not
clobbered by setting of CFLAGS by the user at make time.

Rich