feature request: flag to disable math library build

feature request: flag to disable math library build

[John Spencer]

i'm currently doing lots of testing in qemu, and compiling musl takes a 
*very long time*, especially on MIPS, where either the emulation is much 
slower than ARM, the speed difference is due to the crappy OABI, or the 
kernel for the emulated system is doing something stupid.
in any case, it's 2 - 10 times slower.

currently i'm debugging a segfault in the mips dynamic linker (after i 
spent nearly 12 hours building a static gdb).

for this kind of testing it would be advantageous if i could disable the 
math library entirely (since it is completely unnecessary), which 
accounts for roughly 50% of the build time.

this might also be an option desired by ppl that use tcc or pcc.

maybe --disable-math ?

Re: feature request: flag to disable math library build

[Szabolcs Nagy]

* John Spencer <maillist-musl@barfooze.de> [2012-10-11 19:30:54 +0200]:
> for this kind of testing it would be advantageous if i could disable
> the math library entirely (since it is completely unnecessary),
> which accounts for roughly 50% of the build time.
> 
> this might also be an option desired by ppl that use tcc or pcc.
> 
> maybe --disable-math ?

there is already an option for this:

mv src/math src/.math

now you disabled math

if you don't like such hacks then you can add

SRCS:=$(filter-out src/complex/%,$(SRCS))
SRCS:=$(filter-out src/math/%,$(SRCS))

to your config.mak

i'm not sure how safe is this: libc uses some math
functions (at least stdio uses frexpl for printf)

Re: feature request: flag to disable math library build

[Szabolcs Nagy]

* Szabolcs Nagy <nsz@port70.net> [2012-10-11 20:17:09 +0200]:
> if you don't like such hacks then you can add
> 
> SRCS:=$(filter-out src/complex/%,$(SRCS))
> SRCS:=$(filter-out src/math/%,$(SRCS))
> 
> to your config.mak
> 
> i'm not sure how safe is this: libc uses some math
> functions (at least stdio uses frexpl for printf)

these are the math functions used by other parts
of libc (mostly float scan and print):

M = \
__fpclassifyl \
copysign \
copysignl \
fabs \
fmod \
fmodl \
frexp \
frexpl \
scalbn \
scalbnl \

so you need

SRCS := $(SRCS) $(M:%=src/math/%.c)

(not tested)

Re: feature request: flag to disable math library build

[Rich Felker]

On Thu, Oct 11, 2012 at 09:37:38PM +0200, Szabolcs Nagy wrote:
> * Szabolcs Nagy <nsz@port70.net> [2012-10-11 20:17:09 +0200]:
> > if you don't like such hacks then you can add
> > 
> > SRCS:=$(filter-out src/complex/%,$(SRCS))
> > SRCS:=$(filter-out src/math/%,$(SRCS))
> > 
> > to your config.mak
> > 
> > i'm not sure how safe is this: libc uses some math
> > functions (at least stdio uses frexpl for printf)
> 
> these are the math functions used by other parts
> of libc (mostly float scan and print):
> 
> M = \
> __fpclassifyl \
> copysign \
> copysignl \
> fabs \
> fmod \
> fmodl \
> frexp \
> frexpl \
> scalbn \
> scalbnl \
> 
> so you need
> 
> SRCS := $(SRCS) $(M:%=src/math/%.c)
> 
> (not tested)

Note that this breaks if any of the above functions ever depend on
other math functions or internal math modules, not to mention if the
set of math functions used by the other part of the library ever
changes. I'm generally opposed to this kind of feature switching
because it has huge maintenance cost keeping track of all possible
configurations and ensuring none of them are broken.

I think the problem of extreme slowness building on emulated or
low-end hardware could be better addressed by using distcc, cross
compiling, or other approaches that leverage a more-powerful cpu
instead of omitting parts of libc. (Anyway, isn't musl only a tiny
percentage of the total build time?)

Rich

Re: feature request: flag to disable math library build

[Luca Barbato]

On 10/12/2012 01:28 AM, Rich Felker wrote:
> I think the problem of extreme slowness building on emulated or
> low-end hardware could be better addressed by using distcc, cross
> compiling, or other approaches that leverage a more-powerful cpu
> instead of omitting parts of libc. (Anyway, isn't musl only a tiny
> percentage of the total build time?)

qemu-user usually is a bit faster than the full system emulator.

lu

Re: feature request: flag to disable math library build

[John Spencer]

On 10/12/2012 01:28 AM, Rich Felker wrote:
>
> I think the problem of extreme slowness building on emulated or
> low-end hardware could be better addressed by using distcc, cross
> compiling, or other approaches that leverage a more-powerful cpu

of course having such a setup is preferable.
however i have to build sabotage as a whole, and am testing different 
compilers
that are not necessarily able to compile using distcc with an 
identically built cross-compiler.
engineering such an environment, that is even embeddable into the 
sabotage build environment,
seems like a herculean effort.

also it doesn't solve the problem that you simply don't want to have the 
math library, if it
for example breaks pcc.

> instead of omitting parts of libc. (Anyway, isn't musl only a tiny
> percentage of the total build time?)
it takes a pretty long time, especially when built with .a and .so support.
for example, to compile the things in stage0, you don't need math 
support at all, and the full musl
gets built in stage1 again.
if i can halve the time spent during stage0 musl builds, that will sum 
up soon.

Re: feature request: flag to disable math library build

[Isaac Dunham]

On Thu, 11 Oct 2012 19:28:58 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> On Thu, Oct 11, 2012 at 09:37:38PM +0200, Szabolcs Nagy wrote:
> > these are the math functions used by other parts
> > of libc (mostly float scan and print):
> > 
> > M = \
<snip>
> > so you need
> > 
> > SRCS := $(SRCS) $(M:%=src/math/%.c)
> > 
> > (not tested)
> 
> Note that this breaks if any of the above functions ever depend on
> other math functions or internal math modules, not to mention if the
> set of math functions used by the other part of the library ever
> changes. I'm generally opposed to this kind of feature switching
> because it has huge maintenance cost keeping track of all possible
> configurations and ensuring none of them are broken.

I'd tend to consider src/math to be core components.
And IIRC, src/complex built the last time I tried CVS pcc (but I could be misremembering!)

However, C11 makes complex numbers optional, and there's currently no dependance on src/complex/ - would it make sense to allow 
/configure --disable-complex 
(via the filter-out rule)?
What percent of the build time is that?

-- 
Isaac Dunham <idunham@lavabit.com>

Re: feature request: flag to disable math library build

[Rich Felker]

On Sat, Oct 13, 2012 at 12:23:49AM -0700, Isaac Dunham wrote:
> On Thu, 11 Oct 2012 19:28:58 -0400
> Rich Felker <dalias@aerifal.cx> wrote:
> 
> > On Thu, Oct 11, 2012 at 09:37:38PM +0200, Szabolcs Nagy wrote:
> > > these are the math functions used by other parts
> > > of libc (mostly float scan and print):
> > > 
> > > M = \
> <snip>
> > > so you need
> > > 
> > > SRCS := $(SRCS) $(M:%=src/math/%.c)
> > > 
> > > (not tested)
> > 
> > Note that this breaks if any of the above functions ever depend on
> > other math functions or internal math modules, not to mention if the
> > set of math functions used by the other part of the library ever
> > changes. I'm generally opposed to this kind of feature switching
> > because it has huge maintenance cost keeping track of all possible
> > configurations and ensuring none of them are broken.
> 
> I'd tend to consider src/math to be core components.
> And IIRC, src/complex built the last time I tried CVS pcc (but I
> could be misremembering!)

Nice to hear - I hope that's true!

> However, C11 makes complex numbers optional, and there's currently
> no dependance on src/complex/ - would it make sense to allow
> /configure --disable-complex 
> (via the filter-out rule)?
> What percent of the build time is that?

Should be very small. These functions contain basically no heavy
computation, only a few things for dealing with situations where the
overflow/underflow cases need special handling.

There are plenty of other things in musl that are not required by the
base standards, like GNU extensions, all of the XSI option, and
various POSIX feature groups like spawn, but they're not made
optional, because it increases complexity and has almost no benefits.

Rich

Re: feature request: flag to disable math library build

[Isaac Dunham]

On Sat, 13 Oct 2012 08:26:42 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> > And IIRC, src/complex built the last time I tried CVS pcc (but I
> > could be misremembering!)
> 
> Nice to hear - I hope that's true!

Yes. pcc 20120923 used. 
The problems I'm running into are -m*:

echo 'int main(){ return 0; }' |pcc -march=i386 -
ld: unrecognised emulation mode: arch=i386
Supported emulations: elf_i386 i386linux elf_x86_64 elf_l1om
ld terminated with status 1
(I use -melf_i386, i386linux is not the right mode)

And a segfaulty loader.

> > However, C11 makes complex numbers optional, and there's currently
> > no dependance on src/complex/ - would it make sense to allow
> > /configure --disable-complex 
> > (via the filter-out rule)?

> There are plenty of other things in musl that are not required by the
> base standards, like GNU extensions, all of the XSI option, and
> various POSIX feature groups like spawn, but they're not made
> optional, because it increases complexity and has almost no benefits.

The potential issue is that complex/ requires a compiler capable of handling complex numbers, which C11 makes optional. Of course, it would be possible to just say that the compiler used to build musl must support complex numbers.


-- 
Isaac Dunham <idunham@lavabit.com>

Re: feature request: flag to disable math library build

[Rich Felker]

On Sat, Oct 13, 2012 at 04:49:45PM -0700, Isaac Dunham wrote:
> On Sat, 13 Oct 2012 08:26:42 -0400
> Rich Felker <dalias@aerifal.cx> wrote:
> 
> > > And IIRC, src/complex built the last time I tried CVS pcc (but I
> > > could be misremembering!)
> > 
> > Nice to hear - I hope that's true!
> 
> Yes. pcc 20120923 used. 
> The problems I'm running into are -m*:
> 
> echo 'int main(){ return 0; }' |pcc -march=i386 -
> ld: unrecognised emulation mode: arch=i386
> Supported emulations: elf_i386 i386linux elf_x86_64 elf_l1om
> ld terminated with status 1
> (I use -melf_i386, i386linux is not the right mode)

This sounds like a bug. -m* should not be passed thru to the linker.
I suppose the configure checks should be updated to try linking with
-march etc. rather than just compiling, so this bug is caught.

> And a segfaulty loader.

Any idea why? My guess would be generating bad code that needs
relocations before it can run.

> > > However, C11 makes complex numbers optional, and there's currently
> > > no dependance on src/complex/ - would it make sense to allow
> > > /configure --disable-complex 
> > > (via the filter-out rule)?
> 
> > There are plenty of other things in musl that are not required by the
> > base standards, like GNU extensions, all of the XSI option, and
> > various POSIX feature groups like spawn, but they're not made
> > optional, because it increases complexity and has almost no benefits.
> 
> The potential issue is that complex/ requires a compiler capable of
> handling complex numbers, which C11 makes optional. Of course, it
> would be possible to just say that the compiler used to build musl
> must support complex numbers.

If this is ever an issue, we could consider making the complex code
operate on the representation of complex values as double[2], etc.
rather than directly using complex. IMO this is ugly though. I think
most relevant compilers will want to comply to C99 as well as C11.

Rich

Re: feature request: flag to disable math library build

[John Spencer]

On 10/13/2012 09:23 AM, Isaac Dunham wrote:
>
> I'd tend to consider src/math to be core components.
> And IIRC, src/complex built the last time I tried CVS pcc (but I could be misremembering!)
>
absolutely not, the math library is a pretty recent addition, and it was 
possible
to compile the entirety of sabotage (at that time roughly 5-10% less 
packages) before it was added.
iirc alsa-lib was the very first program i encountered that used math 
functions outside of the scope
offered by gcc builtins.

Re: feature request: flag to disable math library build

[Rich Felker]

On Sat, Oct 13, 2012 at 03:42:32PM +0200, John Spencer wrote:
> On 10/13/2012 09:23 AM, Isaac Dunham wrote:
> >
> >I'd tend to consider src/math to be core components.
> >And IIRC, src/complex built the last time I tried CVS pcc (but I could be misremembering!)
> >
> absolutely not, the math library is a pretty recent addition, and it
> was possible

No, there has always been a math library. It was just previously
incomplete and buggy/non-conformant. I would be surprised if many
packages compile with no math library at all.

Rich

Re: feature request: flag to disable math library build

[John Spencer]

On 10/11/2012 09:37 PM, Szabolcs Nagy wrote:
>
> these are the math functions used by other parts
> of libc (mostly float scan and print):
>
> M = \
> __fpclassifyl \
> copysign \
> copysignl \
> fabs \
> fmod \
> fmodl \
> frexp \
> frexpl \
> scalbn \
> scalbnl \
>
> so you need
>
> SRCS := $(SRCS) $(M:%=src/math/%.c)
>
> (not tested)
>
hmm i'm not entirely following. what is that SRCS line supposed to do ?
would that apply after having renamed math to .math ?

it's unfortunate, that these internal dependencies exist at all...

Re: feature request: flag to disable math library build

[Rich Felker]

On Fri, Oct 12, 2012 at 02:32:15AM +0200, John Spencer wrote:
> On 10/11/2012 09:37 PM, Szabolcs Nagy wrote:
> >
> >these are the math functions used by other parts
> >of libc (mostly float scan and print):
> >
> >M = \
> >__fpclassifyl \
> >copysign \
> >copysignl \
> >fabs \
> >fmod \
> >fmodl \
> >frexp \
> >frexpl \
> >scalbn \
> >scalbnl \
> >
> >so you need
> >
> >SRCS := $(SRCS) $(M:%=src/math/%.c)
> >
> >(not tested)
> >
> hmm i'm not entirely following. what is that SRCS line supposed to do ?
> would that apply after having renamed math to .math ?

No, after using the other trick to selectively remove part of SRCS..

> it's unfortunate, that these internal dependencies exist at all...

The alternative would be bloat and code duplication.

Rich