Building libc separately from libm,librt,libpthread and others

Building libc separately from libm,librt,libpthread and others

[Timerlan Moldobaev]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

Hi,

Can you please help with reducing the size of statically linked libc.a
library ?
Whereas the comparison table located in
http://www.etalabs.net/compare_libcs.html
claims the size of complete .a set as 333k, I got around 2M while building
the library on x86_64 with gcc version 4.1.1.
I suppose that might be caused by including in libc.a  object files that
belong to libm, librt, libpthread and others.
Am I right ?
Is there any way to compile libc.a solely ?

Thanks,
Tim.

[-- Attachment #2: Type: text/html, Size: 725 bytes --]

Re: Building libc separately from libm,librt,libpthread and others

[John Spencer]

On 04/07/2013 04:43 PM, Timerlan Moldobaev wrote:
> Hi,
>
> Can you please help with reducing the size of statically linked libc.a
> library ?
> Whereas the comparison table located in
> http://www.etalabs.net/compare_libcs.html
> claims the size of complete .a set as 333k, I got around 2M while building
> the library on x86_64 with gcc version 4.1.1.

the comparison page also notes that it is using size(1) and not filesize.

> I suppose that might be caused by including in libc.a  object files that
> belong to libm, librt, libpthread and others.
> Am I right ?
> Is there any way to compile libc.a solely ?

the only thing that can theoretically be left away is libm, but this 
would need some effort.
things like pthread support are fundamental to musl's inner workings, so 
they can not be left away.

Re: Building libc separately from libm,librt,libpthread and others

[Szabolcs Nagy]

* John Spencer <maillist-musl@barfooze.de> [2013-04-07 16:52:08 +0200]:

> On 04/07/2013 04:43 PM, Timerlan Moldobaev wrote:
> >Hi,
> >
> >Can you please help with reducing the size of statically linked libc.a
> >library ?
> >Whereas the comparison table located in
> >http://www.etalabs.net/compare_libcs.html
> >claims the size of complete .a set as 333k, I got around 2M while building
> >the library on x86_64 with gcc version 4.1.1.
> 
> the comparison page also notes that it is using size(1) and not filesize.
> 

compiler flags matter as well: libm has 2x bigger size on gcc 4.4
than on gcc 4.8 because before gcc 4.5 there was no
-fexcess-precision=standard so musl has to use -ffloat-store
for correctness which gives larger and slower code
(this is detected by the configure script)

gcc 4.1 has the same issue and most likely -Os was much worse
back then than in recent gcc

> >I suppose that might be caused by including in libc.a  object files that
> >belong to libm, librt, libpthread and others.
> >Am I right ?
> >Is there any way to compile libc.a solely ?
> 
> the only thing that can theoretically be left away is libm, but this
> would need some effort.
> things like pthread support are fundamental to musl's inner
> workings, so they can not be left away.
> 

you can easily drop src/complex (nothing uses that, you are lucky if it
compiles with 4.1 at all, some ppl reported internal compiler errors on
old gccs in the complex code)

most of src/math can be dropped (all the special functions and those can
be big, but i know that at least frexpl is used by the printf code)

you can drop a lot of functions probably (you have to keep the ones that
are used internally: the 'U' symbols from 'nm libc.a'), but your libc.a
will have reduced functionality for no good reason: if you do static
linking only the used parts will get into the binary (and then the size
of the binary matters not the size of libc.a)

if you only want to compile libc.a you can redefine ALL_LIBS in your
config.mak so it does not contain the SHARED_LIBS (see the Makefile)

Re: Building libc separately from libm,librt,libpthread and others

[John Spencer]

On 04/07/2013 05:43 PM, Szabolcs Nagy wrote:
> * John Spencer<maillist-musl@barfooze.de>  [2013-04-07 16:52:08 +0200]:
>
>> On 04/07/2013 04:43 PM, Timerlan Moldobaev wrote:
>>
>>> I suppose that might be caused by including in libc.a  object files that
>>> belong to libm, librt, libpthread and others.
>>> Am I right ?
>>> Is there any way to compile libc.a solely ?
>> the only thing that can theoretically be left away is libm, but this
>> would need some effort.
>> things like pthread support are fundamental to musl's inner
>> workings, so they can not be left away.
>>
> you can easily drop src/complex (nothing uses that, you are lucky if it
> compiles with 4.1 at all, some ppl reported internal compiler errors on
> old gccs in the complex code)

musl compiles fine with gcc 3.4.6 and gcc 4.2.4 (used by sabotage for 
stage0 bootstrap - gcc 4.2.4 is only used on arm though)

>
> if you only want to compile libc.a you can redefine ALL_LIBS in your
> config.mak so it does not contain the SHARED_LIBS (see the Makefile)

the official way to do this is ./configure --disable-shared

Re: Building libc separately from libm,librt,libpthread and others

[Szabolcs Nagy]

* Timerlan Moldobaev <moldobaev@gmail.com> [2013-04-07 17:43:11 +0300]:
> Whereas the comparison table located in
> http://www.etalabs.net/compare_libcs.html
> claims the size of complete .a set as 333k, I got around 2M while building
> the library on x86_64 with gcc version 4.1.1.

btw the comparision was made on i386 and
32bit vs 64bit can make a big difference

i just checked here and libc.a is 1.4M but
the real size of the object code is 338K using
size libc.a |awk '{s+=$4}END{print s/1024}'
(i386, gcc 4.8)

Re: Building libc separately from libm,librt,libpthread and others

[Rich Felker]

On Sun, Apr 07, 2013 at 05:43:11PM +0300, Timerlan Moldobaev wrote:
> Hi,
> 
> Can you please help with reducing the size of statically linked libc.a
> library ?
> Whereas the comparison table located in
> http://www.etalabs.net/compare_libcs.html
> claims the size of complete .a set as 333k, I got around 2M while building
> the library on x86_64 with gcc version 4.1.1.
> I suppose that might be caused by including in libc.a  object files that
> belong to libm, librt, libpthread and others.
> Am I right ?
> Is there any way to compile libc.a solely ?

What is your goal in getting it smaller? With static linking, only the
object files needed by a program end up in the resulting binary, so
compiling less will not make your binaries any smaller. The only
benefits I can think of are (1) reducing time to compile musl, and (2)
storing the development files on an extremely small storage device. If
you tell us what you're trying to do, we can offer better advice on
how to meet your needs.

Rich

Re: Building libc separately from libm,librt,libpthread and others

[nwmcsween]

[-- Attachment #1: Type: text/plain, Size: 685 bytes --]

Musl is a posix + iso c standard library, there is no separation of standards..

On Apr 7, 2013, at 7:43 AM, Timerlan Moldobaev <moldobaev@gmail.com> wrote:

> Hi,
> 
> Can you please help with reducing the size of statically linked libc.a library ?
> Whereas the comparison table located in http://www.etalabs.net/compare_libcs.html  
> claims the size of complete .a set as 333k, I got around 2M while building the library on x86_64 with gcc version 4.1.1.
> I suppose that might be caused by including in libc.a  object files that belong to libm, librt, libpthread and others.
> Am I right ? 
> Is there any way to compile libc.a solely ?
> 
> Thanks,
> Tim.
> 
> 

[-- Attachment #2: Type: text/html, Size: 1143 bytes --]

Re: Building libc separately from libm,librt,libpthread and others

[Timerlan Moldobaev]

[-- Attachment #1: Type: text/plain, Size: 1172 bytes --]

John , thank you for pointing out that size()  is used in the comparison
table, somehow oversaw that.
BTW, out of curiosity where does the extra size come from ? Some elf
specific format data ?


On Sun, Apr 7, 2013 at 5:52 PM, John Spencer <maillist-musl@barfooze.de>wrote:

> On 04/07/2013 04:43 PM, Timerlan Moldobaev wrote:
>
>> Hi,
>>
>> Can you please help with reducing the size of statically linked libc.a
>> library ?
>> Whereas the comparison table located in
>> http://www.etalabs.net/**compare_libcs.html<http://www.etalabs.net/compare_libcs.html>
>> claims the size of complete .a set as 333k, I got around 2M while building
>> the library on x86_64 with gcc version 4.1.1.
>>
>
> the comparison page also notes that it is using size(1) and not filesize.
>
>
>  I suppose that might be caused by including in libc.a  object files that
>> belong to libm, librt, libpthread and others.
>> Am I right ?
>> Is there any way to compile libc.a solely ?
>>
>
> the only thing that can theoretically be left away is libm, but this would
> need some effort.
> things like pthread support are fundamental to musl's inner workings, so
> they can not be left away.
>
>
>

[-- Attachment #2: Type: text/html, Size: 1859 bytes --]

Re: Building libc separately from libm,librt,libpthread and others

[Timerlan Moldobaev]

[-- Attachment #1: Type: text/plain, Size: 1633 bytes --]

Well, the intention is to use musl package with possible source code
modifications and platform specific optimizations as a an implementation
for standard c library. These sources will be visible to 3rd party
developers wishing to use libc services.  This is the opportunity to make
sure that MIT license allows to do that.
As the developers have the freedom to use any compiler /linker they want,
can I  assume that only the required parts of the libc.a objects will be
included in the resulting executable ?


On Mon, Apr 8, 2013 at 6:32 AM, Rich Felker <dalias@aerifal.cx> wrote:

> On Sun, Apr 07, 2013 at 05:43:11PM +0300, Timerlan Moldobaev wrote:
> > Hi,
> >
> > Can you please help with reducing the size of statically linked libc.a
> > library ?
> > Whereas the comparison table located in
> > http://www.etalabs.net/compare_libcs.html
> > claims the size of complete .a set as 333k, I got around 2M while
> building
> > the library on x86_64 with gcc version 4.1.1.
> > I suppose that might be caused by including in libc.a  object files that
> > belong to libm, librt, libpthread and others.
> > Am I right ?
> > Is there any way to compile libc.a solely ?
>
> What is your goal in getting it smaller? With static linking, only the
> object files needed by a program end up in the resulting binary, so
> compiling less will not make your binaries any smaller. The only
> benefits I can think of are (1) reducing time to compile musl, and (2)
> storing the development files on an extremely small storage device. If
> you tell us what you're trying to do, we can offer better advice on
> how to meet your needs.
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 2291 bytes --]

Re: Building libc separately from libm,librt,libpthread and others

[Boris Alesker]

[-- Attachment #1: Type: text/plain, Size: 337 bytes --]

John Spencer maillist-musl@barfooze.de> wrote:

> the comparison page also notes that it is using size(1) and not filesize.

John , thank you for pointing out that size()  is used for
measurements  in the comparison table, somehow oversaw that.
BTW, out of curiosity where does the extra size come from ? Some elf
specific format data ?

[-- Attachment #2: Type: text/html, Size: 1000 bytes --]

Re: Building libc separately from libm,librt,libpthread and others

[Rich Felker]

On Mon, Apr 08, 2013 at 10:43:25AM +0300, Timerlan Moldobaev wrote:
> John , thank you for pointing out that size()  is used in the comparison
> table, somehow oversaw that.
> BTW, out of curiosity where does the extra size come from ? Some elf
> specific format data ?

A .o file contains a lot of information beyond code: it has to have
all the symbols defined or referenced in the file, as well as the
relocations for both symbolic references and addresses of objects
defined in the same file. Even with a very efficient object format
like the old a.out format, this additional data is likely to be nearly
as large as or larger than an object file containing a single function
or just a few small functions. ELF's storage of this additional data
is much more general/flexible, but also much less efficient; often
it's several times as big as the code. In a worst case, it's hundreds
of times larger (e.g. if the code is just a single "ret" instruction).

The reason I use size(1) rather than file size to measure static
library size is twofold:

1. The size of the actual code and data, not the size of the .o files
in the .a file, is what actually ends up in your final binary.
Actually if you don't strip the binary, some of the additional
information (symbols, etc.) will end up in the final binary too, but
it's proportionally much smaller since it's just one set of tables for
the whole binary rather than one set for each translation unit.

2. Any metric based on .a file size rather than size(1) will "reward"
implementations that are poorly-factored and thus yield larger static
binaries. The easiest way to make your .a file look smaller is to
merge lots of source files together. This indeed reduces the size of
the .a file, but the cost is that programs which use one function from
the .a file will be forced to pull in several other functions they may
not need.

Due to misc. costs of having lots of .o files (build time for musl
itself, excess file size of the .a file, etc.) I've actually merged
some functions into common files. The usual criteria for such merging
are that the functions each be tiny/trivial and not likely to be
useful in tiny applications.

Rich

Re: Building libc separately from libm,librt,libpthread and others

[Boris Alesker]

[-- Attachment #1: Type: text/plain, Size: 1002 bytes --]

Rich Felker dalias@aerifal.cx wote:

>What is your goal in getting it smaller? With static linking, only the
> object files needed by a program end up in the resulting binary, so
> compiling less will not make your binaries any smaller. The only
> benefits I can think of are (1) reducing time to compile musl, and (2)
> storing the development files on an extremely small storage device. If
> you tell us what you're trying to do, we can offer better advice on
> how to meet your needs.


Well, the intention is to use musl package with possible source code
modifications and platform specific optimizations as an implementation for
standard c library. These sources will be visible to 3rd party developers
wishing to use libc services.
1. This is the opportunity to ask  whether  MIT license allows to do that.
2. As the developers have the freedom to use any compiler /linker they
want, can I  assume that only the required parts of the libc.a objects will
be included in the resulting executable ?

[-- Attachment #2: Type: text/html, Size: 1857 bytes --]