bbox: musl versus uclibc

bbox: musl versus uclibc

[Denys Vlasenko]

As uclibc is increasingly aging, I am finally forced
to switch to musl: I'm bitten by a nasty bug in
getopt() - hush is using it in a slightly unusual way,
which uclibc does not expect.

I built a toolchain using
    https://github.com/richfelker/musl-cross-make
(Rich, is this the thing I should be using?)
and it worked with no issues at all.

(I can probably only wish for the README
to also mention how to make this a _static_
toolchain... I have a box with 32-bit userspace,
would be awesome to be able to copy this fresh
64-bit toolchain to it and have it working).

Then I built busybox. Impressions:

Only a few options did not build:
EXTRA_COMPAT and FEATURE_VI_REGEX_SEARCH
failed because they need GNU regexp extensions.
FEATURE_MOUNT_NFS and FEATURE_INETD_RPC do not build
because they need rpc/rpc.h.
Not complaining, since them being in libc was a mistake
in the first place.

Now, the good news - musl has smaller data!
6695 bytes versus 7129 bytes for uclibc:

   text  data   bss     dec    hex filename
 894902   465  6664  902031  dc38f busybox.uclibc
 912538   563  6132  919233  e06c1 busybox.musl

Whee!

Re: bbox: musl versus uclibc

[wdlkmpx]

I'm sure there was plenty of people willing to contribute  to uclibc,
there is even an updated fork.

The project has been badly managed.. thats the only reason i can think
of for this situation  to happen

On 8/14/17, Denys Vlasenko <vda.linux@googlemail.com> wrote:
> As uclibc is increasingly aging, I am finally forced
> to switch to musl: I'm bitten by a nasty bug in
> getopt() - hush is using it in a slightly unusual way,
> which uclibc does not expect.
>
> I built a toolchain using
>     https://github.com/richfelker/musl-cross-make
> (Rich, is this the thing I should be using?)
> and it worked with no issues at all.
>
> (I can probably only wish for the README
> to also mention how to make this a _static_
> toolchain... I have a box with 32-bit userspace,
> would be awesome to be able to copy this fresh
> 64-bit toolchain to it and have it working).
>
> Then I built busybox. Impressions:
>
> Only a few options did not build:
> EXTRA_COMPAT and FEATURE_VI_REGEX_SEARCH
> failed because they need GNU regexp extensions.
> FEATURE_MOUNT_NFS and FEATURE_INETD_RPC do not build
> because they need rpc/rpc.h.
> Not complaining, since them being in libc was a mistake
> in the first place.
>
> Now, the good news - musl has smaller data!
> 6695 bytes versus 7129 bytes for uclibc:
>
>    text  data   bss     dec    hex filename
>  894902   465  6664  902031  dc38f busybox.uclibc
>  912538   563  6132  919233  e06c1 busybox.musl
>
> Whee!
> _______________________________________________
> busybox mailing list
> busybox@busybox.net
> http://lists.busybox.net/mailman/listinfo/busybox
>

Re: bbox: musl versus uclibc

[Emmanuel Deloget]

On Mon, Aug 14, 2017 at 7:43 PM, Denys Vlasenko
<vda.linux@googlemail.com> wrote:
> Only a few options did not build:
> EXTRA_COMPAT and FEATURE_VI_REGEX_SEARCH
> failed because they need GNU regexp extensions.

I have a patch somewhere that enable parts of VI_REGEX_SEARCH (it only
does forward regex search while the GNU counterpart implements both
forward and backward regex search). I can submit that if the community
is interested.

BR,

-- Emmanuel Deloget

Re: bbox: musl versus uclibc

[Rich Felker]

On Mon, Aug 14, 2017 at 07:43:39PM +0200, Denys Vlasenko wrote:
> As uclibc is increasingly aging, I am finally forced
> to switch to musl: I'm bitten by a nasty bug in
> getopt() - hush is using it in a slightly unusual way,
> which uclibc does not expect.

While I'm glad musl is working for you, this sounds fragile unless
it's actually just a bug in uclibc, in which case it'll hopefully get
fixed. If you're relying on weird internal-state of getopt there's a
chance this could break in the future on musl or on other libcs.
Can you explain the problem you encountered on uclibc?

> I built a toolchain using
>     https://github.com/richfelker/musl-cross-make
> (Rich, is this the thing I should be using?)
> and it worked with no issues at all.

Yes, it's the easiest and canonical way to get a full toolchain.

> (I can probably only wish for the README
> to also mention how to make this a _static_
> toolchain... I have a box with 32-bit userspace,
> would be awesome to be able to copy this fresh
> 64-bit toolchain to it and have it working).

If you put LDFLAGS="-static --static" (second form is a hack to
workaround libtool's stripping of -static :) in COMMON_CONFIG you
should get a static build of the toolchain, but it's linked to
whatever the libc in your build environment is.

In my config.mak I use:

ifeq ($(HOST),)
COMMON_CONFIG += CC="i486-linux-musl-gcc -static --static" CXX="i486-linux-musl-g++ -static --static"
endif
COMMON_CONFIG += CFLAGS="-g0 -Os" CXXFLAGS="-g0 -Os" LDFLAGS="-s -static --static"

to build the toolchain static-linked using an existing
i486/musl-targeting toolchain. The static in $CC/$CXX is needed
because some broken configure tests in GMP try to run test binaries
built without honoring LDFLAGS, and will fail to run if you don't have
ld-musl-i386 installed.

Now that canadian-cross support works, you could probably instead
treat i486-linux-musl as a cross compiler (host!=build) instead.

If you prefer you can use x86_64 instead, too, but I like the
toolchain binaries themselves being 32-bit simply because it saves
roughly half the ram usage when running them.

> Then I built busybox. Impressions:
> 
> Only a few options did not build:
> EXTRA_COMPAT and FEATURE_VI_REGEX_SEARCH
> failed because they need GNU regexp extensions.

Yes. There was a request for the corresponding version of some of
these extensions in the POSIX regex API, which bb vi could be
converted to use if we adopted them, but there is some nontrivial
runtime cost. It probably doesn't matter now since the regexec core is
fairly slow (well, good big-O but bad constant) right now, but it may
impose a more noticable relative cost once we make regexec faster.

> FEATURE_MOUNT_NFS and FEATURE_INETD_RPC do not build
> because they need rpc/rpc.h.
> Not complaining, since them being in libc was a mistake
> in the first place.

I think glibc has dropped these too now, haven't they? AFAIK they're
just keeping the symbols for ABI compat but they're not an exposed API
for linking new code; you're supposed to use tirpc.

> Now, the good news - musl has smaller data!
> 6695 bytes versus 7129 bytes for uclibc:
> 
>    text  data   bss     dec    hex filename
>  894902   465  6664  902031  dc38f busybox.uclibc
>  912538   563  6132  919233  e06c1 busybox.musl

Probably getpw*/getgr* static buffers or something. musl's backend for
these just uses the buffer out of getline so as not to impose cost in
apps that don't need the legacy (non-_r) functions or an arbitrary
limit on record length.

It would be interesting to know where the text size increase comes
from. Not that big in abs difference, but given that a large portion
of the text is busybox code and not libc, it's a fairly large relative
difference. Come to think about it, it might actually be regex..

Rich

Re: bbox: musl versus uclibc

[Denys Vlasenko]

[-- Attachment #1: Type: text/plain, Size: 683 bytes --]

On Mon, Aug 14, 2017 at 8:48 PM, Rich Felker <dalias@libc.org> wrote:
>> Now, the good news - musl has smaller data!
>> 6695 bytes versus 7129 bytes for uclibc:
>>
>>    text  data   bss     dec    hex filename
>>  894902   465  6664  902031  dc38f busybox.uclibc
>>  912538   563  6132  919233  e06c1 busybox.musl
>
> Probably getpw*/getgr* static buffers or something. musl's backend for
> these just uses the buffer out of getline so as not to impose cost in
> apps that don't need the legacy (non-_r) functions or an arbitrary
> limit on record length.
>
> It would be interesting to know where the text size increase comes
> from.

See attached (only musl data is in the file).

[-- Attachment #2: busybox_unstripped.map.data --]
[-- Type: application/octet-stream, Size: 7510 bytes --]

 .bss.mal       0x0000000008128460      0x410 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(malloc.o)
 .bss.buf       0x0000000008128ce0      0x408 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(stdout.o)
 .bss.buf       0x00000000081288c0      0x408 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(stdin.o)
 .bss.builtin   0x0000000008128340      0x104 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(atexit.o)
 .data.f        0x0000000008128160       0x88 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(stdout.o)
 .data.f        0x00000000081280c0       0x88 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(stdin.o)
 .data.f        0x0000000008128020       0x88 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(stderr.o)
 .bss.tm.1099   0x0000000008129120       0x2c /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(gmtime.o)
 .bss.tm.1097   0x0000000008129160       0x2c /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(localtime.o)
 .bss.buf.1360  0x00000000081288a0       0x20 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(getservbyport.o)
 .bss           0x00000000081292a0       0x20 /usr/app/musl_toolchain-2017-08-12/lib/gcc/i686-linux-musl/6.3.0/crtbeginT.o
 .bss.buf.1073  0x000000000812956c       0x1a /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(asctime.o)
 .bss.mnt.1328  0x0000000008129424       0x18 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(mntent.o)
 .bss.main_tls  0x0000000008129548       0x18 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__init_tls.o)
 .bss.r1        0x00000000081294fc       0x14 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.r0        0x0000000008129510       0x14 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.x.1686    0x0000000000000000       0x12 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(ether.o)
 .bss.se.1359   0x0000000008129470       0x10 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(getservbyport.o)
 .bss.se.1359   0x0000000008129460       0x10 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(getservbyname.o)
 .bss.buf.1404  0x0000000008129480       0x10 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(inet_ntoa.o)
 .bss.__tzname  0x0000000008129538        0x8 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.seed      0x0000000008129298        0x8 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(rand.o)
 .bss.ofl_lock  0x00000000081294a4        0x8 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(ofl.o)
 .bss.lock      0x00000000081294d8        0x8 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.lock      0x0000000008129448        0x8 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(syslog.o)
 .bss.lock      0x00000000081293f0        0x8 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(atexit.o)
 .bss.buf.1360  0x0000000008129458        0x8 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(getservbyname.o)
 .bss.buf       0x00000000081294b0        0x8 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(stderr.o)
 .bss.std_name  0x0000000008129530        0x7 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.dst_name  0x0000000008129528        0x7 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.a.1673    0x0000000000000000        0x6 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(ether.o)
 .data.optind   0x0000000008128200        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(getopt.o)
 .data.opterr   0x00000000081281fc        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(getopt.o)
 .data.old_tz   0x000000000812821c        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .data.log_fd   0x0000000008128204        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(syslog.o)
 .data          0x0000000008128224        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__set_thread_area.o)
 .data          0x0000000000000000        0x4 /usr/app/musl_toolchain-2017-08-12/lib/gcc/i686-linux-musl/6.3.0/crtbeginT.o
 .bss.zi        0x00000000081294f8        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.types     0x00000000081294ec        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.trans     0x00000000081294f4        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.slot      0x00000000081293f8        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(atexit.o)
 .bss.p.1071    0x00000000081294b8        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(strtok.o)
 .bss.ofl_head  0x00000000081294ac        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(ofl.o)
 .bss.map_size  0x00000000081294e0        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.log_opt   0x0000000008129444        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(syslog.o)
 .bss.linesize  0x000000000812940c        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(getusershell.o)
 .bss.line      0x0000000008129410        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(getusershell.o)
 .bss.index     0x00000000081294f0        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.head      0x00000000081294d4        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(synccall.o)
 .bss.head      0x00000000081293fc        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(atexit.o)
 .bss.h.1616    0x0000000008129450        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(gethostbyaddr.o)
 .bss.h.1615    0x0000000008129454        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(gethostbyname2.o)
 .bss.f         0x0000000008129408        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(getusershell.o)
 .bss.end.2954  0x0000000008129414        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(malloc.o)
 .bss.end.1443  0x0000000000000000        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(lite_malloc.o)
 .bss.dummy     0x00000000081294bc        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(synccall.o)
 .bss.dummy     0x0000000000000000        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(fflush.o)
 .bss.dst_off   0x0000000008129524        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)
 .bss.cur.1442  0x0000000000000000        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(lite_malloc.o)
 .bss.context   0x00000000081294c0        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(synccall.o)
 .bss.callback  0x00000000081294c4        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(synccall.o)
 .bss.brk.1556  0x0000000008129564        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(expand_heap.o)
 .bss.abbrevs   0x00000000081294e8        0x4 /usr/app/musl_toolchain-2017-08-12/i686-linux-musl/lib/libc.a(__tz.o)

Re: bbox: musl versus uclibc

[Thomas Petazzoni]

Hello,

On Mon, 14 Aug 2017 12:59:41 -0500, wdlkmpx wrote:

> I'm sure there was plenty of people willing to contribute  to uclibc,
> there is even an updated fork.
> 
> The project has been badly managed.. thats the only reason i can think
> of for this situation  to happen

uClibc-ng is alive at https://uclibc-ng.org/. Regular releases
(actually more regular than musl in recent months!), updated web site,
responsive maintainer, lots of cleanup in the code base, and QA effort.

So saying that uClibc is dead is completely incorrect.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: Re: bbox: musl versus uclibc

[Jorge Almeida]

On Wed, Aug 16, 2017 at 1:34 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:

> On Mon, 14 Aug 2017 12:59:41 -0500, wdlkmpx wrote:
>

>> The project has been badly managed.. thats the only reason i can think
>
> uClibc-ng is alive at https://uclibc-ng.org/. Regular releases
> (actually more regular than musl in recent months!), updated web site,
> responsive maintainer, lots of cleanup in the code base, and QA effort.
>

I just visited the link you provided. I was curious to know whether to
compile a program one still has to "make menuconfig", which IMO is
fine for the linux kernel. So I clicked the "Documentation" link. It
leads to an "Overview over different language related features".

What did I do wrong?

Jorge Almeida

Re: bbox: musl versus uclibc

[Denys Vlasenko]

On Wed, Aug 16, 2017 at 2:34 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> On Mon, 14 Aug 2017 12:59:41 -0500, wdlkmpx wrote:
>
>> I'm sure there was plenty of people willing to contribute  to uclibc,
>> there is even an updated fork.
>>
>> The project has been badly managed.. thats the only reason i can think
>> of for this situation  to happen
>
> uClibc-ng is alive at https://uclibc-ng.org/. Regular releases
> (actually more regular than musl in recent months!), updated web site,
> responsive maintainer, lots of cleanup in the code base, and QA effort.
>
> So saying that uClibc is dead is completely incorrect.

These "ng" names are not the best idea.

Maybe uclibc-ng can just supersede uclibc?

We can give www.uclibc.org domain (or the server itself) to it.

Re: bbox: musl versus uclibc

[Thomas Petazzoni]

Hello,

On Wed, 16 Aug 2017 15:25:20 +0200, Denys Vlasenko wrote:

> These "ng" names are not the best idea.
> 
> Maybe uclibc-ng can just supersede uclibc?

Waldemar proposed many, many times to Bernhard to take over the
project, and Bernhard essentially never replied. So it was either
adding -ng or doing nothing.

I think adding -ng was better than doing nothing :)

And yes, I believe uClibc-ng should just supersede uClibc, or at least
that www.uclibc.org should point to www.uclibc-ng.org to make more
people aware of the fact that uClibc is alive again.

> We can give www.uclibc.org domain (or the server itself) to it.

Cc'ing Waldemar, who maintains uClibc-ng.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: Re: bbox: musl versus uclibc

[Waldemar Brodkorb]

Hi,
Jorge Almeida wrote,

> On Wed, Aug 16, 2017 at 1:34 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> 
> > On Mon, 14 Aug 2017 12:59:41 -0500, wdlkmpx wrote:
> >
> 
> >> The project has been badly managed.. thats the only reason i can think
> >
> > uClibc-ng is alive at https://uclibc-ng.org/. Regular releases
> > (actually more regular than musl in recent months!), updated web site,
> > responsive maintainer, lots of cleanup in the code base, and QA effort.
> >
> 
> I just visited the link you provided. I was curious to know whether to
> compile a program one still has to "make menuconfig", which IMO is
> fine for the linux kernel. So I clicked the "Documentation" link. It
> leads to an "Overview over different language related features".
> 
> What did I do wrong?

Nothing. I updated the page now.
A valid config is still required to build a uClibc-ng toolchain.
You can avoid this by just using Buildroot, Crosstool-ng or OpenADK,
which all have good uClibc-ng toolchain support.

best regards
 Waldemar

Re: [musl] Re: bbox: musl versus uclibc

[Waldemar Brodkorb]

Hi,
Thomas Petazzoni wrote,

> Hello,
> 
> On Wed, 16 Aug 2017 15:25:20 +0200, Denys Vlasenko wrote:
> 
> > These "ng" names are not the best idea.
> > 
> > Maybe uclibc-ng can just supersede uclibc?
> 
> Waldemar proposed many, many times to Bernhard to take over the
> project, and Bernhard essentially never replied. So it was either
> adding -ng or doing nothing.
> 
> I think adding -ng was better than doing nothing :)
> 
> And yes, I believe uClibc-ng should just supersede uClibc, or at least
> that www.uclibc.org should point to www.uclibc-ng.org to make more
> people aware of the fact that uClibc is alive again.
> 
> > We can give www.uclibc.org domain (or the server itself) to it.
> 
> Cc'ing Waldemar, who maintains uClibc-ng.

A link from uclibc.org to uclibc-ng.org would be nice.
Then people can still search the old mailinglists or find
the old git history.

If there would be consense about, the domain uclibc.org could be
transferred to me and I could try to provide a cleaner access to
old and new project data. 

best regards
 Waldemar