musl for ARM

musl for ARM

[Brian Wang]

Hello all,

I am currently thinking of switching to musl from glibc for my target
after some readings on musl.
I would like some advice from musl experts:
  * Does it support gettext stuff?
  * Does it _boost_ the performance on a 400MHz arm926 device?  Or it
is just smaller?
    By _boost_, I mean if the user can actually feel the improvement
in performance.

I did try the musl cross project and successfully built a musl-based
arm linux toolchain.
My kernel (2.6.24) was built successfully (not tried it on my device yet).
However, when building busybox, there are some header files clashes,
resulting in conflicting types.
An example of it:
---------------------
In file included from
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/kd.h:3:0,
                 from console-tools/kbd_mode.c:23:
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/types.h:12:26:
error: conflicting types for ‘fd_set’
In file included from
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/time.h:9:0,
                 from include/libbb.h:45,
                 from console-tools/kbd_mode.c:22:
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/select.h:25:3:
note: previous declaration of ‘fd_set’ was here
---------------------

I read the FAQ, which mentions "After building and installing GCC, any
header files in GCC’s internal include directory
 which clash with headers provided by musl should be removed. Failure
to do so may cause header-related build failures later."
Is there something that I should do to get the headers right?

Sorry for all the questions and thanks in advance.


brian

-- 
brian
------------------

Cool-Karaoke - The smallest recording studio, in your palm, open-sourced
http://cool-idea.com.tw/

iMaGiNaTiOn iS mOrE iMpOrTaNt tHaN kNoWlEdGe

Re: musl for ARM

[John Spencer]

On 10/02/2012 09:27 AM, Brian Wang wrote:
> Hello all,
>
> I am currently thinking of switching to musl from glibc for my target
> after some readings on musl.
> I would like some advice from musl experts:
>    * Does it support gettext stuff?

musl has no libintl built-in, but i put together a replacement that is 
capable of building all packages relying on the gettext tools:
https://github.com/rofl0r/gettext-tiny
at this point, all it does is echo the passed translation string back; 
i.e. everything's english.
however you can compile gnu gettext just fine, if you want to spend 40+ 
min on its compilation.

>    * Does it _boost_ the performance on a 400MHz arm926 device?  Or it
> is just smaller?
>      By _boost_, I mean if the user can actually feel the improvement
> in performance.

it will definitely boost shell performance, in that it has a far lower 
syscall overhead on startup. configure scripts run up to 4x as fast.
also, memory usage will be much smaller. a booted up x86_64 sabotage 
system takes ~2MB RAM, including kernel.

> I did try the musl cross project and successfully built a musl-based
> arm linux toolchain.
> My kernel (2.6.24) was built successfully (not tried it on my device yet).
> However, when building busybox, there are some header files clashes,
> resulting in conflicting types.
> An example of it:
> ---------------------
> In file included from
> /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/kd.h:3:0,
>                   from console-tools/kbd_mode.c:23:
> /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/types.h:12:26:
> error: conflicting types for ‘fd_set’
> In file included from
> /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/time.h:9:0,
>                   from include/libbb.h:45,
>                   from console-tools/kbd_mode.c:22:
> /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/select.h:25:3:
> note: previous declaration of ‘fd_set’ was here
> ---------------------
>

the problem here seems to be the inclusion of

linux/types.h, not GCC-related headers.

here's a busybox build recipe that's known to work with ARM (at least with kernel 3.0+ headers):
  https://github.com/rofl0r/sabotage/blob/master/pkg/busybox

there's also a prebuilt armv7l rootfs available on that site, check README on the repo for info.

Re: musl for ARM

[Rich Felker]

On Tue, Oct 02, 2012 at 03:27:28PM +0800, Brian Wang wrote:
> Hello all,
> 
> I am currently thinking of switching to musl from glibc for my target
> after some readings on musl.
> I would like some advice from musl experts:
>   * Does it support gettext stuff?
>   * Does it _boost_ the performance on a 400MHz arm926 device?  Or it
> is just smaller?
>     By _boost_, I mean if the user can actually feel the improvement
> in performance.

In general, if an application's perceived performance varies heavily
depending on libc, the application is probably doing something wrong
or at least naive. I can think of some exceptions of course, but this
advice is just to say that libc is probably not a place to look for
significantly improving overall performance. If there are libc
bottlenecks, you could probably get a lot more performance out of
changing the application.

The main exceptions I can think of are places where libc has the wrong
big-O: for example, O(n²) qsort or O(nm) strstr, or backtracking
regex implementations, can make completely-sane programs run extremely
slow on a bad libc. Note that musl and glibc are almost always
equivalent in this area; uclibc and dietlibc and perhaps others have
some problems here.

One area you can get vastly better performance with musl is
application startup overhead. Especially with static linking, but even
with dynamic linking if your only .so is libc, the startup time is
2-5x faster than glibc, which really makes a difference to the runtime
of shell scripts (like configure) that invoke tons of external
programs.

> I did try the musl cross project and successfully built a musl-based
> arm linux toolchain.
> My kernel (2.6.24) was built successfully (not tried it on my device yet).
> However, when building busybox, there are some header files clashes,
> resulting in conflicting types.
> An example of it:
> ---------------------
> In file included from
> /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/kd.h:3:0,
>                  from console-tools/kbd_mode.c:23:
> /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/types.h:12:26:
> error: conflicting types for ‘fd_set’
> In file included from
> /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/time.h:9:0,
>                  from include/libbb.h:45,
>                  from console-tools/kbd_mode.c:22:
> /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/select.h:25:3:
> note: previous declaration of ‘fd_set’ was here
> ---------------------

It looks like these kernel headers are not sanitized for compatibility
with userspace..?

Rich

Re: musl for ARM

[Brian Wang]

[-- Attachment #1: Type: text/plain, Size: 3163 bytes --]

On Oct 2, 2012 6:46 PM, "John Spencer" <maillist-musl@barfooze.de> wrote:
>
> On 10/02/2012 09:27 AM, Brian Wang wrote:
>>
>> Hello all,
>>
>> I am currently thinking of switching to musl from glibc for my target
>> after some readings on musl.
>> I would like some advice from musl experts:
>>    * Does it support gettext stuff?
>
>
> musl has no libintl built-in, but i put together a replacement that is
capable of building all packages relying on the gettext tools:
> https://github.com/rofl0r/gettext-tiny
> at this point, all it does is echo the passed translation string back;
i.e. everything's english.
> however you can compile gnu gettext just fine, if you want to spend 40+
min on its compilation.
>

I do need a working gettext support.  I will definitely take a look at the
tiny gettext implementation.  Thank you for the info.

>
>>    * Does it _boost_ the performance on a 400MHz arm926 device?  Or it
>> is just smaller?
>>      By _boost_, I mean if the user can actually feel the improvement
>> in performance.
>
>
> it will definitely boost shell performance, in that it has a far lower
syscall overhead on startup. configure scripts run up to 4x as fast.
> also, memory usage will be much smaller. a booted up x86_64 sabotage
system takes ~2MB RAM, including kernel.

Cool.  The boost in the app startup time makes musl worth the try.

Does sabotage work as a cross development platform?

>
>
>> I did try the musl cross project and successfully built a musl-based
>> arm linux toolchain.
>> My kernel (2.6.24) was built successfully (not tried it on my device
yet).
>> However, when building busybox, there are some header files clashes,
>> resulting in conflicting types.
>> An example of it:
>> ---------------------
>> In file included from
>>
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/kd.h:3:0,
>>                   from console-tools/kbd_mode.c:23:
>>
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/types.h:12:26:
>> error: conflicting types for ‘fd_set’
>> In file included from
>>
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/time.h:9:0,
>>                   from include/libbb.h:45,
>>                   from console-tools/kbd_mode.c:22:
>>
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/select.h:25:3:
>> note: previous declaration of ‘fd_set’ was here
>> ---------------------
>>
>
> the problem here seems to be the inclusion of
>
> linux/types.h, not GCC-related headers.
>
> here's a busybox build recipe that's known to work with ARM (at least
with kernel 3.0+ headers):
>  https://github.com/rofl0r/sabotage/blob/master/pkg/busybox
>

I will try to play with it after my sleep.  It is midnight in Taiwan right
now. :-)

> there's also a prebuilt armv7l rootfs available on that site, check
README on the repo for info.
>
>
ok.  i will skip this since my device is only an armv5te box.

[-- Attachment #2: Type: text/html, Size: 3940 bytes --]

Re: musl for ARM

[Brian Wang]

[-- Attachment #1: Type: text/plain, Size: 3609 bytes --]

On Oct 2, 2012 9:57 PM, "Rich Felker" <dalias@aerifal.cx> wrote:
>
> On Tue, Oct 02, 2012 at 03:27:28PM +0800, Brian Wang wrote:
> > Hello all,
> >
> > I am currently thinking of switching to musl from glibc for my target
> > after some readings on musl.
> > I would like some advice from musl experts:
> >   * Does it support gettext stuff?
> >   * Does it _boost_ the performance on a 400MHz arm926 device?  Or it
> > is just smaller?
> >     By _boost_, I mean if the user can actually feel the improvement
> > in performance.
>
> In general, if an application's perceived performance varies heavily
> depending on libc, the application is probably doing something wrong
> or at least naive. I can think of some exceptions of course, but this
> advice is just to say that libc is probably not a place to look for
> significantly improving overall performance. If there are libc
> bottlenecks, you could probably get a lot more performance out of
> changing the application.
>
> The main exceptions I can think of are places where libc has the wrong
> big-O: for example, O(n²) qsort or O(nm) strstr, or backtracking
> regex implementations, can make completely-sane programs run extremely
> slow on a bad libc. Note that musl and glibc are almost always
> equivalent in this area; uclibc and dietlibc and perhaps others have
> some problems here.
>
> One area you can get vastly better performance with musl is
> application startup overhead. Especially with static linking, but even
> with dynamic linking if your only .so is libc, the startup time is
> 2-5x faster than glibc, which really makes a difference to the runtime
> of shell scripts (like configure) that invoke tons of external
> programs.

ok.  that makes sense.  the faster application startup time is one of the
performance figure that i'm looking for.

the applications on my device generally run quite smoothly once the are up
thanks to the efl devrlopers :-)

>
> > I did try the musl cross project and successfully built a musl-based
> > arm linux toolchain.
> > My kernel (2.6.24) was built successfully (not tried it on my device
yet).
> > However, when building busybox, there are some header files clashes,
> > resulting in conflicting types.
> > An example of it:
> > ---------------------
> > In file included from
> >
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/kd.h:3:0,
> >                  from console-tools/kbd_mode.c:23:
> >
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/types.h:12:26:
> > error: conflicting types for ‘fd_set’
> > In file included from
> >
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/time.h:9:0,
> >                  from include/libbb.h:45,
> >                  from console-tools/kbd_mode.c:22:
> >
/opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/select.h:25:3:
> > note: previous declaration of ‘fd_set’ was here
> > ---------------------
>
> It looks like these kernel headers are not sanitized for compatibility
> with userspace..?

i built the musl toolchain with the musl cross project found on the musl
community wiki.  i did replace the 3.x kernel with my oldish 2.6.24.  any
pointers on how to sanitize kernel headers?  i have not built toolchains
myself since the days when prebuilt toolchains were readily available...

thanks in advance.

brian

>
> Rich

[-- Attachment #2: Type: text/html, Size: 4251 bytes --]

Re: musl for ARM

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2012-10-02 09:48:43 -0400]:
> One area you can get vastly better performance with musl is
> application startup overhead. Especially with static linking, but even
> with dynamic linking if your only .so is libc, the startup time is
> 2-5x faster than glibc, which really makes a difference to the runtime
> of shell scripts (like configure) that invoke tons of external
> programs.
> 

i don't see this mentioned on the libc comparision table

is this the self-exec benchmark?

> > However, when building busybox, there are some header files clashes,
> > resulting in conflicting types.
> > An example of it:
> > ---------------------
> > In file included from
> > /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/kd.h:3:0,
> >                  from console-tools/kbd_mode.c:23:
> > /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/linux/types.h:12:26:
> > error: conflicting types for ???fd_set???
> > In file included from
> > /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/time.h:9:0,
> >                  from include/libbb.h:45,
> >                  from console-tools/kbd_mode.c:22:
> > /opt/cross/arm-linux-musleabi/lib/gcc/arm-linux-musleabi/4.7.1/../../../../arm-linux-musleabi/include/sys/select.h:25:3:
> > note: previous declaration of ???fd_set??? was here
> > ---------------------
> 
> It looks like these kernel headers are not sanitized for compatibility
> with userspace..?
> 


linux/types.h only typedefs fd_set ifdef __KERNEL__
so userspace code shouldnt see fd_set at all,
only __kernel_fd_set

i think busybox is doing something wrong there

Re: musl for ARM

[Rich Felker]

On Wed, Oct 03, 2012 at 12:18:59AM +0800, Brian Wang wrote:
> > One area you can get vastly better performance with musl is
> > application startup overhead. Especially with static linking, but even
> > with dynamic linking if your only .so is libc, the startup time is
> > 2-5x faster than glibc, which really makes a difference to the runtime
> > of shell scripts (like configure) that invoke tons of external
> > programs.
> 
> ok.  that makes sense.  the faster application startup time is one of the
> performance figure that i'm looking for.

For individual applications, the difference is not going to be
perceptible unless you're comparing static-linked musl to
dynamic-linked glibc, on apps that use a ton of libraries. In that
case, static-linked glibc would also be fairly fast at startup, but
setting up glibc for static linking is rather difficult.

The place you'll see a real noticable difference is when starting
thousands of tiny, short-lived programs -- this happens mostly in
configure scripts and bloated init-scripts.

If boot time is an issue, musl (especially with static linking) could
help a lot. Combining that with a more streamlined initscript setup
could make even more of a difference.

> i built the musl toolchain with the musl cross project found on the musl
> community wiki.  i did replace the 3.x kernel with my oldish 2.6.24.  any
> pointers on how to sanitize kernel headers?  i have not built toolchains
> myself since the days when prebuilt toolchains were readily available...

I would be careful with older kernels. There have been important fixes
as late as 2.6.27 or 2.6.31 (I forget exactly...somewhere around
there) and you should at least be aware of the issues and backport
security fixes if using older kernels.

Rich

Re: musl for ARM

[Rich Felker]

On Tue, Oct 02, 2012 at 06:39:32PM +0200, Szabolcs Nagy wrote:
> * Rich Felker <dalias@aerifal.cx> [2012-10-02 09:48:43 -0400]:
> > One area you can get vastly better performance with musl is
> > application startup overhead. Especially with static linking, but even
> > with dynamic linking if your only .so is libc, the startup time is
> > 2-5x faster than glibc, which really makes a difference to the runtime
> > of shell scripts (like configure) that invoke tons of external
> > programs.
> > 
> 
> i don't see this mentioned on the libc comparision table
> 
> is this the self-exec benchmark?

Yes. That definitely could be better-documented. self-exec is the
best/only way I've found to measure the actual startup overhead, as
opposed to just aggregate time to run a simple program.

> linux/types.h only typedefs fd_set ifdef __KERNEL__
> so userspace code shouldnt see fd_set at all,
> only __kernel_fd_set
> 
> i think busybox is doing something wrong there

Yes, something is amiss...

Rich