Draft: musl promo materials

Draft: musl promo materials

[Rich Felker]

Hi all,
Here's a quick draft (and request for input/suggestions) of
points/highlights about musl to eventually be incorporated into the
website and other opportunities to promote musl. With
embedded-relevant ports getting integrated now, I'm thinking this sort
of thing will also be useful for sending announcements to relevant
mailing lists, etc. Anything big I'm missing?

Rich


----------------------------------------------------------------------

Consistent quality and implementation behavior from tiny embedded
systems to full servers.

Minimal machine-specific code, meaning less chance of breakage on
minority architectures and better success with "write once run
everywhere" development.

Realtime-quality robustness. No unnecessary dynamic allocation. No
unrecoverable late failures. No lazy binding or lazy allocation.

MIT license.

Full math library with a focus on correctness. Exact and
correctly-rounded conversion between binary floating point and decimal
strings.

Reentrancy, thread-safety, and async-signal safety well beyond the
requirements of POSIX. Even snprintf and dprintf are fully reentrant
and async-signal-safe.

Highly resource-efficient POSIX threads implementation, making
multi-threaded application design viable even for memory-constrained
systems.

Re: Draft: musl promo materials

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2012-07-13 14:12:54 -0400]:
> Consistent quality and implementation behavior from tiny embedded
> systems to full servers.
> 
> Minimal machine-specific code, meaning less chance of breakage on
> minority architectures and better success with "write once run
> everywhere" development.
> 
> Realtime-quality robustness. No unnecessary dynamic allocation. No
> unrecoverable late failures. No lazy binding or lazy allocation.
> 
> MIT license.
> 
> Full math library with a focus on correctness. Exact and
> correctly-rounded conversion between binary floating point and decimal
> strings.
> 
> Reentrancy, thread-safety, and async-signal safety well beyond the
> requirements of POSIX. Even snprintf and dprintf are fully reentrant
> and async-signal-safe.
> 
> Highly resource-efficient POSIX threads implementation, making
> multi-threaded application design viable even for memory-constrained
> systems.

i'd somehow add that both static and dynamic linking is supported
properly and without bloat as musl is better at it than glibc

i like musl's clean code, clean header files (no gcc specific mess)
and simple build system (even for cross compilation)
again something that glibc is lacking

and there could be a hint that things like security, worst cases
(stack usage, algorithm complexity) and conformance are taken seriously

Re: Draft: musl promo materials

[Rich Felker]

Updated version based on some comments.. I think the list is getting
long enough that it would possibly make sense to reorder/trim it for
the intended target audience in some usages, and only include the full
thing on the website.

Rich


----------------------------------------------------------------------

Consistent quality and implementation behavior from tiny embedded
systems to full servers.

Minimal machine-specific code, meaning less chance of breakage on
minority architectures and better success with "write once run
everywhere" development.

Extremely-efficient static and dynamic linking support, yielding small
binaries and minimal startup overhead.

Realtime-quality robustness. No unnecessary dynamic allocation. No
unrecoverable late failures. No lazy binding or lazy allocation.

MIT license.

Full math library with a focus on correctness. Exact and
correctly-rounded conversion between binary floating point and decimal
strings.

Reentrancy, thread-safety, and async-signal safety well beyond the
requirements of POSIX. Even snprintf and dprintf are fully reentrant
and async-signal-safe.

Highly resource-efficient POSIX threads implementation, making
multi-threaded application design viable even for memory-constrained
systems.

Simple source code and source tree layout, so it's easy to customize
or track down the cause of unexpected behavior or bugs, or simply
learn how the library works.

Re: Draft: musl promo materials

[orc]

On Fri, 13 Jul 2012 21:16:09 +0200
Szabolcs Nagy <nsz@port70.net> wrote:

> * Rich Felker <dalias@aerifal.cx> [2012-07-13 14:12:54 -0400]:
> > Consistent quality and implementation behavior from tiny embedded
> > systems to full servers.
> > 
> > Minimal machine-specific code, meaning less chance of breakage on
> > minority architectures and better success with "write once run
> > everywhere" development.
> > 
> > Realtime-quality robustness. No unnecessary dynamic allocation. No
> > unrecoverable late failures. No lazy binding or lazy allocation.
> > 
> > MIT license.
> > 
> > Full math library with a focus on correctness. Exact and
> > correctly-rounded conversion between binary floating point and
> > decimal strings.
> > 
> > Reentrancy, thread-safety, and async-signal safety well beyond the
> > requirements of POSIX. Even snprintf and dprintf are fully reentrant
> > and async-signal-safe.
> > 
> > Highly resource-efficient POSIX threads implementation, making
> > multi-threaded application design viable even for memory-constrained
> > systems.
> 
> i'd somehow add that both static and dynamic linking is supported
> properly and without bloat as musl is better at it than glibc
> 
> i like musl's clean code, clean header files (no gcc specific mess)
> and simple build system (even for cross compilation)
> again something that glibc is lacking

Agree with that. musl allowed me implement a variant of skein crypt()
for example, and much more things than mess called glibc. It's source
code is easy to understand and good start point to learn how libc works.

> 
> and there could be a hint that things like security, worst cases
> (stack usage, algorithm complexity) and conformance are taken
> seriously

Re: Draft: musl promo materials

[Isaac Dunham]

On Fri, 13 Jul 2012 19:30:52 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> Updated version based on some comments.. I think the list is getting
> long enough that it would possibly make sense to reorder/trim it for
> the intended target audience in some usages, and only include the full
> thing on the website.
> 
> Rich
> 
> 
> ----------------------------------------------------------------------
> 
> Consistent quality and implementation behavior from tiny embedded
> systems to full servers.
> 
> Minimal machine-specific code, meaning less chance of breakage on
> minority architectures and better success with "write once run
> everywhere" development.

One criticism I've heard (not saying I agree!) is that you lose
performance with musl thanks to most functions being in C...

> Extremely efficient static and dynamic linking support, yielding small
> binaries and minimal startup overhead.
> 
> Realtime-quality robustness. No unnecessary dynamic allocation. No
> unrecoverable late failures. No lazy binding or lazy allocation.
> 
This reminded me about _XOPEN_REALTIME:
"This Option Group consists of the set of the following options from
within POSIX.1-2008 (see Options ):

    _POSIX_FSYNC
    _POSIX_MEMLOCK
    _POSIX_MEMLOCK_RANGE
    _POSIX_MESSAGE_PASSING
    _POSIX_PRIORITIZED_IO
    _POSIX_PRIORITY_SCHEDULING
    _POSIX_SHARED_MEMORY_OBJECTS
    _POSIX_SYNCHRONIZED_IO
"
Of these, the MEMLOCK ones are the only ones defined yet.
It seems _POSIX_FSYNC is supported, but not advertised.
The other 4 are not defined yet; what's musl's  support for them?
I'm curious how far from  complete _XOPEN_REALTIME support musl is.

X/Open also defines the Advanced Realtime, Realtime Threads, and
Advanced Realtime Threads options.

> MIT license.
> 
> Full math library with a focus on correctness. Exact and
> correctly rounded conversion between binary floating point and decimal
> strings.
> 
> Reentrancy, thread-safety, and async-signal safety well beyond the
> requirements of POSIX. Even snprintf and dprintf are fully reentrant
> and async-signal-safe.
> 
> Highly resource-efficient POSIX threads implementation, making
> multi-threaded application design viable even for memory-constrained
> systems.
> 
> Simple source code and source tree layout, so it's easy to customize
> or track down the cause of unexpected behavior or bugs, or simply
> learn how the library works.

Other than the mention of realtime when no proper realtime support is
advertised in the headers, seems pretty good.

Re: Draft: musl promo materials

[Charlie Kester]

On 07/13/2012 11:12 AM, Rich Felker wrote:
> Hi all,
> Here's a quick draft (and request for input/suggestions) of
> points/highlights about musl to eventually be incorporated into the
> website and other opportunities to promote musl. With
> embedded-relevant ports getting integrated now, I'm thinking this sort
> of thing will also be useful for sending announcements to relevant
> mailing lists, etc. Anything big I'm missing?
>
> Rich

It would be good to include a list of apps that have been successfully 
compiled with musl.  No need to list every such app, just some 
representative ones that will help dispel any fears about the library's 
ability to replace glibc in the most common uses.

Re: Draft: musl promo materials

[Isaac Dunham]

On Sat, 14 Jul 2012 14:45:40 -0700
Charlie Kester <corky1951@comcast.net> wrote:


> It would be good to include a list of apps that have been
> successfully compiled with musl.  No need to list every such app,
> just some representative ones that will help dispel any fears about
> the library's ability to replace glibc in the most common uses.

Somehow, listing compatible applications in PR does not
sound very encouraging, until you can run the really huge
suites/applications (I'm thinking of glibc-linked NVIDIA + Xorg, as was
recently done with a patched version of musl, Google Earth, or
native-built OpenOffice, Firefox/Chromium, the GIMP, Wine, Java, Mesa,
GNOME, KDE, maybe Xfce, R, GRASS, BRL-CAD, FreeCAD, TeX, GDAL...the
list isn't exclusive or a recommendation of the software, but it
should give an idea of the scale I'm talking about). If you say "can
run GCC 4.7, PHP, Xfbdev, Xfce 4.10, SDL, Tomoyo and grsec" (Xfce is in
Snowflake, the rest are the big names from sabotage), that sure seems
to reinforce the impression you're concerned about by virtue of
omission. 
I'd prefer to see a link to "further information and compatability" or
some such thing.

Isaac Dunham

Re: Draft: musl promo materials

[Gregor Richards]

On 07/14/2012 07:06 PM, Isaac Dunham wrote:
> On Sat, 14 Jul 2012 14:45:40 -0700
> Charlie Kester <corky1951@comcast.net> wrote:
>
>
>> It would be good to include a list of apps that have been
>> successfully compiled with musl.  No need to list every such app,
>> just some representative ones that will help dispel any fears about
>> the library's ability to replace glibc in the most common uses.
> Somehow, listing compatible applications in PR does not
> sound very encouraging, until you can run the really huge
> suites/applications (I'm thinking of glibc-linked NVIDIA + Xorg, as was
> recently done with a patched version of musl, Google Earth, or
> native-built OpenOffice, Firefox/Chromium, the GIMP, Wine, Java, Mesa,
> GNOME, KDE, maybe Xfce, R, GRASS, BRL-CAD, FreeCAD, TeX, GDAL...the
> list isn't exclusive or a recommendation of the software, but it
> should give an idea of the scale I'm talking about). If you say "can
> run GCC 4.7, PHP, Xfbdev, Xfce 4.10, SDL, Tomoyo and grsec" (Xfce is in
> Snowflake, the rest are the big names from sabotage), that sure seems
> to reinforce the impression you're concerned about by virtue of
> omission.
> I'd prefer to see a link to "further information and compatability" or
> some such thing.
>
> Isaac Dunham
>
GIMP and XFCE work. I'm working on Firefox, but I believe it currently 
hits a bug in musl's dlopen, so I'm waiting on that fix. It compiles, it 
just doesn't launch (hyuk). I routinely run the complete pkgsrc suite 
through musl to see what works; the current test is ongoing, but here's 
the results of the most recent one, which unfortunately suffered from 
some bugs in my setup and so has some false build failures:

Tried 10444, depfail 4489, depok 5955, nobuild 2454, testfail 336, 
success 3165 (53.1486%). Breakdown: http://sprunge.us/jgDO . MVD: 
http://sprunge.us/cjRK

(Tried = packages attempted, depfail = deps failed to build, nobuild = 
didn't build, testfail = built but tests failed, success = all tests 
passed, percentage is out of depok, MVD = most valuable dependencies, of 
which I have since fixed security/p5-Crypt-Rijndael and textproc/xmlrpc-c )

I suppose I'll post the results of the latest run on the mailing list 
once it's done. I've mainly stuck to the IRC channel.

With valediction,
  - Gregor Richards

Re: Draft: musl promo materials

[Rich Felker]

On Sat, Jul 14, 2012 at 12:05:15PM -0700, Isaac Dunham wrote:
> On Fri, 13 Jul 2012 19:30:52 -0400
> Rich Felker <dalias@aerifal.cx> wrote:
> 
> > Updated version based on some comments.. I think the list is getting
> > long enough that it would possibly make sense to reorder/trim it for
> > the intended target audience in some usages, and only include the full
> > thing on the website.
> > 
> > Rich
> > 
> > 
> > ----------------------------------------------------------------------
> > 
> > Consistent quality and implementation behavior from tiny embedded
> > systems to full servers.
> > 
> > Minimal machine-specific code, meaning less chance of breakage on
> > minority architectures and better success with "write once run
> > everywhere" development.
> 
> One criticism I've heard (not saying I agree!) is that you lose
> performance with musl thanks to most functions being in C...

I haven't found that to be the case. When it is, asm can be added if
needed. If you're aware of some operations that are significantly
slower than glibc and believe lack of asm is the problem, please
report them.

> > Realtime-quality robustness. No unnecessary dynamic allocation. No
> > unrecoverable late failures. No lazy binding or lazy allocation.
> > 
> This reminded me about _XOPEN_REALTIME:
> "This Option Group consists of the set of the following options from
> within POSIX.1-2008 (see Options ):
> [...]
> I'm curious how far from  complete _XOPEN_REALTIME support musl is.

When I refer to realtime-quality, it's a matter of the entire library
having suitable characteristics for realtime use - things like not
crashing from resouce exhaustion, not having gratuitious resource
contention issues, etc. The _XOPEN_REALTIME option is full of extra
functionality that's useful for programming idioms that are commonly
used/taught for realtime programming, which is a separate issue.
Amusingly, some libcs go out of their way to support the latter and
completely fail to support the former, making the latter utterly
useless.

Rich

Re: Draft: musl promo materials

[Rich Felker]

On Sat, Jul 14, 2012 at 04:06:41PM -0700, Isaac Dunham wrote:
> On Sat, 14 Jul 2012 14:45:40 -0700
> Charlie Kester <corky1951@comcast.net> wrote:
> 
> 
> > It would be good to include a list of apps that have been
> > successfully compiled with musl.  No need to list every such app,
> > just some representative ones that will help dispel any fears about
> > the library's ability to replace glibc in the most common uses.
> 
> Somehow, listing compatible applications in PR does not
> sound very encouraging, until you can run the really huge

I agree completely. A compatibility list should definitely not be part
of the long-term promotional strategy. Early on, it's been useful for
getting people's attention that musl is a rising contender, but I also
believe it presents an impression that compatibility is a
concern/downside of musl.

Rich

Re: Draft: musl promo materials

[Isaac Dunham]

On Sat, 14 Jul 2012 19:30:38 -0400
Gregor Richards <gr@purdue.edu> wrote:

> On 07/14/2012 07:06 PM, Isaac Dunham wrote:
> > On Sat, 14 Jul 2012 14:45:40 -0700
> > Charlie Kester <corky1951@comcast.net> wrote:
> >
> >
> >> It would be good to include a list of apps that have been
> >> successfully compiled with musl.  No need to list every such app,
> >> just some representative ones that will help dispel any fears about
> >> the library's ability to replace glibc in the most common uses.
> > Somehow, listing compatible applications in PR does not
> > sound very encouraging, until you can run the really huge
> > suites/applications (I'm thinking of glibc-linked NVIDIA + Xorg, as
> > was recently done with a patched version of musl, Google Earth, or
> > native-built OpenOffice, Firefox/Chromium, the GIMP, Wine, Java,
> > Mesa, GNOME, KDE, maybe Xfce, R, GRASS, BRL-CAD, FreeCAD, TeX,
> > GDAL...the list isn't exclusive or a recommendation of the
> > 
...
> > Isaac Dunham
> >
> GIMP and XFCE work. I'm working on Firefox, but I believe it
> currently hits a bug in musl's dlopen, so I'm waiting on that fix. It
> compiles, it just doesn't launch (hyuk). I routinely run the complete
> pkgsrc suite through musl to see what works; the current test is
> ongoing, but here's the results of the most recent one, which
> unfortunately suffered from some bugs in my setup and so has some
> false build failures:
> 
> Tried 10444, depfail 4489, depok 5955, nobuild 2454, testfail 336, 
> success 3165 (53.1486%). Breakdown: http://sprunge.us/jgDO . MVD: 
> http://sprunge.us/cjRK
> 
> (Tried = packages attempted, depfail = deps failed to build, nobuild
> = didn't build, testfail = built but tests failed, success = all
> tests passed, percentage is out of depok, MVD = most valuable
> dependencies, of which I have since fixed security/p5-Crypt-Rijndael
> and textproc/xmlrpc-c )
php, sdl - see latest sabotage pkg for patches
perl, mpg123, mutt... - broken setup or wrong config? I've built both
w/out patching, but some are picky about the config.
heimdal-seriously, someone still wants Kerberos 4 ?!
(heimdal is a Kerberos4-only implementation; this means only DES-based
encryption, and MIT Kerberos has deprecated Kerberos 4)
ap-auth-kerb - Presume this is MIT Kerberos; 1.8.x can be built with
one small patch, but 1.10.x fails thanks to Redhat's broken code in
libverto...
heirloom-* - I had to patch it (mainly makefile changes, one or
two source changes) last time I tried, but that's several months ago.
cups - I think this wants libusb, which I haven't seen built...

I'm wondering about how complete C++ support is, or whether
__{BEGIN,END}_DECLS is overused--Fox, Qt, FLTK, and xpdf are all C++.

Overall, I'd expect you've got at least a couple hundred false failures,
but I notice that Mesa and parts of TeX are building, which is pretty
impressive itself.

Status of packages mentioned:
Builds:
Mesa, GIMP, Xfce, TeX (partial)
testfail:
firefox
buildfail/testfail:
GNOME, TeX (partial)
depfail:
OpenOffice, R, Wine, Java, KDE, GRASS?

Isaac Dunham

Re: Draft: musl promo materials

[Gregor Richards]

On 07/14/2012 08:49 PM, Isaac Dunham wrote:
> On Sat, 14 Jul 2012 19:30:38 -0400
> Gregor Richards<gr@purdue.edu>  wrote:
>
>> On 07/14/2012 07:06 PM, Isaac Dunham wrote:
>>> On Sat, 14 Jul 2012 14:45:40 -0700
>>> Charlie Kester<corky1951@comcast.net>  wrote:
>>>
>>>
>>>> It would be good to include a list of apps that have been
>>>> successfully compiled with musl.  No need to list every such app,
>>>> just some representative ones that will help dispel any fears about
>>>> the library's ability to replace glibc in the most common uses.
>>> Somehow, listing compatible applications in PR does not
>>> sound very encouraging, until you can run the really huge
>>> suites/applications (I'm thinking of glibc-linked NVIDIA + Xorg, as
>>> was recently done with a patched version of musl, Google Earth, or
>>> native-built OpenOffice, Firefox/Chromium, the GIMP, Wine, Java,
>>> Mesa, GNOME, KDE, maybe Xfce, R, GRASS, BRL-CAD, FreeCAD, TeX,
>>> GDAL...the list isn't exclusive or a recommendation of the
>>>
> ...
>>> Isaac Dunham
>>>
>> GIMP and XFCE work. I'm working on Firefox, but I believe it
>> currently hits a bug in musl's dlopen, so I'm waiting on that fix. It
>> compiles, it just doesn't launch (hyuk). I routinely run the complete
>> pkgsrc suite through musl to see what works; the current test is
>> ongoing, but here's the results of the most recent one, which
>> unfortunately suffered from some bugs in my setup and so has some
>> false build failures:
>>
>> Tried 10444, depfail 4489, depok 5955, nobuild 2454, testfail 336,
>> success 3165 (53.1486%). Breakdown: http://sprunge.us/jgDO . MVD:
>> http://sprunge.us/cjRK
>>
>> (Tried = packages attempted, depfail = deps failed to build, nobuild
>> = didn't build, testfail = built but tests failed, success = all
>> tests passed, percentage is out of depok, MVD = most valuable
>> dependencies, of which I have since fixed security/p5-Crypt-Rijndael
>> and textproc/xmlrpc-c )
> php, sdl - see latest sabotage pkg for patches
I will, thanks.
> perl, mpg123, mutt... - broken setup or wrong config? I've built both
> w/out patching, but some are picky about the config.
Uhh, perl builds fine for me in pkgsrc? Not sure why you thought 
otherwise. Only its tests failed. mpg123 and mutt probably just build 
with more dependencies and end up exploding.
> heimdal-seriously, someone still wants Kerberos 4 ?!
> (heimdal is a Kerberos4-only implementation; this means only DES-based
> encryption, and MIT Kerberos has deprecated Kerberos 4)
The fact that it exists in pkgsrc does not indicate that it's wildly used.
> ap-auth-kerb - Presume this is MIT Kerberos; 1.8.x can be built with
> one small patch, but 1.10.x fails thanks to Redhat's broken code in
> libverto...
apache2 didn't build, X|-|-| means dependency failure.
> heirloom-* - I had to patch it (mainly makefile changes, one or
> two source changes) last time I tried, but that's several months ago.
I really don't care to get every package in pkgsrc building. I don't 
think anybody cares one iota if this works.
> cups - I think this wants libusb, which I haven't seen built...
Probably. I don't have the full log for this build any more.
>
> I'm wondering about how complete C++ support is, or whether
> __{BEGIN,END}_DECLS is overused--Fox, Qt, FLTK, and xpdf are all C++.
I haven't seen C++ fail significantly, plenty of things work.
>
> Overall, I'd expect you've got at least a couple hundred false failures,
> but I notice that Mesa and parts of TeX are building, which is pretty
> impressive itself.
There are also false failures just because my build setup was all borked 
before, so we'll have to hold off to see more realistic results.
>
> Status of packages mentioned:
> Builds:
> Mesa, GIMP, Xfce, TeX (partial)
> testfail:
> firefox
> buildfail/testfail:
> GNOME, TeX (partial)
> depfail:
> OpenOffice, R, Wine, Java, KDE, GRASS?
>
> Isaac Dunham
>

With valediction,
  - Gregor Richards