Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rob Landley]

Notes from the discussion we had on IRC, plus some further random  
thoughts on telling the world about musl:

- wait until 1.0 so it's most likely to works for them.
   - People who take a look and wander off again are less likely to  
take another look,
     so try to make a spash when you're _ready_, not before.
   - counter this with "rule of 7", people filter out noise and won't  
remember they've
     even heard of you until they've seen it in ~7 different places. So  
once you _ARE_
     ready, get the word out everywhere. (Politely.)

- prepare the website to covert casual browsers into long-term users.
   - press release extoling virtues
     - simple
       - realtime: less code is more deterministic
       - security: less code is easier to audit
       - students/teachers: learn how a posix system works
     - link to the online git browser for the "show me the code" guys.
     - already tested against 8 gazillion packages
     - standards compliant
     - BSD license: static linking ok, android deployment ok
     - works side by side with existing libraries, or static linked
       - easy deployment on android without bionic limitations
     - technical advantages
       - support static and dynamic linking and do _both_ well
       - thread implementation is _not_crazy_, and no legacy baggage.

   - obvious "start here" from main page.
     - Why it's cool (collate)
     - how to use it (collate)
       - HOWTO walkthrough

   - binaries they can try.
     - cross compiler, build hello world
     - livecd of full-ish x86 distro.
       - with working x11 and simple gui (xfce? fvwm?)
     - chroot for each target with native development tools
       - system images for qemu maybe?
       - launch x11 vnc server and display in tightvnc window?
     - jslinux live image on website

- distro conversions
   - leverage existing repositories, don't fall into the buildroot trap
   - approach gentoo guys about a musl build
     - #gentoo-embedded on freenode
     - maybe funtoo would be easier (Daniel Robbins' new project,  
#funtoo on freenode)
   - approach debian guys about musl debootstrap
   - arch linux, slackware, puppy, crunchbang, tinycore...
     - http://distrowatch.com/popularity
   - approach cyanogenmod guys about doing a musl-based cyanogenmod.
     - way into man's heart is through the stomach and up under the  
ribcage,
       one way into android is cyanogenmod.

- push "musl support" patches to other projects upstream all at once
   - sabotage collected a bunch?
   - people who develop on 3 other project seeing musl on all 3 lists
     makes dev community look big and active.

- Write linux from scratch "musl hint", contribute it to LFS, then link
   to it on LFS website from musl website.

- is userbase of glibc, uClibc, klibc, or dietlibc better served by  
musl?
   - contribute musl option to buildroot?
   - contribute musl option to crosstool-ng?
   - Ask mentor graphics (formly code sourcery) to do a musl toolchain?
     - LOTS of proprietary embedded devs use this one, it's  
"professional".
     - windriver.com is now a wholly owned subsidiary of intel
   - klibc guys are initramfs@vger or embedded@vger (see lists)
     - ask clibc author Peter Anvin if musl serves his needs?

- mailing lists you can post a "here's how musl can help _you_" on:
   It's not spam if you tailor a post to each list, especially if  
there's patches
   attached in the case of dash or util-linux...
   - each architecture list for arches you support (linux-arm,  
linux-ppc, etc).
     "musl is pleased to announce support for the $BLAH architecture,  
here are
      a cross compiler, chroot with native compiler, and a system image  
to play with."
     - http://www.arm.linux.org.uk/mailinglists/lists.php
     - http://www.linux-mips.org/wiki/Net_Resources#Mailing_lists
     - https://lists.ozlabs.org/listinfo/linuxppc-dev
     - http://vger.kernel.org/vger-lists.html#linux-x86_64
   - http://vger.kernel.org/vger-lists.html#dash
   - http://vger.kernel.org/vger-lists.html#initramfs
   - http://vger.kernel.org/vger-lists.html#linux-embedded
   - http://vger.kernel.org/vger-lists.html#util-linux
   - and maybe one "OS support" message to linux-kernel.

- websites that might review musl if we ask nicely:
   - linux
     - lwn.net (submit via lwn@lwn.net)
     - h-online (ping @codepope on twitter)
     - Linux Journal
     - Linux Today (they'll just link elsewhere)
   - android
     - not personally familiar, google for "android news" finds several.
     - works well with android kernel, installs side-by-side with  
bionic,
       static links well, doesn't introduce any new licensing issues,
       provides full posix environment, active and responsive dev  
community.
   - paper magazines
     - long shot, but if you send a press release to pc magazine and  
computerworld
       and such explaining how musl might help android bridge the gap  
between phones
       and the desktop they might write a "will android bridge the gap  
between phones
       and the desktop" article mentioning musl. :)
   - tech bloggers
     - cringely.com
   - Consumer Electronic Linux Forum
     - Tim Bird and elinux.org

- do a musl distro that runs well on raspberry pi, tell  
http://www.raspberrypi.org/

- ask people on mailing list and irc to blog/tweet about the 1.0  
release when it
   happens.

- write a syllabus for theoretical "teaching musl" one semester  
comp-sci course.

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Justin Cormack]

[-- Attachment #1: Type: text/plain, Size: 5742 bytes --]

On 29 Nov 2012 20:50, "Rob Landley" <rob@landley.net> wrote:
>
> Notes from the discussion we had on IRC, plus some further random
thoughts on telling the world about musl:
>
> - wait until 1.0 so it's most likely to works for them.
>   - People who take a look and wander off again are less likely to take
another look,
>     so try to make a spash when you're _ready_, not before.
>   - counter this with "rule of 7", people filter out noise and won't
remember they've
>     even heard of you until they've seen it in ~7 different places. So
once you _ARE_
>     ready, get the word out everywhere. (Politely.)
>
> - prepare the website to covert casual browsers into long-term users.
>   - press release extoling virtues
>     - simple
>       - realtime: less code is more deterministic
>       - security: less code is easier to audit
>       - students/teachers: learn how a posix system works
>     - link to the online git browser for the "show me the code" guys.
>     - already tested against 8 gazillion packages
>     - standards compliant
>     - BSD license: static linking ok, android deployment ok
>     - works side by side with existing libraries, or static linked
>       - easy deployment on android without bionic limitations
>     - technical advantages
>       - support static and dynamic linking and do _both_ well
>       - thread implementation is _not_crazy_, and no legacy baggage.
>
>   - obvious "start here" from main page.
>     - Why it's cool (collate)
>     - how to use it (collate)
>       - HOWTO walkthrough
>
>   - binaries they can try.
>     - cross compiler, build hello world
>     - livecd of full-ish x86 distro.
>       - with working x11 and simple gui (xfce? fvwm?)
>     - chroot for each target with native development tools
>       - system images for qemu maybe?
>       - launch x11 vnc server and display in tightvnc window?
>     - jslinux live image on website
>
> - distro conversions
>   - leverage existing repositories, don't fall into the buildroot trap
>   - approach gentoo guys about a musl build
>     - #gentoo-embedded on freenode
>     - maybe funtoo would be easier (Daniel Robbins' new project, #funtoo
on freenode)
>   - approach debian guys about musl debootstrap
>   - arch linux, slackware, puppy, crunchbang, tinycore...
>     - http://distrowatch.com/popularity
>   - approach cyanogenmod guys about doing a musl-based cyanogenmod.
>     - way into man's heart is through the stomach and up under the
ribcage,
>       one way into android is cyanogenmod.
>
> - push "musl support" patches to other projects upstream all at once
>   - sabotage collected a bunch?
>   - people who develop on 3 other project seeing musl on all 3 lists
>     makes dev community look big and active.
>
> - Write linux from scratch "musl hint", contribute it to LFS, then link
>   to it on LFS website from musl website.
>
> - is userbase of glibc, uClibc, klibc, or dietlibc better served by musl?
>   - contribute musl option to buildroot?
>   - contribute musl option to crosstool-ng?
>   - Ask mentor graphics (formly code sourcery) to do a musl toolchain?
>     - LOTS of proprietary embedded devs use this one, it's "professional".
>     - windriver.com is now a wholly owned subsidiary of intel
>   - klibc guys are initramfs@vger or embedded@vger (see lists)
>     - ask clibc author Peter Anvin if musl serves his needs?
>
> - mailing lists you can post a "here's how musl can help _you_" on:
>   It's not spam if you tailor a post to each list, especially if there's
patches
>   attached in the case of dash or util-linux...
>   - each architecture list for arches you support (linux-arm, linux-ppc,
etc).
>     "musl is pleased to announce support for the $BLAH architecture, here
are
>      a cross compiler, chroot with native compiler, and a system image to
play with."
>     - http://www.arm.linux.org.uk/mailinglists/lists.php
>     - http://www.linux-mips.org/wiki/Net_Resources#Mailing_lists
>     - https://lists.ozlabs.org/listinfo/linuxppc-dev
>     - http://vger.kernel.org/vger-lists.html#linux-x86_64
>   - http://vger.kernel.org/vger-lists.html#dash
>   - http://vger.kernel.org/vger-lists.html#initramfs
>   - http://vger.kernel.org/vger-lists.html#linux-embedded
>   - http://vger.kernel.org/vger-lists.html#util-linux
>   - and maybe one "OS support" message to linux-kernel.
>
> - websites that might review musl if we ask nicely:
>   - linux
>     - lwn.net (submit via lwn@lwn.net)
>     - h-online (ping @codepope on twitter)
>     - Linux Journal
>     - Linux Today (they'll just link elsewhere)
>   - android
>     - not personally familiar, google for "android news" finds several.
>     - works well with android kernel, installs side-by-side with bionic,
>       static links well, doesn't introduce any new licensing issues,
>       provides full posix environment, active and responsive dev
community.
>   - paper magazines
>     - long shot, but if you send a press release to pc magazine and
computerworld
>       and such explaining how musl might help android bridge the gap
between phones
>       and the desktop they might write a "will android bridge the gap
between phones
>       and the desktop" article mentioning musl. :)
>   - tech bloggers
>     - cringely.com
>   - Consumer Electronic Linux Forum
>     - Tim Bird and elinux.org
>
> - do a musl distro that runs well on raspberry pi, tell
http://www.raspberrypi.org/
>
> - ask people on mailing list and irc to blog/tweet about the 1.0 release
when it
>   happens.
>
> - write a syllabus for theoretical "teaching musl" one semester comp-sci
cour

That's a great list.

Also conferences. FOSDEM is in Feb and is a good place talks still open.

Will work on what I can...

Justin

[-- Attachment #2: Type: text/html, Size: 7906 bytes --]

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Luca Barbato]

On 11/29/2012 10:15 PM, Justin Cormack wrote:
>> - distro conversions
>>   - leverage existing repositories, don't fall into the buildroot trap
>>   - approach gentoo guys about a musl build
>>     - #gentoo-embedded on freenode
>>     - maybe funtoo would be easier (Daniel Robbins' new project, #funtoo
> on freenode)

Given you have already at least 3 gentoo developers looking at you I'm
not sure what's easier =) (poke me on irc this weekend to complete the
crossdev musl target and ping Flameeyes regarding a gentoo/musl tinderbox)

>>   - paper magazines
>>     - long shot, but if you send a press release to pc magazine and
> computerworld
>>       and such explaining how musl might help android bridge the gap
> between phones
>>       and the desktop they might write a "will android bridge the gap
> between phones
>>       and the desktop" article mentioning musl. :)

I might help there.

> 
> Also conferences. FOSDEM is in Feb and is a good place talks still open.
> 

Fosdem might be good.

lu

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rich Felker]

On Thu, Nov 29, 2012 at 09:15:38PM +0000, Justin Cormack wrote:
> Also conferences. FOSDEM is in Feb and is a good place talks still
> open.

I'd welcome help putting together a list of possible conferences to go
to. I don't have a budget for international conference travel, but if
there are some in the US coming up soon, I could pursue them. For
FOSDEM or others, I'd also consider working with somebody else close
with the project to put together a talk. And of course if anybody has
leads on somebody willing to fund my travel to FOSDEM or other
international conferences, that's even better. :-)

Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rob Landley]

On 11/29/2012 07:30:08 PM, Rich Felker wrote:
> On Thu, Nov 29, 2012 at 09:15:38PM +0000, Justin Cormack wrote:
> > Also conferences. FOSDEM is in Feb and is a good place talks still
> > open.
> 
> I'd welcome help putting together a list of possible conferences to go
> to. I don't have a budget for international conference travel, but if
> there are some in the US coming up soon, I could pursue them. For
> FOSDEM or others, I'd also consider working with somebody else close
> with the project to put together a talk. And of course if anybody has
> leads on somebody willing to fund my travel to FOSDEM or other
> international conferences, that's even better. :-)

Some conferences have travel budgets for speakers.

And https://www.linuxfoundation.org/programs/developer/travel/request  
might still be active. (They're big and clueless, but they have money.)

By the way: recording your own talks is nice too. You can do little 2  
minute segments with your phone and/or

> Rich
> 

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rob Landley]

On 11/29/2012 03:15:38 PM, Justin Cormack wrote:
> Also conferences. FOSDEM is in Feb and is a good place talks still  
> open.

"Lessons learned from writing a new linux/android C library from  
scratch" would be a fun talk. Video could be linked from the musl  
website...

Rob

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Szabolcs Nagy]

* Rob Landley <rob@landley.net> [2012-11-30 03:21:10 -0600]:
> On 11/29/2012 03:15:38 PM, Justin Cormack wrote:
> >Also conferences. FOSDEM is in Feb and is a good place talks still
> >open.
> 
> "Lessons learned from writing a new linux/android C library from
> scratch" would be a fun talk. Video could be linked from the musl
> website...
> 

reminds me that i wanted to collect "software bugs found by musl"

(so far there are about 10 formal bugs against the posix standard
some of them are serious, there is even an accepted editorial
issue in the c11 standard, dalias made several reports against glibc
some of them recorded on ewontfix and there are gcc, clang and many
pcc issues found by musl +the many patches against software packages
some of which already got upstream and various bugs fixed in internal
musl code that might affect others: tre, math, crypto..)

the design or quality issues are more interesting ones but harder
to enumerate
(it would be nice to have a collection of broken/dangerous apis and
general libc issues that affect others not just musl users, this
probably can be part of the documentation effort)

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rob Landley]

On 11/30/2012 05:08:39 AM, Szabolcs Nagy wrote:
> * Rob Landley <rob@landley.net> [2012-11-30 03:21:10 -0600]:
> > On 11/29/2012 03:15:38 PM, Justin Cormack wrote:
> > >Also conferences. FOSDEM is in Feb and is a good place talks still
> > >open.
> >
> > "Lessons learned from writing a new linux/android C library from
> > scratch" would be a fun talk. Video could be linked from the musl
> > website...
> >
> 
> reminds me that i wanted to collect "software bugs found by musl"
> 
> (so far there are about 10 formal bugs against the posix standard
> some of them are serious, there is even an accepted editorial
> issue in the c11 standard, dalias made several reports against glibc
> some of them recorded on ewontfix and there are gcc, clang and many
> pcc issues found by musl +the many patches against software packages
> some of which already got upstream and various bugs fixed in internal
> musl code that might affect others: tre, math, crypto..)

I look forward to seeing this talk.

Rob

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rich Felker]

On Thu, Nov 29, 2012 at 02:50:03PM -0600, Rob Landley wrote:
> Notes from the discussion we had on IRC, plus some further random
> thoughts on telling the world about musl:
> 
> - wait until 1.0 so it's most likely to works for them.
>   - People who take a look and wander off again are less likely to
> take another look,
>     so try to make a spash when you're _ready_, not before.

Agreed. I never really wrote down this principle before, but it's been
in my mind for a long time. The main publicizing I've been doing so
far has been in niche places (like mips and microblaze lists) and
making sure musl is well-indexed in databases like freecode (formerly
freshmeat) and ohloh. I suspect there are a lot more of the latter we
should also hit.

>   - counter this with "rule of 7", people filter out noise and won't
> remember they've
>     even heard of you until they've seen it in ~7 different places.
> So once you _ARE_
>     ready, get the word out everywhere. (Politely.)

Yep. I especially liked your idea of holding off on some of the
upstream portability fix patches and sending them all at once as a
barrage. We might want to find someone in advance to organize the list
of where patches need to go and organize some people to handle posting
and following up on the bug reports. The *politely* element is key.
Anyone involved in this should spend some time practicing how to make
the reports politely rather than busting in with "omg ur software sux"
or similar. Even if many of the packages that need the most work are
packages that _we_ aren't interested in using for our own legacy-free
systems, working diplomatically with them and getting their bugs fixed
so that [potential] musl users don't come back with "why doesn't
such-and-such work with musl??" is important.

> - prepare the website to covert casual browsers into long-term users.
>   - press release extoling virtues
>     - simple
>       - realtime: less code is more deterministic
>       - security: less code is easier to audit
>       - students/teachers: learn how a posix system works
>     - link to the online git browser for the "show me the code" guys.
>     - already tested against 8 gazillion packages
>     - standards compliant
>     - BSD license: static linking ok, android deployment ok
>     - works side by side with existing libraries, or static linked
>       - easy deployment on android without bionic limitations
>     - technical advantages
>       - support static and dynamic linking and do _both_ well
>       - thread implementation is _not_crazy_, and no legacy baggage.

Nice bullet points. I think we could expand on these some, but you
already hit a lot of good ones I was missing or overlooking.

>   - obvious "start here" from main page.
>     - Why it's cool (collate)
>     - how to use it (collate)
>       - HOWTO walkthrough
> 
>   - binaries they can try.
>     - cross compiler, build hello world
>     - livecd of full-ish x86 distro.
>       - with working x11 and simple gui (xfce? fvwm?)

I'd go with xfce or lxde. Last I checked fvwm still didn't even have
modern text support (using legacy X font system). BTW, supposedly the
new HarfBuzz and other text rendering stack stuff has taken a turn in
the direction we like. I saw an article on the redesign that claimed
98-99% drop in bloat for working with fonts. I mention this mainly
because, as much as a lot of us are unhappy with FDO and related
projects, it would be nice to support and praise when that stuff does
move in the right direction.

>     - chroot for each target with native development tools
>       - system images for qemu maybe?

It would be really cool if we had "system root trees" rather than
"system images", with virtual 9p for booting qemu. By the way, getting
qemu working on musl should probably be in our compatibility goals for
1.0.

>       - launch x11 vnc server and display in tightvnc window?
>     - jslinux live image on website

Love this idea. I'd like it even better if we could adapt jslinux to
start with a preinitialized memory/machine-state image rather than
having to spend 15-20 seconds booting. BTW I think we need to look
into whether there are license issues with using jslinux...?

By the way, somebody also mentioned a mips emulator in js we could use
instead or in addition to jslinux, but I don't know if it emulates
enough hardware to get linux up.

> - distro conversions
>   - leverage existing repositories, don't fall into the buildroot trap

I think I've already avoided this trap without avoiding new distros,
by building a community of people who like rolling their own distros
but not getting involved in the distro/build stuff at all myself. This
has been great for getting musl a lot of testing with lower entry
barrier than large existing distros, but adding the latter now would
be a good next step.

>   - approach gentoo guys about a musl build
>     - #gentoo-embedded on freenode
>     - maybe funtoo would be easier (Daniel Robbins' new project,
> #funtoo on freenode)
>   - approach debian guys about musl debootstrap
>   - arch linux, slackware, puppy, crunchbang, tinycore...
>     - http://distrowatch.com/popularity
>   - approach cyanogenmod guys about doing a musl-based cyanogenmod.
>     - way into man's heart is through the stomach and up under the
> ribcage,
>       one way into android is cyanogenmod.

I like the idea of approaching cyanogenmod.

> - push "musl support" patches to other projects upstream all at once
>   - sabotage collected a bunch?
>   - people who develop on 3 other project seeing musl on all 3 lists
>     makes dev community look big and active.

I like this idea a lot. See notes above.

> - Write linux from scratch "musl hint", contribute it to LFS, then link
>   to it on LFS website from musl website.

Can you elaborate on what a "musl hint" would be?

> - is userbase of glibc, uClibc, klibc, or dietlibc better served by
> musl?
>   - contribute musl option to buildroot?
>   - contribute musl option to crosstool-ng?

There's demand for musl in crosstool-ng at least; the maintainer has
expressed this, but is largely unfamiliar with musl and waiting for a
patch from someone willing to do it.

>   - Ask mentor graphics (formly code sourcery) to do a musl toolchain?
>     - LOTS of proprietary embedded devs use this one, it's
> "professional".

I suspect they might be particularly interested since their customers
might need to avoid copyleft code.

>     - windriver.com is now a wholly owned subsidiary of intel
>   - klibc guys are initramfs@vger or embedded@vger (see lists)
>     - ask clibc author Peter Anvin if musl serves his needs?

I think musl essentially obsoletes klibc. The main added cost of musl
over klibc for initrds, etc. is the soft-float code that will get
linked in due to printf on archs with no hard-float. If this is a
problem, we could eventually support an option with musl to inhibit
printf float code from getting linked at static-link time. This is a
bit nasty but could be done with weak symbol tricks. Of course, even
with all the soft-float code, I think binaries linked to musl will be
significantly smaller than all the kernel modules, firmware, etc.
stored on an initrd.

On the other side, the big advantage of musl over klibc is that you
don't have to worry about whether the apps you need will be okay with
the limited/non-conformant functionality provided by klibc.
Well-written apps should "just work", and be small.

> - mailing lists you can post a "here's how musl can help _you_" on:
>   It's not spam if you tailor a post to each list, especially if
> there's patches
>   attached in the case of dash or util-linux...
>   - each architecture list for arches you support (linux-arm,
> linux-ppc, etc).
>     "musl is pleased to announce support for the $BLAH architecture,
> here are
>      a cross compiler, chroot with native compiler, and a system
> image to play with."
>     - http://www.arm.linux.org.uk/mailinglists/lists.php
>     - http://www.linux-mips.org/wiki/Net_Resources#Mailing_lists
>     - https://lists.ozlabs.org/listinfo/linuxppc-dev
>     - http://vger.kernel.org/vger-lists.html#linux-x86_64
>   - http://vger.kernel.org/vger-lists.html#dash
>   - http://vger.kernel.org/vger-lists.html#initramfs
>   - http://vger.kernel.org/vger-lists.html#linux-embedded
>   - http://vger.kernel.org/vger-lists.html#util-linux
>   - and maybe one "OS support" message to linux-kernel.

Agreed. I think the key is tailoring the message not to be spammy.
Presumably all of these lists are already used for announcing FOSS
stuff that's potentially useful to the communities they serve.

> - websites that might review musl if we ask nicely:
>   - linux
>     - lwn.net (submit via lwn@lwn.net)
>     - h-online (ping @codepope on twitter)
>     - Linux Journal
>     - Linux Today (they'll just link elsewhere)

I think getting the 1.0 release on Reddit, Hacker News, etc. would be
useful too. If for nothing else, as one step closer to the magic 7.

>   - android
>     - not personally familiar, google for "android news" finds several.
>     - works well with android kernel, installs side-by-side with
> bionic,
>       static links well, doesn't introduce any new licensing issues,
>       provides full posix environment, active and responsive dev
> community.

Yep. I'm not familiar with these sites either though.

>   - paper magazines
>     - long shot, but if you send a press release to pc magazine and
> computerworld
>       and such explaining how musl might help android bridge the gap
> between phones
>       and the desktop they might write a "will android bridge the
> gap between phones
>       and the desktop" article mentioning musl. :)

Nice talking points there.

>   - tech bloggers
>     - cringely.com
>   - Consumer Electronic Linux Forum
>     - Tim Bird and elinux.org
> 
> - do a musl distro that runs well on raspberry pi, tell
> http://www.raspberrypi.org/
> 
> - ask people on mailing list and irc to blog/tweet about the 1.0
> release when it
>   happens.

Yes, this is definitely an under-utilized aspect of the (now rather
large) community.

> - write a syllabus for theoretical "teaching musl" one semester
> comp-sci course.

:)

Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Truls Becken]

On 2012-11-30, at 02:27, Rich Felker wrote:

> On Thu, Nov 29, 2012 at 02:50:03PM -0600, Rob Landley wrote:
> 
>> - Write linux from scratch "musl hint", contribute it to LFS, then link
>>  to it on LFS website from musl website.
> 
> Can you elaborate on what a "musl hint" would be?


From the LFS hints page:

"The LFS Hints are small documents that explain how to do things that are not covered in the LFS or BLFS books. They provide a variety of information such as alternative ways of building and configuring packages, information on new/unstable packages that haven't yet made it into the books, specialized techniques for specific hardware, and other areas that are of interest to LFS users."

http://www.linuxfromscratch.org/hints/read.html

-Truls

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Luca Barbato]

On 11/30/12 2:27 AM, Rich Felker wrote:
> I'd go with xfce or lxde. Last I checked fvwm still didn't even have
> modern text support (using legacy X font system). BTW, supposedly the
> new HarfBuzz and other text rendering stack stuff has taken a turn in
> the direction we like. I saw an article on the redesign that claimed
> 98-99% drop in bloat for working with fonts. I mention this mainly
> because, as much as a lot of us are unhappy with FDO and related
> projects, it would be nice to support and praise when that stuff does
> move in the right direction.

HarfBuzz isn't that strange not-quite-C++-yet-C++ franken-library?

lu

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rich Felker]

On Fri, Nov 30, 2012 at 03:29:31PM +0100, Luca Barbato wrote:
> On 11/30/12 2:27 AM, Rich Felker wrote:
> > I'd go with xfce or lxde. Last I checked fvwm still didn't even have
> > modern text support (using legacy X font system). BTW, supposedly the
> > new HarfBuzz and other text rendering stack stuff has taken a turn in
> > the direction we like. I saw an article on the redesign that claimed
> > 98-99% drop in bloat for working with fonts. I mention this mainly
> > because, as much as a lot of us are unhappy with FDO and related
> > projects, it would be nice to support and praise when that stuff does
> > move in the right direction.
> 
> HarfBuzz isn't that strange not-quite-C++-yet-C++ franken-library?

Bleh, it looks like it was converted to C++ at some point. This is a
major disappointment. Add "opentype interpreter" to the list of core
system components that need to be replaced... Fortunately, I think
this is something that can be done in at most a couple thousand lines
of C.

.. WTF. On looking at it again, it seems to be C source with .cc
extension on all the source files. No idea what they were thinking.
I'd be interested in hearing the story on this.

Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[nwmcsween]

I would try enlightenment, it's a stack that is reasonably portable and utilized within a few embedded projects which IMO is a good fit with musl. A few things need to be fixed within the codebase though specifically the feature macros from what I glanced at.

On Nov 30, 2012, at 11:05 AM, Rich Felker <dalias@aerifal.cx> wrote:

> On Fri, Nov 30, 2012 at 03:29:31PM +0100, Luca Barbato wrote:
>> On 11/30/12 2:27 AM, Rich Felker wrote:
>>> I'd go with xfce or lxde. Last I checked fvwm still didn't even have
>>> modern text support (using legacy X font system). BTW, supposedly the
>>> new HarfBuzz and other text rendering stack stuff has taken a turn in
>>> the direction we like. I saw an article on the redesign that claimed
>>> 98-99% drop in bloat for working with fonts. I mention this mainly
>>> because, as much as a lot of us are unhappy with FDO and related
>>> projects, it would be nice to support and praise when that stuff does
>>> move in the right direction.
>> 
>> HarfBuzz isn't that strange not-quite-C++-yet-C++ franken-library?
> 
> Bleh, it looks like it was converted to C++ at some point. This is a
> major disappointment. Add "opentype interpreter" to the list of core
> system components that need to be replaced... Fortunately, I think
> this is something that can be done in at most a couple thousand lines
> of C.
> 
> .. WTF. On looking at it again, it seems to be C source with .cc
> extension on all the source files. No idea what they were thinking.
> I'd be interested in hearing the story on this.
> 
> Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rich Felker]

On Fri, Nov 30, 2012 at 12:16:00PM -0800, nwmcsween@gmail.com wrote:
> I would try enlightenment, it's a stack that is reasonably portable
> and utilized within a few embedded projects which IMO is a good fit
> with musl. A few things need to be fixed within the codebase though
> specifically the feature macros from what I glanced at.

Does it have a working text stack? One big problem that led to all of
the modern desktop bloat is that a large portion of legacy software
never got past the "character = glyph" myth and thus can't support
text written in the native languages of nearly half the world's
population. This allowed GNOME/FDO junk to take over the Linux
desktop, since it was not politically viable for major distributions
to say "F you" to entire linguistic groups.

Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[nwmcsween]

From a quick glance (evas_font*) it does. The other reason behind enlightenment is that native applications don't have to swallow a mess of dependencies as with most gtk or qt applications.

On Nov 30, 2012, at 12:20 PM, Rich Felker <dalias@aerifal.cx> wrote:

> On Fri, Nov 30, 2012 at 12:16:00PM -0800, nwmcsween@gmail.com wrote:
>> I would try enlightenment, it's a stack that is reasonably portable
>> and utilized within a few embedded projects which IMO is a good fit
>> with musl. A few things need to be fixed within the codebase though
>> specifically the feature macros from what I glanced at.
> 
> Does it have a working text stack? One big problem that led to all of
> the modern desktop bloat is that a large portion of legacy software
> never got past the "character = glyph" myth and thus can't support
> text written in the native languages of nearly half the world's
> population. This allowed GNOME/FDO junk to take over the Linux
> desktop, since it was not politically viable for major distributions
> to say "F you" to entire linguistic groups.
> 
> Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2012-11-30 14:05:26 -0500]:
> On Fri, Nov 30, 2012 at 03:29:31PM +0100, Luca Barbato wrote:
> > HarfBuzz isn't that strange not-quite-C++-yet-C++ franken-library?
> 
> Bleh, it looks like it was converted to C++ at some point. This is a
> major disappointment. Add "opentype interpreter" to the list of core
> system components that need to be replaced... Fortunately, I think
> this is something that can be done in at most a couple thousand lines
> of C.
> 
> .. WTF. On looking at it again, it seems to be C source with .cc
> extension on all the source files. No idea what they were thinking.
> I'd be interested in hearing the story on this.
> 

it seems it has extern c interface

but internally it uses methods and some templates

(i didnt see constructors, destructors, exceptions
nor any libstdc++ dependency so it's mostly c with
minor c++ syntax sugar)

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rich Felker]

On Fri, Nov 30, 2012 at 09:25:46PM +0100, Szabolcs Nagy wrote:
> * Rich Felker <dalias@aerifal.cx> [2012-11-30 14:05:26 -0500]:
> > On Fri, Nov 30, 2012 at 03:29:31PM +0100, Luca Barbato wrote:
> > > HarfBuzz isn't that strange not-quite-C++-yet-C++ franken-library?
> > 
> > Bleh, it looks like it was converted to C++ at some point. This is a
> > major disappointment. Add "opentype interpreter" to the list of core
> > system components that need to be replaced... Fortunately, I think
> > this is something that can be done in at most a couple thousand lines
> > of C.
> > 
> > .. WTF. On looking at it again, it seems to be C source with .cc
> > extension on all the source files. No idea what they were thinking.
> > I'd be interested in hearing the story on this.
> > 
> 
> it seems it has extern c interface
> 
> but internally it uses methods and some templates
> 
> (i didnt see constructors, destructors, exceptions
> nor any libstdc++ dependency so it's mostly c with
> minor c++ syntax sugar)

I suppose it could be forked and that mess patched out...

Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Luca Barbato]

On 11/30/12 9:26 PM, Rich Felker wrote:
> I suppose it could be forked and that mess patched out...

If you do I'll help as well.

lu

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rich Felker]

On Fri, Nov 30, 2012 at 10:14:01PM +0100, Luca Barbato wrote:
> On 11/30/12 9:26 PM, Rich Felker wrote:
> > I suppose it could be forked and that mess patched out...
> 
> If you do I'll help as well.

I think my staying out of this is part of what Landley meant by not
getting into the distro business. :-) If whoever _is_ working on a
small, self-hosting system with decent modern GUI gets to the point of
dealing with harfbuzz and wanting to keep out all C++ dependencies, I
think the polite way to handle it would be to contact the maintainers
and tell them why it's a burden to require a C++ compiler (especially
if you want to use pcc or something firm-based with no C++ compiler)
and that you're willing to maintain a no-C++ fork, but would also be
interested in whether they'd be willing to phase-out C++ usage.

Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rob Landley]

On 11/29/2012 07:27:01 PM, Rich Felker wrote:
> On Thu, Nov 29, 2012 at 02:50:03PM -0600, Rob Landley wrote:
> > Notes from the discussion we had on IRC, plus some further random
> > thoughts on telling the world about musl:
> >
> > - wait until 1.0 so it's most likely to works for them.
> >   - People who take a look and wander off again are less likely to
> > take another look,
> >     so try to make a spash when you're _ready_, not before.
> 
> Agreed. I never really wrote down this principle before, but it's been
> in my mind for a long time. The main publicizing I've been doing so
> far has been in niche places (like mips and microblaze lists) and
> making sure musl is well-indexed in databases like freecode (formerly
> freshmeat) and ohloh. I suspect there are a lot more of the latter we
> should also hit.

Having a good 'elevator pitch' is nice too.

   Musl is a brand new C library that bridges the gap between Linux and  
Android.
   It's BSD licensed and Posix-2008 compliant, written from scratch to  
be small
   simple and clean with no legacy baggage and good thread support. The  
reason
   you've never heard of it is the first git commit was February 2011.

> >     - chroot for each target with native development tools
> >       - system images for qemu maybe?
> 
> It would be really cool if we had "system root trees" rather than
> "system images", with virtual 9p for booting qemu. By the way, getting
> qemu working on musl should probably be in our compatibility goals for
> 1.0.

It's slightly more complicated than that, or I'd already have done it.  
:)

Making client side initramfs work is easy, but it needs a server. Last  
time I looked into this (circa  
http://landley.net/notes-2011.html#29-04-2011), it would require diod  
on the host acting as a server to work on most targets. While this  
isn't much different than qemu's cifs export mode launching samba,  
people are much less likely to have diod already installed on the host.

QEMU does have a built-in 9p server, virtfs requiring a virtio  
transport. Unfortunately, between qemu-system-blah and the kernel  
virtio transport isn't supported equally on all targets. Specifically,  
I couldn't get virtio to work on arm, which is kinda important.

I still have the "i686-virtio" and powerpc-virto targets Alessio Bogani  
contributed last year, and they mostly work (just fixed minor bit rot  
with qemu getting renamed to qemu-system-i386). That's actually using  
virtio block device and virtio network, not virtfs, but the principle's  
the same. (Think of virtio as a bus and then other things get  
implemented on top of that bus; even though the bus is simpler than PCI  
support for it may not exist in the target we want because nobody's  
bothered yet.)

I also looked at getting virtio console working, but at the time the UI  
for it was appalling:

   http://lists.gnu.org/archive/html/qemu-devel/2011-05/msg00390.html

> >       - launch x11 vnc server and display in tightvnc window?
> >     - jslinux live image on website
> 
> Love this idea. I'd like it even better if we could adapt jslinux to
> start with a preinitialized memory/machine-state image rather than
> having to spend 15-20 seconds booting. BTW I think we need to look
> into whether there are license issues with using jslinux...?

The actual javascript source says redistribution requires permission of  
the author, so yeah. Somebody has to ping fabrice. (Or else reference  
the files off his web page, which is hugely impolite.)

I wonder what the busybox guys did? Let's see, google for "jslinux  
busybox" and bubble up through the archive a bit to find the commit  
that added it to the website is here:

http://git.uclibc.org/busybox-website/commit?id=317871f1a85816e3c0f405f00b2f081978fa9caa

And the jslinux in that still says you need Fabrice's permission to  
redistribute, commit comment is just that Denys added it and no further  
details. Date is March 4 of this year so check the mailing list archive  
on the website...

And this is the start of the thread about it:

   http://lists.busybox.net/pipermail/busybox/2012-March/077498.html

Content of message, and I quote, "Hey guys, Check this out!". One  
reply, "I saw it. Very cool." No mention of license or permission.  
(These are the people who took over the project that's the poster child  
for GPL enforcement. Wheee...)

Yes, by all means, let's ask Fabrice. (As musl maintainer, you should  
probably do it.)

> By the way, somebody also mentioned a mips emulator in js we could use
> instead or in addition to jslinux, but I don't know if it emulates
> enough hardware to get linux up.

Yay!

> > - distro conversions
> >   - leverage existing repositories, don't fall into the buildroot  
> trap
> 
> I think I've already avoided this trap without avoiding new distros,
> by building a community of people who like rolling their own distros
> but not getting involved in the distro/build stuff at all myself. This
> has been great for getting musl a lot of testing with lower entry
> barrier than large existing distros, but adding the latter now would
> be a good next step.

You've done great. My point here is just "how can we leverage the work  
other people have done without distracting ourselves more than  
necessary".

If we can come up with a musl chroot that debootstrap runs in (the  
"compile packages" not just "install binaries from repo" variant), or a  
musl chroot that emerge can build the world in, or koji:

    
http://opensource.com/life/11/7/free-sake-story-koji?sc_cid=70160000000IDmjAAG

Or whatever SuSE's variant is (blanking on the name), or buildroot, or  
arch's pacman, or...

(I'd mentino Yocto's poky but after meego and tizen and such it's  
really hard to _care_ about any of the Linux foundation's "look,  
another not-android!" gyrations.)

I wandered aimlessly through this bit more than once over the years and  
it's why I have the "bootstrap" stuff in my control-images. What I was  
trying to do was extend the basic "simplest self-hosting build  
environment" up to whatever the various distros needed and then let  
them build themselves. It's a finite amount of work. But it's an  
_undocumented_ finite amount of work and all these build systems have  
DEEP assumptions that they are building under themselves. (Yes, even  
gentoo. It's actually worse than Debian about it.)

Basically, I started researching gentoo bootstrapping in 2008:

   http://landley.net/notes-2008.html#17-12-2008

My friend Mirell took a serious stab at getting it working in 2009:

    
http://web.archive.org/web/20090411155149/http://impactlinux.com/hg/gfs

I took my own stab at it in 2010:

  http://landley.net/notes-2010.html#15-09-2010

And then tried a Funtoo version in 2011:

   http://www.funtoo.org/wiki/ReBootstrap

And each time, when you try to create the base system from something  
other than an existing gentoo system it just _melted_down_ and went off  
on a tangent to fix something else.

I eventually went "ok, there are three problems here":

1) I'm trying to build a distro on something other than an existing  
copy of that distro.

2) I'm trying to use a different libc (and different toolchain, but  
that's less of an issue).

3) The existing system I'm providing uses busybox instead of the  
specific versions of the gnu/dammit tools they expect (and boy were  
they version-sensitive and configuration sensitive, things like python  
requiring bzip2 shared libraries to exist on the host when it's built  
or else package extraction won't work because portage doesn't know how  
to pipe stuff through bzip2, just use the optional python shared  
library wrapper).

So I went "if I build Linux From Scratch I can deal with 2 and 3  
separately, and then rebootstrap portage under the resulting LFS system  
on the target, and it's a huge digression but can be completely  
automated, so..." (And that's before getting into trying to wean  
portage off of the requirement of explicitly naming every supported  
target in every single ebuild file in the entire tree, so that adding a  
new target requires touching every file in the portage tree and the  
idea of telling gentoo "just build whatever architecture the host is"  
is fundamentally against the way they designed it... Yeah, that was me  
going "funtoo, how does _that_ work"?)

You may have noticed that my todo list has archaeological strata. Right  
now Toybox has put all this on hold, and the rise of Android has  
probably rendered some of it moot...

An advantage YOU have is existing gentoo developers with interest who  
you may be able to foist this off on. Plus you're just trying to swap  
out _one_ component at a time, and only support a few specific  
architectures.

> > - Write linux from scratch "musl hint", contribute it to LFS, then  
> link
> >   to it on LFS website from musl website.
> 
> Can you elaborate on what a "musl hint" would be?

Basically they're standalone "or you could do it _this_ way"  
explanations for alternatives or extensiosn to the way Linux From  
Scratch does stuff:

   http://hints.linuxfromscratch.org

   http://www.linuxfromscratch.org/hints/read.html

They're referenced from time to time in the main Linux From Scratch  
book, ala:

   http://www.linuxfromscratch.org/lfs/view/stable/chapter08/kernel.html

And at the end it's one of the things they point you at:

    
http://www.linuxfromscratch.org/lfs/view/stable/chapter09/whatnow.html

The "Beyond Linux From Scratch" book more or less evolved out of the  
pile of hints, trying to turn it into a bigger systematic sequel book  
of additional stuff you can build. But the hints profide alternatives  
to the base infrastructure, such as BSD-style init scripts instead of  
SysV style:

   http://www.linuxfromscratch.org/hints/downloads/files/bsd-init.txt

Or using uClibc instead of glibc in the base system:

   http://www.linuxfromscratch.org/hints/downloads/files/uclibc.txt

(Of course BLFS has its own problems. Their last release was in 2008  
and they went to one of those "everybody just use svn and ping us if  
it's broken" things because their development model's become too  
dysfunctional to cut releases anymore.)

Anyway, advantage of this: if they take it you can then link to it from  
musl's website and go "hey build your own system guys, here's how you  
do a musl based LFS". It's not _explicitly_ that LFS would be endorsing  
you, but it feels that way a little bit. :)

> > - is userbase of glibc, uClibc, klibc, or dietlibc better served by
> > musl?
> >   - contribute musl option to buildroot?
> >   - contribute musl option to crosstool-ng?
> 
> There's demand for musl in crosstool-ng at least; the maintainer has
> expressed this, but is largely unfamiliar with musl and waiting for a
> patch from someone willing to do it.

He's a good guy. I'm not a fan of the design of his project (driving it  
makes The Tardis look user-friendly), but a lot of people find it  
useful.

> >   - Ask mentor graphics (formly code sourcery) to do a musl  
> toolchain?
> >     - LOTS of proprietary embedded devs use this one, it's
> > "professional".
> 
> I suspect they might be particularly interested since their customers
> might need to avoid copyleft code.

Or at least the random threat of lawsuits even if they _try_ to conform.

Not gonna rant. Nope. Not gonna.

> >     - windriver.com is now a wholly owned subsidiary of intel
> >   - klibc guys are initramfs@vger or embedded@vger (see lists)
> >     - ask clibc author Peter Anvin if musl serves his needs?
> 
> I think musl essentially obsoletes klibc. The main added cost of musl
> over klibc for initrds, etc. is the soft-float code that will get
> linked in due to printf on archs with no hard-float. If this is a
> problem, we could eventually support an option with musl to inhibit
> printf float code from getting linked at static-link time.

A "./configure --disable-soft-float" would be nice just for bragging  
rights.

> This is a
> bit nasty but could be done with weak symbol tricks. Of course, even
> with all the soft-float code, I think binaries linked to musl will be
> significantly smaller than all the kernel modules, firmware, etc.
> stored on an initrd.
> 
> On the other side, the big advantage of musl over klibc is that you
> don't have to worry about whether the apps you need will be okay with
> the limited/non-conformant functionality provided by klibc.
> Well-written apps should "just work", and be small.

An _ideal_ situation would be to get Peter Anvin to end klibc  
development, point his users at musl, and possibly even become a musl  
user/developer. I admit the diplomacy to arrange such an outcome is  
beyond me.

However, it would be nice to ask him what he gets out of klibc. If it's  
providing a reference implementation or using it to test system call  
generation or some such, possibly musl could fill this role. If there's  
something specific klibc does, it would be nice if musl could also do  
it.

> > - websites that might review musl if we ask nicely:
> >   - linux
> >     - lwn.net (submit via lwn@lwn.net)
> >     - h-online (ping @codepope on twitter)
> >     - Linux Journal
> >     - Linux Today (they'll just link elsewhere)
> 
> I think getting the 1.0 release on Reddit, Hacker News, etc. would be
> useful too. If for nothing else, as one step closer to the magic 7.

Obviously, none of these lists are exhaustive. :)

But if we add to them much, collating to create a master list sounds  
like a job for a wiki.

> >   - android
> >     - not personally familiar, google for "android news" finds  
> several.
> >     - works well with android kernel, installs side-by-side with
> > bionic,
> >       static links well, doesn't introduce any new licensing issues,
> >       provides full posix environment, active and responsive dev
> > community.
> 
> Yep. I'm not familiar with these sites either though.

Enthuse to the cyanogen mod guys, send them to enthuse to others?

   http://www.homeonthestrange.com/view.php?ID=28

> > - ask people on mailing list and irc to blog/tweet about the 1.0
> > release when it
> >   happens.
> 
> Yes, this is definitely an under-utilized aspect of the (now rather
> large) community.

Word of mouth is the best asset, but buzz is all about combo bonuses.

The _other_ thing you might want to do is pull up the original BBC  
"kitchen nightmares" on Netflix streaming and watch Gordon arrange a  
full set of customers for each failing restaurant, at which point  
hilarity inevitably ensues.

"OK, you've achieved basic competence. Now a thundering horde of a  
gazillion people
show up wanting to try your thing. How do you avoid service melting  
into a puddle
under the load?"

Lots of people are not prepared for _success_. You've got to acclimate  
newbies. You've got to have a website that can handle the downloads.  
Mailing list traffic spikes. I dunno if this will happen, but _if_ it  
does, are you prepared?

Ideally you'd have not just a goot FAQ and web pages but "the book of  
musl" which can act as both tutorial and reference (two different  
functions), and designated hand-holders in the community fielding a  
flood of newbies.

This is probably hashing your chickens before counting them, of course,  
but I wanted to at least bring it up.

For example, if you put your musl porting guide online (p.s. put your  
musl porting guide online) and you suddenly have three different people  
submitting ports to different architectures, the testing burden's gonna  
get nuts. The "why doesn't it work with this package" requests could  
spike beyond the ability to field. Bug tracker. Translations, including  
people translating the documentation and web pages themselves (with all  
the version skew that entails). Triaging and merging forks. Patches  
winding up in distro repositories and not necessarily making their way  
back upstream promptly...

Yeah yeah, such problems we should have. Just don't be surprised by  
them. :)

Rob

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Jens Staal]

torsdagen den 29 november 2012 14.50.03 skrev  Rob Landley:
>      - works well with android kernel, installs side-by-side with  
> bionic,
>        static links well, doesn't introduce any new licensing issues,
>        provides full posix environment, active and responsive dev  
> community.
>    - paper magazines
>      - long shot, but if you send a press release to pc magazine and  
> computerworld
>        and such explaining how musl might help android bridge the gap  
> between phones
>        and the desktop they might write a "will android bridge the gap  
> between phones
>        and the desktop" article mentioning musl.

I like this idea :) and I do believe that the CyanogenMod community might be a 
nice start (although my feeling is that it seems somewhat disorganized with 
downloads from weird file-sharing sites and forum posts - my experience with 
the custom builds has however been very positive).

Another project that might be interested is the Zshaolin project. They build 
an impressive ammount of linux tools for Android. I just tried with "ldd zsh" 
on my phone, but that did unfortunately no work (ldd is missing) so I am not 
sure how they built them at the moment - most likely with glibc.

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Daniel Bainton]

On 30 November 2012 06:38, Jens Staal <staal1978@gmail.com> wrote:
> torsdagen den 29 november 2012 14.50.03 skrev  Rob Landley:
>>      - works well with android kernel, installs side-by-side with
>> bionic,
>>        static links well, doesn't introduce any new licensing issues,
>>        provides full posix environment, active and responsive dev
>> community.
>>    - paper magazines
>>      - long shot, but if you send a press release to pc magazine and
>> computerworld
>>        and such explaining how musl might help android bridge the gap
>> between phones
>>        and the desktop they might write a "will android bridge the gap
>> between phones
>>        and the desktop" article mentioning musl.
>
> I like this idea :) and I do believe that the CyanogenMod community might be a
> nice start (although my feeling is that it seems somewhat disorganized with
> downloads from weird file-sharing sites and forum posts - my experience with
> the custom builds has however been very positive).

Huh? CyanogenMod has their own servers where they host their builds.
They are very organized. You must be talking about the tons of other
custom Android builds.

--
Daniel

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Hiltjo Posthuma]

On Thu, Nov 29, 2012 at 9:50 PM, Rob Landley <rob@landley.net> wrote:
> Notes from the discussion we had on IRC, plus some further random thoughts

Maybe it's also an idea to add a donation link on the musl website so
people can donate money.

PS. I've tried Sabotage Linux recently and I'm very impressed on how
much musl has progressed and how much works already, keep it up!

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[idunham]

> Notes from the discussion we had on IRC, plus some further random
> thoughts on telling the world about musl:
>
> - wait until 1.0 so it's most likely to works for them.
>    - People who take a look and wander off again are less likely to
> take another look,
>      so try to make a spash when you're _ready_, not before.
>    - counter this with "rule of 7", people filter out noise and won't
> remember they've
>      even heard of you until they've seen it in ~7 different places. So
> once you _ARE_
>      ready, get the word out everywhere. (Politely.)
>
> - prepare the website to covert casual browsers into long-term users.
>    - press release extoling virtues
>      - simple
>        - realtime: less code is more deterministic
>        - security: less code is easier to audit
>        - students/teachers: learn how a posix system works
>      - link to the online git browser for the "show me the code" guys.
>      - already tested against 8 gazillion packages
>      - standards compliant
>      - BSD license: static linking ok, android deployment ok
Little quibble: MIT + some BSD and some PD code.
>      - works side by side with existing libraries, or static linked
>        - easy deployment on android without bionic limitations
>      - technical advantages
>        - support static and dynamic linking and do _both_ well
>        - thread implementation is _not_crazy_, and no legacy baggage.
>
>    - obvious "start here" from main page.
>      - Why it's cool (collate)
>      - how to use it (collate)
>        - HOWTO walkthrough
>
>    - binaries they can try.
>      - cross compiler, build hello world
>      - livecd of full-ish x86 distro.
>        - with working x11 and simple gui (xfce? fvwm?)
>      - chroot for each target with native development tools
>        - system images for qemu maybe?
>        - launch x11 vnc server and display in tightvnc window?
>      - jslinux live image on website
>
> - distro conversions
>    - leverage existing repositories, don't fall into the buildroot trap
>    - approach gentoo guys about a musl build
>      - #gentoo-embedded on freenode
>      - maybe funtoo would be easier (Daniel Robbins' new project,
> #funtoo on freenode)
Luca Barbado covered this one ;)
>    - approach debian guys about musl debootstrap
>    - arch linux, slackware, puppy, crunchbang, tinycore...
Puppy: Already some awareness. Fatdog64 allegedly includes a musl
toolchain.  At least two of the developers involved in pupngo (an
experimental "puppy project" that produces a _very_ small & minimal
~Puppy-style distro/project) have been working on musl toolchains, and one
of them is involved in a large number of the puppy projects/puplets.
TinyCore: Already some awareness: someone emailed pcc-list about enabling
linux-arm, with the intended usecase being microcore/some variant of "Army
Core" on A10 devices.
(pcc apparently supports ARM but only on some BSD flavor)


> - push "musl support" patches to other projects upstream all at once
>    - sabotage collected a bunch?
And a number in musl-pkgsrc-patches (though I'm dubious about some of them)
>    - people who develop on 3 other project seeing musl on all 3 lists
>      makes dev community look big and active.
>
> - Write linux from scratch "musl hint", contribute it to LFS, then link
>    to it on LFS website from musl website.
>
> - is userbase of glibc, uClibc, klibc, or dietlibc better served by
> musl?
The dietlibc & uclibc section is where the puppy developers are starting
to try musl.
FWIW, I got their attention by mentioning the size of a full-static binary
for tinymp3, a little ffmpeg-based MP3 player; the license change got some
interest too.

>    - contribute musl option to buildroot?
>    - contribute musl option to crosstool-ng?
May be sensible.
Embtoolkit is advertising that musl support is on its way.
>    - Ask mentor graphics (formly code sourcery) to do a musl toolchain?
>      - LOTS of proprietary embedded devs use this one, it's
> "professional".
>      - windriver.com is now a wholly owned subsidiary of intel
>    - klibc guys are initramfs@vger or embedded@vger (see lists)
>      - ask clibc author Peter Anvin if musl serves his needs?
>
> - mailing lists you can post a "here's how musl can help _you_" on:
>    It's not spam if you tailor a post to each list, especially if
> there's patches
>    attached in the case of dash or util-linux...
>    - each architecture list for arches you support (linux-arm,
> linux-ppc, etc).
>      "musl is pleased to announce support for the $BLAH architecture,
> here are
>       a cross compiler, chroot with native compiler, and a system image
> to play with."
I'm assuming this would mean some of your work and some of Gregor's?

>      - http://www.arm.linux.org.uk/mailinglists/lists.php
>      - http://www.linux-mips.org/wiki/Net_Resources#Mailing_lists
>      - https://lists.ozlabs.org/listinfo/linuxppc-dev
>      - http://vger.kernel.org/vger-lists.html#linux-x86_64
>    - http://vger.kernel.org/vger-lists.html#dash
>    - http://vger.kernel.org/vger-lists.html#initramfs
>    - http://vger.kernel.org/vger-lists.html#linux-embedded
>    - http://vger.kernel.org/vger-lists.html#util-linux
>    - and maybe one "OS support" message to linux-kernel.
>
> - websites that might review musl if we ask nicely:
>    - linux
>      - lwn.net (submit via lwn@lwn.net)
>      - h-online (ping @codepope on twitter)
>      - Linux Journal
>      - Linux Today (they'll just link elsewhere)

I'd add Phoronix.  I'd suggest inquiring about a "guest article" (make
sure to mention that it builds with clang as well as GCC, since Micheal
seems to to love anything related to LLVM)
Also getting musl support upstream into apache would help, since one of
the simplest benchmarks PTS does involves building apache from _unpatched_
 source, then testing its performance.
>    - android
>      - not personally familiar, google for "android news" finds several.
>      - works well with android kernel, installs side-by-side with
> bionic,
>        static links well, doesn't introduce any new licensing issues,
>        provides full posix environment, active and responsive dev
> community.

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rob Landley]

On 11/29/2012 08:21:48 PM, idunham@lavabit.com wrote:
> > Notes from the discussion we had on IRC, plus some further random
> > thoughts on telling the world about musl:
> >
> > - wait until 1.0 so it's most likely to works for them.
> >    - People who take a look and wander off again are less likely to
> > take another look,
> >      so try to make a spash when you're _ready_, not before.
> >    - counter this with "rule of 7", people filter out noise and  
> won't
> > remember they've
> >      even heard of you until they've seen it in ~7 different  
> places. So
> > once you _ARE_
> >      ready, get the word out everywhere. (Politely.)
> >
> > - prepare the website to covert casual browsers into long-term  
> users.
> >    - press release extoling virtues
> >      - simple
> >        - realtime: less code is more deterministic
> >        - security: less code is easier to audit
> >        - students/teachers: learn how a posix system works
> >      - link to the online git browser for the "show me the code"  
> guys.
> >      - already tested against 8 gazillion packages
> >      - standards compliant
> >      - BSD license: static linking ok, android deployment ok
> Little quibble: MIT + some BSD and some PD code.

Alas, we don't have a good group term like "copyleft" for "would be  
public domain if our legal system wasn't screwed up".

I poked Dalias on irc to clarify that we can give a single top level  
license and call all the code "compatible" with that, and then link to  
the big long copyright list for everybody who cares but have a clear  
story on the website.

(Gregor is of the opinion that every binary statically linked against  
musl cannot be distributed without including a copy of the complete  
text of every single sub-license in the entire musl tree, in all the  
varied wordings, and that distributing binaries without bundling them  
in an archive with such an array of legal documents would be a license  
violation. I point out that he's not a lawyer, say "the program"  
referring to source versions sounds like a perfectly reasonable  
interpretation to me and anybody trying to prove otherwise in court  
might have an interesting time getting nonzero damages although if they  
asked nicely for me to include it I would happily stop using or  
distributing any of their software, and otherwise ignore him.)

> > - distro conversions
> >    - leverage existing repositories, don't fall into the buildroot  
> trap
> >    - approach gentoo guys about a musl build
> >      - #gentoo-embedded on freenode
> >      - maybe funtoo would be easier (Daniel Robbins' new project,
> > #funtoo on freenode)
> Luca Barbado covered this one ;)

I happily defer to the judgement of domain experts.

> >    - approach debian guys about musl debootstrap
> >    - arch linux, slackware, puppy, crunchbang, tinycore...
> Puppy: Already some awareness. Fatdog64 allegedly includes a musl
> toolchain.  At least two of the developers involved in pupngo (an
> experimental "puppy project" that produces a _very_ small & minimal
> ~Puppy-style distro/project) have been working on musl toolchains,  
> and one
> of them is involved in a large number of the puppy projects/puplets.
> TinyCore: Already some awareness: someone emailed pcc-list about  
> enabling
> linux-arm, with the intended usecase being microcore/some variant of  
> "Army
> Core" on A10 devices.
> (pcc apparently supports ARM but only on some BSD flavor)

I look forward to the toolchain problem being solved. I'm not holding  
my breath.

> > - push "musl support" patches to other projects upstream all at once
> >    - sabotage collected a bunch?
> And a number in musl-pkgsrc-patches (though I'm dubious about some of  
> them)

We can triage, confirm, document, and send them upstream as a batch  
when 1.0 happens.

> >    - people who develop on 3 other project seeing musl on all 3  
> lists
> >      makes dev community look big and active.
> >
> > - Write linux from scratch "musl hint", contribute it to LFS, then  
> link
> >    to it on LFS website from musl website.
> >
> > - is userbase of glibc, uClibc, klibc, or dietlibc better served by
> > musl?
> The dietlibc & uclibc section is where the puppy developers are  
> starting
> to try musl.

I'm tempted to analyze each libc variant: eglibc, uClibc, klibc,  
dietlibc, newlib. Look at it, figure out what specifically its users  
get out of it, figure out if musl can meet their needs.

But I'm just not familiar enough with most of them, and haven't got  
time to do proper research right now...

> FWIW, I got their attention by mentioning the size of a full-static  
> binary
> for tinymp3, a little ffmpeg-based MP3 player; the license change got  
> some
> interest too.
> 
> >    - contribute musl option to buildroot?
> >    - contribute musl option to crosstool-ng?
> May be sensible.
> Embtoolkit is advertising that musl support is on its way.

Yay!

Should we have a page collecting "projects using musl"? The project  
will probably outgrow it, but to start with it might be nice. (There's  
a chicken and egg problem: it would be a great thing to include in a  
release announcement, but if you poke people about it more than maybe a  
week before the actual 1.0 release date or you might blunt your splash.  
Then again there's something to be said for building anticipation, and  
musl isn't currently a _secret_...)

> >    - Ask mentor graphics (formly code sourcery) to do a musl  
> toolchain?
> >      - LOTS of proprietary embedded devs use this one, it's
> > "professional".
> >      - windriver.com is now a wholly owned subsidiary of intel
> >    - klibc guys are initramfs@vger or embedded@vger (see lists)
> >      - ask clibc author Peter Anvin if musl serves his needs?
> >
> > - mailing lists you can post a "here's how musl can help _you_" on:
> >    It's not spam if you tailor a post to each list, especially if
> > there's patches
> >    attached in the case of dash or util-linux...
> >    - each architecture list for arches you support (linux-arm,
> > linux-ppc, etc).
> >      "musl is pleased to announce support for the $BLAH  
> architecture,
> > here are
> >       a cross compiler, chroot with native compiler, and a system  
> image
> > to play with."
> I'm assuming this would mean some of your work and some of Gregor's?

It would mean getting it to work, doesn't matter which implementation.

> >      - http://www.arm.linux.org.uk/mailinglists/lists.php
> >      - http://www.linux-mips.org/wiki/Net_Resources#Mailing_lists
> >      - https://lists.ozlabs.org/listinfo/linuxppc-dev
> >      - http://vger.kernel.org/vger-lists.html#linux-x86_64
> >    - http://vger.kernel.org/vger-lists.html#dash
> >    - http://vger.kernel.org/vger-lists.html#initramfs
> >    - http://vger.kernel.org/vger-lists.html#linux-embedded
> >    - http://vger.kernel.org/vger-lists.html#util-linux
> >    - and maybe one "OS support" message to linux-kernel.
> >
> > - websites that might review musl if we ask nicely:
> >    - linux
> >      - lwn.net (submit via lwn@lwn.net)
> >      - h-online (ping @codepope on twitter)
> >      - Linux Journal
> >      - Linux Today (they'll just link elsewhere)
> 
> I'd add Phoronix.  I'd suggest inquiring about a "guest article" (make
> sure to mention that it builds with clang as well as GCC, since  
> Micheal
> seems to to love anything related to LLVM)

Always good to tailor the article to the outlet. :)

> Also getting musl support upstream into apache would help, since one  
> of
> the simplest benchmarks PTS does involves building apache from  
> _unpatched_
>  source, then testing its performance.

What's needed for that?

Rob

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rich Felker]

On Fri, Nov 30, 2012 at 08:04:44PM -0600, Rob Landley wrote:
> >>      - already tested against 8 gazillion packages
> >>      - standards compliant
> >>      - BSD license: static linking ok, android deployment ok
> >Little quibble: MIT + some BSD and some PD code.
> 
> Alas, we don't have a good group term like "copyleft" for "would be
> public domain if our legal system wasn't screwed up".

We do have a word for this license class; it's called "permissive".
Roughly speaking, "FOSS" breaks down into "permissive" and "copyleft"
licenses, where the former basically allow you to do whatever you
like, and the latter burden you with ensuring that the parties who get
the software from you get the same freedoms you had.

> I poked Dalias on irc to clarify that we can give a single top level
> license and call all the code "compatible" with that, and then link
> to the big long copyright list for everybody who cares but have a
> clear story on the website.

The MIT license text serves this purpose.

> >> - push "musl support" patches to other projects upstream all at once
> >>    - sabotage collected a bunch?
> >And a number in musl-pkgsrc-patches (though I'm dubious about some
> >of them)
> 
> We can triage, confirm, document, and send them upstream as a batch
> when 1.0 happens.



> 
> >>    - people who develop on 3 other project seeing musl on all 3
> >lists
> >>      makes dev community look big and active.
> >>
> >> - Write linux from scratch "musl hint", contribute it to LFS,
> >then link
> >>    to it on LFS website from musl website.
> >>
> >> - is userbase of glibc, uClibc, klibc, or dietlibc better served by
> >> musl?
> >The dietlibc & uclibc section is where the puppy developers are
> >starting
> >to try musl.
> 
> I'm tempted to analyze each libc variant: eglibc, uClibc, klibc,
> dietlibc, newlib. Look at it, figure out what specifically its users
> get out of it, figure out if musl can meet their needs.

I can give you the short version...

eglibc has very few advantages over plain glibc now that Drepper is
gone. Mainly just some build-time options to tweak/omit features. My
impression is that it will become obsolete now that glibc is
improving.

uClibc I think you know. :-)

klibc is probably only relevant to initrd. In principle it's a lot
like Bionic -- a "thin" libc that said "screw standards as long as we
can provide the libc functionality needed in our very-limited domain".

dietlibc's user base seems to be mostly fefe/djb fans, and maybe
people making rescue disks and such. It's not secure or robust enough
for internet-facing use or for many embedded uses.

newlib's niche is systems with no kernel, or kernels very different
from POSIX-oriented ones. I don't think it would be used on any
systems any of the other libcs you mentioned get used on.

Among these, I think the only two against which musl wins in all
respects are klibc and dietlibc. As for the others:

[e]glibc: There may be "enterprise" uses where musl lacks bloated
legacy features glibc has (think things like NIS, locales with
different charsets, utmp, ...), and of course glibc has established
lock-in due to the existing binary ecosystem as well as software with
source-level portability problems.

uClibc: Its main advantage is just compatibility with existing
embedded software that's non-portable and making uClibc-specific
assumptions (probably without the authors even being aware).
Compatibility with ancient Linux kernel versions may also be an issue
for some embedded uses (needing to use an old kernel for compatibility
or size reasons). musl has roughly the same level of compatibility
with 2.4 kernels as uClibc does, except that uClibc also provides the
hackish LinuxThreads for crappy-but-sometimes-usable multithreading on
2.4. On older kernels (2.2, 2.0), uClibc is probably the only working
option.

newlib: As distributed, musl is a full POSIX libc targetting the Linux
kernel syscall API. Newlib can be used on all sorts of systems
provided you fill in the system glue. However, if you're willing to do
the same with musl (which could be as easy as making a __syscall
function which implements the equivalent of the syscalls for stuff you
do), musl could possibly fill newlib's role.

> >>    - contribute musl option to buildroot?
> >>    - contribute musl option to crosstool-ng?
> >May be sensible.
> >Embtoolkit is advertising that musl support is on its way.
> 
> Yay!

Nice. We should add a link back from the website and wiki.

> Should we have a page collecting "projects using musl"? The project
> will probably outgrow it, but to start with it might be nice.

Yes. This is something I'd kind of hoped would evolve on the wiki.

> (There's a chicken and egg problem: it would be a great thing to
> include in a release announcement, but if you poke people about it
> more than maybe a week before the actual 1.0 release date or you
> might blunt your splash. Then again there's something to be said for
> building anticipation, and musl isn't currently a _secret_...)

If Embtoolkit, Aboriginal, and perhaps crosstool-ng or buildroot have
support for it before 1.0, they would certainly be good items to
mention in publicity materials.

> >Also getting musl support upstream into apache would help, since
> >one of
> >the simplest benchmarks PTS does involves building apache from
> >_unpatched_
> > source, then testing its performance.
> 
> What's needed for that?

Yes, I'd like to know too. Aside from this thread on the
marketing/publicity aspects, we should have a pre-1.0 thread on
determining what missing interfaces, bugs, differences from glibc,
etc. are blocking compatibility with important packages so that I can
get around to fixing them. At this point it might be nice if we had a
bug tracker for it, but I think we can manage with just the list for
at least a while longer.

Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Isaac Dunham]

On Fri, 30 Nov 2012 23:06:20 -0500
Rich Felker <dalias@aerifal.cx> wrote:

> > 
> > I'm tempted to analyze each libc variant: eglibc, uClibc, klibc,
> > dietlibc, newlib. Look at it, figure out what specifically its users
> > get out of it, figure out if musl can meet their needs.
> 
> I can give you the short version...
<snip>
> 
> uClibc I think you know. :-)
> 
> klibc is probably only relevant to initrd. In principle it's a lot
> like Bionic -- a "thin" libc that said "screw standards as long as we
> can provide the libc functionality needed in our very-limited domain".
clone() & pthreads are unsupported, because it's not threadsafe.

klibc does have one very large advantage over musl, especially wrt. intrds: It supports most Linux arches (musl would need to port to several more before becoming a full replacement).
Arches they support but we don't: alpha, cris, ia64 (static only), parisc, ppc64, s390, s390x, sparc.  

Of those, Debian officially supports ia64, ppc64, s390, and sparc (they have sparc 32-bit userland on sparc64).
s390x, parisc, and alpha are unofficial ports.

klibc also has a few untested ports:
m32r, m68k, arm-thumb, sh, sparc64. 

> dietlibc's user base seems to be mostly fefe/djb fans, and maybe
> people making rescue disks and such. It's not secure or robust enough
> for internet-facing use or for many embedded uses.
> 
> newlib's niche is systems with no kernel, or kernels very different
> from POSIX-oriented ones. I don't think it would be used on any
> systems any of the other libcs you mentioned get used on.

It also _is_ used on Linux. newlib on Linux is LGPL, and has more features than other platforms.

newlib is a pretty random assortment: it looks like someone tried to collect all the permissive code they could find and call it a libc...

> Among these, I think the only two against which musl wins in all
> respects are klibc and dietlibc. As for the others:

-- 
Isaac Dunham <idunham@lavabit.com>

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rob Landley]

On 11/30/2012 10:06:20 PM, Rich Felker wrote:
> On Fri, Nov 30, 2012 at 08:04:44PM -0600, Rob Landley wrote:
> > >>      - already tested against 8 gazillion packages
> > >>      - standards compliant
> > >>      - BSD license: static linking ok, android deployment ok
> > >Little quibble: MIT + some BSD and some PD code.
> >
> > Alas, we don't have a good group term like "copyleft" for "would be
> > public domain if our legal system wasn't screwed up".
> 
> We do have a word for this license class; it's called "permissive".
> Roughly speaking, "FOSS" breaks down into "permissive" and "copyleft"
> licenses, where the former basically allow you to do whatever you
> like, and the latter burden you with ensuring that the parties who get
> the software from you get the same freedoms you had.
> 
> > I poked Dalias on irc to clarify that we can give a single top level
> > license and call all the code "compatible" with that, and then link
> > to the big long copyright list for everybody who cares but have a
> > clear story on the website.
> 
> The MIT license text serves this purpose.

At some point we might want to collate edits to the original list.

More suggestions:

The musl-libc.org main page should probably just go to the "intro"  
page, possibly with the existing paragraph merged into the entro.


The "what are musl's dependencies" faq entry is a mix of bullet points  
and non-bullet point material that doesn't just clarify the bullet  
points. It doesn't mention what "make" you need (posix-2008 compliance  
should be enough?) and the shell is implicit (does _not_ need bash,  
posix-2008 again good enough?). Did we actually test Linux 2.6.0?

Hmmm... something like:

-----
Compiling and linking programs against musl should work with any  
conformant C compiler.

To build musl itself, you need:

   - Linux 2.6 or later
   - c99 compiler with support for gcc-style assembly language.
   - linker with weak symbol support
   - posix-2008 compliant "make" and "sh"
   - a supported CPU architecture

At present i386, x86_64, arm, mips, microblaze, and powerpc  
architectures are supported by musl.
-----

It would be nice if there was some kind of "musl manual". If you want  
to write a program against the musl libc, what does it provide? (HTML  
is fine, man page format is kinda archaic these days. This is mostly  
posix, but not entirely.)

If we're not up to writing something, w link to the POSIX spec with the  
list of functions we've implemented, plus the man7 pages on the linux  
variations thereof (and system calls) and maybe the Linux Standard Base  
sort of collectively covers it.

http://pubs.opengroup.org/onlinepubs/9699919799/ (see "system  
interfaces")
http://kernel.org/doc/man-pages/ (sections 2 and 3)
http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/book1.html  
(chapter 12)

Beyond the FAQ, there should probably be a page introductory docs. I  
generally link to the old boot floppy howto, the power up to bash  
prompt howto, and Linux From Scratch. (We could always write something  
new, but this is more "understanding how your root filesystem works"  
and "creating a small distro" than "how libc works".)

It's a pity the Linux Foundation's lack of clue keeps bleeding through  
to the point they link to a "registration required" _wrapper_frame_ for  
posix-2008 instead of the actual web version:

   http://refspecs.linuxfoundation.org/

(Yes, if you dig through the susv4 they link to and bother to register,  
it's just linking to the opengroup.org pages via a gratuitous HTML  
frame. Pity, the refspecs page is a good collection of docs other than  
that, modulo some guidance on which of the several versions of each  
thing they link to are actually useful. But that's the Linux Foundation  
for you: a collection of Fortune 500 companies tried to give money to  
this "Linux" thing and couldn't figure out who to make the check out  
to, so they created an entity for that purpose. This entity remains  
very good at cashing checks, still working on the rest.)

Anyway...

More conceptual intros: what is ELF (with a link to the specs and  
mention of alternatives like #!/interpreters, binflt, binmisc, and  
historical a.out) also WHY is ELF (it's an archive format for binary  
code, about as flexible as tar or zip but with metadata about function  
names, offsets, and permission bits instead of files/lengths/owners,  
generic enough that binflt is created from it). Dynamic vs static  
linking, "what is a dynamic linker and wazzit _do_". Something on the  
whole "how to tell if your binary is built against musl or some other  
libc, how to tell if it's statically or dynamically linked" (fun with  
ldd and readelf -a), long ago I wrote an intro to cross compiling if  
that's worth linking to... Stuff.

Rob

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rich Felker]

On Tue, Dec 04, 2012 at 02:48:53PM -0600, Rob Landley wrote:
> On 11/30/2012 10:06:20 PM, Rich Felker wrote:
> >On Fri, Nov 30, 2012 at 08:04:44PM -0600, Rob Landley wrote:
> >> >>      - already tested against 8 gazillion packages
> >> >>      - standards compliant
> >> >>      - BSD license: static linking ok, android deployment ok
> >> >Little quibble: MIT + some BSD and some PD code.
> >>
> >> Alas, we don't have a good group term like "copyleft" for "would be
> >> public domain if our legal system wasn't screwed up".
> >
> >We do have a word for this license class; it's called "permissive".
> >Roughly speaking, "FOSS" breaks down into "permissive" and "copyleft"
> >licenses, where the former basically allow you to do whatever you
> >like, and the latter burden you with ensuring that the parties who get
> >the software from you get the same freedoms you had.
> >
> >> I poked Dalias on irc to clarify that we can give a single top level
> >> license and call all the code "compatible" with that, and then link
> >> to the big long copyright list for everybody who cares but have a
> >> clear story on the website.
> >
> >The MIT license text serves this purpose.
> 
> At some point we might want to collate edits to the original list.
> 
> More suggestions:
> 
> The musl-libc.org main page should probably just go to the "intro"
> page, possibly with the existing paragraph merged into the entro.

I feel like the current contents of the "intro" page are a bit verbose
for a main/landing page, but I'm open to other views on this.

> The "what are musl's dependencies" faq entry is a mix of bullet
> points and non-bullet point material that doesn't just clarify the
> bullet points. It doesn't mention what "make" you need (posix-2008
> compliance should be enough?) and the shell is implicit (does _not_
> need bash, posix-2008 again good enough?). Did we actually test
> Linux 2.6.0?

Make needs to be GNU make or compatible. Writing makefiles for POSIX
make is an exercise in wasting maintenance time; there's no way to
write general or wildcard based rules. As far as I know, it should
work even with ancient versions of GNU make (pre-GPL3), and would
probably work with a minimal clone of GNU make (but GNU make itself is
already very light and portable).

Shell is not really required at all since you can write your own
config.mak based on the sample one. Configure is intended to require
just POSIX sh, not anything from bash; I've never even used it with
bash myself.

I have not tested it with Linux 2.6.0, but based on man syscalls,
2.6.0 has all the functionality needed for programs that don't use
POSIX 2008 functionality to work. 2.6.16 is needed for the "*at"
interfaces, and a number of signal-related bugs were fixed in 2.6.22,
so if you don't want to work around kernel bugs, I think 2.6.22 is the
earliest kernel anybody should actually be using.

> Hmmm... something like:
> 
> -----
> Compiling and linking programs against musl should work with any
> conformant C compiler.
> 
> To build musl itself, you need:
> 
>   - Linux 2.6 or later

Linux is not a build dependency. You can build musl just fine on any
machine with a gcc targetting the ISA/ABI you want musl to run on.

>   - c99 compiler with support for gcc-style assembly language.

I'm actually hoping to make the gcc-style inline asm optional at some
point, but for now, it's required.

> It would be nice if there was some kind of "musl manual". If you
> want to write a program against the musl libc, what does it provide?
> (HTML is fine, man page format is kinda archaic these days. This is
> mostly posix, but not entirely.)

Yes, this would be covered in the proposed manual, an outline for
which I posted to this list a couple weeks back. Right now I'm leaning
towards having it be my main project between the last 0.9.x release
and 1.0, to have a major addition for 1.0 (the manual) without the
risk of regressions from major code changes.

> If we're not up to writing something, w link to the POSIX spec with
> the list of functions we've implemented, plus the man7 pages on the
> linux variations thereof (and system calls) and maybe the Linux
> Standard Base sort of collectively covers it.

Basically, my idea is that the manual will "defer to POSIX and ISO C"
on functions specified by them, with specific notes on extended
behavior in these functions beyond what the standards require. For
nonstandard functions, early versions of the manual will probably
provide a list of what's provided with notes wherever they differ
significantly from the GNU or BSD functions they're modelled on. Later
we could expand this documentation to actually cite the relevant BSD
or Linux man pages.

In addition to this, certain functions in the standards have
implementation-defined behaviors, which means an implementation is
required to document what behavior it provides. One thing we should
definitely document is iconv and locale behavior. Compiling a complete
list of the implementation-defined behavior musl needs to document
would be a good project someone could help out with between now and
1.0.

> Beyond the FAQ, there should probably be a page introductory docs. I
> generally link to the old boot floppy howto, the power up to bash
> prompt howto, and Linux From Scratch. (We could always write
> something new, but this is more "understanding how your root
> filesystem works" and "creating a small distro" than "how libc
> works".)

Sounds useful.

> More conceptual intros: what is ELF (with a link to the specs and
> mention of alternatives like #!/interpreters, binflt, binmisc, and
> historical a.out) also WHY is ELF (it's an archive format for binary
> code, about as flexible as tar or zip but with metadata about
> function names, offsets, and permission bits instead of
> files/lengths/owners, generic enough that binflt is created from

ELF does a lot of things. Your description is somewhat accurate for
ELF object files, but for an ELF program file or shared library, the
only part of the file that's actually used is basically sequence of
instructions for mapping the file into memory, adjusting memory
permissions, and possibly performing relocations and symbol
resolution.

> it). Dynamic vs static linking, "what is a dynamic linker and wazzit
> _do_". Something on the whole "how to tell if your binary is built
> against musl or some other libc, how to tell if it's statically or
> dynamically linked" (fun with ldd and readelf -a), long ago I wrote
> an intro to cross compiling if that's worth linking to... Stuff.

Yes, the "how to tell what libc a binary is linked against?" is
actually a really good FAQ topic (a question that really arises
frequently).

Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Isaac Dunham]

On Tue, 4 Dec 2012 16:45:22 -0500
Rich Felker <dalias@aerifal.cx> wrote:

> > The "what are musl's dependencies" faq entry is a mix of bullet
> > points and non-bullet point material that doesn't just clarify the
> > bullet points. It doesn't mention what "make" you need (posix-2008
> > compliance should be enough?) and the shell is implicit (does _not_
> > need bash, posix-2008 again good enough?). Did we actually test
> > Linux 2.6.0?
> 
> Make needs to be GNU make or compatible. Writing makefiles for POSIX
> make is an exercise in wasting maintenance time; there's no way to
> write general or wildcard based rules. As far as I know, it should
> work even with ancient versions of GNU make (pre-GPL3), and would
> probably work with a minimal clone of GNU make (but GNU make itself is
> already very light and portable).

I've used GNU make 3.80, which is the earliest one is _likely_ to encounter (except on distros from last century).  Debian Squeeze still uses 3.81, which is pre-GPL3.


> Shell is not really required at all since you can write your own
> config.mak based on the sample one. Configure is intended to require
> just POSIX sh, not anything from bash; I've never even used it with
> bash myself.

configure works with Busybox and Debian ash, all 3 ksh variants (PD/MirBSD and ksh93), bash... basically every POSIX-style shell except posh, which apparently doesn't support one of the signals we trap. (If I can figure out which one, I'll report it as a bug: posh is supposed to be "Debian Policy" conformant, which is a minimally extended superset of POSIX.)

> >   - c99 compiler with support for gcc-style assembly language. 

Also needs weak symbols. 
In real life this works out to "gcc, clang, or pcc"; there are unidentified issues with Open64, icc might or might not work, and same for Path64 and Sun Studio for Linux.

> > It would be nice if there was some kind of "musl manual". If you
> > want to write a program against the musl libc, what does it provide?
> > (HTML is fine, man page format is kinda archaic these days. This is
> > mostly posix, but not entirely.)

For some folks, but not all.
(I prefer local documentation that can be viewed without a web-browser...)

> Yes, this would be covered in the proposed manual, an outline for
> which I posted to this list a couple weeks back. Right now I'm leaning
> towards having it be my main project between the last 0.9.x release
> and 1.0, to have a major addition for 1.0 (the manual) without the
> risk of regressions from major code changes.
> 
> > If we're not up to writing something, w link to the POSIX spec with
> > the list of functions we've implemented, plus the man7 pages on the
> > linux variations thereof (and system calls) and maybe the Linux
> > Standard Base sort of collectively covers it.
> 
> Basically, my idea is that the manual will "defer to POSIX and ISO C"
> on functions specified by them, with specific notes on extended
> behavior in these functions beyond what the standards require. For
> nonstandard functions, early versions of the manual will probably
> provide a list of what's provided with notes wherever they differ
> significantly from the GNU or BSD functions they're modelled on. Later
> we could expand this documentation to actually cite the relevant BSD
> or Linux man pages.

I still think that documenting what functions we support should be mainly done in the linux-manpages project, which has fairly good coverage and notes libc differences.

> In addition to this, certain functions in the standards have
> implementation-defined behaviors, which means an implementation is
> required to document what behavior it provides. One thing we should
> definitely document is iconv and locale behavior. Compiling a complete
Speaking of which, will musl support the "ASCII" and "" aliases for the C locale, or is that just bloat?  (iirc, these were the issues with R that made libiconv necessary)
> list of the implementation-defined behavior musl needs to document
> would be a good project someone could help out with between now and
> 1.0.
<snip>
> > Dynamic vs static linking, "what is a dynamic linker and wazzit
> > _do_". Something on the whole "how to tell if your binary is built
> > against musl or some other libc, how to tell if it's statically or
> > dynamically linked" (fun with ldd and readelf -a), long ago I wrote
> > an intro to cross compiling if that's worth linking to... Stuff.
> 
> Yes, the "how to tell what libc a binary is linked against?" is
> actually a really good FAQ topic (a question that really arises
> frequently).

I use "strings $BINARY | grep ld.*so"; this lets me see the libc required to run it (or if it's static) as well as arch.

-- 
Isaac Dunham <idunham@lavabit.com>

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rich Felker]

On Tue, Dec 04, 2012 at 03:01:22PM -0800, Isaac Dunham wrote:
> > >   - c99 compiler with support for gcc-style assembly language.
> 
> Also needs weak symbols. 

Yes, thanks for pointing this out.

> In real life this works out to "gcc, clang, or pcc"; there are
> unidentified issues with Open64, icc might or might not work, and
> same for Path64 and Sun Studio for Linux.

I suspect firm is very close to working, and tinycc mob branch might
also be close to working. icc is almost surely working unless there's
some trivial issue; it supports everything we need. The Open64 (or was
it path?) issues seem to be generating spurious relocations in the
shared library; it should be working fine for static linking.

> > > It would be nice if there was some kind of "musl manual". If you
> > > want to write a program against the musl libc, what does it provide?
> > > (HTML is fine, man page format is kinda archaic these days. This is
> > > mostly posix, but not entirely.)
> 
> For some folks, but not all.
> (I prefer local documentation that can be viewed without a web-browser...)

I think markdown, or something roughly equivalent, would be preferred.
HTML and other formats could be generated from it if needed.

> I still think that documenting what functions we support should be
> mainly done in the linux-manpages project, which has fairly good
> coverage and notes libc differences.

What do you mean by this? You want linux-manpages to cover documenting
what musl supports?

> > In addition to this, certain functions in the standards have
> > implementation-defined behaviors, which means an implementation is
> > required to document what behavior it provides. One thing we should
> > definitely document is iconv and locale behavior. Compiling a complete
> Speaking of which, will musl support the "ASCII" and "" aliases for the C locale, or is that just bloat?  (iirc, these were the issues with R that made libiconv necessary)
> > list of the implementation-defined behavior musl needs to document
> > would be a good project someone could help out with between now and
> > 1.0.
> <snip>
> > > Dynamic vs static linking, "what is a dynamic linker and wazzit
> > > _do_". Something on the whole "how to tell if your binary is built
> > > against musl or some other libc, how to tell if it's statically or
> > > dynamically linked" (fun with ldd and readelf -a), long ago I wrote
> > > an intro to cross compiling if that's worth linking to... Stuff.
> > 
> > Yes, the "how to tell what libc a binary is linked against?" is
> > actually a really good FAQ topic (a question that really arises
> > frequently).
> 
> I use "strings $BINARY | grep ld.*so"; this lets me see the libc
> required to run it (or if it's static) as well as arch.

Well for static binaries, there's a lot more work. Also, incorrectly
built (broken toolchain) dynamic-linked musl binaries could show
/lib/ld-linux.so.2 as the program interpreter and "mostly" work
(breaking in subtle ways) as long as the glibc dynamic linker is
present on the system, so a good FAQ answer would at least cover this
possibility and how to check for it.

Rich

Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC.

[Rob Landley]

On 12/04/2012 03:45:22 PM, Rich Felker wrote:
> On Tue, Dec 04, 2012 at 02:48:53PM -0600, Rob Landley wrote:
> > On 11/30/2012 10:06:20 PM, Rich Felker wrote:
> > >On Fri, Nov 30, 2012 at 08:04:44PM -0600, Rob Landley wrote:
> > >> >>      - already tested against 8 gazillion packages
> > >> >>      - standards compliant
> > >> >>      - BSD license: static linking ok, android deployment ok
> > >> >Little quibble: MIT + some BSD and some PD code.
> > >>
> > >> Alas, we don't have a good group term like "copyleft" for "would  
> be
> > >> public domain if our legal system wasn't screwed up".
> > >
> > >We do have a word for this license class; it's called "permissive".
> > >Roughly speaking, "FOSS" breaks down into "permissive" and  
> "copyleft"
> > >licenses, where the former basically allow you to do whatever you
> > >like, and the latter burden you with ensuring that the parties who  
> get
> > >the software from you get the same freedoms you had.
> > >
> > >> I poked Dalias on irc to clarify that we can give a single top  
> level
> > >> license and call all the code "compatible" with that, and then  
> link
> > >> to the big long copyright list for everybody who cares but have a
> > >> clear story on the website.
> > >
> > >The MIT license text serves this purpose.
> >
> > At some point we might want to collate edits to the original list.
> >
> > More suggestions:
> >
> > The musl-libc.org main page should probably just go to the "intro"
> > page, possibly with the existing paragraph merged into the entro.
> 
> I feel like the current contents of the "intro" page are a bit verbose
> for a main/landing page, but I'm open to other views on this.

Something I tried to do on the aboriginal linux about page was have a  
"what is it?" box that drew the eye.

   http://landley.net/aboriginal/about.html

I dunno how _successful_ I was, but that was the intent. It bypasses  
the "where do I go next" hurdle when people glance at the page but  
aren't actually hooked yet. (I don't want to require a decision out of  
anybody quite that early. :)

> > The "what are musl's dependencies" faq entry is a mix of bullet
> > points and non-bullet point material that doesn't just clarify the
> > bullet points. It doesn't mention what "make" you need (posix-2008
> > compliance should be enough?) and the shell is implicit (does _not_
> > need bash, posix-2008 again good enough?). Did we actually test
> > Linux 2.6.0?
> 
> Make needs to be GNU make or compatible. Writing makefiles for POSIX
> make is an exercise in wasting maintenance time; there's no way to
> write general or wildcard based rules. As far as I know, it should
> work even with ancient versions of GNU make (pre-GPL3), and would
> probably work with a minimal clone of GNU make (but GNU make itself is
> already very light and portable).
> 
> Shell is not really required at all since you can write your own
> config.mak based on the sample one. Configure is intended to require
> just POSIX sh, not anything from bash; I've never even used it with
> bash myself.

Make calls sh. All those "rm -rf" entries under clean: get washed  
through /bin/sh (unless you set SHELL), and it's easy to  
leak/{bashisms,in,there} if you're not careful. :)

But if it builds on systems with the Defective Annoying SHell as  
#!/bin/sh, that probably covers it.

> I have not tested it with Linux 2.6.0, but based on man syscalls,
> 2.6.0 has all the functionality needed for programs that don't use
> POSIX 2008 functionality to work.

My "old" regressino test is a Red Hat 9 kvm image (2.4.20 kernel).  
Let's see, 2.6.0 was 2003 and knoppix 3.6 was 2004... and is using  
2.4.27. (I vaguely recall that distros were reluctant to move to 2.6  
and basically didn't until around 2005...)

> 2.6.16 is needed for the "*at"
> interfaces, and a number of signal-related bugs were fixed in 2.6.22,
> so if you don't want to work around kernel bugs, I think 2.6.22 is the
> earliest kernel anybody should actually be using.

Probably good to document this then.

> > Hmmm... something like:
> >
> > -----
> > Compiling and linking programs against musl should work with any
> > conformant C compiler.
> >
> > To build musl itself, you need:
> >
> >   - Linux 2.6 or later
> 
> Linux is not a build dependency. You can build musl just fine on any
> machine with a gcc targetting the ISA/ABI you want musl to run on.
> 
> >   - c99 compiler with support for gcc-style assembly language.
> 
> I'm actually hoping to make the gcc-style inline asm optional at some
> point, but for now, it's required.
> 
> > It would be nice if there was some kind of "musl manual". If you
> > want to write a program against the musl libc, what does it provide?
> > (HTML is fine, man page format is kinda archaic these days. This is
> > mostly posix, but not entirely.)
> 
> Yes, this would be covered in the proposed manual, an outline for
> which I posted to this list a couple weeks back. Right now I'm leaning
> towards having it be my main project between the last 0.9.x release
> and 1.0, to have a major addition for 1.0 (the manual) without the
> risk of regressions from major code changes.
> 
> > If we're not up to writing something, w link to the POSIX spec with
> > the list of functions we've implemented, plus the man7 pages on the
> > linux variations thereof (and system calls) and maybe the Linux
> > Standard Base sort of collectively covers it.
> 
> Basically, my idea is that the manual will "defer to POSIX and ISO C"
> on functions specified by them, with specific notes on extended
> behavior in these functions beyond what the standards require.

I'm thinking a resource for people writing code. Somewhere they can  
look up "what knobs do I have a available and what do they do?"

When writing (or indexing) documentation I find it useful to keep the  
two different roles in mind, tutorial and reference. A great reference  
isn't always good to learn from, and the thing you learned from can be  
horrible to try to look something up in afterwards. It's possible for  
one thing to do both, but at least twice as hard.

> For
> nonstandard functions, early versions of the manual will probably
> provide a list of what's provided with notes wherever they differ
> significantly from the GNU or BSD functions they're modelled on. Later
> we could expand this documentation to actually cite the relevant BSD
> or Linux man pages.
> 
> In addition to this, certain functions in the standards have
> implementation-defined behaviors, which means an implementation is
> required to document what behavior it provides. One thing we should
> definitely document is iconv and locale behavior. Compiling a complete
> list of the implementation-defined behavior musl needs to document
> would be a good project someone could help out with between now and
> 1.0.
> 
> > Beyond the FAQ, there should probably be a page introductory docs. I
> > generally link to the old boot floppy howto, the power up to bash
> > prompt howto, and Linux From Scratch. (We could always write
> > something new, but this is more "understanding how your root
> > filesystem works" and "creating a small distro" than "how libc
> > works".)
> 
> Sounds useful.

http://www.tldp.org/HOWTO/Bootdisk-HOWTO/
http://www.tldp.org/HOWTO/From-PowerUp-To-Bash-Prompt-HOWTO.html
http://www.linuxfromscratch.org/lfs/view/stable/

> > More conceptual intros: what is ELF (with a link to the specs and
> > mention of alternatives like #!/interpreters, binflt, binmisc, and
> > historical a.out) also WHY is ELF (it's an archive format for binary
> > code, about as flexible as tar or zip but with metadata about
> > function names, offsets, and permission bits instead of
> > files/lengths/owners, generic enough that binflt is created from
> 
> ELF does a lot of things. Your description is somewhat accurate for
> ELF object files, but for an ELF program file or shared library, the
> only part of the file that's actually used is basically sequence of
> instructions for mapping the file into memory, adjusting memory
> permissions, and possibly performing relocations and symbol
> resolution.

I also didn't mention the difference betwee .o, .so, and exe files. :)

There's a linkers and loaders book online, and an actual ELF spec on  
refspecs.linuxfoundation.org (although you need to read the base spec  
_and_ a target spec to get anything useful).

http://www.linuxjournal.com/article/6463
http://www.iecc.com/linker/

http://www.linuxjournal.com/article/1059
http://www.linuxjournal.com/article/1060
http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html

Rob