From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/2386 Path: news.gmane.org!not-for-mail From: Rob Landley Newsgroups: gmane.linux.lib.musl.general Subject: Re: Summary of 1.0 marketing plan/scheme/nefarious plot from IRC. Date: Fri, 30 Nov 2012 18:04:54 -0600 Message-ID: <1354320294.2190.24@driftwood> References: <20121130012701.GY20323@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; DelSp=Yes; Format=Flowed Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1354320320 2750 80.91.229.3 (1 Dec 2012 00:05:20 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 1 Dec 2012 00:05:20 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-2386-gllmg-musl=m.gmane.org@lists.openwall.com Sat Dec 01 01:05:31 2012 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1TeaaI-0008QI-ST for gllmg-musl@plane.gmane.org; Sat, 01 Dec 2012 01:05:31 +0100 Original-Received: (qmail 3243 invoked by uid 550); 1 Dec 2012 00:05:17 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 3231 invoked from network); 1 Dec 2012 00:05:14 -0000 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:subject:to:in-reply-to:x-mailer:message-id:mime-version :content-type:content-disposition:content-transfer-encoding :x-gm-message-state; bh=hR9oQMmB0I29lAjgaqqquQMvn1cfIv8LlhJRmaiGbD0=; b=mAahU/j1B3WNFIDBvnNUH9XUMKFFUTUZHsbqUU5jenPN1OvJE5fsUYvd3jcq+AIX2q lEcPcVfXOxgmLhQ62kWwsDryYNvNy1N4ayzchF+HWgz27ulZnofOzJuyQBD3FehH+Kqp eb0PYoL9I3cNanhcD00UgcevdNOkzL3Sg5oLT8pq38MoEoRv01mE/dW3acJmMuIP7JMt t9auxTKteSs2eBWGWqd/preyTvmRbyQF0rGKdrxsmyZb9gIDROJA+BrLHMc/QE7uNkV0 mlqfj2BIjb5HRyebxfxN0UFFNfXSX/pQKdMkO994mskPX9XHAuCKiM862jbCFRuF1yYa JwNg== In-Reply-To: <20121130012701.GY20323@brightrain.aerifal.cx> (from dalias@aerifal.cx on Thu Nov 29 19:27:01 2012) X-Mailer: Balsa 2.4.11 Content-Disposition: inline X-Gm-Message-State: ALoCoQlyX87mn578Oc1JB9Tvetq8K/jk2d7510ln/80DUsTCr7UsB4ggD6Op59Xyl2f8frqphBAM Xref: news.gmane.org gmane.linux.lib.musl.general:2386 Archived-At: On 11/29/2012 07:27:01 PM, Rich Felker wrote: > On Thu, Nov 29, 2012 at 02:50:03PM -0600, Rob Landley wrote: > > Notes from the discussion we had on IRC, plus some further random > > thoughts on telling the world about musl: > > > > - wait until 1.0 so it's most likely to works for them. > > - People who take a look and wander off again are less likely to > > take another look, > > so try to make a spash when you're _ready_, not before. >=20 > Agreed. I never really wrote down this principle before, but it's been > in my mind for a long time. The main publicizing I've been doing so > far has been in niche places (like mips and microblaze lists) and > making sure musl is well-indexed in databases like freecode (formerly > freshmeat) and ohloh. I suspect there are a lot more of the latter we > should also hit. Having a good 'elevator pitch' is nice too. Musl is a brand new C library that bridges the gap between Linux and =20 Android. It's BSD licensed and Posix-2008 compliant, written from scratch to =20 be small simple and clean with no legacy baggage and good thread support. The =20 reason you've never heard of it is the first git commit was February 2011. > > - chroot for each target with native development tools > > - system images for qemu maybe? >=20 > It would be really cool if we had "system root trees" rather than > "system images", with virtual 9p for booting qemu. By the way, getting > qemu working on musl should probably be in our compatibility goals for > 1.0. It's slightly more complicated than that, or I'd already have done it. =20 :) Making client side initramfs work is easy, but it needs a server. Last =20 time I looked into this (circa =20 http://landley.net/notes-2011.html#29-04-2011), it would require diod =20 on the host acting as a server to work on most targets. While this =20 isn't much different than qemu's cifs export mode launching samba, =20 people are much less likely to have diod already installed on the host. QEMU does have a built-in 9p server, virtfs requiring a virtio =20 transport. Unfortunately, between qemu-system-blah and the kernel =20 virtio transport isn't supported equally on all targets. Specifically, =20 I couldn't get virtio to work on arm, which is kinda important. I still have the "i686-virtio" and powerpc-virto targets Alessio Bogani =20 contributed last year, and they mostly work (just fixed minor bit rot =20 with qemu getting renamed to qemu-system-i386). That's actually using =20 virtio block device and virtio network, not virtfs, but the principle's =20 the same. (Think of virtio as a bus and then other things get =20 implemented on top of that bus; even though the bus is simpler than PCI =20 support for it may not exist in the target we want because nobody's =20 bothered yet.) I also looked at getting virtio console working, but at the time the UI =20 for it was appalling: http://lists.gnu.org/archive/html/qemu-devel/2011-05/msg00390.html > > - launch x11 vnc server and display in tightvnc window? > > - jslinux live image on website >=20 > Love this idea. I'd like it even better if we could adapt jslinux to > start with a preinitialized memory/machine-state image rather than > having to spend 15-20 seconds booting. BTW I think we need to look > into whether there are license issues with using jslinux...? The actual javascript source says redistribution requires permission of =20 the author, so yeah. Somebody has to ping fabrice. (Or else reference =20 the files off his web page, which is hugely impolite.) I wonder what the busybox guys did? Let's see, google for "jslinux =20 busybox" and bubble up through the archive a bit to find the commit =20 that added it to the website is here: http://git.uclibc.org/busybox-website/commit?id=3D317871f1a85816e3c0f405f00= b2f081978fa9caa And the jslinux in that still says you need Fabrice's permission to =20 redistribute, commit comment is just that Denys added it and no further =20 details. Date is March 4 of this year so check the mailing list archive =20 on the website... And this is the start of the thread about it: http://lists.busybox.net/pipermail/busybox/2012-March/077498.html Content of message, and I quote, "Hey guys, Check this out!". One =20 reply, "I saw it. Very cool." No mention of license or permission. =20 (These are the people who took over the project that's the poster child =20 for GPL enforcement. Wheee...) Yes, by all means, let's ask Fabrice. (As musl maintainer, you should =20 probably do it.) > By the way, somebody also mentioned a mips emulator in js we could use > instead or in addition to jslinux, but I don't know if it emulates > enough hardware to get linux up. Yay! > > - distro conversions > > - leverage existing repositories, don't fall into the buildroot =20 > trap >=20 > I think I've already avoided this trap without avoiding new distros, > by building a community of people who like rolling their own distros > but not getting involved in the distro/build stuff at all myself. This > has been great for getting musl a lot of testing with lower entry > barrier than large existing distros, but adding the latter now would > be a good next step. You've done great. My point here is just "how can we leverage the work =20 other people have done without distracting ourselves more than =20 necessary". If we can come up with a musl chroot that debootstrap runs in (the =20 "compile packages" not just "install binaries from repo" variant), or a =20 musl chroot that emerge can build the world in, or koji: =20 http://opensource.com/life/11/7/free-sake-story-koji?sc_cid=3D70160000000ID= mjAAG Or whatever SuSE's variant is (blanking on the name), or buildroot, or =20 arch's pacman, or... (I'd mentino Yocto's poky but after meego and tizen and such it's =20 really hard to _care_ about any of the Linux foundation's "look, =20 another not-android!" gyrations.) I wandered aimlessly through this bit more than once over the years and =20 it's why I have the "bootstrap" stuff in my control-images. What I was =20 trying to do was extend the basic "simplest self-hosting build =20 environment" up to whatever the various distros needed and then let =20 them build themselves. It's a finite amount of work. But it's an =20 _undocumented_ finite amount of work and all these build systems have =20 DEEP assumptions that they are building under themselves. (Yes, even =20 gentoo. It's actually worse than Debian about it.) Basically, I started researching gentoo bootstrapping in 2008: http://landley.net/notes-2008.html#17-12-2008 My friend Mirell took a serious stab at getting it working in 2009: =20 http://web.archive.org/web/20090411155149/http://impactlinux.com/hg/gfs I took my own stab at it in 2010: http://landley.net/notes-2010.html#15-09-2010 And then tried a Funtoo version in 2011: http://www.funtoo.org/wiki/ReBootstrap And each time, when you try to create the base system from something =20 other than an existing gentoo system it just _melted_down_ and went off =20 on a tangent to fix something else. I eventually went "ok, there are three problems here": 1) I'm trying to build a distro on something other than an existing =20 copy of that distro. 2) I'm trying to use a different libc (and different toolchain, but =20 that's less of an issue). 3) The existing system I'm providing uses busybox instead of the =20 specific versions of the gnu/dammit tools they expect (and boy were =20 they version-sensitive and configuration sensitive, things like python =20 requiring bzip2 shared libraries to exist on the host when it's built =20 or else package extraction won't work because portage doesn't know how =20 to pipe stuff through bzip2, just use the optional python shared =20 library wrapper). So I went "if I build Linux From Scratch I can deal with 2 and 3 =20 separately, and then rebootstrap portage under the resulting LFS system =20 on the target, and it's a huge digression but can be completely =20 automated, so..." (And that's before getting into trying to wean =20 portage off of the requirement of explicitly naming every supported =20 target in every single ebuild file in the entire tree, so that adding a =20 new target requires touching every file in the portage tree and the =20 idea of telling gentoo "just build whatever architecture the host is" =20 is fundamentally against the way they designed it... Yeah, that was me =20 going "funtoo, how does _that_ work"?) You may have noticed that my todo list has archaeological strata. Right =20 now Toybox has put all this on hold, and the rise of Android has =20 probably rendered some of it moot... An advantage YOU have is existing gentoo developers with interest who =20 you may be able to foist this off on. Plus you're just trying to swap =20 out _one_ component at a time, and only support a few specific =20 architectures. > > - Write linux from scratch "musl hint", contribute it to LFS, then =20 > link > > to it on LFS website from musl website. >=20 > Can you elaborate on what a "musl hint" would be? Basically they're standalone "or you could do it _this_ way" =20 explanations for alternatives or extensiosn to the way Linux From =20 Scratch does stuff: http://hints.linuxfromscratch.org http://www.linuxfromscratch.org/hints/read.html They're referenced from time to time in the main Linux From Scratch =20 book, ala: http://www.linuxfromscratch.org/lfs/view/stable/chapter08/kernel.html And at the end it's one of the things they point you at: =20 http://www.linuxfromscratch.org/lfs/view/stable/chapter09/whatnow.html The "Beyond Linux From Scratch" book more or less evolved out of the =20 pile of hints, trying to turn it into a bigger systematic sequel book =20 of additional stuff you can build. But the hints profide alternatives =20 to the base infrastructure, such as BSD-style init scripts instead of =20 SysV style: http://www.linuxfromscratch.org/hints/downloads/files/bsd-init.txt Or using uClibc instead of glibc in the base system: http://www.linuxfromscratch.org/hints/downloads/files/uclibc.txt (Of course BLFS has its own problems. Their last release was in 2008 =20 and they went to one of those "everybody just use svn and ping us if =20 it's broken" things because their development model's become too =20 dysfunctional to cut releases anymore.) Anyway, advantage of this: if they take it you can then link to it from =20 musl's website and go "hey build your own system guys, here's how you =20 do a musl based LFS". It's not _explicitly_ that LFS would be endorsing =20 you, but it feels that way a little bit. :) > > - is userbase of glibc, uClibc, klibc, or dietlibc better served by > > musl? > > - contribute musl option to buildroot? > > - contribute musl option to crosstool-ng? >=20 > There's demand for musl in crosstool-ng at least; the maintainer has > expressed this, but is largely unfamiliar with musl and waiting for a > patch from someone willing to do it. He's a good guy. I'm not a fan of the design of his project (driving it =20 makes The Tardis look user-friendly), but a lot of people find it =20 useful. > > - Ask mentor graphics (formly code sourcery) to do a musl =20 > toolchain? > > - LOTS of proprietary embedded devs use this one, it's > > "professional". >=20 > I suspect they might be particularly interested since their customers > might need to avoid copyleft code. Or at least the random threat of lawsuits even if they _try_ to conform. Not gonna rant. Nope. Not gonna. > > - windriver.com is now a wholly owned subsidiary of intel > > - klibc guys are initramfs@vger or embedded@vger (see lists) > > - ask clibc author Peter Anvin if musl serves his needs? >=20 > I think musl essentially obsoletes klibc. The main added cost of musl > over klibc for initrds, etc. is the soft-float code that will get > linked in due to printf on archs with no hard-float. If this is a > problem, we could eventually support an option with musl to inhibit > printf float code from getting linked at static-link time. A "./configure --disable-soft-float" would be nice just for bragging =20 rights. > This is a > bit nasty but could be done with weak symbol tricks. Of course, even > with all the soft-float code, I think binaries linked to musl will be > significantly smaller than all the kernel modules, firmware, etc. > stored on an initrd. >=20 > On the other side, the big advantage of musl over klibc is that you > don't have to worry about whether the apps you need will be okay with > the limited/non-conformant functionality provided by klibc. > Well-written apps should "just work", and be small. An _ideal_ situation would be to get Peter Anvin to end klibc =20 development, point his users at musl, and possibly even become a musl =20 user/developer. I admit the diplomacy to arrange such an outcome is =20 beyond me. However, it would be nice to ask him what he gets out of klibc. If it's =20 providing a reference implementation or using it to test system call =20 generation or some such, possibly musl could fill this role. If there's =20 something specific klibc does, it would be nice if musl could also do =20 it. > > - websites that might review musl if we ask nicely: > > - linux > > - lwn.net (submit via lwn@lwn.net) > > - h-online (ping @codepope on twitter) > > - Linux Journal > > - Linux Today (they'll just link elsewhere) >=20 > I think getting the 1.0 release on Reddit, Hacker News, etc. would be > useful too. If for nothing else, as one step closer to the magic 7. Obviously, none of these lists are exhaustive. :) But if we add to them much, collating to create a master list sounds =20 like a job for a wiki. > > - android > > - not personally familiar, google for "android news" finds =20 > several. > > - works well with android kernel, installs side-by-side with > > bionic, > > static links well, doesn't introduce any new licensing issues, > > provides full posix environment, active and responsive dev > > community. >=20 > Yep. I'm not familiar with these sites either though. Enthuse to the cyanogen mod guys, send them to enthuse to others? http://www.homeonthestrange.com/view.php?ID=3D28 > > - ask people on mailing list and irc to blog/tweet about the 1.0 > > release when it > > happens. >=20 > Yes, this is definitely an under-utilized aspect of the (now rather > large) community. Word of mouth is the best asset, but buzz is all about combo bonuses. The _other_ thing you might want to do is pull up the original BBC =20 "kitchen nightmares" on Netflix streaming and watch Gordon arrange a =20 full set of customers for each failing restaurant, at which point =20 hilarity inevitably ensues. "OK, you've achieved basic competence. Now a thundering horde of a =20 gazillion people show up wanting to try your thing. How do you avoid service melting =20 into a puddle under the load?" Lots of people are not prepared for _success_. You've got to acclimate =20 newbies. You've got to have a website that can handle the downloads. =20 Mailing list traffic spikes. I dunno if this will happen, but _if_ it =20 does, are you prepared? Ideally you'd have not just a goot FAQ and web pages but "the book of =20 musl" which can act as both tutorial and reference (two different =20 functions), and designated hand-holders in the community fielding a =20 flood of newbies. This is probably hashing your chickens before counting them, of course, =20 but I wanted to at least bring it up. For example, if you put your musl porting guide online (p.s. put your =20 musl porting guide online) and you suddenly have three different people =20 submitting ports to different architectures, the testing burden's gonna =20 get nuts. The "why doesn't it work with this package" requests could =20 spike beyond the ability to field. Bug tracker. Translations, including =20 people translating the documentation and web pages themselves (with all =20 the version skew that entails). Triaging and merging forks. Patches =20 winding up in distro repositories and not necessarily making their way =20 back upstream promptly... Yeah yeah, such problems we should have. Just don't be surprised by =20 them. :) Rob=