From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/3703 Path: news.gmane.org!not-for-mail From: Rob Landley Newsgroups: gmane.linux.lib.musl.general Subject: Re: embedded newbies site. Date: Mon, 22 Jul 2013 01:00:04 -0500 Message-ID: <1374472804.3719.28@driftwood> References: Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; DelSp=Yes; Format=Flowed Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1374472818 22051 80.91.229.3 (22 Jul 2013 06:00:18 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 22 Jul 2013 06:00:18 +0000 (UTC) Cc: musl@lists.openwall.com To: musl@lists.openwall.com Original-X-From: musl-return-3707-gllmg-musl=m.gmane.org@lists.openwall.com Mon Jul 22 08:00:21 2013 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1V19AS-0007VG-C0 for gllmg-musl@plane.gmane.org; Mon, 22 Jul 2013 08:00:20 +0200 Original-Received: (qmail 17870 invoked by uid 550); 22 Jul 2013 06:00:19 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 17860 invoked from network); 22 Jul 2013 06:00:18 -0000 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:subject:to:cc:in-reply-to:x-mailer:message-id :mime-version:content-type:content-disposition :content-transfer-encoding:x-gm-message-state; bh=CQaXiLASYdDyf+q0sFaJ9HgaxuCLQGfjABRvG6p6fvQ=; b=Lizy0QWDLlDt/zVHvcOS+aDAxuZtO2xrF0TmHSFb2xtwDz8JgrnFcdsdJvSpsdfAl9 RMb4qGEerkz0kJSteuv6KAZJPuOq0bQe4jdcgqzWs8eWuma4zHifuJPYnWNAkaVk39C7 YfSa7Tzr+l7wONXFnMBvsuNdMUlMduiWjRDtLa7QSVr6tD7GQEznHU5Gvx5N2l86CfpD RnvMh9FuStk9R7wKqH9qR3yPPWG/76VAvTVJJaG3gAkEUrcrpw8Ki95H8hKdWpPQTjj2 CqrF0IOd5q7vojJkVhJMS3hENltGm7IiaxDCrTyi8toCc1xfJHAegpewjML3K3HBb599 LspQ== X-Received: by 10.42.228.68 with SMTP id jd4mr16569104icb.44.1374472806747; Sun, 21 Jul 2013 23:00:06 -0700 (PDT) In-Reply-To: (from lmemsm@gmail.com on Tue Jul 16 06:50:29 2013) X-Mailer: Balsa 2.4.11 Content-Disposition: inline X-Gm-Message-State: ALoCoQmCA+wKYqRHZySw5QODGcGe/cwlwyJHx5fgpKZ/0t+LV4vjR0FdveP4Y84KrZ21Pu+ApwSf Xref: news.gmane.org gmane.linux.lib.musl.general:3703 Archived-At: On 07/16/2013 06:50:29 AM, LM wrote: > On Mon, Jul 15, 2013 at 10:03 PM, Rob Landley wrote: >=20 > > I'd like an explicit a place to collect and preserve information =20 > about > > this sort of thing, and a place we can send newbies to ask all the =20 > stupid > > questions. The main page should teach somebody what embedded =20 > development > > _is_ and how to do it, starting with how to build and install the =20 > simplest > > Linux system that boots to a shell prompt (three packages: linux, =20 > musl, and > > toybox). > > >=20 > Sounds like a great idea. Would be interested in reading articles on =20 > some > of the topics mentioned. Sites like suckless.org state what they =20 > consider > are better and worse software choices. I lurked on #suckless irq channel on OTP or whatever it was for a week. =20 It seems to be support for some window manager (dwm?). Nothing else was =20 ever discussed... > Would be nice to see some actual > statistics and rationale backing up what is considered better or worse > design. For instance, there are some negative mentions about the PCRE > library, but when I tried to track down the cons for using it, I only =20 > found > dated performance comparisons showing how poorly it worked if you =20 > don't use > the newer JIT implementation. The great thing about Linux From Scratch is it's practical. It's a =20 procedure you can actually reproduce for yourself, and when you try it =20 you get a running system that you built and are in a position to =20 modify. It mostly explains why you're doing what you're doing, and =20 provides some alternatives along the way. But Linux From Scratch 3.x was a better learning experince than the =20 current one, because these days it's much biger and much more =20 complicated to get a running system, and you don't really learn much =20 more. Plus the "hints" files about things like BSD init scripts are =20 sort of deprecated now. And it doesn't really present stuff like tcl as =20 optional, even though it's only ever used to run test suites... Beyond Linux From Scratch is about adding stuff to the base linux =20 system, but there's nothing in there about _removing_ stuff. Or =20 swapping out base packages for alternatives. (Again, the "hints" used =20 to go into this, but they seem to have tailed off over the past few =20 years...) Oh, we should totally be linking to =20 http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html and =20 possibly trying to reproduce a current version under 3.x kernels. A lot of stuff, anybody can take and just do the legwork. For example, =20 we really need a current version of =20 https://www.kernel.org/doc/mirror/lki-single.html and somebody could =20 just _take_ that and compare it with the current kernel and do an =20 updated version based on what they learn by reading current kernel =20 source using the old 2.4 version as a guide... > What might be a positive for a system that's > optimized for a particular processor might be a negative if you're > interested in software that ports to multiple processors and vice =20 > versa. I've yet to find a per-processor optimization that buys you one =20 interation of Moore's Law. And I _have_ seen years of seesawing back and forth over "here's a =20 lookup table of 256 angles your ship can be at where we've done all the =20 triginometry... oh the new processor has a floating point coprocessor =20 and tiny l1 cache so it's actually faster to calculate it than thrash =20 the cache with our lookup table, oh now the NEW processor has a big L2 =20 cache so the lookup table is faster again, oh now they've added 3D =20 hardware so all this mess is slower than having the 3D hardware do =20 it..." I've seen optimizations where the pendulum went back and forth a =20 half dozen times on the same issue with RISC vs CISC and whether loop =20 unrolling is a win and... And what it keeps coming back to is "simple code you understand beats =20 clever code you don't". Do a simple implementation, let the compiler =20 optimize it for you, the bit you're doing is not the hard part (if it =20 is, you're doing it wrong, which means back up and rethink), so just =20 get it done and stay out of the way while Big Important Programmers do =20 things they find Terribly Hard To Do because they're straining at gnats =20 and all that... (Premature optimization is the root of all evil, when in doubt use =20 brute force, etc.) > Musl's useful not just for embedded systems but for older machines =20 > that > want to run efficient desktop environments. However, what works for a > desktop environment might not work well for an embedded system and so =20 > on. Knoppix was a fine desktop. I used it as such for over two years. I =20 installed it to the hard drive because running from CD was slow and =20 tying up the CD drive was inconvenient, but operating under a space =20 constraint on the image size meant they had to figure out what they =20 really NEEDED, and it made a better system. > Would like to see actual lists of pros and cons, less opinions and =20 > let the > user decide if the software is really a bad fit with his/her needs or =20 > not. I wouldn't presume to advise people without knowing what they wanted to =20 use a system _for_. For an awful lot of people Red Hat Enterprise is =20 the correct choice, albeit for financial rather than technical reasons. =20 (You know why Red Hat drove Sun Microsystems out of business, right?) > Would also love to see a forum where one could discuss pros and cons =20 > of > various software and library choices, alternatives already out there =20 > and if > the user wants to rewrite some of these himself or herself for =20 > specific > needs, a place to discuss design issues. I'm not sure you're asking well-defined questions here. Ok, simple mindset: complexity is a cost. Treat complexity as something =20 you spend to get functionality, and you're 3/4 of the way to making =20 good decisions. There's some fuzziness in measuring complexity, but lines of source =20 code maps pretty well to "amount of human thought required to =20 understand what this thing is doing". If you have a project that gets =20 the job done with 100,000 lines of code, and another one that requires =20 2 million lines of code, to _me_ it's pretty darn clear which of the =20 two is superior. You can then say 'but the big one has features X, Y, and Z I need, and =20 we benchmarked the performance in our deployment environment and the =20 big one performs 12.7% faster", and then you go "do you really need =20 those features? How much work would adding them to the small one be, =20 and would the upstream project take it or just consider it bloat, and =20 if you were to maintain your own patchset to add that feature to the =20 small one would it change your answer about whether or not you actually =20 need it?" And of course there's complexity you directly engage with and =20 complexity you indirectly engage with; your _local_ complexity may be =20 "this giant black box works for everybody else so just using it is very =20 easy for us as long as it never breaks, and if it does we have a vendor =20 for that". And of course if you _are_ the vendor, deploying dropbear =20 instead of openssh can have a negative PR effect because openssh is The =20 Standard but if that's such a big deal why aren't you using Windows... And really all this infrastructure is generally stuff that should just =20 work, and should be an existing solved problem so you can focus on your =20 app... See how it spirals through a gazillion topics? As I said: not sure what =20 questions you're really asking. > There is an lfs-chat list. Think it would probably be a good idea to =20 > post > something about the idea of an LFS for embedded systems there and see =20 > if > any of the regular LFS users would be interested in getting involved. =20 > A > start might be to take the outline of possible topics Rob Landley =20 > supplied, > put it up on a wiki and see if people will volunteer to fill in some =20 > of the > blanks. Might also be useful to get together a list of what tasks =20 > need to > be done to get something started and ask for actual volunteers for =20 > each > task to help get things rolling. I do think a mailing list or forum =20 > would > be useful as well. That way, one can get discussions going and =20 > brainstorm > ideas about how best to program something or find information on a =20 > topic. > I tend to prefer mailing lists and forums to IRC. It's easier to read > through past information. Good concrete questions to answer are a good start. Not "maybe people =20 would want to know X" but "I want to know X." > I've been talking with another developer about the possibility of =20 > building > (yet another) lightweight Linux distribution for older machines. I =20 > really > haven't been happy with what's currently out there. Aboriginal Linux is the lightest-weight development environment I know =20 how to make. (And switching to musl should make it lighter.) > The average definition > of a lightweight Linux desktop for older machines is to use a lot of =20 > GTK+ > programs (with a lightweight desktop like XFCE (not my definition of > lightweight), LXDE or razorQT) and even interpreted programs (as long =20 > as > they look like they're in console mode or like they might somehow be > lighter or more useful than their compiled equivalents). X11 is a windowing system. It draws graphics onna screen; lines, fonts, =20 boxes, stamping images from bitmaps and such, and the bitblts and =20 double buffering used for dragging and scrolling windows and such. Then you have a window manager that draws borders and title bars and =20 menus, and gives them behavior so when you grab the corner and drag it =20 the window resizes, or grab the title bar the window moves, or handles =20 the z-order stuff so windows draw in front of other windows (which =20 pragmatically means you hide or clip window areas and only draw parts =20 of 'em). Then you have a toolkit, which is a shared library of graphics =20 primitives and associated behavior when they get mouseovers or clicks =20 or keys on the keyboard are pressed while it has focus. (Window manager =20 defines what "focus" is and sending keypresses and clicks to the right =20 thing.) Your toolkit is where you find code to implement a button or a =20 scrollbar or a pulldown menu. Then you have a desktop program, which is the thing that runs to _use_ =20 X11, a window manager, and a toolkit to provide behavior for an =20 otherwise empty screen. It provides the bar alogn the top that shows =20 you your list of pen windows, and provides a menu of stuff you can =20 launch, and a dock for tiny icons associated with programs that know =20 about that type of desktop and can do the right transaction with it to =20 register themselves. I'm running the xubuntu linux distro. It's using xfce as the desktop =20 program, which uses the gtk toolkit, and xfwm4 is the window manager. =20 All running on top of x.org which is the windowing system. It's possible _not_ to use all these layers of stuff, but generally =20 when a program doesn't it's because its reinventing it. You don't have =20 to use gtk, you can have your program draw its own buttons and respond =20 to mouse clicks in that area of its window manually: and that means no =20 two programs look or behave remotely the same. Once again, defining "simple" requires understand what it is you're =20 trying to _do_. Simple is an adjective, not a noun. > They typically > use the KISS principle which means (according to their take on it) I'm > stuck with the one graphics editor, the one music player, etc. that =20 > the > distribution creator happens to like. A Gimp or a Photoshop style =20 > program > has a lot of functionality. So does an Office Suite like LibreOffice. Star Office (a german company that Sun bought and renamed Open Office) =20 was the first non-microsoft program that actually had good support for =20 reading and writing Word files, due to YEARS of careful reverse =20 engineering effort that started back on OS/2 before being ported to =20 Linux. The opening of Open Office had the same failure mode Mozilla did (long =20 story, I did a talk about it once if you're bored) and the resulting =20 code bloat is epic. But getting the "reads, edits, and writes word =20 documents well" functionality out of anything _else_ turns out to be =20 really hard. > If you're going to replace heavyweights with a program that does one =20 > thing > well, you're typically going to need more than one application with =20 > each > application designed to perform a specific piece of the functionality > well. You need more than one type of graphics program if you're doing > serious graphics editing, more than one type of music program if =20 > you're > doing serious music creation, etc. A lot of the topics such as how =20 > to put > together a system from scratch, what boot and init programs to go =20 > with, > which userspace utilities to use, which package manager to use, which > libraries are efficient would be of great interest for the project. Linux From Scratch and Beyond LInux From Scratch already cover this. =20 And Gentoo set about trying to automate it. Both have serious failings, =20 but they're an existing starting point to acquire this knowledge. What neither does is says how to set up a simple base system that isn't =20 infested with gnu crap, and then extend it towards providing the =20 prerequisites packages such as OpenOffice require. Learning how to swap =20 busybox for coreutils and make that work to run postgresql on the =20 resulting system... > Another concern to me is which projects are open to accepting patches =20 > and > which aren't so open, making it prudent to look into more friendly > alternatives. I'd also been interested in discussing when it pays to > rewrite something from scratch and when it's better to reuse what's =20 > already > been done. I've been picking up ideas by looking at the code embedded > systems use. However, the end goal for this particular project is =20 > not an > embedded system but a GUI desktop that an average end user will be > comfortable working with. There's a lot of overlap, but definitely > different goals with different design tradeoffs. Embedded and non-embedded systems are distinguishable by the =20 "complexity is a cost" mindset. Desktop systems seem to think they have =20 unlimited storage, memory, bandwidth, processing power, and so on due =20 to Moore's Law, and that they also have unlimited warm bodies capable =20 of maintaining the result due to open source and the internet. Embedded systems are designed with the idea that fitting those 15 =20 million lines of code into 30 cents worth of flash memory could be =20 painful. That running it on a processor running off a watch battery may =20 be slow. That one junior engineer alotted 3 days to port it and every =20 prerequisite package it requires to a brand new processor implemented =20 in an FPGA with a beta compiler fork based on gcc 3.4 might have a =20 rough time of it. That there's some exploit lurking in those 15 million =20 lines of code, and when you put it on a system that no human being will =20 log into for two years, that doesn't get upgraded in all that time, but =20 has a broadband connection to the internet, bad things will happen. Think about Mozilla vs webkit. Mozilla is based around the idea that =20 writing a good browser is hard, and there should be only one, and it =20 must do everything for everybody and be perfect. Webkit is based on the idea that a browser is disposable and gets =20 reimplemented from scratch every few years. Webkit started life as the =20 KHTML engine in Konqueror, the browser built into KDE which went from =20 zero to usable in about a year. Then Apple grabbed it and forked it and =20 did Safari out of it. Then Google grabbed it and forked it and did =20 Chrome out of it. I expect in a couple years people will throw chrome =20 out and do a new one. Google designed Chrome to work with Android, on phones and tablets. You =20 can kill individual tab processes, because they acknowledge they're =20 going to break and it won't be perfect so that's a thing you may want =20 to _do_. It's got a lot more of the embedded mindset than Mozilla, as =20 Scott McCloud explained back at the launch: http://www.scottmccloud.com/googlechrome/ Rob=