From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/3157 Path: news.gmane.org!not-for-mail From: Rob Landley Newsgroups: gmane.linux.lib.musl.general Subject: Re: Best place to discuss other lightweight libraries? Date: Mon, 22 Apr 2013 21:14:27 -0500 Message-ID: <1366683267.18069.155@driftwood> References: <5175D52D.3050107@gentoo.org> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; DelSp=Yes; Format=Flowed Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1366725607 6957 80.91.229.3 (23 Apr 2013 14:00:07 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 23 Apr 2013 14:00:07 +0000 (UTC) Cc: musl@lists.openwall.com To: musl@lists.openwall.com Original-X-From: musl-return-3161-gllmg-musl=m.gmane.org@lists.openwall.com Tue Apr 23 16:00:10 2013 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1UUdlR-0000aa-V1 for gllmg-musl@plane.gmane.org; Tue, 23 Apr 2013 16:00:10 +0200 Original-Received: (qmail 18028 invoked by uid 550); 23 Apr 2013 14:00:09 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 18020 invoked from network); 23 Apr 2013 14:00:08 -0000 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:date:from:subject:to:cc:in-reply-to:x-mailer:message-id :mime-version:content-type:content-disposition :content-transfer-encoding:x-gm-message-state; bh=sqr8qWZskm0z8du3zzSXGDcNdvizBvI6OSM8RuCwgXk=; b=HsD8EmpiolW0/bKYicnSo5QU0TlW59YrCLbEJ5At3F5/jMRLAo5uPEYDM/cJsKItkL OUKF2KpYcRO8JuiFupp4dPuRljO4jQQlzfed+t/800oyQvflK+mrGRn/peGez+8of0gl jgk+ghRjvh75RB7dTrCsh5aQFOYMHNrFPl2KVMoGTwlBTog3v7MqRtx3p5ZsWjPfHBvh jN2+CWmk9cxuioiCc+I9esKFVDuD2JA8v5p+pJ+NjysySctjjGe5lbOL6a+x2gk4LxFp VP/jf52ziLWarhZljNteJQisve0AcblOaasmqcT8XCnIx0e6y5oAHP8CsGcy57YwkjLQ iW+A== X-Received: by 10.50.17.71 with SMTP id m7mr18304966igd.14.1366725596537; Tue, 23 Apr 2013 06:59:56 -0700 (PDT) In-Reply-To: <5175D52D.3050107@gentoo.org> (from lu_zero@gentoo.org on Mon Apr 22 19:26:21 2013) X-Mailer: Balsa 2.4.11 Content-Disposition: inline X-Gm-Message-State: ALoCoQns13/kjHTUXaiOZ/gVeI58wQnmiRzwJ9IBLrqo/vb2WhWI1rEjHZqHIkfcROE9M3infQOS Xref: news.gmane.org gmane.linux.lib.musl.general:3157 Archived-At: On 04/22/2013 07:26:21 PM, Luca Barbato wrote: > On 04/23/2013 01:06 AM, Rich Felker wrote: > > On Tue, Apr 23, 2013 at 12:42:01AM +0200, Luca Barbato wrote: > >> On 04/22/2013 11:52 PM, Rich Felker wrote: > >>>> For this there aren't solution that won't cause different =20 > problems I'm > >>>> afraid. > >>> > >>> Sure there are. I get the impression you can tell I was talking =20 > about > >>> libav/ffmpeg's log interface. :-) The obvious solution is to bind =20 > log > >>> contexts to the object you're acting on. See this nice talk: > >>> > >>> =20 > http://misko.hevery.com/2008/11/21/clean-code-talks-global-state-and-sing= letons/ > >>> > >>> If I remember right, part of the problem way back was that there =20 > were > >>> deep function calls that had no context available to them, and =20 > that > >>> didn't _need_ a context for anything but logging warnings or =20 > whatnot. > >> > >> In the specific case yes. I tried to foster proper return error > >> propagation, so you get something more meaningful than EINVAL/-1 =20 > and > >> that is usually enough in those specific cases. > >> > >> The general problem is that the library user wants to be the only =20 > one > >> having a say on what goes where so single point overrides are =20 > useful. > > > > The problem with your comment here is the phrase "library user". Who > > is the library user? You may be thinking from a standpoint (in our > > example) of MPlayer, but what if instead the application you were > > looking at were a file manager that itself had no awareness of video > > files, and only ended up processing them as part of a library pulled > > in from a plugin for file previews? Obviously there is no way the =20 > app > > can be aware of where the log messages should go since it's not =20 > aware > > the library even exists. The user is the library that depends on the > > library doing the logging, not the app, and it's very possible that > > there is more than once such library. In which case, you have =20 > madness. >=20 > Usually (at least that's what I do in those case) the global logger is > overridden to use the outer library logger then you -end-user- =20 > override > it as well and then everything goes where you want. >=20 > The other widespread solution is to eat stderr and if something =20 > appears > show to the user, crude but not so bad. >=20 > >> When you start using those libraries in situations in which you'd =20 > like > >> to have per-$situation logging then you start to scream. > > > > Keep in mind it might not even be "per-situation logging". It might =20 > be > > something like one plugin needing to send the message back up to the > > application as a popup message to display, and another plugin just > > wanting to render the message text as a file preview in place of an > > image... >=20 > Yeah, logging messages properly is terrible. >=20 > >>> Yes, basically. Dependency on glib means your library will impose > >>> bloat and it will preclude robustness. > >> > >> Yet glib gives you oh-so-many-features (I fell for it once), sadly =20 > there > >> aren't many utility libs that provide sort of useful data =20 > structures, > > > > If you want the data structures, I think that means you should be > > using C++, not C. >=20 > C++ stock data structures are too runtime-dependent, crafting your own > means getting the worst of both words if you aren't extremely careful =20 > =3D\ >=20 > Hopefully the new crop of system languages would try to capitalize on > the experience... What new crop of system languages? C is a portable assembly language with minimal abstraction between the =20 programmer and what the hardware is actually doing. It uses static =20 typing, static memory allocation, and if you really care you can =20 explicitly specify integer sizes (uint16_t or LP64) and handle =20 endianness and alignment and so on down to memory mapped bitmasks. It =20 provides simple container types based on pointer math: arrays are =20 simple pointer arithmetic, and structs concatenate a group of variables =20 so each member name corresponds to a fixed offset and size (static, =20 determined at compile time) where the value is to be found relative to =20 the pointer to the start of the struct. Scripting languages like python/lua/ruby have opaque abstractions where =20 you honestly don't need to know how it's implemented. They replace =20 poitners with references, and build garbage collection and dynamic =20 typing on top of that. Their built-in container types are resizeable, =20 including an array variant and a dictionary variant. The dictionaries =20 aggregate via keyword/value association, so you can add and remove =20 members on the fly. In C, types are a property of pointers. In scripting languages, types =20 are a property of objects, meaning _references_have_no_type_. You =20 dereference to find out what type it is. So when you implement =20 functions, you find out what type it is when you try to use it, but =20 asking for a member and performing an operation on that member. If the =20 member isn't there, or doesn't support that operation, it throws an =20 exception. You can catch that exception and handle it however you like, =20 up to and including adjusting the object to add the member in question =20 so it _can_ succeed. But if you don't catch the exception locally, no =20 problem: it's all garbage collected. References that fall out of scope =20 are naturally freed by the system. These are two fundamentally different ways of programming. scripting =20 languages are dynamic, everything interesting determineda t runtime, to =20 the point where they don't even have a compilation step. You set the =20 executable bit on your source code. (Is there a bytecode compilation =20 step at load time with an optimized interpreter doing batched code =20 translation with buffering that Sun's marketing department called "just =20 in time" or some such nonsense but which Apple's 68k emulator for the =20 PowerPC was already doing in 1994? Maybe. Again: it doesn't matter, the =20 abstractions are opaque, it all just works.) So with C: pointers, everything statically compiled to machine =20 language, no abstraction. With scripting langauges: references, =20 interpreted at runtime, opaque abstraction and often multiple different =20 but compatible implementations (python/jython). Then you have C++, which can't decide which it is. C is a local peak in =20 language design space. scripting languages are another. C++ is in the =20 valley between them, neither fish nor fowl, with the worst =20 characteristics of both. It's a static language like C, statically =20 typed and based on pointers, with thick layers of _leaky_ abstractions. =20 If anything goes wrong, you have to know all the implementation details =20 of the standard template library in order to debug it. Your global =20 constructors are called before main() and those have zeroed memory but =20 when you new() an object it doesn't have zeroed memory and you must =20 initialize every single member in the constructor and of coure you =20 can't memset(this, 0, sizeof(this)) because there's magic data in the =20 object for RTTI and virtual methods which you can't _see_ but which you =20 can trivially damage if you don't know the magic invisible =20 implementation details. ALL of C++ is magic invisible implementation details. The only way to =20 safely use the language is to know enough about it you could have =20 written the compiler and all the libraries. Otherwise, it's going to =20 break and you won't know why, although following magic "design =20 patterns" from your local cargo cult leader may help shield you from =20 the wrath of the compiler for another day, if you're lucky and turn =20 widdershins twice every tuesday before noon but after having the =20 _right_ cup of coffee while wearing lucky socks. C++ saw scripting languages and tried to ape their features =20 (Exceptions!) but doing dynamic typing at compile time is every bit as =20 stupid as doing dynamic memory management at compile time, and their =20 attempt (templates) is TURING COMPLETE AT COMPILE TIME meaning you can =20 write 10 lines of C++ that will keep the compiler busy until your hard =20 drive fills up, and detecting this is equivalent to solving the halting =20 problem. Even when it does NOT do that, a couple lines of C++ template =20 making your binary ten times larger is considered _normal_. Note: Java is also in the no man's land between C and scripting =20 languages, but it's in the foothills of scripting languages instead of =20 the foothills of C: it did dynamic memory management but _kept_ static =20 typing, then realized how dumb taht was and punched holes in its type =20 system with "interfaces", and then made code generators to spit out =20 reams of interface code and designed new tools (Eclipse!) to handle =20 multi-million line code bases for 2 year old projects made by 3 people. =20 Alas, when Y2K happened and all that Cobol needed to be rewritten Java =20 was the hot new fad (Pogs! Beanie Babies! Bitcoin!) and looked good in =20 comparison to cobol, so it's the new mainframe punchcard language. Oh =20 well. Steve Yegge eviscerated Java so I don't have to here: http://steve-yegge.blogspot.com/2007/12/codes-worst-enemy.html So back to the "new generation of system languages": C is a portable =20 assembly language. It's a category killer in that niche, the best there =20 is at what it does that's already killed off competitors like Pascal. =20 The only real survivors are derivatives of C whose main selling point =20 is that they CONTAIN THE WHOLE OF C, VERBATIM. (By that logic a mud pie =20 is a good beverage, because each mud pie contains a glass of water.) Scripting langauges (even ugly ones like Javascript, Perl, and PHP) =20 rely on opaque abstractions independent of what the hardware is doing. =20 Java is the new Cobol. Which direction is your new system language going in? > > strl* considered harmful, for 3 reasons: ... > > I'm aware some people like strl* and we have them in musl because =20 > it's > > better to provide them than to deal with people rolling their own =20 > and > > doing it in wrong and insecure ways. But I still would recommend > > against using them in new code. Toybox has xstrncpy(): If the string doesn't fit in the buffer, kill =20 the program with an error message. Rob=