From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/3141 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: Best place to discuss other lightweight libraries? Date: Mon, 22 Apr 2013 17:52:48 -0400 Message-ID: <20130422215248.GR20323@brightrain.aerifal.cx> References: <20130422145346.GP20323@brightrain.aerifal.cx> <51755575.9030809@gentoo.org> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1366667579 21818 80.91.229.3 (22 Apr 2013 21:52:59 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 22 Apr 2013 21:52:59 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-3145-gllmg-musl=m.gmane.org@lists.openwall.com Mon Apr 22 23:53:03 2013 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1UUOfW-0002lu-49 for gllmg-musl@plane.gmane.org; Mon, 22 Apr 2013 23:53:02 +0200 Original-Received: (qmail 19595 invoked by uid 550); 22 Apr 2013 21:53:00 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 19581 invoked from network); 22 Apr 2013 21:53:00 -0000 Content-Disposition: inline In-Reply-To: <51755575.9030809@gentoo.org> User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:3141 Archived-At: On Mon, Apr 22, 2013 at 05:21:25PM +0200, Luca Barbato wrote: > On 04/22/2013 04:53 PM, Rich Felker wrote: > > - Thread allergies, i.e. horribly over-complicating program logic to > > avoid threads. The best examples I can think of are the added logic > > needed to generalize a program that's reading from ordinary file > > descriptors (e.g. connection sockets) in an event loop to support > > SSL sockets or zlib-compressed streams. (Note: there are ways to > > address this kind of problem more cleanly without threads too, but > > nobody does it. I can elaborate if anybody's interested.) > > I'm interested to read about it. Well the canonical multi-threaded way to deal with such cases would be to replace your file descriptor with a pipe or socket file descriptor connected to a thread that's doing all the translation work (SSL, zlib, etc.). Then, your main event loop just sees a normal file descriptor, so you don't have to invade your code that handles streams/connections/whatever with a new abstraction framework around file descriptors that also supports SSL or whatever. Not only are such abstraction layers bloated; they also add a lot of places for bugs to hide, and they're difficult to get right. For a great example from just a few days ago, see Bitlbee bug #1046: http://bugs.bitlbee.org/bitlbee/ticket/1046 But suppose you want to solve this problem without threads. Well, the basic idea is to do the same thing, but put the translation layer in your event loop rather than in a thread. That is, make the pipe or socketpair like you would have done for the threaded solution, but instead of creating a thread, register the file descriptors with an event handler in your main event loop. For an example with SSL, when data comes in on the SSL socket, first the SSL-processing even handler gets fired, and writes the decrypted stream to a pipe or socketpair. Then, the event handler for the other end of that pipe/socket runs and handles it like an ordinary connection. Obviously this has some degree of additional overhead compared to an abstraction layer, so I wouldn't recommend doing it for applications whose JOB is to handle connections or streaming data as efficiently as possible (e.g. a webserver). But for something like an IRC client or web browser, where socket performance is irrelevant compared to other things, it's idiotic to design fancy abstraction layers for reading and writing to connections when you could just do a design like what I described above. > > - DBus. > > Sadly nobody is pushing for a better local socket multicast abstraction > to send notifications back and forth in an efficient fashion. > > I'm hoping for nanomsg once it is complete or Binder once it is > correctly documented ^^; (and thus implemented in more than few forks of > linux and maybe haiku) Yes, Binder looks really promising, but I also think part of the problem is that the need for this kind of interface is exaggerated... > > - Use of global state. Even seemingly-harmless things like a global > > registered log function are harmful, because two different libraries > > (or the main program and a library) might be trying to use the > > library with the global log destination, and clobbering each other's > > choices. > > For this there aren't solution that won't cause different problems I'm > afraid. Sure there are. I get the impression you can tell I was talking about libav/ffmpeg's log interface. :-) The obvious solution is to bind log contexts to the object you're acting on. See this nice talk: http://misko.hevery.com/2008/11/21/clean-code-talks-global-state-and-singletons/ If I remember right, part of the problem way back was that there were deep function calls that had no context available to them, and that didn't _need_ a context for anything but logging warnings or whatnot. Really, the fact that they can fail and want to be able to report failure means they _do_ need a context, but I can understand the desire to cheat, especially if there's a performance cost to passing the context all the way down. In that case, hidden non-global state, namely thread-local storage, might be an appropriate solution. It's still hidden state (and thus A Bad Thing), but at least it's no longer global state. > > - Designs based on shared libraries, especially lots of them. This > > creates bloat and often interferes with the ability to use static > > linking. > > Special mention to those that want to do clever stuff on the init > section (e.g. change a program global state from there) Did whatever lib did that (OpenAL, was it?) ever fix their bugs? > > - Dependency on any library with the above problems. :-) > > And that kills everybody using glib? *runs and hides* Yes, basically. Dependency on glib means your library will impose bloat and it will preclude robustness. Rich