From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/1043 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: printf POSIX compliance Date: Sat, 9 Jun 2012 17:11:37 -0400 Message-ID: <20120609211137.GZ163@brightrain.aerifal.cx> References: <20120608144423.GN163@brightrain.aerifal.cx> <20120608145519.GP163@brightrain.aerifal.cx> <20120608150618.GB17860@port70.net> <4FD22C6C.5040704@barfooze.de> <20120608193357.40fb538d@newbook> <20120609024556.GY163@brightrain.aerifal.cx> <20120609125854.GC17860@port70.net> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: dough.gmane.org 1339276699 29110 80.91.229.3 (9 Jun 2012 21:18:19 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sat, 9 Jun 2012 21:18:19 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-1044-gllmg-musl=m.gmane.org@lists.openwall.com Sat Jun 09 23:18:17 2012 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1SdT12-0000Pd-4Y for gllmg-musl@plane.gmane.org; Sat, 09 Jun 2012 23:16:12 +0200 Original-Received: (qmail 32463 invoked by uid 550); 9 Jun 2012 21:16:11 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 32444 invoked from network); 9 Jun 2012 21:16:11 -0000 Content-Disposition: inline In-Reply-To: <20120609125854.GC17860@port70.net> User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:1043 Archived-At: On Sat, Jun 09, 2012 at 02:58:55PM +0200, Szabolcs Nagy wrote: > i don't like the gnulib approach to portability > > 1) it implements broken functions which simply must not > be used in an application with portability requirements > (eg alloca is implemented using assumptions about the > stack mechanism and invokes undefined behaviour) This is definitely a problem, perhaps the biggest problem with gnulib. Since most programs that use gnulib almost require GCC anyway, I'm not sure alloca is the best example, but it surely applies to things like freadahead. Anyway this sort of junk is at best orthogonal to goal of writing multi-platform applications. > 2) instead of sharing code between platforms they use ifdefs > and depend on internals of the implementation > (eg. broken gnu stdio functions: gnulib will never work on > a new platform out of the box, you have to port it whenever > there is a new libc or even a new version of a libc) Actually the legacy unices shared a surprising amount of stdio internals, largely because of broken code like gnulib that poked at the internals... > 3) they duplicate a lot of the libc writing effort Are you sure? I think most of the libc-like code is copied from (and shared with) glibc... > and provide bad quality code > (eg. math functions: a mathematical software will only be > portable if libm is high quality so if portability matters > you need a libm written in portable c code (and maybe even > softfloat emulation) I haven't looked at their math functions, but if they really are that broken, it brings up a major point you missed: The gnulib configure tests are not just extremely pedantic in judging a system as non-conformant, but _selectively_ pedantic. That is, they test every single corner case that glibc went to the trouble to get right, but ignore the (equally many) cases glibc got horrible wrong. And sometimes they even test for non-conformant, glibc-specific behavior... How does this relate to math? Well, it seems they check for all sorts of obscure breakage in the host libm, but then replace it with their own broken math code... > in my opinion the more correct approaches to portability are > > 1) have a high quality libc (like musl) and port it > to all the required targets (eg by emulating syscalls) I don't think enforcing implementation homogeneity is a viable solution for applications. > 2) write the application with portability in mind: instead > of using posix api use a 'libportability' api that covers > enough areas so the application can work and simple enough > so it can be implemented on any target (so eg. it has no > alloca and freadahead in it, and only has limited form of > networking, process or file management etc) This approach is not just wrong but extremely harmful. It's the approach of APR, NSPR, and basically any library whose name ends in "PR" (portable runtime). I've not read the Apache code, but I've read SVN's code and its use of APR, and the level of idiocy it involves is astounding. The equivalent of a simple printf call ends up as this giant chain of allocations and concatenations building strings to send to the output system. I seriously doubt there's any way a program that uses APR (Apache itself included) could avoid crashing under OOM conditions. Surely there's no fundamental reason it _has_ to be this way. After all, someone making a portable system interface library could end up making it just like POSIX. The problem is that the task attracts people with the wrong traits: particularly, obsessiveness with designing something absolutely general and no intuition for the actual necessary usage cases. More fundamentally (i.e. even if it didn't attract the wrong people) the approach is wrong for the following two reasons: 1. It penalizes correct systems by putting an added layer of bloat between the application and the underlying correct POSIX implementation. 2. It makes it impossible to reuse code that was written to C99/POSIX in your application without "porting" the code to your new portability library, and conversely, impossible to reuse code written to your portability in projects that want to be light-weight and pure-C-only or POSIX-only (or else forces them to pull in an ugly library they don't want and that might have portability issues by virtue of not being ported to some systems they want/need to support). Ultimately, gnulib does actually have the right basic approach; they just botched a lot of the details. The correct approach from an application side (where you can't just fix the whole broken OS) is to write your code to standard C/POSIX assuming it all works (possibly avoiding making unnecessary assumptions for interfaces that are "hard to fix"), and include patch-up code for a known finite set of broken systems you want to support. Today, that set would probably be just "Windows, Android, and OSX/iOS". This minimizes (to zero) the cost of portability on clean, non-broken systems and avoids doing anything that could cause your program to break on such systems. It also keeps the cost reasonably low (much lower than the *PR solutions) even on broken systems. And it works well in practice. This is not something I just made up; it's a policy I pushed during the entire time I was working on MPlayer, which usually took the form of "Keep the hacks to fix broken stuff as close to the broken component as possible, and avoid polluting code that has nothing to do with the broken stuff." The main differences from gnulib are: - Don't try to "fix" any system except the known-broken ones. - Avoid unnecessary fixes whose costs are disproportionately large compared to their benefits. - Don't add all sorts of unnecessary functions that have nothing to do with portability. I don't expect to "revolutionize" gnulib, but since the right (in my opinion) approach is reasonably similar to gnulib's basic idea, I think we could influence the evolution of gnulib in a more positive direction. Some things that might be nice to see... - Clear distinction in gnulib between "broken libc replacement" stuff and "convenience utility" stuff where the latter is implemented portably without any #ifdeffery. - Automatically-generated option to configure that inhibits all gnulib tests and replacement functions and forcibly assumes the underlying system is C and POSIX conformant. - And of course cleanup/fixing of existing known bugs and portability issues (and thread-safety issues). Rich