From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/1749 Path: news.gmane.org!not-for-mail From: Gregor Richards Newsgroups: gmane.linux.lib.musl.general Subject: Re: Best bikeshed ever (feature test macros) Date: Fri, 24 Aug 2012 18:24:49 -0400 Message-ID: <5037FF31.1060409@purdue.edu> References: <20120824214138.GA17792@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Trace: ger.gmane.org 1345847107 16966 80.91.229.3 (24 Aug 2012 22:25:07 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 24 Aug 2012 22:25:07 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-1750-gllmg-musl=m.gmane.org@lists.openwall.com Sat Aug 25 00:25:08 2012 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1T52JP-0003No-0X for gllmg-musl@plane.gmane.org; Sat, 25 Aug 2012 00:25:07 +0200 Original-Received: (qmail 13647 invoked by uid 550); 24 Aug 2012 22:25:04 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 13639 invoked from network); 24 Aug 2012 22:25:04 -0000 User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:14.0) Gecko/20120713 Thunderbird/14.0 In-Reply-To: <20120824214138.GA17792@brightrain.aerifal.cx> X-PMX-Version: 5.5.9.388399 X-PerlMx-Virus-Scanned: Yes Xref: news.gmane.org gmane.linux.lib.musl.general:1749 Archived-At: On 08/24/2012 05:41 PM, Rich Felker wrote: > Hi all, > > Feature test macros (the fun -D_POSIX_C_SOURCE=200809L, -D_GNU_SOURCE, > etc. things everybody gets wrong) have been one of the more > controversial aspects of musl, particularly the fact that musl > presents by default a straight ISO C conforming environment with no > POSIX, traditional Unix, etc. stuff offending the pristine C > namespace, and requires the use of one or more feature test macros to > get basically _ANY_ typical unixy software to build. > > There's been some (mostly dead-end) discussion over the past few weeks > from folks who are unhappy with this situation or want it to change; I > suspect there are also some purists who want every application out > there to change and make explicit what features it depends on. > > In this thread I'd like to gauge opinions on the matter. In other > words, this is the ultimate bikeshed thread. > > To give it some direction, I'd like to start off with some pros and > cons of the different options... > > > 1. Leaving everything as it is. > > PROS: Obtaining conforming standard C environment is easy. Detecting > (for the purpose of flaming or fixing) programs failing to use feature > test macros correctly is also easy. > > CONS: Basically every program requires a feature test macro to be > added to CFLAGS in order to compile it. Using -D_GNU_SOURCE works 99% > of the time, but the other 1% of the time it will _break_ programs > that are already correctly using -D_XOPEN_SOURCE=700 or similar by > introducing nonstandard functions that pollute the namespace and > conflict with the application. Thus it becomes really hard to have a > universal working build procedure. It's also very hard to work around > broken build systems (like GCC's bootstrapping) that refuse to honor > your custom CFLAGS. > > > 2. Making the kitchen sink (_GNU_SOURCE) available by default. > > PROS: Works with most software and won't break software that's already > correctly using feature test macros. > > CONS: The preprocessor logic in the headers becomes MUCH uglier. And > purists may object to this on moral grounds. > > > 3. Making only some limited subset (e.g. POSIX base) available by > default. > > PROS: Easy to do, e.g. by adding "|| !defined(__STRICT_ANSI__)" to all > POSIX functionality #ifs. Cannot break any correct code in the default > configuration except pure ISO C code that's non-POSIX, and even then, > -std=c99 fixes it. Might cause applications to be built with less GNU > interface dependencies. > > CONS: Probably fails to get a significant portion of apps working. > > > Much like the last thread I created to assess opinion (the license > one), this is all fairly open-ended and not necessarily going to lead > to any short- or long-term change in direction, but then again it > could... Replies don't have to be of the form 1/2/3; alternative ideas > are welcome, as are replies that just address which goals/criteria are > most important to you. > > Rich My vote is for something like (3). My “ideal” would be an imaginary macro that's “every symbol ever exposed by POSIX+X/Open, plus some specific version of BSD”, so it's POSIX.2008 plus everything removed from POSIX.2001 etc and some very classical set of BSD extensions. If there were a _BSD_SOURCE=, so that _BSD_SOURCE actually meant something, I would want that, but since there's not that's a bit grotty. _XOPEN_SOURCE=700 does actually cover most things, and I think many programs would work with it. _XOPEN_SOURCE=700|_BSD_SOURCE is /very/ close to what every other libc will give you. I strongly object to (2), as even glibc does not expose GNU extensions by default. Plenty of libcs expose everything by default, but we should not strive to be even worse than glibc… I actually don't have a problem with (1) in theory, but in practice it's a pain, because you wind up in situations where it is impossible to write a program which correctly compiles on both musl and [name random other libc]. This makes musl a nightmare to port to, and for such an absurd reason. Here's why, in examples: glibc: Default is not _GNU_SOURCE, contrary to popular misconception. The default is _BSD_SOURCE|_SVID_SOURCE|_POSIX_C_SOURCE=200809L . _BSD_SOURCE|_SVID_SOURCE covers most of the things in _XOPEN_SOURCE too, so you can effectively think of it as something similar to _BSD_SOURCE|_XOPEN_SOURCE=700. If you set anything or use -std=c*, you lose everything; but if you want everything back, it's just a _GNU_SOURCE away. BSDs: Default is (approximately) everything. Again, if you specify -std=c* OR any feature test macros, you lose BSD functions (and some POSIX.2008 functions where it's not up to date). In the case of FreeBSD, you /cannot get them back/ without poking at internal defines that you're not supposed to touch. Other BSDs support _BSD_SOURCE, but FreeBSD does not. This means that if your BSD is out of date, as they all are, and you specify a POSIX version that includes things it has under _BSD_SOURCE, you're just plain screwed. So, to make things work on BSD, the safest thing to do is define nothing. Except that then it won't compile on musl. FreeBSD and other BSDs provide /broken functions/ if you specify _GNU_SOURCE, bug compatible with long-since fixed bugs, so it is almost always a bad idea to do so. Solaris: Default is everything. Otherwise, this libc is a nightmare. The -std=c* version has to correspond to the _POSIX_C_SOURCE or _XOPEN_SOURCE version (even so far as you cannot use -std=c99 with _POSIX_C_SOURCE=199506L), which means that code which is properly labeled but happens to have been compiled on a new enough compiler will break for no reason at all. Further, like FreeBSD, if you specify anything, you lose everything, and to get it back you set __EXTENSIONS__, which is arbitrary and just yet another “everything” macro. It does not recognize _BSD_SOURCE, _SVID_SOURCE, _GNU_SOURCE or any other non-standard macros, so the only way to get everything safely is to specify nothing. Oh, and here's a nice trick: It's not up to date with _XOPEN_SOURCE=700 yet, and if you set _XOPEN_SOURCE=700, IT DOESN'T RECOGNIZE IT. It doesn't check >= 600, it checks == 600. Yeesh. musl: Default is standard C only. There is /no way/ to get everything, except insofar as _GNU_SOURCE|_BSD_SOURCE happens to cover everything, until you find something that it doesn't. -std=c* have no effect. Code that is properly labeled works perfectly. So here's the issue. If I have a program that is strictly POSIX version whatever, then I can safely set _POSIX_C_SOURCE=whatever and it will do the right thing everywhere (leaving out Solaris's stupid -std=c* nonsense). If I use ANY extensions, I'm screwed. Consider Microcosm, where in one part of the source I generate bindings to the native structures by scraping them from the headers with a generator program. Naturally, since I'm actually generating system-specific bindings here, I want /everything/. Before I ported Microcosm to be hostable on musl, I simply didn't set any feature macros, and all was well; EVERY other libc gave me everything I wanted. Then I ported it to musl, and then I got it to work everywhere else again. Its feature macros look like this now: #define _POSIX_C_SOURCE 200809L #define _XOPEN_SOURCE 600 /* do not set to 700: Solaris is awful */ #define _GNU_SOURCE #define _BSD_SOURCE #define __EXTENSIONS__ 1 #define __BSD_VISIBLE 1 Note how I essentially have one line per libc. I had to change the feature macros every time I ported, THEN I had to go back and change them again. That's stupid. That's beyond stupid. Feature macros are nice in theory, but in practice they're implemented too inconsistently. They're designed to make your programs more portable, but in reality, other than _POSIX_C_SOURCE, they only serve to make your programs less portable. It's unfortunate that this is a “you have to do it this way because otherwise reality will boot you in the face” situation, but you have to do it this way. Otherwise, reality will boot you in the face. With valediction, - Gregor Richards