From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/637 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: [PATCH 0/4] Fix function definitions. Date: Fri, 9 Mar 2012 11:38:53 -0500 Message-ID: <20120309163853.GF184@brightrain.aerifal.cx> References: <1331280854-9080-1-git-send-email-gf@unixsol.org> <4F59BD70.9020105@unixsol.org> <20120309083358.GA184@brightrain.aerifal.cx> <4F59CA50.5060407@unixsol.org> <4F5A1D89.9090304@landley.net> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: dough.gmane.org 1331311158 28351 80.91.229.3 (9 Mar 2012 16:39:18 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Fri, 9 Mar 2012 16:39:18 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-638-gllmg-musl=m.gmane.org@lists.openwall.com Fri Mar 09 17:39:15 2012 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1S62qY-0002Jr-Gy for gllmg-musl@plane.gmane.org; Fri, 09 Mar 2012 17:39:14 +0100 Original-Received: (qmail 14051 invoked by uid 550); 9 Mar 2012 16:39:12 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 14034 invoked from network); 9 Mar 2012 16:39:10 -0000 Content-Disposition: inline In-Reply-To: <4F5A1D89.9090304@landley.net> User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:637 Archived-At: On Fri, Mar 09, 2012 at 09:11:05AM -0600, Rob Landley wrote: > Why are you doing feature test macros at _all_ in a 2012 Linux libc? If > you don't #define any of them (which most code doesn't), you get (from > /usr/lib/features.h): features.h is part of glibc, and not a part I like, for several reasons: 1. Each additional nested header increases base compile time significantly, and makes a big difference for small translation units that only includes one header themselves, due to extra path search, open, read, parse, close, etc. 2. The way glibc does features.h, with _xxx_SOURCE macros as the input and __USE_xxx macros as the output, obfuscates in the actual headers which macros are necessary or sufficient to get the desired interface declared. This makes them less useful as documentation, and encourages programmers both to define extraneous/unwanted feature test macros due to lack of awareness that one implies another, and (much worse) to actually #define __USE_POSIX, etc. in their own sources rather than defining the correct feature test macro. 3. Others I'm forgetting at the moment.. :-) > /* If nothing (other than _GNU_SOURCE) is defined, > define _BSD_SOURCE and _SVID_SOURCE. */ > #if (!defined __STRICT_ANSI__ && !defined _ISOC99_SOURCE && \ > !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE && \ > !defined _XOPEN_SOURCE && !defined _XOPEN_SOURCE_EXTENDED && \ > !defined _BSD_SOURCE && !defined _SVID_SOURCE) > # define _BSD_SOURCE 1 > # define _SVID_SOURCE 1 > #endif > > and then a bit later... > > /* If none of the ANSI/POSIX macros are defined, use POSIX.1 and POSIX.2 > (and IEEE Std 1003.1b-1993 unless _XOPEN_SOURCE is defined). */ > #if ((!defined __STRICT_ANSI__ || (_XOPEN_SOURCE - 0) >= 500) && \ > !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE) > # define _POSIX_SOURCE 1 > # if defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 500 > # define _POSIX_C_SOURCE 2 > # elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 600 > # define _POSIX_C_SOURCE 199506L > # elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 700 > # define _POSIX_C_SOURCE 200112L > # else > # define _POSIX_C_SOURCE 200809L > # endif > # define __USE_POSIX_IMPLICITLY 1 > #endif > > I.E. you get _BSD_SOURCE, _SVID_SOURCE, _POSIX_SOURCE, and > _POSIX_C_SOURCE=200809L all #defined _FOR_ you if you just avoid > mentioning any of these macros ever. Relying on that is not portable, and musl does not do it. On Linux/glibc you get the kitchen sink when you define nothing. On most other unices, you get an intentionally-broken legacy environment that's not conformant to modern standards. All projects should explicitly define the feature test macros they want. This also assists you in profiling the portability of your code (switching out _GNU_SOURCE and you see exactly what extensions you're using). Thankfully for ones that fail to do this, you can just add the appropriate -D's to CPPFLAGS. There's been some discussion of the "kitchen sink by default" approach in musl, and I'm not entirely opposed to it, but the main issue is that there's no portable way to select "plain ISO C" with a feature test macro. There are glibc/gcc ways to request specific versions of ISO C with feature test macros, but they took the silly approach of renaming the macro for each version rather than something like #define _ISO_C_SOURCE 199901L which would have been much cleaner... Also note that glibc's version of "kitchen sink by default" is broken. Several functions like strerror_r, basename, and perhaps still scanf get *broken* GNU versions by default unless you specify an appropriate feature test macro. So if you want to use them, you're stuck; you have to use _POSIX_C_SOURCE or _XOPEN_SOURCE and then define some additional macros to try to get back what you lost. And I'm not even sure it's possible without breaking the standard functions again... > And that's exactly what my code did before Georgi tried to build it > against musl and suddenly that didn't work anymore because musl is weird. > > (A sane development team would have implemented _negative_ macros, ala > _ANSI_ONLY, to switch stuff _off_.) There are very good reasons why this was not done; you could ask the committees if you care. > >> At present musl makes no attempt to support(*) the _BSD_SOURCE or > >> _SVID_SOURCE feature test macros; every nonstandard (non-POSIX) > >> extention offered by glibc is grouped together under _GNU_SOURCE, and > >> this works for musl because (unlike with glibc), musl's _GNU_SOURCE > >> only enables features; it does not alter standard interfaces like > >> strerror_r or basename to be broken GNU versions of the functions. > > A lot of those "nonstandard" things were in BSD circa 1982, before the > FSF even existed. I agree. But most of them were omitted from the standards for good reasons, and even many of the ones that were included (like bcopy) were redundant. Anyway, the reason I've not focused on adding _BSD_SOURCE is mainly lack of necessity. I don't think I've encountered a single package that was defining _BSD_SOURCE and failed to build because of things it expected to get from _BSD_SOURCE. If there are major packages that could be made easier to build by adding it, I'd be a lot more interested in doing so. > I don't care about granularity, I just want my code to work. But I > refuse to #define _ALL_HAIL_RICHARD_STALLMAN as a condition of using > standard Linux functions. > > I will not say _GNU_SOURCE when implementing a project that has nothing > whatsoever to do with the FSF's GNU project, and acts to replace large > chunks of its' userspace. OK, so basically you want alternate macros to request the kitchen sink to avoid writing the word "GNU" anywhere in your project. :-) All joking aside, at least this is the first *reason* I've seen for possible changes in this area, so it's worth considering. > >> I think there's a good (nontrivial) discussion to be had about whether > >> it's worthwhile to have the _BSD_SOURCE and _SVID_SOURCE feature test > >> macros supported in musl. > > Given that the "nothing is #defined" behavior gives you most of this > anyway on both glibc and uClibc, why musl is bothering with it at _all_ > is kinda inexplicable to me. Because unlike uClibc, musl does not just copy all the mess from glibc. I like the uClibc project and the people involved, but I don't like the fact that basically everything but the internal implementation details of stdio and locale are nearly direct copies from glibc. > Yes, I'm aware that Posix 2008 mentions feature test macros: > > http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_02 > > And silently ignoring them is reasonably compliant behavior. (You can > even be technically compliant with Item 3 under _POSIX_C_SOURCE if you > consider __linux__ a feature test macro. It's got the double underscore!) As long as you don't define additional things outside the reserved namespace, it's compliant to ignore them. I agree totally. Of course then your implementation complies only to POSIX, but not to ISO C. For instance, this strictly conforming program cannot be compiled (intentionally detailed to make it plausible): #include #include int debugging = 1; int dprintf(const char *fmt, ...) { int ret; if (debugging) return 0; va_list ap = va_start(fmt); ret = vprintf(fmt, ap); va_end(ap); return ret; } int main() { dprintf("hello, world\n"); } The same issue happens with conforming POSIX programs if you add extensions outside the reserved namespaces. > Which shows you all the compiler builtin #defines. See __linux__ in > that list? If you _have_ to test for something, what's wrong with > testing for __linux__? This is not POSIX conformant. POSIX requires invoking the compiler with -D_POSIX_C_SOURCE=200809L to respect the namespace. With your suggestion, it would also be necessary to add -U__linux__ to the command line. In other words, your proposed change would break all correct programs that are using feature test macros properly, for the sake of avoiding adding a -D option to the CPPFLAGS of programs that aren't doing it right. > >>> Should the headers be filled with feature checks (that would make them > >>> quite ugly) or assume we have _GNU_SOURCE defined and remove any > >>> _GNU_SOURCE > >>> checks? > >> > >> I'm confused what you mean by "assume we have _GNU_SOURCE defined". > > > > Well, ignore my suggestion. I was thinking of removing most _GNU_SOURCE > > checks in headers (except for pure GNU extensions) but reading what you > > wrote above I remembered what PITA feature_macros were. Unfortunately > > it seems that defining _GNU_SOURCE to get the kitchen sink is easier... > > It's also wrong. And it means that Musl can't build code that not just > glibc and uClibc can build out of the box, but BSD can too. (chroot() > was _not_ invented by the gnu/dammit project.) It's not my intent to imply that any of these features were invented by the GNU project. Rather, _GNU_SOURCE is simply the only existing feature test macro for "kitchen sink" that exists on Linux systems. (Blame glibc for naming it this rather than _LINUX_SOURCE!) As such, existing real-world software that's conforming to the standards and using _POSIX_C_SOURCE or _XOPEN_SOURCE is already testing for the availability of _GNU_SOURCE and adding that when it wants the kitchen sink. While using a different name would be *cosmetically* better, it would simply have the effect of making you add -D_BETTER_NAME_HERE to all apps' CPPFLAGS since they'd no longer be able to find a working option themselves... > The feature test macros actually means that musl goes out of its _way_ > not to build them. You have extra code that's just there to break stuff. One could also say that not having the internals of FILE defined in stdio.h, and not using the same struct member names as glibc, is "going out of its way" to break stuff... In any case, the feature test macro issue is easily solved, if you always want full extensions, by just using "CC=gcc -D_GNU_SOURCE". If there's demand for a way to do this without ever having to write that dirty word "GNU", we could consider adding an alias.. Rich