[PATCH 0/4] Fix function definitions.

[PATCH 0/4] Fix function definitions.

[Georgi Chorbadzhiyski]

Define several functions if _BSD_SOURCE is set. _XOPEN_SOURCE
should also be checked but I can't be bothered, because _BSD_SOURCE
works.

Background for these patches:
  http://lists.landley.net/pipermail/toybox-landley.net/2012-March/000309.html

Georgi Chorbadzhiyski (4):
  include/stdlib.h: Define clearenv() if (_SVID_SOURCE || _BSD_SOURCE)
    is set.
  include/unistd.h: Define sethostname() if _BSD_SOURCE is set.
  include/unistd.h: Define vfork() if _BSD_SOURCE is set.
  include/sys/types.h: major(), minor() and makedev() should be defined
    if _BSD_SOURCE is set.

 include/stdlib.h    |    5 ++++-
 include/sys/types.h |    5 ++++-
 include/unistd.h    |   10 ++++++++--
 3 files changed, 16 insertions(+), 4 deletions(-)

-- 
1.7.5.1

[PATCH 1/4] include/stdlib.h: Define clearenv() if (_SVID_SOURCE || _BSD_SOURCE) is set.

[Georgi Chorbadzhiyski]

According to clearenv(3) man page, the function should be defined
if (_SVID_SOURCE || _BSD_SOURCE) is set.
---
 include/stdlib.h |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/include/stdlib.h b/include/stdlib.h
index 9c8a118..441eef6 100644
--- a/include/stdlib.h
+++ b/include/stdlib.h
@@ -127,12 +127,15 @@ unsigned short *seed48 (unsigned short [3]);
 void lcong48 (unsigned short [7]);
 #endif
 
+#if defined(_GNU_SOURCE) || defined(_SVID_SOURCE) || defined(_BSD_SOURCE)
+int clearenv(void);
+#endif
+
 #if defined(_GNU_SOURCE)
 void *alloca(size_t);
 char *mktemp (char *);
 void *valloc (size_t);
 void *memalign(size_t, size_t);
-int clearenv(void);
 int ptsname_r(int, char *, size_t);
 char *ecvt(double, int, int *, int *);
 char *fcvt(double, int, int *, int *);
-- 
1.7.5.1

[PATCH 2/4] include/unistd.h: Define sethostname() if _BSD_SOURCE is set.

[Georgi Chorbadzhiyski]

According to sethostname(2) man page, the function should be defined
if _BSD_SOURCE is set.
---
 include/unistd.h |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/include/unistd.h b/include/unistd.h
index 7662e51..3fda5aa 100644
--- a/include/unistd.h
+++ b/include/unistd.h
@@ -144,6 +144,10 @@ void sync(void);
 int getdtablesize(void);
 #endif
 
+#if defined(_GNU_SOURCE) || defined(_BSD_SOURCE)
+int sethostname(const char *, size_t);
+#endif
+
 #ifdef _GNU_SOURCE
 int brk(void *);
 void *sbrk(intptr_t);
@@ -152,7 +156,6 @@ pid_t vfork(void);
 int vhangup(void);
 int chroot(const char *);
 int getpagesize(void);
-int sethostname(const char *, size_t);
 int usleep(unsigned);
 unsigned ualarm(unsigned, unsigned);
 int setgroups(size_t, const gid_t []);
-- 
1.7.5.1

[PATCH 3/4] include/unistd.h: Define vfork() if _BSD_SOURCE is set.

[Georgi Chorbadzhiyski]

According to vfork(2) man page, the function should be defined
if _BSD_SOURCE is set.
---
 include/unistd.h |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/include/unistd.h b/include/unistd.h
index 3fda5aa..083eae4 100644
--- a/include/unistd.h
+++ b/include/unistd.h
@@ -152,7 +152,6 @@ int sethostname(const char *, size_t);
 int brk(void *);
 void *sbrk(intptr_t);
 pid_t forkall(void);
-pid_t vfork(void);
 int vhangup(void);
 int chroot(const char *);
 int getpagesize(void);
@@ -164,6 +163,10 @@ int setresgid(gid_t, gid_t, gid_t);
 char *get_current_dir_name(void);
 #endif
 
+#if defined(_GNU_SOURCE) || defined(_BSD_SOURCE)
+pid_t vfork(void);
+#endif
+
 #define _XOPEN_VERSION          700
 #define _XOPEN_UNIX             1
 #define _XOPEN_ENH_I18N         1
-- 
1.7.5.1

[PATCH 4/4] include/sys/types.h: major(), minor() and makedev() should be defined if _BSD_SOURCE is set.

[Georgi Chorbadzhiyski]

---
 include/sys/types.h |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/include/sys/types.h b/include/sys/types.h
index 18433f8..cef38d1 100644
--- a/include/sys/types.h
+++ b/include/sys/types.h
@@ -58,6 +58,10 @@ typedef __uint16_t u_int16_t;
 typedef __uint32_t u_int32_t;
 typedef __uint64_t u_int64_t;
 
+#ifdef _BSD_SOURCE
+#include <sys/sysmacros.h>
+#endif
+
 #ifdef _GNU_SOURCE
 typedef char *caddr_t;
 typedef unsigned char u_char;
@@ -69,7 +73,6 @@ typedef unsigned long long u_quad_t;
 typedef long register_t;
 #include <endian.h>
 #include <sys/select.h>
-#include <sys/sysmacros.h>
 #endif
 
 #ifdef __cplusplus
-- 
1.7.5.1

Re: [PATCH 0/4] Fix function definitions.

[Georgi Chorbadzhiyski]

On 3/9/12 10:14 AM, Georgi Chorbadzhiyski wrote:
> Define several functions if _BSD_SOURCE is set. _XOPEN_SOURCE
> should also be checked but I can't be bothered, because _BSD_SOURCE
> works.
>
> Background for these patches:
>    http://lists.landley.net/pipermail/toybox-landley.net/2012-March/000309.html
>
> Georgi Chorbadzhiyski (4):
>    include/stdlib.h: Define clearenv() if (_SVID_SOURCE || _BSD_SOURCE)
>      is set.
>    include/unistd.h: Define sethostname() if _BSD_SOURCE is set.
>    include/unistd.h: Define vfork() if _BSD_SOURCE is set.
>    include/sys/types.h: major(), minor() and makedev() should be defined
>      if _BSD_SOURCE is set.
>
>   include/stdlib.h    |    5 ++++-
>   include/sys/types.h |    5 ++++-
>   include/unistd.h    |   10 ++++++++--
>   3 files changed, 16 insertions(+), 4 deletions(-)
>

Hmm...it seems this is not enough. See include/unistd.h

#ifdef _GNU_SOURCE
int brk(void *);
void *sbrk(intptr_t);
pid_t forkall(void);
int vhangup(void);
int getpagesize(void);
int usleep(unsigned);
unsigned ualarm(unsigned, unsigned);
int setgroups(size_t, const gid_t []);
int setresuid(uid_t, uid_t, uid_t);
int setresgid(gid_t, gid_t, gid_t);
char *get_current_dir_name(void);
#endif

Most of the above according to their man pages should be defined if
_BSD_SOURCE is set.

Should the headers be filled with feature checks (that would make them
quite ugly) or assume we have _GNU_SOURCE defined and remove any _GNU_SOURCE
checks?

-- 
Georgi Chorbadzhiyski
http://georgi.unixsol.org/

Re: [PATCH 0/4] Fix function definitions.

[Georgi Chorbadzhiyski]

On 3/9/12 10:21 AM, Georgi Chorbadzhiyski wrote:
> On 3/9/12 10:14 AM, Georgi Chorbadzhiyski wrote:
>> Define several functions if _BSD_SOURCE is set. _XOPEN_SOURCE
>> should also be checked but I can't be bothered, because _BSD_SOURCE
>> works.
>>
>> Background for these patches:
>> http://lists.landley.net/pipermail/toybox-landley.net/2012-March/000309.html
>>
>> Georgi Chorbadzhiyski (4):
>> include/stdlib.h: Define clearenv() if (_SVID_SOURCE || _BSD_SOURCE)
>> is set.
>> include/unistd.h: Define sethostname() if _BSD_SOURCE is set.
>> include/unistd.h: Define vfork() if _BSD_SOURCE is set.
>> include/sys/types.h: major(), minor() and makedev() should be defined
>> if _BSD_SOURCE is set.
>>
>> include/stdlib.h | 5 ++++-
>> include/sys/types.h | 5 ++++-
>> include/unistd.h | 10 ++++++++--
>> 3 files changed, 16 insertions(+), 4 deletions(-)
>>
>
> Hmm...it seems this is not enough. See include/unistd.h
>
> #ifdef _GNU_SOURCE
> int brk(void *);
> void *sbrk(intptr_t);
> pid_t forkall(void);
> int vhangup(void);
> int getpagesize(void);
> int usleep(unsigned);
> unsigned ualarm(unsigned, unsigned);
> int setgroups(size_t, const gid_t []);
> int setresuid(uid_t, uid_t, uid_t);
> int setresgid(gid_t, gid_t, gid_t);
> char *get_current_dir_name(void);
> #endif
>
> Most of the above according to their man pages should be defined if
> _BSD_SOURCE is set.
>
> Should the headers be filled with feature checks (that would make them
> quite ugly) or assume we have _GNU_SOURCE defined and remove any _GNU_SOURCE
> checks?

...for functions that are not pure GNU extensions like strndupa.

-- 
Georgi Chorbadzhiyski
http://georgi.unixsol.org/

Re: [PATCH 0/4] Fix function definitions.

[Rich Felker]

On Fri, Mar 09, 2012 at 10:21:04AM +0200, Georgi Chorbadzhiyski wrote:
> Subject: Re: [musl] [PATCH 0/4] Fix function definitions.

I think you mean declarations. :-)

> Hmm...it seems this is not enough. See include/unistd.h
> 
> #ifdef _GNU_SOURCE
> int brk(void *);
> void *sbrk(intptr_t);
> pid_t forkall(void);
> int vhangup(void);
> int getpagesize(void);
> int usleep(unsigned);
> unsigned ualarm(unsigned, unsigned);
> int setgroups(size_t, const gid_t []);
> int setresuid(uid_t, uid_t, uid_t);
> int setresgid(gid_t, gid_t, gid_t);
> char *get_current_dir_name(void);
> #endif
> 
> Most of the above according to their man pages should be defined if
> _BSD_SOURCE is set.

At present musl makes no attempt to support(*) the _BSD_SOURCE or
_SVID_SOURCE feature test macros; every nonstandard (non-POSIX)
extention offered by glibc is grouped together under _GNU_SOURCE, and
this works for musl because (unlike with glibc), musl's _GNU_SOURCE
only enables features; it does not alter standard interfaces like
strerror_r or basename to be broken GNU versions of the functions.

I think there's a good (nontrivial) discussion to be had about whether
it's worthwhile to have the _BSD_SOURCE and _SVID_SOURCE feature test
macros supported in musl. The main benefit I can see is that
applications which define _BSD_SOURCE or _SVID_SOURCE in their default
CFLAGS for the sake of getting certain important traditional
interfaces (like MAP_ANONYMOUS) but without bringing in broken GNU
behavior could perhaps be made to work out-of-the-box on musl without
manually adding -D_GNU_SOURCE to the CFLAGS. On the other hand, since
musl's _GNU_SOURCE is "non-destructive", I'm not sure that's a huge
benefit.


(*) _BSD_SOURCE appears in some places in musl's headers now, but that
was due to a (rather ill-thought-out) attempt to add some BSD
functions like strlcpy which glibc refuses to support, and avoid
making them visible when just _GNU_SOURCE is used. I'm largely
convinced this approach was a mistake, but how it should work is still
a topic for discussion...

> Should the headers be filled with feature checks (that would make them
> quite ugly) or assume we have _GNU_SOURCE defined and remove any _GNU_SOURCE
> checks?

I'm confused what you mean by "assume we have _GNU_SOURCE defined".

Rich

Re: [PATCH 0/4] Fix function definitions.

[Georgi Chorbadzhiyski]

On 3/9/12 10:33 AM, Rich Felker wrote:
> On Fri, Mar 09, 2012 at 10:21:04AM +0200, Georgi Chorbadzhiyski wrote:
>> Subject: Re: [musl] [PATCH 0/4] Fix function definitions.
>
> I think you mean declarations. :-)
>
>> Hmm...it seems this is not enough. See include/unistd.h
>>
>> #ifdef _GNU_SOURCE
>> int brk(void *);
>> void *sbrk(intptr_t);
>> pid_t forkall(void);
>> int vhangup(void);
>> int getpagesize(void);
>> int usleep(unsigned);
>> unsigned ualarm(unsigned, unsigned);
>> int setgroups(size_t, const gid_t []);
>> int setresuid(uid_t, uid_t, uid_t);
>> int setresgid(gid_t, gid_t, gid_t);
>> char *get_current_dir_name(void);
>> #endif
>>
>> Most of the above according to their man pages should be defined if
>> _BSD_SOURCE is set.
>
> At present musl makes no attempt to support(*) the _BSD_SOURCE or
> _SVID_SOURCE feature test macros; every nonstandard (non-POSIX)
> extention offered by glibc is grouped together under _GNU_SOURCE, and
> this works for musl because (unlike with glibc), musl's _GNU_SOURCE
> only enables features; it does not alter standard interfaces like
> strerror_r or basename to be broken GNU versions of the functions.
>
> I think there's a good (nontrivial) discussion to be had about whether
> it's worthwhile to have the _BSD_SOURCE and _SVID_SOURCE feature test
> macros supported in musl. The main benefit I can see is that
> applications which define _BSD_SOURCE or _SVID_SOURCE in their default
> CFLAGS for the sake of getting certain important traditional
> interfaces (like MAP_ANONYMOUS) but without bringing in broken GNU
> behavior could perhaps be made to work out-of-the-box on musl without
> manually adding -D_GNU_SOURCE to the CFLAGS. On the other hand, since
> musl's _GNU_SOURCE is "non-destructive", I'm not sure that's a huge
> benefit.
>
> (*) _BSD_SOURCE appears in some places in musl's headers now, but that
> was due to a (rather ill-thought-out) attempt to add some BSD
> functions like strlcpy which glibc refuses to support, and avoid
> making them visible when just _GNU_SOURCE is used. I'm largely
> convinced this approach was a mistake, but how it should work is still
> a topic for discussion...

Umm, like I expected all those declarations open a big can of worms :(

>> Should the headers be filled with feature checks (that would make them
>> quite ugly) or assume we have _GNU_SOURCE defined and remove any _GNU_SOURCE
>> checks?
>
> I'm confused what you mean by "assume we have _GNU_SOURCE defined".

Well, ignore my suggestion. I was thinking of removing most _GNU_SOURCE
checks in headers (except for pure GNU extensions) but reading what you
wrote above I remembered what PITA feature_macros were. Unfortunately
it seems that defining _GNU_SOURCE to get the kitchen sink is easier...

-- 
Georgi Chorbadzhiyski
http://georgi.unixsol.org/

Re: [PATCH 0/4] Fix function definitions.

[Rob Landley]

On 03/09/2012 03:16 AM, Georgi Chorbadzhiyski wrote:
> On 3/9/12 10:33 AM, Rich Felker wrote:
>> On Fri, Mar 09, 2012 at 10:21:04AM +0200, Georgi Chorbadzhiyski wrote:
>>> Subject: Re: [musl] [PATCH 0/4] Fix function definitions.
>>
>> I think you mean declarations. :-)
>>
>>> Hmm...it seems this is not enough. See include/unistd.h
>>>
>>> #ifdef _GNU_SOURCE
>>> int brk(void *);
>>> void *sbrk(intptr_t);
>>> pid_t forkall(void);
>>> int vhangup(void);
>>> int getpagesize(void);
>>> int usleep(unsigned);
>>> unsigned ualarm(unsigned, unsigned);
>>> int setgroups(size_t, const gid_t []);
>>> int setresuid(uid_t, uid_t, uid_t);
>>> int setresgid(gid_t, gid_t, gid_t);
>>> char *get_current_dir_name(void);
>>> #endif

Why are you doing feature test macros at _all_ in a 2012 Linux libc?  If
you don't #define any of them (which most code doesn't), you get (from
/usr/lib/features.h):

/* If nothing (other than _GNU_SOURCE) is defined,
   define _BSD_SOURCE and _SVID_SOURCE.  */
#if (!defined __STRICT_ANSI__ && !defined _ISOC99_SOURCE && \
     !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE && \
     !defined _XOPEN_SOURCE && !defined _XOPEN_SOURCE_EXTENDED && \
     !defined _BSD_SOURCE && !defined _SVID_SOURCE)
# define _BSD_SOURCE    1
# define _SVID_SOURCE   1
#endif

and then a bit later...

/* If none of the ANSI/POSIX macros are defined, use POSIX.1 and POSIX.2
   (and IEEE Std 1003.1b-1993 unless _XOPEN_SOURCE is defined).  */
#if ((!defined __STRICT_ANSI__ || (_XOPEN_SOURCE - 0) >= 500) && \
     !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE)
# define _POSIX_SOURCE  1
# if defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 500
#  define _POSIX_C_SOURCE       2
# elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 600
#  define _POSIX_C_SOURCE       199506L
# elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 700
#  define _POSIX_C_SOURCE       200112L
# else
#  define _POSIX_C_SOURCE       200809L
# endif
# define __USE_POSIX_IMPLICITLY 1
#endif

I.E. you get _BSD_SOURCE, _SVID_SOURCE, _POSIX_SOURCE, and
_POSIX_C_SOURCE=200809L all #defined _FOR_ you if you just avoid
mentioning any of these macros ever.

And that's exactly what my code did before Georgi tried to build it
against musl and suddenly that didn't work anymore because musl is weird.

(A sane development team would have implemented _negative_ macros, ala
_ANSI_ONLY, to switch stuff _off_.)

>>> Most of the above according to their man pages should be defined if
>>> _BSD_SOURCE is set.
>>
>> At present musl makes no attempt to support(*) the _BSD_SOURCE or
>> _SVID_SOURCE feature test macros; every nonstandard (non-POSIX)
>> extention offered by glibc is grouped together under _GNU_SOURCE, and
>> this works for musl because (unlike with glibc), musl's _GNU_SOURCE
>> only enables features; it does not alter standard interfaces like
>> strerror_r or basename to be broken GNU versions of the functions.

A lot of those "nonstandard" things were in BSD circa 1982, before the
FSF even existed.

I do not write code for Gnu Hurd.  I write code for Linux.  Saying
"GNU/Linux" is an obvious oxymoron because the licenses aren't even
compatible: GNU is GPLv3 (only), Linux is GPLv2 (only), they can't share
code with each other.

I don't care about granularity, I just want my code to work.  But I
refuse to #define _ALL_HAIL_RICHARD_STALLMAN as a condition of using
standard Linux functions.

I will not say _GNU_SOURCE when implementing a project that has nothing
whatsoever to do with the FSF's GNU project, and acts to replace large
chunks of its' userspace.

>> I think there's a good (nontrivial) discussion to be had about whether
>> it's worthwhile to have the _BSD_SOURCE and _SVID_SOURCE feature test
>> macros supported in musl.

Given that the "nothing is #defined" behavior gives you most of this
anyway on both glibc and uClibc, why musl is bothering with it at _all_
is kinda inexplicable to me.

Yes, I'm aware that Posix 2008 mentions feature test macros:

http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_02

And silently ignoring them is reasonably compliant behavior.  (You can
even be technically compliant with Item 3 under _POSIX_C_SOURCE if you
consider __linux__ a feature test macro.  It's got the double underscore!)

>> The main benefit I can see is that
>> applications which define _BSD_SOURCE or _SVID_SOURCE in their default
>> CFLAGS for the sake of getting certain important traditional
>> interfaces (like MAP_ANONYMOUS) but without bringing in broken GNU
>> behavior could perhaps be made to work out-of-the-box on musl without
>> manually adding -D_GNU_SOURCE to the CFLAGS. On the other hand, since
>> musl's _GNU_SOURCE is "non-destructive", I'm not sure that's a huge
>> benefit.

It's not non-destructive to a program that didn't #define _any_ of this
stuff.  It requires you to #define stuff other libc's don't to get what
should be default behavior.

>> (*) _BSD_SOURCE appears in some places in musl's headers now, but that
>> was due to a (rather ill-thought-out) attempt to add some BSD
>> functions like strlcpy which glibc refuses to support, and avoid
>> making them visible when just _GNU_SOURCE is used. I'm largely
>> convinced this approach was a mistake, but how it should work is still
>> a topic for discussion...
> 
> Umm, like I expected all those declarations open a big can of worms :(

Allow me to introduce you to:

  cc -E -dM - < /dev/null

Which shows you all the compiler builtin #defines.  See __linux__ in
that list?  If you _have_ to test for something, what's wrong with
testing for __linux__?

>>> Should the headers be filled with feature checks (that would make them
>>> quite ugly) or assume we have _GNU_SOURCE defined and remove any
>>> _GNU_SOURCE
>>> checks?
>>
>> I'm confused what you mean by "assume we have _GNU_SOURCE defined".
> 
> Well, ignore my suggestion. I was thinking of removing most _GNU_SOURCE
> checks in headers (except for pure GNU extensions) but reading what you
> wrote above I remembered what PITA feature_macros were. Unfortunately
> it seems that defining _GNU_SOURCE to get the kitchen sink is easier...

It's also wrong.  And it means that Musl can't build code that not just
glibc and uClibc can build out of the box, but BSD can too.  (chroot()
was _not_ invented by the gnu/dammit project.)

The feature test macros actually means that musl goes out of its _way_
not to build them. You have extra code that's just there to break stuff.

Rob

Re: [PATCH 0/4] Fix function definitions.

[Rich Felker]

On Fri, Mar 09, 2012 at 09:11:05AM -0600, Rob Landley wrote:
> Why are you doing feature test macros at _all_ in a 2012 Linux libc?  If
> you don't #define any of them (which most code doesn't), you get (from
> /usr/lib/features.h):

features.h is part of glibc, and not a part I like, for several
reasons:

1. Each additional nested header increases base compile time
significantly, and makes a big difference for small translation units
that only includes one header themselves, due to extra path search,
open, read, parse, close, etc.

2. The way glibc does features.h, with _xxx_SOURCE macros as the input
and __USE_xxx macros as the output, obfuscates in the actual headers
which macros are necessary or sufficient to get the desired interface
declared. This makes them less useful as documentation, and encourages
programmers both to define extraneous/unwanted feature test macros due
to lack of awareness that one implies another, and (much worse) to
actually #define __USE_POSIX, etc. in their own sources rather than
defining the correct feature test macro.

3. Others I'm forgetting at the moment.. :-)

> /* If nothing (other than _GNU_SOURCE) is defined,
>    define _BSD_SOURCE and _SVID_SOURCE.  */
> #if (!defined __STRICT_ANSI__ && !defined _ISOC99_SOURCE && \
>      !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE && \
>      !defined _XOPEN_SOURCE && !defined _XOPEN_SOURCE_EXTENDED && \
>      !defined _BSD_SOURCE && !defined _SVID_SOURCE)
> # define _BSD_SOURCE    1
> # define _SVID_SOURCE   1
> #endif
> 
> and then a bit later...
> 
> /* If none of the ANSI/POSIX macros are defined, use POSIX.1 and POSIX.2
>    (and IEEE Std 1003.1b-1993 unless _XOPEN_SOURCE is defined).  */
> #if ((!defined __STRICT_ANSI__ || (_XOPEN_SOURCE - 0) >= 500) && \
>      !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE)
> # define _POSIX_SOURCE  1
> # if defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 500
> #  define _POSIX_C_SOURCE       2
> # elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 600
> #  define _POSIX_C_SOURCE       199506L
> # elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 700
> #  define _POSIX_C_SOURCE       200112L
> # else
> #  define _POSIX_C_SOURCE       200809L
> # endif
> # define __USE_POSIX_IMPLICITLY 1
> #endif
> 
> I.E. you get _BSD_SOURCE, _SVID_SOURCE, _POSIX_SOURCE, and
> _POSIX_C_SOURCE=200809L all #defined _FOR_ you if you just avoid
> mentioning any of these macros ever.

Relying on that is not portable, and musl does not do it. On
Linux/glibc you get the kitchen sink when you define nothing. On most
other unices, you get an intentionally-broken legacy environment
that's not conformant to modern standards.

All projects should explicitly define the feature test macros they
want. This also assists you in profiling the portability of your code
(switching out _GNU_SOURCE and you see exactly what extensions you're
using). Thankfully for ones that fail to do this, you can just add the
appropriate -D's to CPPFLAGS.

There's been some discussion of the "kitchen sink by default" approach
in musl, and I'm not entirely opposed to it, but the main issue is
that there's no portable way to select "plain ISO C" with a feature
test macro. There are glibc/gcc ways to request specific versions of
ISO C with feature test macros, but they took the silly approach of
renaming the macro for each version rather than something like #define
_ISO_C_SOURCE 199901L which would have been much cleaner...

Also note that glibc's version of "kitchen sink by default" is broken.
Several functions like strerror_r, basename, and perhaps still scanf
get *broken* GNU versions by default unless you specify an appropriate
feature test macro. So if you want to use them, you're stuck; you have
to use _POSIX_C_SOURCE or _XOPEN_SOURCE and then define some
additional macros to try to get back what you lost. And I'm not even
sure it's possible without breaking the standard functions again...

> And that's exactly what my code did before Georgi tried to build it
> against musl and suddenly that didn't work anymore because musl is weird.
> 
> (A sane development team would have implemented _negative_ macros, ala
> _ANSI_ONLY, to switch stuff _off_.)

There are very good reasons why this was not done; you could ask the
committees if you care.

> >> At present musl makes no attempt to support(*) the _BSD_SOURCE or
> >> _SVID_SOURCE feature test macros; every nonstandard (non-POSIX)
> >> extention offered by glibc is grouped together under _GNU_SOURCE, and
> >> this works for musl because (unlike with glibc), musl's _GNU_SOURCE
> >> only enables features; it does not alter standard interfaces like
> >> strerror_r or basename to be broken GNU versions of the functions.
> 
> A lot of those "nonstandard" things were in BSD circa 1982, before the
> FSF even existed.

I agree. But most of them were omitted from the standards for good
reasons, and even many of the ones that were included (like bcopy)
were redundant.

Anyway, the reason I've not focused on adding _BSD_SOURCE is mainly
lack of necessity. I don't think I've encountered a single package
that was defining _BSD_SOURCE and failed to build because of things it
expected to get from _BSD_SOURCE. If there are major packages that
could be made easier to build by adding it, I'd be a lot more
interested in doing so.

> I don't care about granularity, I just want my code to work.  But I
> refuse to #define _ALL_HAIL_RICHARD_STALLMAN as a condition of using
> standard Linux functions.
> 
> I will not say _GNU_SOURCE when implementing a project that has nothing
> whatsoever to do with the FSF's GNU project, and acts to replace large
> chunks of its' userspace.

OK, so basically you want alternate macros to request the kitchen sink
to avoid writing the word "GNU" anywhere in your project. :-)

All joking aside, at least this is the first *reason* I've seen for
possible changes in this area, so it's worth considering.

> >> I think there's a good (nontrivial) discussion to be had about whether
> >> it's worthwhile to have the _BSD_SOURCE and _SVID_SOURCE feature test
> >> macros supported in musl.
> 
> Given that the "nothing is #defined" behavior gives you most of this
> anyway on both glibc and uClibc, why musl is bothering with it at _all_
> is kinda inexplicable to me.

Because unlike uClibc, musl does not just copy all the mess from
glibc. I like the uClibc project and the people involved, but I don't
like the fact that basically everything but the internal
implementation details of stdio and locale are nearly direct copies
from glibc.

> Yes, I'm aware that Posix 2008 mentions feature test macros:
> 
> http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_02
> 
> And silently ignoring them is reasonably compliant behavior.  (You can
> even be technically compliant with Item 3 under _POSIX_C_SOURCE if you
> consider __linux__ a feature test macro.  It's got the double underscore!)

As long as you don't define additional things outside the reserved
namespace, it's compliant to ignore them. I agree totally.

Of course then your implementation complies only to POSIX, but not to
ISO C. For instance, this strictly conforming program cannot be
compiled (intentionally detailed to make it plausible):

#include <stdio.h>
#include <stdarg.h>
int debugging = 1;
int dprintf(const char *fmt, ...)
{
	int ret;
	if (debugging) return 0;
	va_list ap = va_start(fmt);
	ret = vprintf(fmt, ap);
	va_end(ap);
	return ret;
}
int main()
{
	dprintf("hello, world\n");
}

The same issue happens with conforming POSIX programs if you add
extensions outside the reserved namespaces.

> Which shows you all the compiler builtin #defines.  See __linux__ in
> that list?  If you _have_ to test for something, what's wrong with
> testing for __linux__?

This is not POSIX conformant. POSIX requires invoking the compiler
with -D_POSIX_C_SOURCE=200809L to respect the namespace. With your
suggestion, it would also be necessary to add -U__linux__ to the
command line.

In other words, your proposed change would break all correct programs
that are using feature test macros properly, for the sake of avoiding
adding a -D option to the CPPFLAGS of programs that aren't doing it
right.

> >>> Should the headers be filled with feature checks (that would make them
> >>> quite ugly) or assume we have _GNU_SOURCE defined and remove any
> >>> _GNU_SOURCE
> >>> checks?
> >>
> >> I'm confused what you mean by "assume we have _GNU_SOURCE defined".
> > 
> > Well, ignore my suggestion. I was thinking of removing most _GNU_SOURCE
> > checks in headers (except for pure GNU extensions) but reading what you
> > wrote above I remembered what PITA feature_macros were. Unfortunately
> > it seems that defining _GNU_SOURCE to get the kitchen sink is easier...
> 
> It's also wrong.  And it means that Musl can't build code that not just
> glibc and uClibc can build out of the box, but BSD can too.  (chroot()
> was _not_ invented by the gnu/dammit project.)

It's not my intent to imply that any of these features were invented
by the GNU project. Rather, _GNU_SOURCE is simply the only existing
feature test macro for "kitchen sink" that exists on Linux systems.
(Blame glibc for naming it this rather than _LINUX_SOURCE!) As such,
existing real-world software that's conforming to the standards and
using _POSIX_C_SOURCE or _XOPEN_SOURCE is already testing for the
availability of _GNU_SOURCE and adding that when it wants the kitchen
sink. While using a different name would be *cosmetically* better, it
would simply have the effect of making you add -D_BETTER_NAME_HERE to
all apps' CPPFLAGS since they'd no longer be able to find a working
option themselves...

> The feature test macros actually means that musl goes out of its _way_
> not to build them. You have extra code that's just there to break stuff.

One could also say that not having the internals of FILE defined in
stdio.h, and not using the same struct member names as glibc, is
"going out of its way" to break stuff...

In any case, the feature test macro issue is easily solved, if you
always want full extensions, by just using "CC=gcc -D_GNU_SOURCE".

If there's demand for a way to do this without ever having to write
that dirty word "GNU", we could consider adding an alias..

Rich

Re: [PATCH 0/4] Fix function definitions.

[Isaac Dunham]

On Fri, 9 Mar 2012 11:38:53 -0500
Rich Felker <dalias@aerifal.cx> wrote:

> On Fri, Mar 09, 2012 at 09:11:05AM -0600, Rob Landley wrote:
> > Why are you doing feature test macros at _all_ in a 2012 Linux libc?  If
> > you don't #define any of them (which most code doesn't), you get (from
> > /usr/lib/features.h):
> 
> features.h is part of glibc, and not a part I like, for several
> reasons:
> 
> 1. Each additional nested header increases base compile time
> significantly, and makes a big difference for small translation units
> that only includes one header themselves, due to extra path search,
> open, read, parse, close, etc.
> 
> 2. The way glibc does features.h, with _xxx_SOURCE macros as the input
> and __USE_xxx macros as the output, obfuscates in the actual headers
> which macros are necessary or sufficient to get the desired interface
> declared. 
> 
> 3. Others I'm forgetting at the moment.. :-)

> 
> Relying on that is not portable, and musl does not do it. On
> Linux/glibc you get the kitchen sink when you define nothing. On most
> other unices, you get an intentionally-broken legacy environment
> that's not conformant to modern standards.
> 
> All projects should explicitly define the feature test macros they
> want. This also assists you in profiling the portability of your code
> (switching out _GNU_SOURCE and you see exactly what extensions you're
> using). Thankfully for ones that fail to do this, you can just add the
> appropriate -D's to CPPFLAGS.


Strikes me that just because you don't #include <features.h> in every other header doesn't require making it a NOP.
I don't think features.h is in the reserved namespace.
So why not have #define _KITCHEN_SINK_SOURCE in there, and let whoever wants that go
#include <features.h>
?
Just because it converts macros internally, or GNU includes it, doesn't mean you have to include it or handle macros same way to ship it in a semi-compatible state.
IF a programmer #include's features.h themselves, they asked for the kitchen sink; ignoring that is wrong.


> Anyway, the reason I've not focused on adding _BSD_SOURCE is mainly
> lack of necessity. I don't think I've encountered a single package
> that was defining _BSD_SOURCE and failed to build because of things it
> expected to get from _BSD_SOURCE. If there are major packages that
> could be made easier to build by adding it, I'd be a lot more
> interested in doing so.
> 
> > I don't care about granularity, I just want my code to work.  But I
> > refuse to #define _ALL_HAIL_RICHARD_STALLMAN as a condition of using
> > standard Linux functions.
> > 
> > I will not say _GNU_SOURCE when implementing a project that has nothing
> > whatsoever to do with the FSF's GNU project, and acts to replace large
> > chunks of its' userspace.
..
> > >> I think there's a good (nontrivial) discussion to be had about whether
> > >> it's worthwhile to have the _BSD_SOURCE and _SVID_SOURCE feature test
> > >> macros supported in musl.
> > 
> > Given that the "nothing is #defined" behavior gives you most of this
> > anyway on both glibc and uClibc, why musl is bothering with it at _all_
> > is kinda inexplicable to me.
> 
> Because unlike uClibc, musl does not just copy all the mess from
> glibc. 

I'd like to see at least _BSD_SOURCE available, provided that
1. It actually is a BSD-ish API, with similar scope to NetBSD or glibc's _BSD_SOURCE
2. It isn't autodefined when using standard headers.


> > ... And it means that Musl can't build code that not just
> > glibc and uClibc can build out of the box, but BSD can too.  (chroot()
> > was _not_ invented by the gnu/dammit project.)
>
> It's not my intent to imply that any of these features were invented
> by the GNU project. Rather, _GNU_SOURCE is simply the only existing
> feature test macro for "kitchen sink" that exists on Linux systems.
..

The ONLY existing macro?
_BSD_SOURCE is an existing macro, and while it isn't the FULL kitchen sink, it's a lot closer.
(and apparently close enough for toybox?)

Also, you seem to be arguing that if it's automatically defined, then it's not standards compliant, and assume that if the standards say we can't do that, there's no reason to implement it at all.

Here's my proposal:
Would it work to implement _BSD_SOURCE and leave features.h lying around (read: implemented, but NOT included in any system headers) with this content:

#warn "features.h is not portable- use -D_BSD_SOURCE"
#if ((!defined _POSIX_C_SOURCE || _XOPEN_SOURCE || _BSD_SOURCE || 
	_GNU_SOURCE || __STRICT_ANSI__ || _POSIX_SOURCE ))
#define _BSD_SOURCE
#define _POSIX_C_SOURCE 200809L
#endif

-- 
Isaac Dunham <idunham@lavabit.com>

Re: [PATCH 0/4] Fix function definitions.

[Rich Felker]

On Fri, Mar 16, 2012 at 12:05:42PM -0700, Isaac Dunham wrote:
> Strikes me that just because you don't #include <features.h> in
> every other header doesn't require making it a NOP.
> I don't think features.h is in the reserved namespace.

It's not.

> So why not have #define _KITCHEN_SINK_SOURCE in there, and let
> whoever wants that go
> #include <features.h>
> ?
> Just because it converts macros internally, or GNU includes it,
> doesn't mean you have to include it or handle macros same way to
> ship it in a semi-compatible state.
> IF a programmer #include's features.h themselves, they asked for the
> kitchen sink; ignoring that is wrong.

Actually, if a program is including features.h, the author is either
(1) unclear what they were trying to accomplish, or (2) trying to
force the feature test macros to be processed early, before other
files are included or defines take effect. With glibc, all feature
test macros take effect the first time features.h is included, so
including it early could suppress consideration of feature test macros
getting defined (or undefined) later (e.g. within a library header
file). Of course such usage is extremely non-portable; the only valid
way to use feature test macros is defining them before any system
header is included, and never undefining or redefining them later.

Perhaps including features.h without defining any feature test macros
at all could be construed as "asking for the kitchen sink", but I find
that interpretation a bit doubtful since the behavior (on glibc) is
unchanged from what they would have gotten without including it.

> I'd like to see at least _BSD_SOURCE available, provided that
> 1. It actually is a BSD-ish API, with similar scope to NetBSD or
> glibc's _BSD_SOURCE
> 2. It isn't autodefined when using standard headers.

Could you give a summary of what the differences in _BSD_SOURCE and
_GNU_SOURCE are? My impression (possibly wrong) is that on glibc it's
pretty close to _GNU_SOURCE but without the nasty intentional GNU
incompatibilities (like GNU basename) and with some/most of the GNU or
Linux-specific extensions missing.

I'm quite willing to add proper _BSD_SOURCE support if you (or someone
else) is willing to put in the effort to figuring out what it should
include.

> > It's not my intent to imply that any of these features were invented
> > by the GNU project. Rather, _GNU_SOURCE is simply the only existing
> > feature test macro for "kitchen sink" that exists on Linux systems.
> ...
> 
> The ONLY existing macro?
> _BSD_SOURCE is an existing macro, and while it isn't the FULL
> kitchen sink, it's a lot closer.
> (and apparently close enough for toybox?)

Indeed, it's not as much, but it may be sufficient for a lot of things
that are highly desirable like MAP_ANONYMOUS.

> Also, you seem to be arguing that if it's automatically defined,
> then it's not standards compliant, and assume that if the standards
> say we can't do that, there's no reason to implement it at all.

I never meant to imply that. Sorry.

> Here's my proposal:
> Would it work to implement _BSD_SOURCE and leave features.h lying
> around (read: implemented, but NOT included in any system headers)
> with this content:
> 
> #warn "features.h is not portable- use -D_BSD_SOURCE"
> #if ((!defined _POSIX_C_SOURCE || _XOPEN_SOURCE || _BSD_SOURCE || 
> 	_GNU_SOURCE || __STRICT_ANSI__ || _POSIX_SOURCE ))
> #define _BSD_SOURCE
> #define _POSIX_C_SOURCE 200809L
> #endif

I don't see how this really helps. If the goal is to give behavior
closer to glibc, you'd want it to define _GNU_SOURCE, not _BSD_SOURCE.
But this will fix very few programs (only the ones that include
features.h explicitly, which probably means they're doing something
very wrong...) when the alternate workaround (adding -D_GNU_SOURCE or
whatever is needed to $CC or $CPPFLAGS) is easier and less invasive to
programs.

In any case I'm interested in adding _BSD_SOURCE if you or someone
else will help with getting it right.

Rich

Re: Fix function definitions.

[Isaac Dunham]

On Fri, 16 Mar 2012 18:48:55 -0400
Rich Felker <dalias@aerifal.cx> wrote:
..
> Perhaps including features.h without defining any feature test macros
> at all could be construed as "asking for the kitchen sink", but I find
> that interpretation a bit doubtful since the behavior (on glibc) is
> unchanged from what they would have gotten without including it.
> 
It's at least asking for BSD-style stuff, instead of straight ANSI.
It certainly isn't asking for the full _GNU_SOURCE.
 
> Could you give a summary of what the differences in _BSD_SOURCE and
> _GNU_SOURCE are? My impression (possibly wrong) is that on glibc it's
> pretty close to _GNU_SOURCE but without the nasty intentional GNU
> incompatibilities (like GNU basename) and with some/most of the GNU or
> Linux-specific extensions missing.
...
> In any case I'm interested in adding _BSD_SOURCE if you or someone
> else will help with getting it right.

I started on <unistd.h>, and noticed a namespace issue:
ualarm, usleep, and a few other functions should actually NOT be defined
on _GNU_SOURCE (per the glibc headers and manpages), but only on
_BSD_SOURCE or _XOPEN_SOURCE >= 500
Instead, they are defined only on _GNU_SOURCE.
(There's also the L_SET & co. aliases for SEEK_* to do, but that's
trivial).

I'll have a patch once unistd.h is ready.  There are a few more
functions to move around, though--I have to locate all the problem ones.
Should be ready in a day or two.

Isaac Dunham