Fwd: Re: Detecting musl at compile and/or configure time

Fwd: Re: Detecting musl at compile and/or configure time

[Chris Hall]

On 29/06/2019 14:27, A. Wilcox wrote:
> On 06/29/19 06:48, Chris Hall wrote:
>>
>> Is there a supported way of detecting that one is building against 
>> the musl libc ?
...
> The musl libc specifically does not have a FTM (feature test macro)
> because it aims to be an exact implementation of C11 / POSIX and
> therefore it has no "quirks" to detect.

I too aim for perfection, but find that I approach it asymptotically :-)

However, restating the question in those terms: how should I detect that 
one is *not* building against some library which is *not* musl ?  [Given 
that other libraries may be equally discrete.]

...
> See here for more information:
> 
> https://wiki.musl-libc.org/faq.html#Q:-Why-is-there-no-%3Ccode%3E__MUSL__%3C/code%3E-macro?

FWIW: I read https://www.musl-libc.org/faq.html, but did not also find 
the wiki's FAQ :-(

> Is there a reason you wish to detect musl at configure/compile time?
> Perhaps if we knew the reason, we could help you make your code more
> portable and/or not require such detection.

I have a little build system which tries to detect the "usual suspects" 
automatically, even without a full configure/cmake/etc. step.  If the 
detection process fails, it generates a warning and the user must (at 
least) add a '-Dxxx' to suppress that.

I confess I have only recently stumbled across musl.  Perhaps systems 
which default to musl are so rare that I can, as a practical matter, 
ignore them ?  The question then is whether to add a '-DqLIB_MUSL' gizmo 
to my build stuff -- so that "musl-gcc -DqLIB_MUSL" will do the trick.

Thanks,

Chris

Re: Detecting musl at compile and/or configure time

[Samuel Holland]

Hi,

On 6/30/19 7:03 AM, Chris Hall wrote:
> I have a little build system which tries to detect the "usual suspects"
> automatically, even without a full configure/cmake/etc. step.  If the detection
> process fails, it generates a warning and the user must (at least) add a '-Dxxx'
> to suppress that.
> 
> I confess I have only recently stumbled across musl.  Perhaps systems which
> default to musl are so rare that I can, as a practical matter, ignore them ? 
> The question then is whether to add a '-DqLIB_MUSL' gizmo to my build stuff --
> so that "musl-gcc -DqLIB_MUSL" will do the trick.

The fundamental problem with that is that you're not really checking for musl.
You're checking for "the supported version (and options) of POSIX", "the
supported version of the Linux UAPI", and "the set of extensions currently
supported by musl", which all change over time. And some of them depend on who's
distributing musl, as they may add their own customizations.

For POSIX, you should declare which version of the standard you follow with
_POSIX_C_SOURCE, which you can verify is supported with _POSIX_VERSION, or for
optional parts, the other macros in unistd.h[1].

The Linux UAPI is stable, so you can use whatever features you want as long as
you gracefully handle ENOSYS/EINVAL (or document a minimum supported version).

Extensions (anything not specified by POSIX or in the UAPI) must be detected
individually, because they may appear (or disappear! like sys/sysmacros.h) at
any time. If you know which extensions you rely on, and have fallbacks as
possible/needed, then it doesn't matter what the libc is named, or what it's
version number is. And it will be much easier to port your application to a new
environment.

[1]: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/unistd.h.html

> Thanks,
> 
> Chris
> 

Cheers,
Samuel

Re: Detecting musl at compile and/or configure time

[Chris Hall]

On 30/06/2019 13:28, Samuel Holland wrote:
> On 6/30/19 7:03 AM, Chris Hall wrote:
>> I have a little build system which tries to detect the "usual suspects"
>> automatically, even without a full configure/cmake/etc. step.  If the detection
>> process fails, it generates a warning and the user must (at least) add a '-Dxxx'
>> to suppress that.
...
> For POSIX, you should declare which version of the standard you follow with
> _POSIX_C_SOURCE, which you can verify is supported with _POSIX_VERSION, or for
> optional parts, the other macros in unistd.h[1].
> 
> The Linux UAPI is stable, so you can use whatever features you want as long as
> you gracefully handle ENOSYS/EINVAL (or document a minimum supported version).

My experience with Linux and BSD is that setting _POSIX_C_SOURCE and/or 
_XOPEN_SOURCE does not work very well.  For BSD it tends to rule out 
stuff I need -- notably in the area of sockets.  So, I have found it is 
more practical to declare _GNU_SOURCE for Linux and nothing at all for 
the BSDs and try to stick to the common areas.  The big differences 
between Linux and the BSDs you just have to deal with, in any case.

> Extensions (anything not specified by POSIX or in the UAPI) must be detected
> individually, because they may appear (or disappear! like sys/sysmacros.h) at
> any time. If you know which extensions you rely on, and have fallbacks as
> possible/needed, then it doesn't matter what the libc is named, or what it's
> version number is. And it will be much easier to port your application to a new
> environment.

I accept that the only way to be absolutely sure of any given extension 
(or local interpretation of some Standard) is to do the required tests 
at configure/cmake/etc time.

Nevertheless, I think it is possible to get a long way without a 
configure/cmake/etc step... much/most of the time.  And where something 
very specific needs to be configured, the more common stuff still does 
not (simplifying the configure step).

I agree that, in this context, the libc is of less interest than the 
underlying OS.

Nevertheless, I see no reason not to detect the libc, where I can.

Clearly musl falls into the (small) category of libc where I cannot.

Thanks,

Chris

Re: Detecting musl at compile and/or configure time

[Jeffrey Walton]

On Sun, Jun 30, 2019 at 8:29 AM Samuel Holland <samuel@sholland.org> wrote:
>
> On 6/30/19 7:03 AM, Chris Hall wrote:
> > I have a little build system which tries to detect the "usual suspects"
> > automatically, even without a full configure/cmake/etc. step.  If the detection
> > process fails, it generates a warning and the user must (at least) add a '-Dxxx'
> > to suppress that.
> >
> > I confess I have only recently stumbled across musl.  Perhaps systems which
> > default to musl are so rare that I can, as a practical matter, ignore them ?
> > The question then is whether to add a '-DqLIB_MUSL' gizmo to my build stuff --
> > so that "musl-gcc -DqLIB_MUSL" will do the trick.
>
> The fundamental problem with that is that you're not really checking for musl.
> You're checking for "the supported version (and options) of POSIX", "the
> supported version of the Linux UAPI", and "the set of extensions currently
> supported by musl", which all change over time. And some of them depend on who's
> distributing musl, as they may add their own customizations.

Or perhaps crafting workarounds for a buggy versions of musl. For
example, https://www.openwall.com/lists/musl/2019/06/26/10 .

Jeff

Re: Detecting musl at compile and/or configure time

[Florian Weimer]

* Samuel Holland:

> The Linux UAPI is stable, so you can use whatever features you want as long as
> you gracefully handle ENOSYS/EINVAL (or document a minimum supported version).

Note that this only applies to the numbered syscall interface.  For
example, it is not true for vsyscall (which is now considered optional,
despite part of the userspace ABI at first, and really hard to detect
properly).  Netlink packet layout also keeps changing for some
subsystems.

Thanks,
Florian

Re: Re: Detecting musl at compile and/or configure time

[Jeffrey Walton]

On Sun, Jun 30, 2019 at 8:03 AM Chris Hall <musl@gmch.uk> wrote:
> ...
>
> > Is there a reason you wish to detect musl at configure/compile time?
> > Perhaps if we knew the reason, we could help you make your code more
> > portable and/or not require such detection.
>
> I have a little build system which tries to detect the "usual suspects"
> automatically, even without a full configure/cmake/etc. step.  If the
> detection process fails, it generates a warning and the user must (at
> least) add a '-Dxxx' to suppress that.

Alpine Linux might make a good test platform for you because it uses
musl out of the box. https://alpinelinux.org/downloads/.

Jeff

Detecting musl at compile and/or configure time

[Chris Hall]

Is there a supported way of detecting that one is building against the 
musl libc ?

At compile time I can, of course, detect glibc because it sets __GLIBC__ 
etc.  For macOS I can see _DARWIN_FEATURE_xxx macros.  The BSDs mostly 
give themselves away.  But for musl I have not found anything :-(

Chris

Re: Detecting musl at compile and/or configure time

[A. Wilcox]

[-- Attachment #1.1: Type: text/plain, Size: 1249 bytes --]

On 06/29/19 06:48, Chris Hall wrote:
> 
> Is there a supported way of detecting that one is building against the
> musl libc ?
> 
> At compile time I can, of course, detect glibc because it sets __GLIBC__
> etc.  For macOS I can see _DARWIN_FEATURE_xxx macros.  The BSDs mostly
> give themselves away.  But for musl I have not found anything :-(
> 
> Chris


The musl libc specifically does not have a FTM (feature test macro)
because it aims to be an exact implementation of C11 / POSIX and
therefore it has no "quirks" to detect.

If there are bugs in musl libc, the authors prefer if you report them
and have them fixed, instead of making your software look for __MUSL__
and then assume all musl versions will always have that bug.

See here for more information:

https://wiki.musl-libc.org/faq.html#Q:-Why-is-there-no-%3Ccode%3E__MUSL__%3C/code%3E-macro?

https://www.openwall.com/lists/musl/2013/03/29/13

Is there a reason you wish to detect musl at configure/compile time?
Perhaps if we knew the reason, we could help you make your code more
portable and/or not require such detection.

Best to you and yours,
--arw


-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Detecting musl at compile and/or configure time

[Szabolcs Nagy]

* A. Wilcox <awilfox@adelielinux.org> [2019-06-29 08:27:50 -0500]:
> On 06/29/19 06:48, Chris Hall wrote:
> > 
> > Is there a supported way of detecting that one is building against the
> > musl libc ?
> > 
> > At compile time I can, of course, detect glibc because it sets __GLIBC__
> > etc.  For macOS I can see _DARWIN_FEATURE_xxx macros.  The BSDs mostly
> > give themselves away.  But for musl I have not found anything :-(
> > 
> > Chris
> 
> 
> The musl libc specifically does not have a FTM (feature test macro)
> because it aims to be an exact implementation of C11 / POSIX and
> therefore it has no "quirks" to detect.

that's not entirely true.

there are some standard macros in posix to test optional features.

and musl implements various extensions to the standard.

note that __GLIBC__ does not detect glibc: uclibc defines it too.

you really want to configure check for particular features
instead of detecting the libc (which is a moving target,
ppl can patch it or change it over time)

> 
> If there are bugs in musl libc, the authors prefer if you report them
> and have them fixed, instead of making your software look for __MUSL__
> and then assume all musl versions will always have that bug.
> 
> See here for more information:
> 
> https://wiki.musl-libc.org/faq.html#Q:-Why-is-there-no-%3Ccode%3E__MUSL__%3C/code%3E-macro?
> 
> https://www.openwall.com/lists/musl/2013/03/29/13
> 
> Is there a reason you wish to detect musl at configure/compile time?
> Perhaps if we knew the reason, we could help you make your code more
> portable and/or not require such detection.
> 
> Best to you and yours,
> --arw
> 
> 
> -- 
> A. Wilcox (awilfox)
> Project Lead, Adélie Linux
> https://www.adelielinux.org
> 

Re: Detecting musl at compile and/or configure time

[Chris Hall]

On 29/06/2019 14:58, Szabolcs Nagy wrote:
>> On 06/29/19 06:48, Chris Hall wrote:
>>>
>>> Is there a supported way of detecting that one is building against the
>>> musl libc ?
>>>
>>> At compile time I can, of course, detect glibc because it sets __GLIBC__
>>> etc.  For macOS I can see _DARWIN_FEATURE_xxx macros.  The BSDs mostly
>>> give themselves away.  But for musl I have not found anything :-(
...
> note that __GLIBC__ does not detect glibc: uclibc defines it too.

Happily it has the courtesy to also:

   #define __UCLIBC__ 1

> you really want to configure check for particular features
> instead of detecting the libc (which is a moving target,
> ppl can patch it or change it over time)

In the limit, yes.  I'm interested in how far one can get without the 
need for configure/cmake/etc. -- accepting that means being quite 
conservative in the choice of features to use (and assuming that the 
target system is reasonably up to date).

Of course, for major issues -- eg epoll vs kqueue or netlink vs routeing 
sockets -- it's the underlying OS that really matters.  Detecting the OS 
at compile time seems reasonably effective (also the compiler and the 
processor).  Detecting the libc feels like it should be a small step to 
complete the picture.

Chris