Re: gnulib cross-compiling issue with musl

Re: gnulib cross-compiling issue with musl

[Rich Felker]

On Tue, Jun 18, 2013 at 11:32:50AM -0700, Paul Eggert wrote:
> On 06/18/13 11:07, Rich Felker wrote:
> 
> > Of my two
> > proposed fixes, the first would fix the issue on any future system
> > that's not broken (not just existing ones), but would obviously not
> > support future broken systems.
> 
> But we're not talking about "future" systems here; we're merely talking
> about unported-to systems, not all of which are future systems.
> 
> Nor are we talking about "broken" systems; we're merely talking about
> systems that do not conform to POSIX -- they may conform to C11, say,
> without conforming to POSIX.
> 
> So this first fix sounds problematic.

I think the advent of new implementations that don't aim for at least
a reasonable level of POSIX compatibility is fairly unlikely. But if
you think it matters, you could add #elif defined _POSIX_VERSION
containing the if(0), and leave #else at the end with #error. This
should be entirely safe.

> > The second proposed fix should support
> > any future system, but would be a bit more costly, assuming it's even
> > possible.
> 
> That would be better, yes.  But it'd be more work to implement than
> method (3) would be.

I agree that it would be more work, but if you really don't accept
solution #1, I'd be happy to work with the gnulib team on figuring out
what solution #2 would entail, whether it's possible, and implementing
it.

> > To revisit why I don't like your proposed fix, for every bug we could
> > get fixed by making an easy way for applications to test "is this
> > musl?", we would have something like 10 new bugs created by people
> > doing that. This is not just speculation; it's based on questions we
> > get on the list and on IRC. Your idea of using such a test in the form
> > of "if (is_musl) assume_non_broken();" is the first proposed use of
> > this form I've seen. Every other request for an easy "if (is_musl)"
> > test have been from people who wanted to do something that would cause
> > bad breakage with future versions of musl.
> 
> I'm afraid I don't follow this reasoning.  Are you saying that,
> method (3) is OK here but its use here might encourage people
> to use it in other places where it's not OK?  If so, let's add
> a comment warning people to use method (3) carefully.

I'm saying that musl intentionally does not provide any macros to
identify itself. Changing this to help gnulib do one correct thing
would not be worth dealing with all the incorrect usages it would lead
to. And I'm confident that presence of such a macro would lead to lots
of bugs in lots of applications, and to people blaming us for changing
things when such incorrect applications break, on the basis of all the
requests we have received from users for a __MUSL__ macro. In every
case, we have been able to recommend clean portable solutions that did
not involve hard-coding assumptions about musl.

The same issue was discussed in the thread on musl and gnulib
compatibility last year, and we reached mutually acceptable
conclusions: instead of adding __MUSL__ and documenting ways for
gnulib to poke at stdio internals (which are documented by musl as
being subject to change between versions and not acceptable for
applications to poke at), we added functions like __freadahead etc.
and gnulib is simply probing for their existence.

Rich

Re: gnulib cross-compiling issue with musl

[Paul Eggert]

On 06/18/13 11:42, Rich Felker wrote:
> if
> you think it matters, you could add #elif defined _POSIX_VERSION
> containing the if(0)

Yes, that should work, though it needs to check a couple ore
things as well.  I pushed the following: does it work for you?

fflush, fseeko: port to musl cross-compiles
* lib/fseeko.c (fseeko): Assume that fflushing stdin works if
on some implementation that (1) is not known to be buggy,
(2) claims conformance to POSIX.1-2008 or later, and (3) is being
cross-compiled to so we can't easily check for lack of
conformance.  This is for cross-compiling to musl.
Reported by Rich Felker in
<http://lists.gnu.org/archive/html/bug-gnulib/2013-06/msg00043.html>.
* m4/fclose.m4 (gl_FUNC_FCLOSE):
* m4/fflush.m4 (gl_FUNC_FFLUSH):
* m4/fseeko.m4 (gl_FUNC_FSEEKO):
Adjust to above change.
* m4/fflush.m4 (gl_FUNC_FFLUSH_STDIN): Set gl_cv_func_fflush_stdin
to 'cross', not to 'no', when cross-compiling.  AC_DEFINE
FUNC_FFLUSH_STDIN to 1, 0, -1 if fflushing stdin is known to work,
known not to work, or unknown.
diff --git a/lib/fseeko.c b/lib/fseeko.c
index a625803..ec5a6aa 100644
--- a/lib/fseeko.c
+++ b/lib/fseeko.c
@@ -19,7 +19,7 @@
 /* Specification.  */
 #include <stdio.h>

-/* Get off_t and lseek.  */
+/* Get off_t, lseek, _POSIX_VERSION.  */
 #include <unistd.h>

 #include "stdio-impl.h"
@@ -99,8 +99,14 @@ fseeko (FILE *fp, off_t offset, int whence)
 #elif defined EPLAN9                /* Plan9 */
   if (fp->rp == fp->buf
       && fp->wp == fp->buf)
+#elif FUNC_FFLUSH_STDIN < 0 && 200809 <= _POSIX_VERSION
+  /* Cross-compiling to some other system advertising conformance to
+     POSIX.1-2008 or later.  Assume fseeko and fflush work as advertised.
+     If this assumption is incorrect, please report the bug to
+     bug-gnulib.  */
+  if (0)
 #else
-  #error "Please port gnulib fseeko.c to your platform! Look at the code in fpurge.c, then report this to bug-gnulib."
+  #error "Please port gnulib fseeko.c to your platform! Look at the code in fseeko.c, then report this to bug-gnulib."
 #endif
     {
       /* We get here when an fflush() call immediately preceded this one (or
diff --git a/m4/fclose.m4 b/m4/fclose.m4
index 2cc2e12..b306b6d 100644
--- a/m4/fclose.m4
+++ b/m4/fclose.m4
@@ -1,4 +1,4 @@
-# fclose.m4 serial 5
+# fclose.m4 serial 6
 dnl Copyright (C) 2008-2013 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -9,7 +9,7 @@ AC_DEFUN([gl_FUNC_FCLOSE],
   AC_REQUIRE([gl_STDIO_H_DEFAULTS])

   gl_FUNC_FFLUSH_STDIN
-  if test $gl_cv_func_fflush_stdin = no; then
+  if test $gl_cv_func_fflush_stdin != yes; then
     REPLACE_FCLOSE=1
   fi

diff --git a/m4/fflush.m4 b/m4/fflush.m4
index 6df5173..c16b314 100644
--- a/m4/fflush.m4
+++ b/m4/fflush.m4
@@ -1,4 +1,4 @@
-# fflush.m4 serial 14
+# fflush.m4 serial 15

 # Copyright (C) 2007-2013 Free Software Foundation, Inc.
 # This file is free software; the Free Software Foundation
@@ -14,7 +14,7 @@ AC_DEFUN([gl_FUNC_FFLUSH],
 [
   AC_REQUIRE([gl_STDIO_H_DEFAULTS])
   gl_FUNC_FFLUSH_STDIN
-  if test $gl_cv_func_fflush_stdin = no; then
+  if test $gl_cv_func_fflush_stdin != yes; then
     REPLACE_FFLUSH=1
   fi
 ])
@@ -72,10 +72,17 @@ AC_DEFUN([gl_FUNC_FFLUSH_STDIN],
            return 7;
          return 0;
        ]])], [gl_cv_func_fflush_stdin=yes], [gl_cv_func_fflush_stdin=no],
-     [dnl Pessimistically assume fflush is broken.
-      gl_cv_func_fflush_stdin=no])
+     [gl_cv_func_fflush_stdin=cross])
      rm conftest.txt
     ])
+  case $gl_cv_func_fflush_stdin in
+    yes) gl_func_fflush_stdin=1 ;;
+    no)  gl_func_fflush_stdin=0 ;;
+    *)   gl_func_fflush_stdin='(-1)' ;;
+  esac
+  AC_DEFINE_UNQUOTED([FUNC_FFLUSH_STDIN], [$gl_func_fflush_stdin],
+    [Define to 1 if fflush is known to work on stdin as per POSIX.1-2008,
+     0 if fflush is known to not work, -1 if unknown.])
 ])

 # Prerequisites of lib/fflush.c.
diff --git a/m4/fseeko.m4 b/m4/fseeko.m4
index e0f2dfb..ca9da28 100644
--- a/m4/fseeko.m4
+++ b/m4/fseeko.m4
@@ -1,4 +1,4 @@
-# fseeko.m4 serial 16
+# fseeko.m4 serial 17
 dnl Copyright (C) 2007-2013 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -37,7 +37,7 @@ AC_DEFUN([gl_FUNC_FSEEKO],
     fi
     m4_ifdef([gl_FUNC_FFLUSH_STDIN], [
       gl_FUNC_FFLUSH_STDIN
-      if test $gl_cv_func_fflush_stdin = no; then
+      if test $gl_cv_func_fflush_stdin != yes; then
         REPLACE_FSEEKO=1
       fi
     ])

Re: gnulib cross-compiling issue with musl

[Rich Felker]

On Tue, Jun 18, 2013 at 01:09:10PM -0700, Paul Eggert wrote:
> On 06/18/13 11:42, Rich Felker wrote:
> > if
> > you think it matters, you could add #elif defined _POSIX_VERSION
> > containing the if(0)
> 
> Yes, that should work, though it needs to check a couple ore
> things as well.  I pushed the following: does it work for you?
> 
> fflush, fseeko: port to musl cross-compiles
> * lib/fseeko.c (fseeko): Assume that fflushing stdin works if
> on some implementation that (1) is not known to be buggy,
> (2) claims conformance to POSIX.1-2008 or later, and (3) is being
> cross-compiled to so we can't easily check for lack of
> conformance.  This is for cross-compiling to musl.
> Reported by Rich Felker in
> <http://lists.gnu.org/archive/html/bug-gnulib/2013-06/msg00043.html>.

Thank you. This patch looks like it should solve the problem. I'll
make sure it gets tested with someone on our team doing cross compiles
and let you know if we run into any problems.

Rich

Re: Re: gnulib cross-compiling issue with musl

[Anthony G. Basile]

On 06/18/2013 02:42 PM, Rich Felker wrote:
> On Tue, Jun 18, 2013 at 11:32:50AM -0700, Paul Eggert wrote:
>
> I'm saying that musl intentionally does not provide any macros to
> identify itself. Changing this to help gnulib do one correct thing
> would not be worth dealing with all the incorrect usages it would lead
> to. And I'm confident that presence of such a macro would lead to lots
> of bugs in lots of applications, and to people blaming us for changing
> things when such incorrect applications break, on the basis of all the
> requests we have received from users for a __MUSL__ macro. In every
> case, we have been able to recommend clean portable solutions that did
> not involve hard-coding assumptions about musl.
>

Hi Rich,

I see your point wrt no __MUSL__ macro, but in practice this is a tough 
one to live with.  It is not just gnulib that makes assumptions about 
libc, although needing the internals of FILE is extreme given that 
implementation details are subject to change.  Most of what I'm hitting 
porting Gentoo over are assumptions about what is provided or where it 
is provide or how it is provided (and standards be damned), and I can't 
play the usual game of #ifdef __MUSL__ as I did for uclibc.  I know you 
might respond with "that's what build systems are for", but ... ouch ... 
  Not to derail, but gnulib was the my first experience of a sequence.


-- 
Anthony G. Basile, Ph. D.
Chair of Information Technology
D'Youville College
Buffalo, NY 14201
(716) 829-8197

Re: Re: gnulib cross-compiling issue with musl

[Rich Felker]

On Thu, Jun 20, 2013 at 12:08:51PM -0400, Anthony G. Basile wrote:
> I see your point wrt no __MUSL__ macro, but in practice this is a
> tough one to live with.  It is not just gnulib that makes
> assumptions about libc, although needing the internals of FILE is
> extreme given that implementation details are subject to change.
> Most of what I'm hitting porting Gentoo over are assumptions about
> what is provided or where it is provide or how it is provided (and
> standards be damned), and I can't play the usual game of #ifdef
> __MUSL__ as I did for uclibc.

Can you give me some examples? My experience with people trying to
determine these things has been that the assumption they wanted to
make using "#ifdef __MUSL__" was outdated by the time the next version
of musl was released. Tests of the form

#ifdef __MUSL__
/* interface foo is available */
#endif

would of course be relatively stable, as long as "foo" is a public
interface and not some internal interface you found and decided to
play with. However, tests of the form

#ifdef __MUSL__
/* assume interface or feature foo is not available */
#endif

would almost surely be wrong at some point in the future. Depending on
how you use the assumption, either build might break with future
versions of musl (for example, if you redefine something that was
missing from the headers), or you might omit features of the
application that would otherwise work with future versions of musl,
and if you fail to fully maintain the #ifdefs, a user of musl 1.5.0
next year building your package might be stuck with the featureset of
musl 0.9.10.

> I know you might respond with "that's
> what build systems are for", but ... ouch ...  Not to derail, but
> gnulib was the my first experience of a sequence.

Most packages have configure scripts that can detect the presence or
absence of public interfaces easily, in ways that fully work with
cross-compiling. The packages that fail to do so are mostly core
Linux-specific components where the author assumed Linux==glibc. For
anything non-Linux-specific, lack of a configure script (or similar
build process) would basically mean either it only runs on the
author's system, or it's taken extreme care only to use standard POSIX
interfaces (and only the ones that aren't broken on major pseudo-POSIX
systems like OSX...).

Rich