'make install' fails when build directory is differnet from that of source tree.

'make install' fails when build directory is differnet from that of source tree.

[Yasuhiro KIMURA]

Hello All,

'make install' fails when build directory is differnet from that of
source tree.

At first I found this problem when building with 5.8 source archive.

For example if I take following steps,

1. cd /tmp/zsh/source
2. tar xfpvJ /somewhere/zsh-5.8.tar.xz
3. cd /tmp/zsh/build
4. /tmp/zsh/source/zsh-5.8/configure --srcdir=/tmp/zsh/source/zsh-5.8
5. make
6. make DESTDIR=/tmp/zsh/install install

then step 6 fails as following.

----------------------------------------------------------------------
make[1]: Entering directory '/tmp/zsh/build/Doc'
/bin/sh /tmp/zsh/source/zsh-5.8/mkinstalldirs /tmp/zsh/install/usr/local/share/man/man1
mkdir -m 755 -p -- /tmp/zsh/install/usr/local/share/man/man1
for file in zsh.1 zshbuiltins.1 zshcalsys.1 zshcompctl.1 zshcompwid.1 zshcompsys.1 zshcontrib.1 zshexpn.1 zshmisc.1 zshmodules.1 zshoptions.1 zshparam.1 zshroadmap.1 zshtcpsys.1 zshzftpsys.1 zshzle.1 zshall.1; do \
    test -s $file || exit 1; \
    /bin/install -c -m 644 $file /tmp/zsh/install/usr/local/share/man/man1/`echo $file | sed 's|zsh|zsh|'` || exit 1; \done
make[1]: *** [Makefile:474: install.man] Error 1
make[1]: Leaving directory '/tmp/zsh/build/Doc'
make: *** [Makefile:222: install.man] Error 2
----------------------------------------------------------------------

But if I checked out source tree from git repository such problem
doesn't happen.

For example if I take following steps,

1. cd /tmp/zsh/source
2. git clone https://git.code.sf.net/p/zsh/code zsh
3. cd zsh
4. git checkout zsh-5.8
5. ./Util/preconfig
6. cd /tmp/zsh/build
7. /tmp/zsh/source/zsh/configure --srcdir=/tmp/zsh/source/zsh
8. make
9. make DESTDIR=/tmp/zsh/install install

then everything completes without any errors.

Why such difference happens? And how it is fixed?

Best Regards.

---
Yasuhiro KIMURA

Re: 'make install' fails when build directory is differnet from that of source tree.

[Daniel Shahaf]

Yasuhiro KIMURA wrote on Fri, 29 May 2020 15:07 +0900:
> 1. cd /tmp/zsh/source
> 2. tar xfpvJ /somewhere/zsh-5.8.tar.xz
> 3. cd /tmp/zsh/build
> 4. /tmp/zsh/source/zsh-5.8/configure --srcdir=/tmp/zsh/source/zsh-5.8
> 5. make
> 6. make DESTDIR=/tmp/zsh/install install

> ----------------------------------------------------------------------
> make[1]: Entering directory '/tmp/zsh/build/Doc'
> /bin/sh /tmp/zsh/source/zsh-5.8/mkinstalldirs /tmp/zsh/install/usr/local/share/man/man1
> mkdir -m 755 -p -- /tmp/zsh/install/usr/local/share/man/man1
> for file in zsh.1 zshbuiltins.1 zshcalsys.1 zshcompctl.1 zshcompwid.1 zshcompsys.1 zshcontrib.1 zshexpn.1 zshmisc.1 zshmodules.1 zshoptions.1 zshparam.1 zshroadmap.1 zshtcpsys.1 zshzftpsys.1 zshzle.1 zshall.1; do \
>     test -s $file || exit 1; \
>     /bin/install -c -m 644 $file /tmp/zsh/install/usr/local/share/man/man1/`echo $file | sed 's|zsh|zsh|'` || exit 1; \done
> make[1]: *** [Makefile:474: install.man] Error 1
> make[1]: Leaving directory '/tmp/zsh/build/Doc'
> make: *** [Makefile:222: install.man] Error 2

I can reproduce this.

I added «set -x» to Doc/Makefile [sic] in front of the loop.  It
fails on «test -s zsh.1».

> But if I checked out source tree from git repository such problem
> doesn't happen.

Good catch.

> Why such difference happens? And how it is fixed?

In git builds the manual pages are generated in the build directory.
I made it work this way a few years ago (workers/43692, commit
zsh-5.6.2-54-g76a16498d) on the principle that the source directory
should never be written to by out-of-tree builds.  (That fix only was a
step in the right direction, rather than a complete fix.  «make» still
puts some files in the source directory.)

In contrast, in tarball builds pre-build manual pages are shipped in the
tarball, so they'll be found in the source directory (/tmp/zsh/source/Doc/*.1)
rather than the build directory (/tmp/zsh/build/Doc/*.1).  I suppose
we don't require yodl for tarball builds.

In Doc/Makefile rule there is ".yo.1" suffix rule that's supposed to
generate man pages.  In tarball out-of-tree builds, the commands of that
rule don't run because neither the target file nor any of its dependencies
exist.  I'm appending the relevant part of make(1) debug output for
reference.

I would prefer to fix this by making tarball builds copy the *.1 files
from the source directory («$(sdir)» in Doc/Makefile) to the build
directory.  This would minimize the difference: tarball builds would
compile the man pages using yodl and git builds would copy the
pregenerated ones.  The following patch does this.

[[[
diff --git a/configure.ac b/configure.ac
index e78ebf86b..c72fe72b2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -3178,6 +3178,13 @@ AC_CONFIG_COMMANDS([stamp-h], [echo >stamp-h])
 
 AC_OUTPUT
 
+dnl Copy pre-built man pages, for tarball out-of-tree builds.
+for manpage in ${srcdir}/Doc/*.1; do
+  if test x"$manpage" != x*"*"*; then
+    cp ${manpage} ./Doc/
+  fi
+done
+
 eval "zshbin1=${bindir}"
 eval "zshbin2=${zshbin1}"
 eval "zshman1=${mandir}"
]]]

If you could test it (by applying it to «configure»), that'd be great.

Thanks for the bug report and for pinpointing the problem!

Daniel



[[[
% make -dC Doc
    Considering target file 'zsh.1'.
     Looking for an implicit rule for 'zsh.1'.
     Trying pattern rule with stem 'zsh'.
     Trying implicit prerequisite 'zsh.yo'.
     Found an implicit rule for 'zsh.1'.
      Considering target file 'zsh.yo'.
       Looking for an implicit rule for 'zsh.yo'.
       Trying pattern rule with stem 'zsh.yo'.
       Trying implicit prerequisite 'zsh.yo,v'.
       Trying pattern rule with stem 'zsh.yo'.
       Trying implicit prerequisite 'RCS/zsh.yo,v'.
       Trying pattern rule with stem 'zsh.yo'.
       Trying implicit prerequisite 'RCS/zsh.yo'.
       Trying pattern rule with stem 'zsh.yo'.
       Trying implicit prerequisite 's.zsh.yo'.
       Trying pattern rule with stem 'zsh.yo'.
       Trying implicit prerequisite 'SCCS/s.zsh.yo'.
       No implicit rule found for 'zsh.yo'.
       Finished prerequisites of target file 'zsh.yo'.
      No need to remake target 'zsh.yo'; using VPATH name '../../tarball/Doc/zsh.yo'.
      Considering target file 'version.yo'.
        Considering target file '../../tarball/Config/version.mk'.
         Looking for an implicit rule for '../../tarball/Config/version.mk'.
         Trying pattern rule with stem 'version.mk'.
         Trying implicit prerequisite '../../tarball/Config/version.mk,v'.
         Trying pattern rule with stem 'version.mk'.
         Trying implicit prerequisite '../../tarball/Config/RCS/version.mk,v'.
         Trying pattern rule with stem 'version.mk'.
         Trying implicit prerequisite '../../tarball/Config/RCS/version.mk'.
         Trying pattern rule with stem 'version.mk'.
         Trying implicit prerequisite '../../tarball/Config/s.version.mk'.
         Trying pattern rule with stem 'version.mk'.
         Trying implicit prerequisite '../../tarball/Config/SCCS/s.version.mk'.
         No implicit rule found for '../../tarball/Config/version.mk'.
         Finished prerequisites of target file '../../tarball/Config/version.mk'.
        No need to remake target '../../tarball/Config/version.mk'.
       Finished prerequisites of target file 'version.yo'.
       Prerequisite '../../tarball/Config/version.mk' is older than target 'version.yo'.
      No need to remake target 'version.yo'; using VPATH name '../../tarball/Doc/version.yo'.
      Considering target file 'zmacros.yo'.
       Looking for an implicit rule for 'zmacros.yo'.
       Trying pattern rule with stem 'zmacros.yo'.
       Trying implicit prerequisite 'zmacros.yo,v'.
       Trying pattern rule with stem 'zmacros.yo'.
       Trying implicit prerequisite 'RCS/zmacros.yo,v'.
       Trying pattern rule with stem 'zmacros.yo'.
       Trying implicit prerequisite 'RCS/zmacros.yo'.
       Trying pattern rule with stem 'zmacros.yo'.
       Trying implicit prerequisite 's.zmacros.yo'.
       Trying pattern rule with stem 'zmacros.yo'.
       Trying implicit prerequisite 'SCCS/s.zmacros.yo'.
       No implicit rule found for 'zmacros.yo'.
       Finished prerequisites of target file 'zmacros.yo'.
      No need to remake target 'zmacros.yo'; using VPATH name '../../tarball/Doc/zmacros.yo'.
      Considering target file 'zman.yo'.
       Looking for an implicit rule for 'zman.yo'.
       Trying pattern rule with stem 'zman.yo'.
       Trying implicit prerequisite 'zman.yo,v'.
       Trying pattern rule with stem 'zman.yo'.
       Trying implicit prerequisite 'RCS/zman.yo,v'.
       Trying pattern rule with stem 'zman.yo'.
       Trying implicit prerequisite 'RCS/zman.yo'.
       Trying pattern rule with stem 'zman.yo'.
       Trying implicit prerequisite 's.zman.yo'.
       Trying pattern rule with stem 'zman.yo'.
       Trying implicit prerequisite 'SCCS/s.zman.yo'.
       No implicit rule found for 'zman.yo'.
       Finished prerequisites of target file 'zman.yo'.
      No need to remake target 'zman.yo'; using VPATH name '../../tarball/Doc/zman.yo'.
      Considering target file 'Zsh/intro.yo'.
       Looking for an implicit rule for 'Zsh/intro.yo'.
       Trying pattern rule with stem 'intro.yo'.
       Trying implicit prerequisite 'Zsh/intro.yo,v'.
       Trying pattern rule with stem 'intro.yo'.
       Trying implicit prerequisite 'Zsh/RCS/intro.yo,v'.
       Trying pattern rule with stem 'intro.yo'.
       Trying implicit prerequisite 'Zsh/RCS/intro.yo'.
       Trying pattern rule with stem 'intro.yo'.
       Trying implicit prerequisite 'Zsh/s.intro.yo'.
       Trying pattern rule with stem 'intro.yo'.
       Trying implicit prerequisite 'Zsh/SCCS/s.intro.yo'.
       No implicit rule found for 'Zsh/intro.yo'.
       Finished prerequisites of target file 'Zsh/intro.yo'.
      No need to remake target 'Zsh/intro.yo'; using VPATH name '../../tarball/Doc/Zsh/intro.yo'.
      Considering target file 'Zsh/metafaq.yo'.
       Looking for an implicit rule for 'Zsh/metafaq.yo'.
       Trying pattern rule with stem 'metafaq.yo'.
       Trying implicit prerequisite 'Zsh/metafaq.yo,v'.
       Trying pattern rule with stem 'metafaq.yo'.
       Trying implicit prerequisite 'Zsh/RCS/metafaq.yo,v'.
       Trying pattern rule with stem 'metafaq.yo'.
       Trying implicit prerequisite 'Zsh/RCS/metafaq.yo'.
       Trying pattern rule with stem 'metafaq.yo'.
       Trying implicit prerequisite 'Zsh/s.metafaq.yo'.
       Trying pattern rule with stem 'metafaq.yo'.
       Trying implicit prerequisite 'Zsh/SCCS/s.metafaq.yo'.
       No implicit rule found for 'Zsh/metafaq.yo'.
       Finished prerequisites of target file 'Zsh/metafaq.yo'.
      No need to remake target 'Zsh/metafaq.yo'; using VPATH name '../../tarball/Doc/Zsh/metafaq.yo'.
      Considering target file 'Zsh/invoke.yo'.
       Looking for an implicit rule for 'Zsh/invoke.yo'.
       Trying pattern rule with stem 'invoke.yo'.
       Trying implicit prerequisite 'Zsh/invoke.yo,v'.
       Trying pattern rule with stem 'invoke.yo'.
       Trying implicit prerequisite 'Zsh/RCS/invoke.yo,v'.
       Trying pattern rule with stem 'invoke.yo'.
       Trying implicit prerequisite 'Zsh/RCS/invoke.yo'.
       Trying pattern rule with stem 'invoke.yo'.
       Trying implicit prerequisite 'Zsh/s.invoke.yo'.
       Trying pattern rule with stem 'invoke.yo'.
       Trying implicit prerequisite 'Zsh/SCCS/s.invoke.yo'.
       No implicit rule found for 'Zsh/invoke.yo'.
       Finished prerequisites of target file 'Zsh/invoke.yo'.
      No need to remake target 'Zsh/invoke.yo'; using VPATH name '../../tarball/Doc/Zsh/invoke.yo'.
      Considering target file 'Zsh/files.yo'.
       Looking for an implicit rule for 'Zsh/files.yo'.
       Trying pattern rule with stem 'files.yo'.
       Trying implicit prerequisite 'Zsh/files.yo,v'.
       Trying pattern rule with stem 'files.yo'.
       Trying implicit prerequisite 'Zsh/RCS/files.yo,v'.
       Trying pattern rule with stem 'files.yo'.
       Trying implicit prerequisite 'Zsh/RCS/files.yo'.
       Trying pattern rule with stem 'files.yo'.
       Trying implicit prerequisite 'Zsh/s.files.yo'.
       Trying pattern rule with stem 'files.yo'.
       Trying implicit prerequisite 'Zsh/SCCS/s.files.yo'.
       No implicit rule found for 'Zsh/files.yo'.
       Finished prerequisites of target file 'Zsh/files.yo'.
      No need to remake target 'Zsh/files.yo'; using VPATH name '../../tarball/Doc/Zsh/files.yo'.
      Considering target file 'Zsh/filelist.yo'.
       Looking for an implicit rule for 'Zsh/filelist.yo'.
       Trying pattern rule with stem 'filelist.yo'.
       Trying implicit prerequisite 'Zsh/filelist.yo,v'.
       Trying pattern rule with stem 'filelist.yo'.
       Trying implicit prerequisite 'Zsh/RCS/filelist.yo,v'.
       Trying pattern rule with stem 'filelist.yo'.
       Trying implicit prerequisite 'Zsh/RCS/filelist.yo'.
       Trying pattern rule with stem 'filelist.yo'.
       Trying implicit prerequisite 'Zsh/s.filelist.yo'.
       Trying pattern rule with stem 'filelist.yo'.
       Trying implicit prerequisite 'Zsh/SCCS/s.filelist.yo'.
       No implicit rule found for 'Zsh/filelist.yo'.
       Finished prerequisites of target file 'Zsh/filelist.yo'.
      No need to remake target 'Zsh/filelist.yo'; using VPATH name '../../tarball/Doc/Zsh/filelist.yo'.
      Pruning file '../../tarball/Doc/Zsh/filelist.yo'.
      Considering target file 'Zsh/seealso.yo'.
       Looking for an implicit rule for 'Zsh/seealso.yo'.
       Trying pattern rule with stem 'seealso.yo'.
       Trying implicit prerequisite 'Zsh/seealso.yo,v'.
       Trying pattern rule with stem 'seealso.yo'.
       Trying implicit prerequisite 'Zsh/RCS/seealso.yo,v'.
       Trying pattern rule with stem 'seealso.yo'.
       Trying implicit prerequisite 'Zsh/RCS/seealso.yo'.
       Trying pattern rule with stem 'seealso.yo'.
       Trying implicit prerequisite 'Zsh/s.seealso.yo'.
       Trying pattern rule with stem 'seealso.yo'.
       Trying implicit prerequisite 'Zsh/SCCS/s.seealso.yo'.
       No implicit rule found for 'Zsh/seealso.yo'.
       Finished prerequisites of target file 'Zsh/seealso.yo'.
      No need to remake target 'Zsh/seealso.yo'; using VPATH name '../../tarball/Doc/Zsh/seealso.yo'.
      Considering target file 'Zsh/compat.yo'.
       Looking for an implicit rule for 'Zsh/compat.yo'.
       Trying pattern rule with stem 'compat.yo'.
       Trying implicit prerequisite 'Zsh/compat.yo,v'.
       Trying pattern rule with stem 'compat.yo'.
       Trying implicit prerequisite 'Zsh/RCS/compat.yo,v'.
       Trying pattern rule with stem 'compat.yo'.
       Trying implicit prerequisite 'Zsh/RCS/compat.yo'.
       Trying pattern rule with stem 'compat.yo'.
       Trying implicit prerequisite 'Zsh/s.compat.yo'.
       Trying pattern rule with stem 'compat.yo'.
       Trying implicit prerequisite 'Zsh/SCCS/s.compat.yo'.
       No implicit rule found for 'Zsh/compat.yo'.
       Finished prerequisites of target file 'Zsh/compat.yo'.
      No need to remake target 'Zsh/compat.yo'; using VPATH name '../../tarball/Doc/Zsh/compat.yo'.
      Considering target file 'Zsh/restricted.yo'.
       Looking for an implicit rule for 'Zsh/restricted.yo'.
       Trying pattern rule with stem 'restricted.yo'.
       Trying implicit prerequisite 'Zsh/restricted.yo,v'.
       Trying pattern rule with stem 'restricted.yo'.
       Trying implicit prerequisite 'Zsh/RCS/restricted.yo,v'.
       Trying pattern rule with stem 'restricted.yo'.
       Trying implicit prerequisite 'Zsh/RCS/restricted.yo'.
       Trying pattern rule with stem 'restricted.yo'.
       Trying implicit prerequisite 'Zsh/s.restricted.yo'.
       Trying pattern rule with stem 'restricted.yo'.
       Trying implicit prerequisite 'Zsh/SCCS/s.restricted.yo'.
       No implicit rule found for 'Zsh/restricted.yo'.
       Finished prerequisites of target file 'Zsh/restricted.yo'.
      No need to remake target 'Zsh/restricted.yo'; using VPATH name '../../tarball/Doc/Zsh/restricted.yo'.
     Finished prerequisites of target file 'zsh.1'.
     Prerequisite '../../tarball/Doc/zsh.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/version.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/zmacros.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/zman.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/Zsh/intro.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/Zsh/metafaq.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/Zsh/invoke.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/Zsh/files.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/Zsh/filelist.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/Zsh/filelist.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/Zsh/seealso.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/Zsh/compat.yo' is older than target 'zsh.1'.
     Prerequisite '../../tarball/Doc/Zsh/restricted.yo' is older than target 'zsh.1'.
    No need to remake target 'zsh.1'.
]]]

Re: 'make install' fails when build directory is differnet from that of source tree.

[Daniel Shahaf]

Daniel Shahaf wrote on Fri, 29 May 2020 22:54 +0000:
> +++ b/configure.ac
> @@ -3178,6 +3178,13 @@ AC_CONFIG_COMMANDS([stamp-h], [echo >stamp-h])
> +dnl Copy pre-built man pages, for tarball out-of-tree builds.
> +for manpage in ${srcdir}/Doc/*.1; do
> +  if test x"$manpage" != x*"*"*; then
> +    cp ${manpage} ./Doc/
> +  fi
> +done

In second thought, this is going to break _in-tree_ tarball builds,
isn't it? —

% cp ./foo ./
cp: './foo' and './foo' are the same file
% 

So, how'd I check whether the build is in-tree or out-of-tree?  Can I
just compare $abs_srcdir to $abs_builddir, or should I touch a file in
one and see whether it appears in the other?

Cheers,

Daniel

Re: 'make install' fails when build directory is differnet from that of source tree.

[Yasuhiro KIMURA]

From: Daniel Shahaf <d.s@daniel.shahaf.name>
Subject: Re: 'make install' fails when build directory is differnet from that of source tree.
Date: Fri, 29 May 2020 22:54:01 +0000

> I would prefer to fix this by making tarball builds copy the *.1 files
> from the source directory («$(sdir)» in Doc/Makefile) to the build
> directory.  This would minimize the difference: tarball builds would
> compile the man pages using yodl and git builds would copy the
> pregenerated ones.  The following patch does this.
> 
> [[[
> diff --git a/configure.ac b/configure.ac
> index e78ebf86b..c72fe72b2 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -3178,6 +3178,13 @@ AC_CONFIG_COMMANDS([stamp-h], [echo >stamp-h])
>  
>  AC_OUTPUT
>  
> +dnl Copy pre-built man pages, for tarball out-of-tree builds.
> +for manpage in ${srcdir}/Doc/*.1; do
> +  if test x"$manpage" != x*"*"*; then
> +    cp ${manpage} ./Doc/
> +  fi
> +done
> +
>  eval "zshbin1=${bindir}"
>  eval "zshbin2=${zshbin1}"
>  eval "zshman1=${mandir}"
> ]]]
> 
> If you could test it (by applying it to «configure»), that'd be great.

Thank you for patch. I tested it and there are 3 items to report.

[Testing patch with source from tarball]

I applied the patch to source from tarball with following steps.

cd /tmp/zsh/source
tar xfpvJ /somewhere/zsh-5.8.tar.xz
cd zsh-5.8
patch -p1 -s -i /path/to/patch
./Util/preconfig

Then 'make install' succeeds with both in-tree and out-of-tree
builds. However, as you mentioned in other mail './configure' of
in-tree build complains as following.

cp: './Doc/zsh.1' and './Doc/zsh.1' are the same file
cp: './Doc/zshall.1' and './Doc/zshall.1' are the same file
cp: './Doc/zshbuiltins.1' and './Doc/zshbuiltins.1' are the same file
cp: './Doc/zshcalsys.1' and './Doc/zshcalsys.1' are the same file
cp: './Doc/zshcompctl.1' and './Doc/zshcompctl.1' are the same file
cp: './Doc/zshcompsys.1' and './Doc/zshcompsys.1' are the same file
cp: './Doc/zshcompwid.1' and './Doc/zshcompwid.1' are the same file
cp: './Doc/zshcontrib.1' and './Doc/zshcontrib.1' are the same file
cp: './Doc/zshexpn.1' and './Doc/zshexpn.1' are the same file
cp: './Doc/zshmisc.1' and './Doc/zshmisc.1' are the same file
cp: './Doc/zshmodules.1' and './Doc/zshmodules.1' are the same file
cp: './Doc/zshoptions.1' and './Doc/zshoptions.1' are the same file
cp: './Doc/zshparam.1' and './Doc/zshparam.1' are the same file
cp: './Doc/zshroadmap.1' and './Doc/zshroadmap.1' are the same file
cp: './Doc/zshtcpsys.1' and './Doc/zshtcpsys.1' are the same file
cp: './Doc/zshzftpsys.1' and './Doc/zshzftpsys.1' are the same file
cp: './Doc/zshzle.1' and './Doc/zshzle.1' are the same file 

[Testing patch with source from git repository]

I applied the patch to source from git repository with following
steps.

cd /tmp/zsh/source
cd zsh
git clone https://git.code.sf.net/p/zsh/code zsh
git checkout zsh-5.8
git apply /path/to/patch
./Util/preconfig

As with tarball case, 'make install' succeeds with both in-tree and
out-of-ree builds. In this case, however, './configure' of both builds
complains such as following.

cp: cannot stat '/tmp/zsh/source/zsh/Doc/*.1': No such file or directory

[Similar problem with help files]

Similar problem happens with 'make install.runhelp' when build is
out-of-source and source is from tarball.

cd /tmp/zsh/source
tar xfpvJ /somewhere/zsh-5.8.tar.xz
cd /tmp/zsh/build
/tmp/zsh/source/zsh-5.8/configure --srcdir=/tmp/zsh/source/zsh-5.8
make DESTDIR=/tmp/zsh/install install.runhelp

These steps result in following error.

make[1]: Entering directory '/tmp/zsh/build/Doc'
if test x"/usr/local/share/zsh/5.8/help" != x""; then \
    /bin/sh /tmp/zsh/source/zsh-5.8/mkinstalldirs /tmp/zsh/install/usr/local/share/zsh/5.8/help; \
    /bin/install -c -m 644 help/* /tmp/zsh/install/usr/local/share/zsh/5.8/help; \
    while read from to; do \
        rm -f /tmp/zsh/install/usr/local/share/zsh/5.8/help/$to || : ; \
        ln -s $from /tmp/zsh/install/usr/local/share/zsh/5.8/help/$to; \
    done < help.txt; \
fi
/bin/install: cannot stat 'help/*': No such file or directory
/bin/sh: 4: cannot open help.txt: No such file
make[1]: *** [Makefile:482: install.runhelp] Error 2
make[1]: Leaving directory '/tmp/zsh/build/Doc'
make: *** [Makefile:226: install.runhelp] Error 2

Best Regards.

---
Yasuhiro KIMURA

Re: 'make install' fails when build directory is differnet from that of source tree.

[Daniel Shahaf]

Yasuhiro KIMURA wrote on Sat, 30 May 2020 18:05 +0900:
> From: Daniel Shahaf <d.s@daniel.shahaf.name>
> Subject: Re: 'make install' fails when build directory is differnet from that of source tree.
> Date: Fri, 29 May 2020 22:54:01 +0000
> 
> > I would prefer to fix this by making tarball builds copy the *.1 files
> > from the source directory («$(sdir)» in Doc/Makefile) to the build
> > directory.  This would minimize the difference: tarball builds would
> > compile the man pages using yodl and git builds would copy the
> > pregenerated ones.  The following patch does this.
> > 
> > [[[
> > diff --git a/configure.ac b/configure.ac
> > index e78ebf86b..c72fe72b2 100644
> > --- a/configure.ac
> > +++ b/configure.ac
> > @@ -3178,6 +3178,13 @@ AC_CONFIG_COMMANDS([stamp-h], [echo >stamp-h])
> >  
> >  AC_OUTPUT
> >  
> > +dnl Copy pre-built man pages, for tarball out-of-tree builds.
> > +for manpage in ${srcdir}/Doc/*.1; do
> > +  if test x"$manpage" != x*"*"*; then
> > +    cp ${manpage} ./Doc/
> > +  fi
> > +done
> > +
> >  eval "zshbin1=${bindir}"
> >  eval "zshbin2=${zshbin1}"
> >  eval "zshman1=${mandir}"
> > ]]]
> > 
> > If you could test it (by applying it to «configure»), that'd be great.  
> 
> Thank you for patch. I tested it and there are 3 items to report.
> 

Thanks for testing.  More below.

> [Testing patch with source from tarball]
> 
> I applied the patch to source from tarball with following steps.
> 
> cd /tmp/zsh/source
> tar xfpvJ /somewhere/zsh-5.8.tar.xz
> cd zsh-5.8
> patch -p1 -s -i /path/to/patch
> ./Util/preconfig
> 
> Then 'make install' succeeds with both in-tree and out-of-tree
> builds. However, as you mentioned in other mail './configure' of
> in-tree build complains as following.
> 
> cp: './Doc/zsh.1' and './Doc/zsh.1' are the same file
> cp: './Doc/zshall.1' and './Doc/zshall.1' are the same file
⋮
> cp: './Doc/zshzle.1' and './Doc/zshzle.1' are the same file 

I'm attaching a revised patch which should resolve this.

> [Testing patch with source from git repository]
> 
> I applied the patch to source from git repository with following
> steps.
> 
> cd /tmp/zsh/source
> cd zsh
> git clone https://git.code.sf.net/p/zsh/code zsh
> git checkout zsh-5.8
> git apply /path/to/patch
> ./Util/preconfig
> 
> As with tarball case, 'make install' succeeds with both in-tree and
> out-of-ree builds. In this case, however, './configure' of both builds
> complains such as following.
> 
> cp: cannot stat '/tmp/zsh/source/zsh/Doc/*.1': No such file or directory

The revised patch attached should resolve this, too.

I'll reply to the third issue separately.

Thanks for the thorough testing,

Daniel

diff --git a/configure.ac b/configure.ac
index e78ebf86b..eed458849 100644
--- a/configure.ac
+++ b/configure.ac
@@ -3178,6 +3178,13 @@ AC_CONFIG_COMMANDS([stamp-h], [echo >stamp-h])
 
 AC_OUTPUT
 
+dnl Copy pre-built man pages, for tarball out-of-tree builds.
+for manpage in `cd ${srcdir} && echo Doc/*.1`; do
+  if test x"$manpage" != x"Doc/*.1" && ! test -e "${manpage}"; then
+    cp ${srcdir}/${manpage} ./Doc/
+  fi
+done
+
 eval "zshbin1=${bindir}"
 eval "zshbin2=${zshbin1}"
 eval "zshman1=${mandir}"

Re: 'make install' fails when build directory is differnet from that of source tree.

[Daniel Shahaf]

Yasuhiro KIMURA wrote on Sat, 30 May 2020 18:05 +0900:
> [Similar problem with help files]
> 
> Similar problem happens with 'make install.runhelp' when build is
> out-of-source and source is from tarball.
> 
> cd /tmp/zsh/source
> tar xfpvJ /somewhere/zsh-5.8.tar.xz
> cd /tmp/zsh/build
> /tmp/zsh/source/zsh-5.8/configure --srcdir=/tmp/zsh/source/zsh-5.8
> make DESTDIR=/tmp/zsh/install install.runhelp
> 
> These steps result in following error.
> 
> make[1]: Entering directory '/tmp/zsh/build/Doc'
> if test x"/usr/local/share/zsh/5.8/help" != x""; then \
>     /bin/sh /tmp/zsh/source/zsh-5.8/mkinstalldirs /tmp/zsh/install/usr/local/share/zsh/5.8/help; \
>     /bin/install -c -m 644 help/* /tmp/zsh/install/usr/local/share/zsh/5.8/help; \
>     while read from to; do \
>         rm -f /tmp/zsh/install/usr/local/share/zsh/5.8/help/$to || : ; \
>         ln -s $from /tmp/zsh/install/usr/local/share/zsh/5.8/help/$to; \
>     done < help.txt; \
> fi
> /bin/install: cannot stat 'help/*': No such file or directory
> /bin/sh: 4: cannot open help.txt: No such file
> make[1]: *** [Makefile:482: install.runhelp] Error 2
> make[1]: Leaving directory '/tmp/zsh/build/Doc'
> make: *** [Makefile:226: install.runhelp] Error 2
> 

Thanks for the report.

This looks like another effect of the same series of Doc/Makefile.in commits:

    76a16498d 43692: Generate man pages in the build dir, rather than the source dir.
    b2ddd9717 43404 (after 43392): Fix 'make install' in out-of-tree builds
    85ae84e89 43392: dist: Generate doc/help/* and doc/help.txt in builddir rather than srcdir.

For me, the patch I just posted (the second revision, from today) fixes
this issue as well.  I also confirmed that `cd $DESTDIR/$prefix/share/zsh/5.8/help && find`
and `cd /usr/share/zsh/help && find` [the latter being my distro's
build] produce identical outputs.

Do let us know if anything else seems amiss.

Cheers,

Daniel

> Best Regards.
> 
> ---
> Yasuhiro KIMURA

Re: 'make install' fails when build directory is differnet from that of source tree.

[Yasuhiro KIMURA]

From: Daniel Shahaf <d.s@daniel.shahaf.name>
Subject: Re: 'make install' fails when build directory is differnet from that of source tree.
Date: Sun, 31 May 2020 01:39:19 +0000

> The revised patch attached should resolve this, too.

I tried new patch and confirmed that now in each case 'make install'
succeeds and './configure' doesn't complain anything.

> I'll reply to the third issue separately.

As for 3rd issue I'm still testing. So I also will report the result
separately.

---
Yasuhiro KIMURA

Re: 'make install' fails when build directory is differnet from that of source tree.

[Yasuhiro KIMURA]

From: Daniel Shahaf <d.s@daniel.shahaf.name>
Subject: Re: 'make install' fails when build directory is differnet from that of source tree.
Date: Sun, 31 May 2020 01:57:39 +0000

> Thanks for the report.
> 
> This looks like another effect of the same series of Doc/Makefile.in commits:
> 
>     76a16498d 43692: Generate man pages in the build dir, rather than the source dir.
>     b2ddd9717 43404 (after 43392): Fix 'make install' in out-of-tree builds
>     85ae84e89 43392: dist: Generate doc/help/* and doc/help.txt in builddir rather than srcdir.
> 
> For me, the patch I just posted (the second revision, from today) fixes
> this issue as well.  I also confirmed that `cd $DESTDIR/$prefix/share/zsh/5.8/help && find`
> and `cd /usr/share/zsh/help && find` [the latter being my distro's
> build] produce identical outputs.
> 
> Do let us know if anything else seems amiss.

I applied new patch to tarball source with following step.

1. cd /tmp/zsh/source
2. tar xfpvJ /somewhere/zsh-5.8.tar.xz
3. cd zsh-5.8
4. ./Util/preconfig

And tried both in-tree and out-of-tree build. Then 'make
install.runhelp' succeeded with both cases. But there are difference
about output of 'make instlal.runhelp'.

in-tree build produced following output.

----------------------------------------------------------------------
yasu@rolling-vm-debian2[2167]% make DESTDIR=/tmp/zsh/install install.runhelp
make[1]: Entering directory '/tmp/zsh/source/zsh-5.8/Doc'
if test x"/usr/local/share/zsh/5.8/help" != x""; then \
    /bin/sh ../mkinstalldirs /tmp/zsh/install/usr/local/share/zsh/5.8/help; \
    /bin/install -c -m 644 help/* /tmp/zsh/install/usr/local/share/zsh/5.8/help; \
    while read from to; do \
        rm -f /tmp/zsh/install/usr/local/share/zsh/5.8/help/$to || : ; \
        ln -s $from /tmp/zsh/install/usr/local/share/zsh/5.8/help/$to; \
    done < help.txt; \
fi
mkdir -m 755 -p -- /tmp/zsh/install/usr/local/share/zsh/5.8/help
make[1]: Leaving directory '/tmp/zsh/source/zsh-5.8/Doc'
yasu@rolling-vm-debian2[2168]%
----------------------------------------------------------------------

And out-of-tree build produced following output.

----------------------------------------------------------------------
yasu@rolling-vm-debian2[2161]% make DESTDIR=/tmp/zsh/install install.runhelp
make[1]: Entering directory '/tmp/zsh/build/Doc'
perl /tmp/zsh/source/zsh-5.8/Util/helpfiles \
    zshbuiltins.1 help help.txt \
    || rm -f help.txt help/*
helpfiles: attempting man zshbuiltins.1 >help/man.tmp
helpfiles: attempting col -bx <help/man.tmp >help/col.tmp
if test x"/usr/local/share/zsh/5.8/help" != x""; then \
    /bin/sh /tmp/zsh/source/zsh-5.8/mkinstalldirs /tmp/zsh/install/usr/local/share/zsh/5.8/help; \
    /bin/install -c -m 644 help/* /tmp/zsh/install/usr/local/share/zsh/5.8/help; \
    while read from to; do \
        rm -f /tmp/zsh/install/usr/local/share/zsh/5.8/help/$to || : ; \
        ln -s $from /tmp/zsh/install/usr/local/share/zsh/5.8/help/$to; \
    done < help.txt; \
fi
mkdir -m 755 -p -- /tmp/zsh/install/usr/local/share/zsh/5.8/help
make[1]: Leaving directory '/tmp/zsh/build/Doc'
yasu@rolling-vm-debian2[2162]%
----------------------------------------------------------------------

I you compare these outputs, there are following extra output in the
later.

----------------------------------------------------------------------
perl /tmp/zsh/source/zsh-5.8/Util/helpfiles \
    zshbuiltins.1 help help.txt \
    || rm -f help.txt help/*
----------------------------------------------------------------------

This means that on out-of-source build help files are generated before
installing them and that out-of-source build has extra dependency on
perl which isn't necessary on in-tree build.

Usually it doesn't matter since perl is included in normal install of
most Unix-like OSes. But, for example, FreeBSD doesn't include perl in
its base system and user sepalately needs to install it by using
FreeBSD ports/packages collection if he want to use it. So if you make
out-of-source build with tarball source on base system only FreeBSD
environment, then 'make install.runhelp' will fail with same error as
previously reported.

Actually I created tarball source that the patch is applied with above
steps, made base sytem only install of FreeBSD 12.1-RELEASE amd64,
copied created source to the environment and tried out-of-tree build.
Then 'make install.runhelp' resulted in following error.

----------------------------------------------------------------------
root@rolling-vm-freebsd5:/tmp/zsh/build # make DESTDIR=/tmp/zsh/install install.runhelp
perl /tmp/zsh/source/zsh-5.8/Util/helpfiles  zshbuiltins.1 help help.txt  || rm -f help.txt help/*
/bin/sh: perl: not found
if test x"/usr/local/share/zsh/5.8/help" != x""; then  /bin/sh /tmp/zsh/source/zsh-5.8/mkinstalldirs /tmp/zsh/install/usr/local/share/zsh/5.8/help;  /usr/bin/install -c -m 644 help/* /tmp/zsh/install/usr/local/share/zsh/5.8/help;  while read from to; do  rm -f /tmp/zsh/install/usr/local/share/zsh/5.8/help/$to || : ;  ln -s $from /tmp/zsh/install/usr/local/share/zsh/5.8/help/$to;  done < help.txt;  fi
mkdir /tmp/zsh/install/usr
chmod 755 /tmp/zsh/install/usr
mkdir /tmp/zsh/install/usr/local
chmod 755 /tmp/zsh/install/usr/local
mkdir /tmp/zsh/install/usr/local/share
chmod 755 /tmp/zsh/install/usr/local/share
mkdir /tmp/zsh/install/usr/local/share/zsh
chmod 755 /tmp/zsh/install/usr/local/share/zsh
mkdir /tmp/zsh/install/usr/local/share/zsh/5.8
chmod 755 /tmp/zsh/install/usr/local/share/zsh/5.8
mkdir /tmp/zsh/install/usr/local/share/zsh/5.8/help
chmod 755 /tmp/zsh/install/usr/local/share/zsh/5.8/help
install: help/*: No such file or directory
*** Error code 71

Stop.
make[1]: stopped in /tmp/zsh/build/Doc
*** Error code 1

Stop.
make: stopped in /tmp/zsh/build
root@rolling-vm-freebsd5:/tmp/zsh/build # 
----------------------------------------------------------------------

---
Yasuhiro KIMURA

Re: 'make install' fails when build directory is differnet from that of source tree.

[Daniel Shahaf]

Yasuhiro KIMURA wrote on Sun, 31 May 2020 16:46 +0900:
> in-tree build produced following output.
⋮
> And out-of-tree build produced following output.
⋮:
> I you compare these outputs, there are following extra output in the
> later.
> 
> ----------------------------------------------------------------------
> perl /tmp/zsh/source/zsh-5.8/Util/helpfiles \
>     zshbuiltins.1 help help.txt \
>     || rm -f help.txt help/*
> ----------------------------------------------------------------------

Thanks again for the thorough investigation and clear presentation.  The
cause of this difference seems to be straightforward: the cp(1)
invocation added by the patch creates [builddir]/Doc/zshbultins.1 with
an mtime of "now", which causes the following rule to run:

   204	help.txt: zshbuiltins.1
   205		@-rm -f $@ help/*
   206		perl $(sdir_top)/Util/helpfiles \
   207		    zshbuiltins.1 help help.txt \
   208		    || rm -f help.txt help/*

I suspect that extending the last patch (workers/45943) to (1) copy
Doc/help/ and Doc/help.txt [due to the aforementioned workers/43392],
and (2) s/cp/cp -p/, would address this.  However, I'm afraid I don't
currently have time to implement and test this.  If you have time to do
so, you'll be very welcome; if not, I'll implement that and post the
patch when I have a minute.

Cheers,

Daniel

Re: 'make install' fails when build directory is differnet from that of source tree.

[Yasuhiro KIMURA]

From: Daniel Shahaf <d.s@daniel.shahaf.name>
Subject: Re: 'make install' fails when build directory is differnet from that of source tree.
Date: Mon, 1 Jun 2020 08:03:42 +0000

> I suspect that extending the last patch (workers/45943) to (1) copy
> Doc/help/ and Doc/help.txt [due to the aforementioned workers/43392],
> and (2) s/cp/cp -p/, would address this.  However, I'm afraid I don't
> currently have time to implement and test this.  If you have time to do
> so, you'll be very welcome; if not, I'll implement that and post the
> patch when I have a minute.

I changed your patch as following.

----------------------------------------------------------------------
diff --git a/configure.ac b/configure.ac
index e78ebf86b..995f010b9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -3178,6 +3178,22 @@ AC_CONFIG_COMMANDS([stamp-h], [echo >stamp-h])
 
 AC_OUTPUT
 
+dnl Copy pre-built man pages and help files, for tarball out-of-tree builds.
+for manpage in `cd ${srcdir} && echo Doc/*.1`; do
+  if test x"$manpage" != x"Doc/*.1" && ! test -e "${manpage}"; then
+    cp ${srcdir}/${manpage} ./Doc/
+  fi
+done
+if ! test -e Doc/help.txt; then
+  cp ${srcdir}/Doc/help.txt ./Doc/
+fi
+mkdir -p ./Doc/help
+for helpfile in `cd ${srcdir} && echo Doc/help/*`; do
+  if test x"$helpfile" != x"Doc/help/*" && ! test -e "${helpfile}"; then
+    cp ${srcdir}/${helpfile} ./Doc/help/
+  fi
+done
+
 eval "zshbin1=${bindir}"
 eval "zshbin2=${zshbin1}"
 eval "zshman1=${mandir}"
----------------------------------------------------------------------

Then tarball out-of-tree 'make install.runhelp' succeeds on base
system only FreeBSD 12.1 amd.

---
Yasuhiro KIMURA

Re: 'make install' fails when build directory is differnet from that of source tree.

[Daniel Shahaf]

Yasuhiro KIMURA wrote on Tue, 02 Jun 2020 03:23 +0900:
> From: Daniel Shahaf <d.s@daniel.shahaf.name>
> Subject: Re: 'make install' fails when build directory is differnet from that of source tree.
> Date: Mon, 1 Jun 2020 08:03:42 +0000
> 
> > I suspect that extending the last patch (workers/45943) to (1) copy
> > Doc/help/ and Doc/help.txt [due to the aforementioned workers/43392],
> > and (2) s/cp/cp -p/, would address this.  However, I'm afraid I don't
> > currently have time to implement and test this.  If you have time to do
> > so, you'll be very welcome; if not, I'll implement that and post the
> > patch when I have a minute.  
> 
> I changed your patch as following.
> 
> ----------------------------------------------------------------------
⋮
> ----------------------------------------------------------------------
> 
> Then tarball out-of-tree 'make install.runhelp' succeeds on base
> system only FreeBSD 12.1 amd.

Thanks!  Merged.

Daniel