Zsh requires Bash? 2002 revisited

Zsh requires Bash? 2002 revisited

[Jim]

[-- Attachment #1: Type: text/plain, Size: 983 bytes --]

Hi all,

After having to rebuild my system, I was unable to build zsh from
source.  Web searches were getting me no where. Checking man
pages, guide, faq, and anything else I could think of reading, again
nothing.  Since I have another life, besides zsh, I would come back
to this as time permitted. I finally found from the 2002 archives:

http://www.zsh.org/mla/workers/2002/msg00922.html
http://www.zsh.org/mla/workers/2002/msg00924.html  Bart's reply.

Not exactly the same as back in 2002, but it was the key for me.
I had been testing different shells to be used as the default "sh" on
my system. The last one I was testing before starting zsh builds
was "posh". Out of the six shells(bash,dash,ksh,mksh,posh,zsh) I
have installed, posh is the only shell that fails in the same way as
described in the archives. Bad luck? Murphy's Law?

Not sure what can or should be done.  I thought I would at least
report it. I just wish it had been easier to find.

Sincerely,

Jim Murphy

Re: Zsh requires Bash? 2002 revisited

[Bart Schaefer]

On Sat, Jan 6, 2018 at 5:47 AM, Jim <linux.tech.guy@gmail.com> wrote:
>
> I had been testing different shells to be used as the default "sh" on
> my system.

Of course the real issue is that autoconf depends on bash, which is
not entirely surprising given that those are both GNU components.

If this is failing in one of our scripts, as opposed to in a stock
autoconf test, we might be able to fix it; since it's "not exactly the
same" would you provide more details?

Re: Zsh requires Bash? 2002 revisited

[Jim]

[-- Attachment #1: Type: text/plain, Size: 3011 bytes --]

On Sat, Jan 6, 2018 at 11:23 AM, Bart Schaefer <schaefer@brasslantern.com>
wrote:

> On Sat, Jan 6, 2018 at 5:47 AM, Jim <linux.tech.guy@gmail.com> wrote:
> >
> > I had been testing different shells to be used as the default "sh" on
> > my system.
>
> If this is failing in one of our scripts, as opposed to in a stock
> autoconf test, we might be able to fix it; since it's "not exactly the
> same" would you provide more details?
>

As I said I was testing different shells, it was by chance that /bin/sh and
/bin/posh were
linked when I started the build. I also tested dash, ksh, mksh, zsh and of
course bash by
linking them to /bin/sh in turn. The only time the zsh build failed was
when /bin/sh and
/bin/posh were linked.

Last lines of make:

make[3]: Leaving directory '/home/friartek/work/zsh/Repository/code/Src/Zle'
make[2]: Leaving directory '/home/friartek/work/zsh/Repository/code/Src'
mv: cannot stat ‘stamp-modobjs.tmp’: No such file or directory
Updated `stamp-modobjs'.
make[2]: Entering directory '/home/friartek/work/zsh/Repository/code/Src'
gawk -f ../Src/makepro.awk main.c Src > main.syms
(echo '/* Generated automatically */'; sed -n '/^E/{s/^E//;p;}' <
main.syms) \
    > main.epro
(echo '/* Generated automatically */'; sed -n '/^L/{s/^L//;p;}' <
main.syms) \
    > `echo main.epro | sed 's/\.epro$/.pro/'`
gcc -c -I. -I../Src  -DHAVE_CONFIG_H -Wall -Wmissing-prototypes -O2 -o
main.o ./main.c
make[2]: Leaving directory '/home/friartek/work/zsh/Repository/code/Src'
make[2]: Entering directory '/home/friartek/work/zsh/Repository/code/Src'
make[2]: Leaving directory '/home/friartek/work/zsh/Repository/code/Src'
rm -f zsh
gcc -s  -rdynamic -o zsh main.o  `cat stamp-modobjs`   -lgdbm -L/usr/lib64
-lpcre -ldl -lncursesw -lrt -lm  -lc
cat: stamp-modobjs: No such file or directory
main.o: In function `main':
main.c:(.text.startup+0x1): undefined reference to `zsh_main'
collect2: error: ld returned 1 exit status
Makefile:227: recipe for target 'zsh' failed
make[1]: *** [zsh] Error 1
make[1]: Leaving directory '/home/friartek/work/zsh/Repository/code/Src'
Makefile:188: recipe for target 'all' failed
make: *** [all] Error 1

The common thing I saw, between this output and the output of make from
2002 was the file name "stamp-modobjs". Coincidence? So my next step
was to link sh to bash and verify that would allow the build to complete.
It did.

Sequence for each build of zsh-5.4.2-146-gc8e2f3c3d:

git reset --hard HEAD
git clean -f -d -x >/dev/null
Util/preconfig
configure <with and without options - no difference>
make

By the way, if there is a better way of preparing and doing builds, I'm
open to suggestions. Also building on a newly created branch.

Shell versions tested:

posh:  0.12.6
mksh: @(#)MIRBSD KSH R56 2017/08/24
dash: 0.5.9.1-r3
ksh: Version ABIJM 93v- 2014-12-24
bash: 4.4.12(1)-release
zsh: 5.3.1

Hope this was the info you needed. If you need more info, let me know.

Re: Zsh requires Bash? 2002 revisited

[Bart Schaefer]

On Sat, Jan 6, 2018 at 12:36 PM, Jim <linux.tech.guy@gmail.com> wrote:
>
> Last lines of make:

Given that you're not seeing shell syntax errors or the like, my guess
is that posh is mishandling the -e command line option (errexit) and
causing some branches of make to fail prematurely.  Or, posh has rm as
a built-in and that's mishandling the -f option.  Try building with
"make -k"?

If it's the "rm" problem, there are two places in Makefile.in
./Src/Makefile.in:    rm -f stamp-modobjs.tmp
./Src/Makefile.in:    rm -f stamp-modobjs stamp-modobjs.tmp
where you can try "[ -f stamp-modobjs.tmp ] && rm ..." (you'll have to
split the latter of the above into two separate "rm" calls).

If that still fails it might help to see a diff of config.log from
bash vs. posh, but it seems much more likely that it's make that's
giving up here.

Re: Zsh requires Bash? 2002 revisited

[Martijn Dekker]

Op 06-01-18 om 21:36 schreef Jim:
> As I said I was testing different shells, it was by chance that /bin/sh and
> /bin/posh were
> linked when I started the build.

You're right, autoconf scripts do not depend on bash and work fine on
any reasonably compliant POSIX shell, including recent zsh versions.

However, posh is non-compliant, bug-ridden, unmaintained, generally
broken, and should not be used. Your problem would be solved by deleting
posh and using dash for /bin/sh instead.

- M.

Re: Zsh requires Bash? 2002 revisited

[Stephane Chazelas]

2018-01-06 22:21:36 +0100, Martijn Dekker:
> Op 06-01-18 om 21:36 schreef Jim:
> > As I said I was testing different shells, it was by chance that /bin/sh and
> > /bin/posh were
> > linked when I started the build.
> 
> You're right, autoconf scripts do not depend on bash and work fine on
> any reasonably compliant POSIX shell, including recent zsh versions.
> 
> However, posh is non-compliant, bug-ridden, unmaintained, generally
> broken, and should not be used. Your problem would be solved by deleting
> posh and using dash for /bin/sh instead.
[...]

FWIW, with sh as dash, if I remove bash and ksh, then the
configure fails with

./configure:13185: functions: attempt to set associative array to scalar

IOW, it seems configure tries to rerun itself as bash, ksh or
zsh whichever is available when running as dash, but not when
running as posh, possibly because it detects that it is already
a variant of ksh? I've not looked in much more details.

Given that autoconf spends a lot of effort working around bugs
in various shell implementations, I suspect the problem is
coming from zsh specific code.

-- 
Stephane

Re: Zsh requires Bash? 2002 revisited

[Stephane Chazelas]

2018-01-06 22:21:36 +0100, Martijn Dekker:
[...]
> However, posh is non-compliant, bug-ridden, unmaintained, generally
> broken, and should not be used. Your problem would be solved by deleting
> posh and using dash for /bin/sh instead.
[...]

One of those bugs at least causes the wrong options for yodl to
be used and "-s" (strip) to be used.

$ posh -c 'case abc in *"b"*) echo yes; esac'
$ posh -c 'case abc in *b*) echo yes; esac'
yes

The patch below works around those, but it still doesn't build
with posh. I'd tend to agree it's probably not worth the effort
working around those if posh is no longer maintained.

diff --git a/configure.ac b/configure.ac
index 1a498f8..e73c4d1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -515,21 +515,21 @@ sed=':1
      s/ *$//'
 
 case " $LDFLAGS " in
-  *" -s "*) strip_exeldflags=true strip_libldflags=true
+  *\ -s\ *) strip_exeldflags=true strip_libldflags=true
     LDFLAGS=`echo " $LDFLAGS " | sed "$sed"` ;;
   *) strip_exeldflags=false strip_libldflags=false ;;
 esac
 
 case " ${EXELDFLAGS+$EXELDFLAGS }" in
   " ") ;;
-  *" -s "*) strip_exeldflags=true
+  *\ -s\ *) strip_exeldflags=true
     EXELDFLAGS=`echo " $EXELDFLAGS " | sed "$sed"` ;;
   *) strip_exeldflags=false ;;
 esac
 
 case " ${LIBLDFLAGS+$LIBLDFLAGS }" in
   " ") ;;
-  *" -s "*) strip_libldflags=true
+  *\ -s\ *) strip_libldflags=true
     LIBLDFLAGS=`echo " $LIBLDFLAGS " | sed "$sed"` ;;
   *) strip_libldflags=false ;;
 esac
@@ -626,9 +626,9 @@ AC_CHECK_PROGS([YODL], [yodl], [: yodl])
 YODL_OPTIONS=''
 if test "x$ac_cv_prog_YODL" = xyodl; then
     case `yodl --version` in
-      *"version 2."*) YODL_OPTIONS='-k' ;;
-      *"version 3."*) YODL_OPTIONS='-k -L' ;;
-      *"version 4."*) YODL_OPTIONS='-k -L' ;;
+      *version\ 2.*) YODL_OPTIONS='-k' ;;
+      *version\ 3.*) YODL_OPTIONS='-k -L' ;;
+      *version\ 4.*) YODL_OPTIONS='-k -L' ;;
     esac
 fi
 AC_SUBST(YODL_OPTIONS)
@@ -3223,7 +3223,7 @@ dnl $result is the default output for config.modules.  We generate
 dnl code to check if we should use this.
 cat <<EOM >> ${CONFIG_MODULES}.sh
 case "\$userlist" in
-  *" $name "*) grep "^name=$name " \${CONFIG_MODULES}.old;;
+  *\ $name\ *) grep "^name=$name " \${CONFIG_MODULES}.old;;
   *) echo "$result";;
 esac
 EOM

Re: Zsh requires Bash? 2002 revisited

[Stephane Chazelas]

2018-01-07 08:48:34 +0000, Stephane Chazelas:
[...]
> One of those bugs at least causes the wrong options for yodl to
> be used and "-s" (strip) to be used.
> 
> $ posh -c 'case abc in *"b"*) echo yes; esac'
> $ posh -c 'case abc in *b*) echo yes; esac'
> yes
> 
> The patch below works around those, but it still doesn't build
> with posh.
[...]

With a fix to the posh issue
(https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=886531#10),
zsh builds. So the other cases that break the build are probably
other instances of case something in *"x"* or *'x'*.

-- 
Stephane

Re: Zsh requires Bash? 2002 revisited

[Bart Schaefer]

On Sun, Jan 7, 2018 at 12:10 AM, Stephane Chazelas
<stephane.chazelas@gmail.com> wrote:
>
> FWIW, with sh as dash, if I remove bash and ksh, then the
> configure fails with
>
> ./configure:13185: functions: attempt to set associative array to scalar

That's coming from configure.ac in the section that processes
config.modules.  Somehow zsh is getting started in such a way that the
zsh/parameter module loads itself and turns "functions" into a special
variable.

Fixing this would require changing the format of the config.modules
file to stop using a variable named "functions".

Re: Zsh requires Bash? 2002 revisited

[Jim]

[-- Attachment #1.1: Type: text/plain, Size: 2102 bytes --]

On Sat, Jan 6, 2018 at 2:57 PM, Bart Schaefer <schaefer@brasslantern.com>
wrote:

> Given that you're not seeing shell syntax errors or the like, my guess
> is that posh is mishandling the -e command line option (errexit) and
> causing some branches of make to fail prematurely.  Or, posh has rm as
> a built-in and that's mishandling the -f option.  Try building with
> "make -k"?
>

After reading the other posts in this thread, I don't believe the following
is all that relevant any more, but just in case.

The man page for posh does not list "rm" as one of its builtins. Nor does it
have "mv". So most likely, not here.?

"make -k" also fails.


> If it's the "rm" problem, there are two places in Makefile.in
> ./Src/Makefile.in:    rm -f stamp-modobjs.tmp
> ./Src/Makefile.in:    rm -f stamp-modobjs stamp-modobjs.tmp
> where you can try "[ -f stamp-modobjs.tmp ] && rm ..." (you'll have to
> split the latter of the above into two separate "rm" calls).
>

Looks like Src/mkmakemod.sh is called at least four times. This is where
stamp-modobjs.tmp is supposed to get created.  Then Src/Makefile.in
does a cmp of stamp-modobjs and stamp-modobjs.tmp. If they compare
it removes stamp-modobjs.tmp, if not it moves stamp-modobjs.tmp to
stamp-modobjs. When moved it also outputs the message
"Updated `stamp-modobjs", which is happening.  Then later the message
"cat: stamp-modobjs: No such file or directory" is output indicating the
file
isn't there.  Trying to figure out what happens in between. Neither file is
present when make exits. A normal exit by the other shells, has at least
stamp-modobjs present.


> If that still fails it might help to see a diff of config.log from
> bash vs. posh, but it seems much more likely that it's make that's
> giving up here.
>

If its still of any interest, attached is the compressed diff comparing
bash vs posh
outputs of config.log. Mail bounced my previous post with the uncompressed
file attached(to large).

If you need any more info, I will try to get you what you need. For now I
have a
couple of versions of zsh to compile, sans posh.

Jim Murphy

[-- Attachment #1.2: Type: text/html, Size: 3287 bytes --]

[-- Attachment #2: config.log-diff.xz --]
[-- Type: application/x-xz, Size: 9048 bytes --]