zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

[Axel Beckert]

Hi,

while trying to prepare an official Debian package of zsh 5.0.8 I ran
into interesting test suite failures which only happen under very
specific circumstances:

The following test suite failures so far only happened to me if

* I try to build zsh inside a chroot (using pbuilder to setup that
  chroot -- the version of pbuilder seems not to matter and is the
  latest one which is from July 2014);

* the host system on which the chroot is installed runs systemd as
  init system (having any version of systemd installed but booting
  the classic sysvinit init system works fine, too); and

* systemd is at version 220. (The version of udev doesn't seem to
  matter and compiling under systemd 215 works fine.)

Aynone has an idea what goes wrong here? Is that zsh's fault or is
that a bug (or new "feature") in systemd 220?

I know that ArchLinux always uses quite new systemd, so it'd be nice
to know if anyone, who's building zsh on that distribution,
experienced similar issues.

../../Test/V08zpty.ztst: starting.
Running test: zpty with a process that does not set up the terminal: internal write
Test ../../Test/V08zpty.ztst failed: bad status 1, expected 0 from:
  zpty cat cat
  zpty -w cat a line of text
  var=
  zpty -r cat var && print -r -- ${var%%$'\r\n'}
  zpty -d cat
Error output:
(zpty):zpty:1: can't open pseudo terminal: no such file or directory
(eval):zpty:2: no such pty command: cat
(eval):zpty:4: no such pty command: cat
(eval):zpty:5: no such pty command: cat
Was testing: zpty with a process that does not set up the terminal: internal write
../../Test/V08zpty.ztst: test failed.
../../Test/W01history.ztst: starting.
Running test: History word references
Test successful.
Running test: History line numbering
Test successful.
Running test: History substitution
Test successful.
Running test: Regression test for history references in command substitution
Test successful.
../../Test/W01history.ztst: all tests successful.
../../Test/X02zlevi.ztst: starting.
(zpty):zpty:21: can't open pseudo terminal: no such file or directory
comptestinit:zpty:23: no such pty command: zsh
Test ../../Test/X02zlevi.ztst failed: non-zero status from preparation code:
  if [[ $OSTYPE = cygwin ]]; then
    ZTST_unimplemented="the zsh/zpty module does not work on Cygwin"
  elif ( zmodload -i zsh/zpty ) >/dev/null 2>&1; then
    . $ZTST_srcdir/comptest
    comptestinit -v -z $ZTST_testdir/../Src/zsh
  else
    ZTST_unimplemented="the zsh/zpty module is not available"
  fi
../../Test/X02zlevi.ztst: test failed.
Running test: change last character in buffer
Test ../../Test/X02zlevi.ztst failed: bad status 1, expected 0 from:
  zletest $'goox\ecld'
Error output:
zletest:zpty:6: no such pty command: zsh
zletest:zpty:8: no such pty command: zsh
zletest:zpty:9: no such pty command: zsh
Was testing: change last character in buffer
../../Test/X02zlevi.ztst: test failed.
../../Test/Y01completion.ztst: starting.
(zpty):zpty:21: can't open pseudo terminal: no such file or directory
comptestinit:zpty:23: no such pty command: zsh
Test ../../Test/Y01completion.ztst failed: non-zero status from preparation code:
  if [[ $OSTYPE = cygwin ]]; then
    ZTST_unimplemented="the zsh/zpty module does not work on Cygwin"
  elif ( zmodload -i zsh/zpty ) >/dev/null 2>&1; then
    . $ZTST_srcdir/comptest
    mkdir comp.tmp
    cd comp.tmp
    comptestinit -z $ZTST_testdir/../Src/zsh &&
    {
      mkdir dir1 &&
      mkdir dir2 &&
      touch file1 &&
      touch file2
    }
  else
    ZTST_unimplemented="the zsh/zpty module is not available"
  fi
../../Test/Y01completion.ztst: test failed.
Running test: directories and files
Test ../../Test/Y01completion.ztst failed: bad status 1, expected 0 from:
  comptest $': \t\t\t\t\t\t\t'
Error output:
comptest:zpty:2: no such pty command: zsh
comptest:zpty:3: no such pty command: zsh
Was testing: directories and files
../../Test/Y01completion.ztst: test failed.
../../Test/Y02compmatch.ztst: starting.
(zpty):zpty:21: can't open pseudo terminal: no such file or directory
comptestinit:zpty:23: no such pty command: zsh
Test ../../Test/Y02compmatch.ztst failed: non-zero status from preparation code:
  if [[ $OSTYPE = cygwin ]]; then
    ZTST_unimplemented="the zsh/zpty module does not work on Cygwin"
  elif ( zmodload -i zsh/zpty ) >/dev/null 2>&1; then
    . $ZTST_srcdir/comptest
    mkdir match.tmp
    cd match.tmp
    comptestinit -z $ZTST_testdir/../Src/zsh &&
    {
      list1=(IndianRed IndianRed2 IndianRed3 IndianRed4)
      test_code () {
    	matcher=$1;
    	list=$2;
    	code="compdef _tst tst ; _tst () { echo -n '<COMPADD>';compadd -M '"
    	code="$code$matcher"
    	code="$code'  - ${(P)list} ; echo  -n '</COMPADD>'"
          code="$code; $extra_cmd"
          code="$code; echo -n '<INSERT_POSITIONS>'"
          code="$code; echo \$compstate[insert_positions]"
          code="$code; echo -n '</INSERT_POSITIONS>'"
          code="$code}"
    	comptesteval "$code"
      }
    }
  else
    ZTST_unimplemented="the zsh/zpty module is not available"
  fi
../../Test/Y02compmatch.ztst: test failed.
Running test: Match Error for "z:"
Test ../../Test/Y02compmatch.ztst failed: bad status 1, expected 0 from:
 test_code z: list1
 comptest  $'tst \t'
Error output:
(eval):1: command not found: test_code
comptest:zpty:2: no such pty command: zsh
comptest:zpty:3: no such pty command: zsh
Was testing: Match Error for "z:"
../../Test/Y02compmatch.ztst: test failed.
../../Test/Y03arguments.ztst: starting.
(zpty):zpty:21: can't open pseudo terminal: no such file or directory
comptestinit:zpty:23: no such pty command: zsh
Test ../../Test/Y03arguments.ztst failed: non-zero status from preparation code:
  if [[ $OSTYPE = cygwin ]]; then
    ZTST_unimplemented="the zsh/zpty module does not work on Cygwin"
  elif ( zmodload -i zsh/zpty ) >/dev/null 2>&1; then
    . $ZTST_srcdir/comptest
    mkdir comp.tmp
    cd comp.tmp
    comptestinit -z $ZTST_testdir/../Src/zsh &&
    {
      comptesteval 'compdef _tst tst'
      tst_arguments () { comptesteval "_tst () { _arguments ${${(@qq)*}} }" }
    }
  else
    ZTST_unimplemented="the zsh/zpty module is not available"
  fi
../../Test/Y03arguments.ztst: test failed.
Running test: one non-option argument
Test ../../Test/Y03arguments.ztst failed: bad status 1, expected 0 from:
 tst_arguments ':desc1:(arg1)'
 comptest $'tst \t\C-wa\t\C-war\t\C-warg\t\C-warg1\t\C-wr\t\C-wx\t \ty \t'
Error output:
(eval):1: command not found: tst_arguments
comptest:zpty:2: no such pty command: zsh
comptest:zpty:3: no such pty command: zsh
Was testing: one non-option argument
../../Test/Y03arguments.ztst: test failed.
**************************************
39 successful test scripts, 5 failures, 0 skipped
**************************************
Makefile:189: recipe for target 'check' failed
make[2]: *** [check] Error 1
make[2]: Leaving directory '/tmp/buildd/zsh-5.0.8/obj/Test'
Makefile:265: recipe for target 'test' failed
make[1]: *** [test] Error 2
make[1]: Leaving directory '/tmp/buildd/zsh-5.0.8/obj'
dh_auto_test: make -j4 test returned exit code 2

		Kind regards, Axel
-- 
/~\  Plain Text Ribbon Campaign                   | Axel Beckert
\ /  Say No to HTML in E-Mail and News            | abe@deuxchevaux.org  (Mail)
 X   See http://www.nonhtmlmail.org/campaign.html | abe@noone.org (Mail+Jabber)
/ \  I love long mails: http://email.is-not-s.ms/ | http://abe.noone.org/ (Web)

Re: zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

[Axel Beckert]

Hi,

some more details:

On Fri, Jun 05, 2015 at 05:02:12PM +0200, Axel Beckert wrote:
> while trying to prepare an official Debian package of zsh 5.0.8 I ran
> into interesting test suite failures which only happen under very
> specific circumstances:

I've also tried 5.0.7-dev-2 and 5.0.7-dev-4 -- no difference.

> ../../Test/V08zpty.ztst: starting.
> Running test: zpty with a process that does not set up the terminal: internal write
> Test ../../Test/V08zpty.ztst failed: bad status 1, expected 0 from:
>   zpty cat cat
>   zpty -w cat a line of text
>   var=
>   zpty -r cat var && print -r -- ${var%%$'\r\n'}
>   zpty -d cat
> Error output:
> (zpty):zpty:1: can't open pseudo terminal: no such file or directory
> (eval):zpty:2: no such pty command: cat
> (eval):zpty:4: no such pty command: cat
> (eval):zpty:5: no such pty command: cat
[...]
> 39 successful test scripts, 5 failures, 0 skipped

While these failures sound similar to what Baptiste Daroussin wrote in
35355 (http://www.zsh.org/mla/workers/2015/msg01279.html, both times 5
zpty-related tests), the fix from Oliver Kiddle in 35360
(http://www.zsh.org/mla/workers/2015/msg01284.html) didn't change
anything for me.

		Kind regards, Axel
-- 
/~\  Plain Text Ribbon Campaign                   | Axel Beckert
\ /  Say No to HTML in E-Mail and News            | abe@deuxchevaux.org  (Mail)
 X   See http://www.nonhtmlmail.org/campaign.html | abe@noone.org (Mail+Jabber)
/ \  I love long mails: http://email.is-not-s.ms/ | http://abe.noone.org/ (Web)

Re: zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

[Oliver Kiddle]

Axel Beckert wrote:
> 
> while trying to prepare an official Debian package of zsh 5.0.8 I ran
> into interesting test suite failures which only happen under very
> specific circumstances:

Does zsh 5.0.7 build under those circumstances? The only actual change
to the workings of zpty in that time is in 33743 / a526a220. That makes
it use posix_openpt instead of opening /dev/ptmx manually if it is
available.

> * systemd is at version 220. (The version of udev doesn't seem to
>   matter and compiling under systemd 215 works fine.)

Are there any differences in the results of configure between the two
versions of systemd or is it only when the tests are run that there's a
difference.

Oliver

Re: zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

[Axel Beckert]

Hi,

On Sat, Jun 06, 2015 at 12:49:55PM +0200, Oliver Kiddle wrote:
> Axel Beckert wrote:
> > while trying to prepare an official Debian package of zsh 5.0.8 I ran
> > into interesting test suite failures which only happen under very
> > specific circumstances:
> 
> Does zsh 5.0.7 build under those circumstances?

No. Fails with 5 zpty related tests, too:

(zpty):zpty:21: can't open pseudo terminal: no such file or directory

> > * systemd is at version 220. (The version of udev doesn't seem to
> >   matter and compiling under systemd 215 works fine.)
> 
> Are there any differences in the results of configure between the two
> versions of systemd or is it only when the tests are run that there's a
> difference.

Will have to figure out. I though suspect that the chroot behaves
differently under systemd 220. One indicator for that is this error
message in chroot cleanup after the (failed) build:

W: Could not unmount dev/pts: umount:
/var/cache/pbuilder/build//17909/dev/pts: not mounted

		Kind regards, Axel
-- 
/~\  Plain Text Ribbon Campaign                   | Axel Beckert
\ /  Say No to HTML in E-Mail and News            | abe@deuxchevaux.org  (Mail)
 X   See http://www.nonhtmlmail.org/campaign.html | abe@noone.org (Mail+Jabber)
/ \  I love long mails: http://email.is-not-s.ms/ | http://abe.noone.org/ (Web)

Re: zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

[Axel Beckert]

Hi,

On Sat, Jun 06, 2015 at 08:48:33PM +0200, Axel Beckert wrote:
> > > * systemd is at version 220. (The version of udev doesn't seem to
> > >   matter and compiling under systemd 215 works fine.)
> > 
> > Are there any differences in the results of configure between the two
> > versions of systemd or is it only when the tests are run that there's a
> > difference.
> 
> Will have to figure out. I though suspect that the chroot behaves
> differently under systemd 220. One indicator for that is this error
> message in chroot cleanup after the (failed) build:
> 
> W: Could not unmount dev/pts: umount:
> /var/cache/pbuilder/build//17909/dev/pts: not mounted

Actually if I call "mount -t devpts none /dev/pts" inside the chroot
before running the testsuite (even if ran after building zsh in the
chroot), the testsuite passes.

So for some reason the testsuite fails if /dev/pts is just an empty
directory, i.e. /dev/pts/ptmx does not exist in the chroot, but
/dev/pts does.

		Kind regards, Axel
-- 
/~\  Plain Text Ribbon Campaign                   | Axel Beckert
\ /  Say No to HTML in E-Mail and News            | abe@deuxchevaux.org  (Mail)
 X   See http://www.nonhtmlmail.org/campaign.html | abe@noone.org (Mail+Jabber)
/ \  I love long mails: http://email.is-not-s.ms/ | http://abe.noone.org/ (Web)

Re: zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

[Axel Beckert]

Hi again,

On Fri, Jun 12, 2015 at 10:52:28PM +0200, Axel Beckert wrote:
> > W: Could not unmount dev/pts: umount:
> > /var/cache/pbuilder/build//17909/dev/pts: not mounted
> 
> Actually if I call "mount -t devpts none /dev/pts" inside the chroot
> before running the testsuite (even if ran after building zsh in the
> chroot), the testsuite passes.
> 
> So for some reason the testsuite fails if /dev/pts is just an empty
> directory, i.e. /dev/pts/ptmx does not exist in the chroot, but
> /dev/pts does.

In the meanwhile I consider the missing /dev/pts mount a bug in the
script which sets up the chroot and have reported it as such:
https://bugs.debian.org/788580 (FTBFS = Debian slang for "Fails to
build from source").

The only thing which is IMHO left on the zsh side is why the testsuite
fails (instead of e.g. skipping the according tests) if /dev/pts/ptmx
is missing. Is this on purpose? Or just never occurred before? (Mostly
curious. :-)

		Kind regards, Axel
-- 
/~\  Plain Text Ribbon Campaign                   | Axel Beckert
\ /  Say No to HTML in E-Mail and News            | abe@deuxchevaux.org  (Mail)
 X   See http://www.nonhtmlmail.org/campaign.html | abe@noone.org (Mail+Jabber)
/ \  I love long mails: http://email.is-not-s.ms/ | http://abe.noone.org/ (Web)

Re: zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

[Peter Stephenson]

On Sat, 13 Jun 2015 12:48:58 +0200
Axel Beckert <abe@deuxchevaux.org> wrote:
> The only thing which is IMHO left on the zsh side is why the testsuite
> fails (instead of e.g. skipping the according tests) if /dev/pts/ptmx
> is missing. Is this on purpose? Or just never occurred before? (Mostly
> curious. :-)

Sounds like you've got to the bottom of this, which is good.

The current problem hasn't occurred before, I don't think, but yes, this
is how it's designed to work --- at least if HAVE_POSIX_OPENPT is
defined, which presumably it is.  In that case zsh is checking for the
API and expects it to work if present.

I guess it would be possible to have a "broken posxix_openpt" test ---
we've done such things before.  But surely if it's possible for the
system to fix posix_openpt, which must be how it's intended to work,
we shouldn't need to test...?

pws

Re: zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

[Bart Schaefer]

On Jun 13,  6:01pm, Peter Stephenson wrote:
}
} I guess it would be possible to have a "broken posxix_openpt" test ---

Not at configure time -- that would result in the module being left
uncompiled, when in fact outside of the chroot it would work.

} we've done such things before.  But surely if it's possible for the
} system to fix posix_openpt, which must be how it's intended to work,
} we shouldn't need to test...?

I think the test failing here was exactly the right thing, as it led to
the origin of the problem being discovered.  The only viable alternative
would be to report the test as "skipped" which I guess would be OK for
the secondary tests that rely on zpty, but not for V08zpty itself.

I suppose the test suite could remember that V08 failed and therefore
skip X02 and all of Y, so as to isolate the problem better.  We could
add some sort of dependency check.

Re: zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

[Axel Beckert]

Hi,

On Sat, Jun 13, 2015 at 06:01:54PM +0100, Peter Stephenson wrote:
> On Sat, 13 Jun 2015 12:48:58 +0200
> Axel Beckert <abe@deuxchevaux.org> wrote:
> > The only thing which is IMHO left on the zsh side is why the testsuite
> > fails (instead of e.g. skipping the according tests) if /dev/pts/ptmx
> > is missing. Is this on purpose? Or just never occurred before? (Mostly
> > curious. :-)
> 
> Sounds like you've got to the bottom of this, which is good.

Yes, I'm glad, too. Especially because afterwards I could upload zsh
5.0.8 to Debian Unstable with good conscience. :-)

It so far build fine on all architectures except GNU Hurd:
https://buildd.debian.org/status/package.php?p=zsh

5.0.8 on sparc hasn't been built yet and the build failure on GNU Hurd
seems to have been an endless loop in the configure script which means
that the issue is rather in autotools than in zsh.

On Sat, Jun 13, 2015 at 10:43:33AM -0700, Bart Schaefer wrote:
> On Jun 13,  6:01pm, Peter Stephenson wrote:
> }
> } I guess it would be possible to have a "broken posxix_openpt" test ---
> 
> Not at configure time -- that would result in the module being left
> uncompiled, when in fact outside of the chroot it would work.

Yeah, I'd say the test suite should not expect to be run in the same
environment as the build. E.g. in Debian we run it directly after the
build inside the build environment, but also "as installed"
afterwards: http://ci.debian.net/packages/z/zsh/unstable/amd64/ (5.0.8
is still missing there, as there may occur lags up to a few days.)

> I think the test failing here was exactly the right thing, as it led to
> the origin of the problem being discovered.

Yes and no. I'm glad about it failing, yes.

> The only viable alternative would be to report the test as "skipped"

That's what I imagined.

> which I guess would be OK for the secondary tests that rely on zpty,
> but not for V08zpty itself.

Hrm. If I know, I the zpty tests would fail due to the environment,
I'd tend to skip them as I would skip 64-bit-only tests in a 32-bit
environment.

> I suppose the test suite could remember that V08 failed and therefore
> skip X02 and all of Y, so as to isolate the problem better.  We could
> add some sort of dependency check.

Actually I think that's overkill. It was clear for me in this case
that all failing tests were zpty related.

		Kind regards, Axel
-- 
/~\  Plain Text Ribbon Campaign                   | Axel Beckert
\ /  Say No to HTML in E-Mail and News            | abe@deuxchevaux.org  (Mail)
 X   See http://www.nonhtmlmail.org/campaign.html | abe@noone.org (Mail+Jabber)
/ \  I love long mails: http://email.is-not-s.ms/ | http://abe.noone.org/ (Web)

Re: zpty-related testsuite failures if building in a chroot on a host running systemd 220 as init system

[Bart Schaefer]

On Jun 15, 12:55pm, Axel Beckert wrote:
}
} On Sat, Jun 13, 2015 at 10:43:33AM -0700, Bart Schaefer wrote:
} > The only viable alternative would be to report the test as "skipped"
} > which I guess would be OK for the secondary tests that rely on zpty,
} > but not for V08zpty itself.
} 
} Hrm. If I know, I the zpty tests would fail due to the environment,
} I'd tend to skip them as I would skip 64-bit-only tests in a 32-bit
} environment.

How would the test suite know in advance that the test was going to fail?
Don't say "by checking for /dev/pts/*" -- that's implementation-specific
information that's supposed to be the purview of the openpt() internals.
Whether that works is the whole point of running the test.