W02jobs.ztst hangs with zsh 5.6.1

W02jobs.ztst hangs with zsh 5.6.1

[İsmail Dönmez]

Hi,

This test worked with the 5.6.0 release on Linux x86_64 (openSUSE
Tumbleweed), here is what I get:

./W02jobs.ztst: starting.
##########################

I did a 5 test run it passed only once. Let me know if I can provide
any further information.

Regards,
ismail

Re: W02jobs.ztst hangs with zsh 5.6.1

[Peter Stephenson]

On Mon, 10 Sep 2018 12:20:21 +0200
İsmail Dönmez <ismail@i10z.com> wrote:
> This test worked with the 5.6.0 release on Linux x86_64 (openSUSE
> Tumbleweed), here is what I get:
> 
> ./W02jobs.ztst: starting.
> ##########################
> 
> I did a 5 test run it passed only once. Let me know if I can provide
> any further information.

You'll need to run with

make ZTST_verbose=2 TESTNUM=W02

to see what failed.

The problem *may* be in the tests --- they're quite brittle.  So when we
find what's failing it'll be useful to run that by hand and see what
happens.

Cheers
pws

Re: W02jobs.ztst hangs with zsh 5.6.1

[İsmail Dönmez]

On Mon, Sep 10, 2018 at 12:32 PM Peter Stephenson
<p.stephenson@samsung.com> wrote:
> You'll need to run with
>
> make ZTST_verbose=2 TESTNUM=W02
>
> to see what failed.
>
> The problem *may* be in the tests --- they're quite brittle.  So when we
> find what's failing it'll be useful to run that by hand and see what
> happens.

Ok here is the log:

Running test: `jobs -r` and `jobs -s` with suspended job
ZTST_test: expecting status: 0
Input: /tmp/zsh.ztst.13823/ztst.in, output:
/tmp/zsh.ztst.13823/ztst.out, error: /tmp/zsh.ztst.13823/ztst.terr

(*.in, *.terr is empty), so:
havana ~ > cat /tmp/zsh.ztst.13823/ztst.out
zsh:*(stopped|suspended)*sleep*
-
\[1]  + (stopped|suspended)*sleep*


Regards,
ismail

Re: W02jobs.ztst hangs with zsh 5.6.1

[Peter Stephenson]

> On 10 September 2018 at 11:39 İsmail Dönmez <ismail@i10z.com> wrote:
> Running test: `jobs -r` and `jobs -s` with suspended job
> ZTST_test: expecting status: 0
> Input: /tmp/zsh.ztst.13823/ztst.in, output:
> /tmp/zsh.ztst.13823/ztst.out, error: /tmp/zsh.ztst.13823/ztst.terr
> 
> (*.in, *.terr is empty), so:
> havana ~ > cat /tmp/zsh.ztst.13823/ztst.out
> zsh:*(stopped|suspended)*sleep*
> -
> \[1]  + (stopped|suspended)*sleep*

Thanks --- the test is doing the following.

sleep 5
^Z
# Should print stopped / suspended message.
jobs -r
# Shouldn't print anything.
print -- -
# Should print "-" --- not really important here, it's
# just to sync up the output within the test system.
jobs -s
# Should show the job.
# The test doesn't bother with the following, it just exits, but
# to try it out by hand I suggest...
kill %1

Could you try this by hand to check?  I'd be very surprised if
it failed, as all the hairiness here is in the use of zpty in the
test system, which I've had a lot of trouble with when trying to
add new tests.  Again, I suspect pre-existing races rather than
the explicit effect of new changes.

In that case I think we'll just have to disable much of W02.

Thanks!
pws

Re: W02jobs.ztst hangs with zsh 5.6.1

[İsmail Dönmez]

Hi,
On Mon, Sep 10, 2018 at 1:19 PM Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
>
> > On 10 September 2018 at 11:39 İsmail Dönmez <ismail@i10z.com> wrote:
> > Running test: `jobs -r` and `jobs -s` with suspended job
> > ZTST_test: expecting status: 0
> > Input: /tmp/zsh.ztst.13823/ztst.in, output:
> > /tmp/zsh.ztst.13823/ztst.out, error: /tmp/zsh.ztst.13823/ztst.terr
> >
> > (*.in, *.terr is empty), so:
> > havana ~ > cat /tmp/zsh.ztst.13823/ztst.out
> > zsh:*(stopped|suspended)*sleep*
> > -
> > \[1]  + (stopped|suspended)*sleep*
>
> Thanks --- the test is doing the following.
>
> sleep 5
> ^Z
> # Should print stopped / suspended message.
> jobs -r
> # Shouldn't print anything.
> print -- -
> # Should print "-" --- not really important here, it's
> # just to sync up the output within the test system.
> jobs -s
> # Should show the job.
> # The test doesn't bother with the following, it just exits, but
> # to try it out by hand I suggest...
> kill %1

Inded that doesn't fail when I manually run it. I put the W02 under a
loop to see if I can get it to fail more but surprisingly it failed
elsewhere:

Running test: suspended job with check_jobs + check_running_jobs
ZTST_test: expecting status: 0
Input: /tmp/zsh.ztst.8136/ztst.in, output:
/tmp/zsh.ztst.8136/ztst.out, error: /tmp/zsh.ztst.8136/ztst.terr

So indeed this could be a zpty problem in the end.

Regards,
ismail

Re: W02jobs.ztst hangs with zsh 5.6.1

[Peter Stephenson]

> On 10 September 2018 at 12:32 İsmail Dönmez <ismail@i10z.com> wrote:
> Inded that doesn't fail when I manually run it. I put the W02 under a
> loop to see if I can get it to fail more but surprisingly it failed
> elsewhere:
> 
> Running test: suspended job with check_jobs + check_running_jobs
> ZTST_test: expecting status: 0
> Input: /tmp/zsh.ztst.8136/ztst.in, output:
> /tmp/zsh.ztst.8136/ztst.out, error: /tmp/zsh.ztst.8136/ztst.terr
> 
> So indeed this could be a zpty problem in the end.

Thanks for that.

I'll disable the tests that attempt to test suspended jobs and see where that
gets us.

diff --git a/Test/W02jobs.ztst b/Test/W02jobs.ztst
index 1e7ac76..fe12f97 100644
--- a/Test/W02jobs.ztst
+++ b/Test/W02jobs.ztst
@@ -85,27 +85,29 @@
 *>zsh:*running jobs*
 *>zsh:*SIGHUPed*
 
-  zpty_start
-  zpty_input 'setopt check_jobs no_check_running_jobs'
-  zpty_input 'sleep 3'
-  sleep 0.1
-  zpty_input $'\C-z'
-  zpty_stop
-0:suspended job with check_jobs + no_check_running_jobs
-*>zsh:*(stopped|suspended)*sleep*
-*>zsh:*(stopped|suspended) jobs*
-# no 'SIGHUPed' message for suspended jobs
-
-  zpty_start
-  zpty_input 'setopt check_jobs check_running_jobs'
-  zpty_input 'sleep 3'
-  sleep 0.1
-  zpty_input $'\C-z'
-  zpty_stop
-0:suspended job with check_jobs + check_running_jobs
-*>zsh:*(stopped|suspended)*sleep*
-*>zsh:*(stopped|suspended) jobs*
-# no 'SIGHUPed' message for suspended jobs
+#### Races presumed to be associated with zpty mean that
+#### tests involving suspending jobs are not safe.
+##   zpty_start
+##   zpty_input 'setopt check_jobs no_check_running_jobs'
+##   zpty_input 'sleep 3'
+##   sleep 0.1
+##   zpty_input $'\C-z'
+##   zpty_stop
+## 0:suspended job with check_jobs + no_check_running_jobs
+## *>zsh:*(stopped|suspended)*sleep*
+## *>zsh:*(stopped|suspended) jobs*
+## # no 'SIGHUPed' message for suspended jobs
+## 
+##   zpty_start
+##   zpty_input 'setopt check_jobs check_running_jobs'
+##   zpty_input 'sleep 3'
+##   sleep 0.1
+##   zpty_input $'\C-z'
+##   zpty_stop
+## 0:suspended job with check_jobs + check_running_jobs
+## *>zsh:*(stopped|suspended)*sleep*
+## *>zsh:*(stopped|suspended) jobs*
+## # no 'SIGHUPed' message for suspended jobs
 
   zpty_start
   zpty_input 'sleep 5 & sleep 4 & sleep 3 &'
@@ -153,19 +155,19 @@
 *>-
 *>zsh:*SIGHUPed*
 
-  zpty_start
-  zpty_input 'sleep 5'
-  sleep 0.1
-  zpty_input $'\C-z'
-  zpty_input 'jobs -r'
-  zpty_input 'print -- -'
-  zpty_input 'jobs -s'
-  zpty_stop
-0:`jobs -r` and `jobs -s` with suspended job
-*>zsh:*(stopped|suspended)*sleep*
-*>-
-*>\[1]  + (stopped|suspended)*sleep*
-# no 'SIGHUPed' message for suspended jobs
+##   zpty_start
+##   zpty_input 'sleep 5'
+##   sleep 0.1
+##   zpty_input $'\C-z'
+##   zpty_input 'jobs -r'
+##   zpty_input 'print -- -'
+##   zpty_input 'jobs -s'
+##   zpty_stop
+## 0:`jobs -r` and `jobs -s` with suspended job
+## *>zsh:*(stopped|suspended)*sleep*
+## *>-
+## *>\[1]  + (stopped|suspended)*sleep*
+## # no 'SIGHUPed' message for suspended jobs
 
   zpty_start
   zpty_input 'sleep 10 & sleep 9 & sleep 8 & sleep 7 &'

Re: W02jobs.ztst hangs with zsh 5.6.1

[İsmail Dönmez]

Hi,
On Mon, Sep 10, 2018 at 2:46 PM Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
>
> > On 10 September 2018 at 12:32 İsmail Dönmez <ismail@i10z.com> wrote:
> > Inded that doesn't fail when I manually run it. I put the W02 under a
> > loop to see if I can get it to fail more but surprisingly it failed
> > elsewhere:
> >
> > Running test: suspended job with check_jobs + check_running_jobs
> > ZTST_test: expecting status: 0
> > Input: /tmp/zsh.ztst.8136/ztst.in, output:
> > /tmp/zsh.ztst.8136/ztst.out, error: /tmp/zsh.ztst.8136/ztst.terr
> >
> > So indeed this could be a zpty problem in the end.
>
> Thanks for that.
>
> I'll disable the tests that attempt to test suspended jobs and see where that
> gets us.

Cloned zsh.git and run W02 under a loop and so far it passed every
time. Thank you!

Regards,
ismail

Re: W02jobs.ztst hangs with zsh 5.6.1

[Peter Stephenson]

On Mon, 10 Sep 2018 12:19:49 +0100 (BST)
Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> I'd be very surprised if it failed, as all the hairiness here is in
> the use of zpty in the test system, which I've had a lot of trouble
> with when trying to add new tests.

It's unlikely zpty is going to get enough love and attention to be
robust enough for what we really need here.

I wonder if there is some replacement we could use that someone might
already know about?  The Python pty module looked like it might be
usable, but I encountered half a dozen Python-style illogicalities before
getting a dozen lines into a programme, so we'll need some existing
script to start with.

pws

Re: W02jobs.ztst hangs with zsh 5.6.1

[Daniel Shahaf]

Peter Stephenson wrote on Mon, 10 Sep 2018 16:14 +0100:
> It's unlikely zpty is going to get enough love and attention to be
> robust enough for what we really need here.
> 
> I wonder if there is some replacement we could use that someone might
> already know about?  The Python pty module looked like it might be
> usable, but I encountered half a dozen Python-style illogicalities before
> getting a dozen lines into a programme, so we'll need some existing
> script to start with.

Could you explain what that replacement (in Python or anything else)
needs to do?  Are you asking for a program that launches ../Src/zsh in a
pty and then writes the stty(1) 'susp' character to that pty?  (At this
point the program could just send SIGTSTP directly, too; I'm not sure
if that's an option.)

Re: W02jobs.ztst hangs with zsh 5.6.1

[Peter Stephenson]

On Mon, 10 Sep 2018 15:50:32 +0000
Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> Peter Stephenson wrote on Mon, 10 Sep 2018 16:14 +0100:
> > It's unlikely zpty is going to get enough love and attention to be
> > robust enough for what we really need here.
> > 
> > I wonder if there is some replacement we could use that someone might
> > already know about?  The Python pty module looked like it might be
> > usable, but I encountered half a dozen Python-style illogicalities before
> > getting a dozen lines into a programme, so we'll need some existing
> > script to start with.  
> 
> Could you explain what that replacement (in Python or anything else)
> needs to do?  Are you asking for a program that launches ../Src/zsh in a
> pty and then writes the stty(1) 'susp' character to that pty?  (At this
> point the program could just send SIGTSTP directly, too; I'm not sure
> if that's an option.)

As near a direct replacement for ztpy as possible, but basically all it
needs to be able to do is spwan a slave running zsh, write to that, and
read back from it.  You'd have thought that wasn't too hard, but
experience suggests otherwise.

I got some life with the Ruby pty module, based on the example in the
documentation.  I haven't worked out how to get the right file
descriptors in the right place, so this is a non-working exmaple of
particular things we want to do more generically --- i.e. pass in
instructions to read and write, then look at the results returned,
rather than hard wire.

pws


require 'pty'

master, slave = PTY.open
read, write = IO.pipe
::ENV["PS1"] = ""
pid = spawn("zsh", "-fV", "+Z", :in=>read, :out=>slave)
read.close     # we dont need the read
slave.close    # or the slave

write.puts 'print $ZSH_VERSION'
p master.gets

# Write is a normal pipe so this doesn't actually work.
write.puts "stty susp \^z"

write.puts 'sleep 5'
# \C-z: again, this isn't actually going to the terminal.
write.printf "%c", 26
p master.gets
# hoping to see something involving a stopped / suspended message.

Re: W02jobs.ztst hangs with zsh 5.6.1

[Bart Schaefer]

On Mon, Sep 10, 2018 at 9:22 AM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
>
> I got some life with the Ruby pty module

I'm a bit leery of having the test suite depend on the availability of
python or ruby.

Re: W02jobs.ztst hangs with zsh 5.6.1

[Peter Stephenson]

On Mon, 10 Sep 2018 09:55:34 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Mon, Sep 10, 2018 at 9:22 AM, Peter Stephenson
> <p.stephenson@samsung.com> wrote:
> >
> > I got some life with the Ruby pty module  
> 
> I'm a bit leery of having the test suite depend on the availability of
> python or ruby.

Yes, it would have to be separate from the main distribution, for use
by those who really needed the extra testing like maintainers and
distributors. It seems clear it's not going to work with just what
we've got immediately to hand.

pws