PATCH: fg and bg tests

PATCH: fg and bg tests

[Peter Stephenson]

Here's a way of testing fg and bg.  See if you can spot any problems
before the tests become too hairy to debug.

The time waiting for the ^z is already there in previous tests ---
nobody's apparently noticed any problems there.  Maybe we can avoid
them all with carefully chosen output, but I don't think it's trivial.

pws

diff --git a/Test/W02jobs.ztst b/Test/W02jobs.ztst
index 65b8600..43f2c4d 100644
--- a/Test/W02jobs.ztst
+++ b/Test/W02jobs.ztst
@@ -181,6 +181,50 @@
 *>\[2]  ? interrupt*sleep*
 *>\[1]  ? kill*sleep*
 
+  zpty_start
+  zpty_input 'ed 2>/dev/null'
+  zpty_input $'a\nstuff1\n.\nw tmpfile\nq\n'
+  zpty_stop
+  cat tmpfile
+  rm -f tmpfile
+0:Sanity check of "ed" as foreground process
+>stuff1
+F:This test checks we can run the simple editor "ed" as a way of testing
+F:user interaction for later job control tests .  The test itself is
+F:trivial; its failure may simply indicate our test methodology does not
+F:work on this system.
+
+  zpty_start
+  zpty_input 'ed 2>/dev/null'
+  sleep 0.1
+  zpty_input $'\C-z'
+  zpty_input 'fg'
+  zpty_input $'a\nstuff2\n.\nw tmpfile\nq\n'
+  zpty_stop
+  cat tmpfile
+  rm -f tmpfile
+0:Basic fg with subsequent user interaction
+*>zsh:*(stopped|suspended)*ed*
+*>*continued*ed*
+*>stuff2
+
+  zpty_start
+  zpty_input 'ed 2>/dev/null'
+  sleep 0.1
+  zpty_input $'\C-z'
+  zpty_input 'bg'
+  zpty_input 'fg'
+  zpty_input $'a\nstuff3\n.\nw tmpfile\nq\n'
+  zpty_stop
+  cat tmpfile
+  rm -f tmpfile
+0:bg and fg with user input
+*>zsh:*(stopped|suspended)*ed*
+*>*continued*ed*
+*>*(stopped|suspended)*ed*
+*>*continued*ed*
+*>stuff3
+
 %clean
 
   zmodload -ui zsh/zpty

Re: PATCH: fg and bg tests

[Peter Stephenson]

On Tue, 24 Apr 2018 13:59:44 +0100
Peter Stephenson <p.stephenson@samsung.com> wrote:
> Here's a way of testing fg and bg.  See if you can spot any problems
> before the tests become too hairy to debug.
> 
> The time waiting for the ^z is already there in previous tests ---
> nobody's apparently noticed any problems there.  Maybe we can avoid
> them all with carefully chosen output, but I don't think it's trivial.

This improves matters in this case by using output from the zpty
process, but it's still there in other cases and is all a bit scary ---
the sleep with the comment next to it was needed because otherwise the
reported termination status of the background job with the "kill -HUP"
mysteriously turned from "hangup" into "done".  However, the new tests
I've added seem to be working OK and as far as I can see I haven't used
any non-standard ed commands.

The last change improves visibility of differences if a pattern match on
multiple lines fails.

I will commit this to see what happens on other systems.

pws

diff --git a/Test/W02jobs.ztst b/Test/W02jobs.ztst
index 65b8600..1ec0b8e 100644
--- a/Test/W02jobs.ztst
+++ b/Test/W02jobs.ztst
@@ -13,6 +13,14 @@
     zpty_input() {
       zpty -w zsh "${(F)@}" $'\n'
     }
+    zpty_line() {
+      local REPLY
+      integer i
+      for (( i = 0; i < ${1:-1}; ++i )); do
+        zpty -r zsh REPLY
+        print -r -- ${REPLY%%($'\r\n'|$'\n')}
+      done
+    }
     zpty_stop() {
       # exit twice in case of check_jobs
       zpty -w zsh $'exit\nexit\n'
@@ -161,15 +169,17 @@
 
   zpty_start
   zpty_input 'sleep 10 & sleep 9 & sleep 8 & sleep 7 &'
-  sleep 0.1
+  zpty_line 4
   zpty_input 'kill %4'
+  # Dunno why we need this when checking input...
   sleep 0.1
+  zpty_line
   zpty_input 'kill -HUP %3'
-  sleep 0.1
+  zpty_line
   zpty_input 'kill -INT %2'
-  sleep 0.1
+  zpty_line
   zpty_input 'kill -KILL %1'
-  sleep 0.1
+  zpty_line
   zpty_stop
 0:various `kill` signals with multiple running jobs
 *>\[1] [0-9]##
@@ -181,6 +191,54 @@
 *>\[2]  ? interrupt*sleep*
 *>\[1]  ? kill*sleep*
 
+  zpty_start
+  zpty_input $'ed 2>/dev/null\n=\n'
+  zpty_line
+  zpty_input $'a\nstuff1\n.\nw tmpfile\nq\n'
+  zpty_stop
+  cat tmpfile
+  rm -f tmpfile
+0:Sanity check of "ed" as foreground process
+*>0
+*>stuff1
+F:This test checks we can run the simple editor "ed" as a way of testing
+F:user interaction for later job control tests .  The test itself is
+F:trivial; its failure may simply indicate our test methodology does not
+F:work on this system.
+
+  zpty_start
+  zpty_input $'ed 2>/dev/null\n=\n'
+  zpty_line
+  zpty_input $'\C-z'
+  zpty_input 'fg'
+  zpty_input $'a\nstuff2\n.\nw tmpfile\nq\n'
+  zpty_stop
+  cat tmpfile
+  rm -f tmpfile
+0:Basic fg with subsequent user interaction
+*>0
+*>zsh:*(stopped|suspended)*ed*
+*>*continued*ed*
+*>stuff2
+
+  zpty_start
+  zpty_input $'ed 2>/dev/null\n=\n'
+  zpty_line
+  zpty_input $'\C-z'
+  zpty_input 'bg'
+  zpty_input 'fg'
+  zpty_input $'a\nstuff3\n.\nw tmpfile\nq\n'
+  zpty_stop
+  cat tmpfile
+  rm -f tmpfile
+0:bg and fg with user input
+*>0
+*>zsh:*(stopped|suspended)*ed*
+*>*continued*ed*
+*>*(stopped|suspended)*ed*
+*>*continued*ed*
+*>stuff3
+
 %clean
 
   zmodload -ui zsh/zpty
diff --git a/Test/ztst.zsh b/Test/ztst.zsh
index 0b26799..1a6a0b5 100755
--- a/Test/ztst.zsh
+++ b/Test/ztst.zsh
@@ -315,7 +315,8 @@ ZTST_diff() {
       
   if (( diff_pat )); then
     local -a diff_lines1 diff_lines2
-    integer failed i
+    integer failed i l n
+    local p
 
     diff_lines1=("${(f)$(<$argv[-2])}")
     diff_lines2=("${(f)$(<$argv[-1])}")
@@ -330,7 +331,25 @@ ZTST_diff() {
       done
     fi
     if (( failed )); then
-      print -rl "Pattern match failed:" \<${^diff_lines1} \>${^diff_lines2}
+      print -r "Pattern match failed, line $i:"
+      n=${#diff_lines1}
+      (( ${#diff_lines2} > $n )) && n=${#diff_lines2}
+      for (( l = 1; l <= n; ++l )); do
+	if (( l == i )); then
+	  p="-"
+	else
+	  p=" "
+	fi
+	print -r -- "$p<${diff_lines1[l]}"
+      done
+      for (( l = 1; l <= n; ++l )); do
+	if (( l == i )); then
+	  p="+"
+	else
+	  p=" "
+	fi
+	print -r -- "$p>${diff_lines2[l]}"
+      done
       diff_ret=1
     fi
   else

Re: PATCH: fg and bg tests

[Stephane Chazelas]

2018-04-25 10:09:40 +0100, Peter Stephenson:
[...]
>    zpty_input 'kill %4'
> +  # Dunno why we need this when checking input...
>    sleep 0.1
[...]

Note that sleep 0.1 is not portable. One could use zselect -t 10
instead.

-- 
Stephane

Re: PATCH: fg and bg tests

[Bart Schaefer]

On Wed, Apr 25, 2018 at 2:09 AM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
>
> This improves matters in this case by using output from the zpty

This fails on MacOS:

./W02jobs.ztst: starting.
Pattern match failed, line 0:
 <0
 <stuff1
 <
 <
 <
 >0
 >?
 >?
 >7
 >stuff1
Test ./W02jobs.ztst failed: output differs from expected as shown above for:
  zpty_start
  zpty_input $'ed 2>/dev/null\n=\n'
  zpty_line
  zpty_input $'a\nstuff1\n.\nw tmpfile\nq\n'
  zpty_stop
  cat tmpfile
  rm -f tmpfile
Was testing: Sanity check of "ed" as foreground process
./W02jobs.ztst: test failed.
The following may (or may not) help identifying the cause:
  This test checks we can run the simple editor "ed" as a way of testing
  user interaction for later job control tests .  The test itself is
  trivial; its failure may simply indicate our test methodology does not
  work on this system.

Re: PATCH: fg and bg tests

[Peter Stephenson]

On Wed, 25 Apr 2018 07:28:32 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Wed, Apr 25, 2018 at 2:09 AM, Peter Stephenson
> <p.stephenson@samsung.com> wrote:
> >
> > This improves matters in this case by using output from the zpty  
> 
> This fails on MacOS:

>0
>?
>?
>7
>stuff1

The ed commands aren't very sophisticated and it's apparently writing
the right thing --- and indeed the 7 is expected but seems to have gone
missing here.

I'm unlikely to find a simpler command that does direct user
interaction and I'm not going to be doing remote debugging.  I could
simply ignore the ed output after 0 and rely on the final write ---
which is really the point of doing that.

pws

Re: PATCH: fg and bg tests

[Peter Stephenson]

On Wed, 25 Apr 2018 15:56:47 +0100
Peter Stephenson <p.stephenson@samsung.com> wrote:
> I'm unlikely to find a simpler command that does direct user
> interaction and I'm not going to be doing remote debugging.

dc is even less verbose (and at least in the GNU version a bit more
helpful if there is an error) and it's even easier to change the
internal state and report, but still has the minimum level of user
interaction I need.  So let's try that.

I assume everyone has dc but in any case it's easy to check that and
skip tests if necessary.

If I get "This doesn't work on my system<EOM>" reports on this I think
it will be time to back the tests off and give up on them as not portable.

Yet more fixes for pattern comparison failures.

pws


diff --git a/Test/W02jobs.ztst b/Test/W02jobs.ztst
index 1ec0b8e..6d5a41f 100644
--- a/Test/W02jobs.ztst
+++ b/Test/W02jobs.ztst
@@ -192,52 +192,46 @@
 *>\[1]  ? kill*sleep*
 
   zpty_start
-  zpty_input $'ed 2>/dev/null\n=\n'
+  zpty_input $'dc 2>/dev/null\n1\np\n'
   zpty_line
-  zpty_input $'a\nstuff1\n.\nw tmpfile\nq\n'
+  zpty_input $'1\n+\np\nq\n'
   zpty_stop
-  cat tmpfile
-  rm -f tmpfile
 0:Sanity check of "ed" as foreground process
-*>0
-*>stuff1
-F:This test checks we can run the simple editor "ed" as a way of testing
+*>1
+*>2
+F:This test checks we can run the simple calculator dc as a way of testing
 F:user interaction for later job control tests .  The test itself is
 F:trivial; its failure may simply indicate our test methodology does not
 F:work on this system.
 
   zpty_start
-  zpty_input $'ed 2>/dev/null\n=\n'
+  zpty_input $'dc 2>/dev/null\n1\np\n'
   zpty_line
   zpty_input $'\C-z'
   zpty_input 'fg'
-  zpty_input $'a\nstuff2\n.\nw tmpfile\nq\n'
+  zpty_input $'1\n+\np\nq\n'
   zpty_stop
-  cat tmpfile
-  rm -f tmpfile
 0:Basic fg with subsequent user interaction
-*>0
-*>zsh:*(stopped|suspended)*ed*
-*>*continued*ed*
-*>stuff2
+*>1
+*>zsh:*(stopped|suspended)*dc*
+*>*continued*dc*
+*>2
 
   zpty_start
-  zpty_input $'ed 2>/dev/null\n=\n'
+  zpty_input $'dc 2>/dev/null\n1\np\n'
   zpty_line
   zpty_input $'\C-z'
   zpty_input 'bg'
   zpty_input 'fg'
-  zpty_input $'a\nstuff3\n.\nw tmpfile\nq\n'
+  zpty_input $'1\n+\np\nq\n'
   zpty_stop
-  cat tmpfile
-  rm -f tmpfile
 0:bg and fg with user input
-*>0
-*>zsh:*(stopped|suspended)*ed*
-*>*continued*ed*
-*>*(stopped|suspended)*ed*
-*>*continued*ed*
-*>stuff3
+*>1
+*>zsh:*(stopped|suspended)*dc*
+*>*continued*dc*
+*>*(stopped|suspended)*dc*
+*>*continued*dc*
+*>2
 
 %clean
 
diff --git a/Test/ztst.zsh b/Test/ztst.zsh
index 1a6a0b5..d835e0e 100755
--- a/Test/ztst.zsh
+++ b/Test/ztst.zsh
@@ -315,26 +315,25 @@ ZTST_diff() {
       
   if (( diff_pat )); then
     local -a diff_lines1 diff_lines2
-    integer failed i l n
+    integer failed i l
     local p
 
-    diff_lines1=("${(f)$(<$argv[-2])}")
-    diff_lines2=("${(f)$(<$argv[-1])}")
+    diff_lines1=("${(f@)$(<$argv[-2])}")
+    diff_lines2=("${(f@)$(<$argv[-1])}")
     if (( ${#diff_lines1} != ${#diff_lines2} )); then
       failed=1
+      print -r "Pattern match filead, line mismatch (${#diff_lines1}/${#diff_lines2}):"
     else
       for (( i = 1; i <= ${#diff_lines1}; i++ )); do
 	if [[ ${diff_lines2[i]} != ${~diff_lines1[i]} ]]; then
 	  failed=1
+	  print -r "Pattern match failed, line $i:"
 	  break
 	fi
       done
     fi
     if (( failed )); then
-      print -r "Pattern match failed, line $i:"
-      n=${#diff_lines1}
-      (( ${#diff_lines2} > $n )) && n=${#diff_lines2}
-      for (( l = 1; l <= n; ++l )); do
+      for (( l = 1; l <= ${#diff_lines1}; ++l )); do
 	if (( l == i )); then
 	  p="-"
 	else
@@ -342,7 +341,7 @@ ZTST_diff() {
 	fi
 	print -r -- "$p<${diff_lines1[l]}"
       done
-      for (( l = 1; l <= n; ++l )); do
+      for (( l = 1; l <= ${#diff_lines2}; ++l )); do
 	if (( l == i )); then
 	  p="+"
 	else

Re: PATCH: fg and bg tests

[Bart Schaefer]

(Regular work hours for me here now so promptness and length of reply
are limited.)

On Wed, Apr 25, 2018 at 10:01 AM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
>
> dc is even less verbose [...]  So let's try that.

For what it's worth, "ed" on MacOS seems to be prompting with a "?"
after every command.  Your first though about ignoring everything
until the final result would probably have worked.

Re: PATCH: fg and bg tests

[Peter Stephenson]

On Wed, 25 Apr 2018 10:16:34 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:

> (Regular work hours for me here now so promptness and length of reply
> are limited.)
> 
> On Wed, Apr 25, 2018 at 10:01 AM, Peter Stephenson
> <p.stephenson@samsung.com> wrote:
> >
> > dc is even less verbose [...]  So let's try that.  
> 
> For what it's worth, "ed" on MacOS seems to be prompting with a "?"
> after every command.  Your first though about ignoring everything
> until the final result would probably have worked.

Hmm... the non-MacOS ed documentation seem fairly clear that prompting
is off by default and when it's on it's a "*" (unless there's an error).

Anyway, ignoring output turned out to be harder than expected ---
finding a consistent point in the code that didn't give blank
lines (that I couldn't necessarily rely on being present) wasn't that
easy. I'd probably have needed to edit the output to compare only
relevant lines.

With a bit of luck, dc is easier to manage all round.

pws

Re: PATCH: fg and bg tests

[Bart Schaefer]

On Wed, Apr 25, 2018 at 10:41 AM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
>
> With a bit of luck, dc is easier to manage all round.

Sadly:

./W02jobs.ztst: starting.
Pattern match filead, line mismatch (6/9):
 <1
 <zsh:*(stopped|suspended)*dc*
 <*continued*dc*
 <*(stopped|suspended)*dc*
 <*continued*dc*
 <2
 >1
 >zsh: suspended  dc 2> /dev/null
 >[1]  + continued  dc 2> /dev/null
 >[1]  + suspended (tty input)  dc 2> /dev/null
 >[1]  + continued  dc 2> /dev/null
 >zsh: command not found: 1
 >zsh: command not found: +
 >zsh: command not found: p
 >zsh: command not found: q
Test ./W02jobs.ztst failed: output differs from expected as shown above for:
  zpty_start
  zpty_input $'dc 2>/dev/null\n1\np\n'
  zpty_line
  zpty_input $'\C-z'
  zpty_input 'bg'
  zpty_input 'fg'
  zpty_input $'1\n+\np\nq\n'
  zpty_stop
Was testing: bg and fg with user input

Re: PATCH: fg and bg tests

[Peter Stephenson]

On Wed, 25 Apr 2018 10:55:55 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Wed, Apr 25, 2018 at 10:41 AM, Peter Stephenson
> <p.stephenson@samsung.com> wrote:
> >
> > With a bit of luck, dc is easier to manage all round.  
> 
> Sadly:
> 
> ./W02jobs.ztst: starting.
> Pattern match filead, line mismatch (6/9):
>  <1
>  <zsh:*(stopped|suspended)*dc*
>  <*continued*dc*
>  <*(stopped|suspended)*dc*
>  <*continued*dc*
>  <2
>  >1

Bart's later experiments suggest this is the helpful behaviour of
exiting on SIGTTIN.

I did a little more experimentation, including attempting to use
subshells reading and writing.  Every attempt to make the input more
manageable resulted in either missing lines or empty blank lines.  The
real killer is bg --- I couldn't see a robust way of ensuring that the
process had gone into the background, got SIGTTIN, then suspended
again.  Needless to say I don't want just to add sleeps.

I think I'm going to have to remove those tests and give up on this for
now.

pws

Re: PATCH: fg and bg tests

[Bart Schaefer]

On Thu, Apr 26, 2018 at 1:08 AM, Peter Stephenson
<p.stephenson@samsung.com> wrote:
>
> I think I'm going to have to remove those tests and give up on this for
> now.

Could have kept the ctrl-Z / fg part of the test and just skipped bg.  Ah well.