Test failures on MacOS with latest pull from git

Test failures on MacOS with latest pull from git

[Bart Schaefer]

This is actually the first time I've tried building the pcre module on
MacOS, so the first error is probably related to my system setup.

./V07pcre.ztst: starting.
Testing PCRE multibyte with locale en_US.UTF-8
Test ./V07pcre.ztst failed: bad status 2, expected 0 from:
  [[ 'foobar' =~ .([^[:ascii:]]). ]]
  print $MATCH
  print $match[1]
Error output:
(eval):1: module `zsh/pcre' has no such feature: `C:pcre-match'
(eval):1: -pcre-match not available for regex
Was testing: Basic non-ASCII regexp matching
./V07pcre.ztst: test failed.

This one looks to be from Oliver's recent vi-mode changes:

./X02zlevi.ztst: starting.
*** /tmp/zsh.ztst.out.53255	Thu Oct 30 16:25:08 2014
--- /tmp/zsh.ztst.tout.53255	Thu Oct 30 16:25:08 2014
***************
*** 1,2 ****
! BUFFER: SwordE
! CURSOR: 4
--- 1,2 ----
! BUFFER: SeE
! CURSOR: 1
Test ./X02zlevi.ztst failed: output differs from expected as shown above for:
  zpty_run 'bindkey -a "^P" yank-pop'
  zletest $'word\C-wline\eddiSE\eP\C-P'
Was testing: line based put before followed by character based yank-pop
./X02zlevi.ztst: test failed.

Re: Test failures on MacOS with latest pull from git

[Bart Schaefer]

On Oct 30,  4:27pm, Bart Schaefer wrote:
}
} This one looks to be from Oliver's recent vi-mode changes:
} 
} *** 1,2 ****
} ! BUFFER: SwordE
} ! CURSOR: 4
} --- 1,2 ----
} ! BUFFER: SeE
} ! CURSOR: 1

Same test fails in a slightly different way on my CentOS box:

../../zsh-5.0/Test/X02zlevi.ztst: starting.
*** /tmp/zsh.ztst.out.28573     Thu Oct 30 17:39:04 2014
--- /tmp/zsh.ztst.tout.28573    Thu Oct 30 17:39:04 2014
***************
*** 1,2 ****
  BUFFER: SwordE
! CURSOR: 5
--- 1,2 ----
  BUFFER: SwordE
! CURSOR: 4
Test ../../zsh-5.0/Test/X02zlevi.ztst failed: output differs from expected as
shown above for:
  zletest $'word\C-wline\eddiSE\ehp\C-P'
Was testing: line based put after followed by character based yank-pop
../../zsh-5.0/Test/X02zlevi.ztst: test failed.

Re: Test failures on MacOS with latest pull from git

[Oliver Kiddle]

Bart wrote:
> }
> } *** 1,2 ****
> } ! BUFFER: SwordE
> } ! CURSOR: 4
> } --- 1,2 ----
> } ! BUFFER: SeE
> } ! CURSOR: 1

I can't work out what could be happening here. I don't have access to a
MacOS box. Could you perhaps step through it slowly to get an idea if
the problem is vi-put-before or yank-pop. The steps are:
zsh -f
bindkey -v
bindkey -a "^P" yank-pop
word<Ctrl-W>line<Esc>ddiSE<Esc>P<Ctrl-P>

The paste should put "line" above the current line and the yank-pop
should remove that line and insert "word" with the cursor over the d.
 
> Same test fails in a slightly different way on my CentOS box:

Actually that's a different test. Almost the same but doing hp instead
of P. In this case, I made a mistake and didn't copy the latest test
script over. Sorry, I've checked in the correction.

Thanks

Oliver

Re: Test failures on MacOS with latest pull from git

[Vin Shelton]

Dear Oliver,

On Fri, Oct 31, 2014 at 8:56 AM, Oliver Kiddle <okiddle@yahoo.co.uk> wrote:
>> Same test fails in a slightly different way on my CentOS box:
>
> Actually that's a different test. Almost the same but doing hp instead
> of P. In this case, I made a mistake and didn't copy the latest test
> script over. Sorry, I've checked in the correction.

Thanks.  The test now passes on mint 17.

  - Vin

Re: Test failures on MacOS with latest pull from git

[Bart Schaefer]

[-- Attachment #1: Type: text/plain, Size: 453 bytes --]

On Oct 31, 2014 6:03 AM, "Oliver Kiddle" <okiddle@yahoo.co.uk> wrote:
>
> Bart wrote:
> > }
> > } *** 1,2 ****
> > } ! BUFFER: SwordE
> > } ! CURSOR: 4
> > } --- 1,2 ----
> > } ! BUFFER: SeE
> > } ! CURSOR: 1
>
> I can't work out what could be happening here. I don't have access to a
> MacOS box. Could you perhaps step through it slowly

I won't be back to that machine until Monday, so if someone else has a
chance to reproduce this please go ahead.

Re: Test failures on MacOS with latest pull from git

[Jun T.]

I got the same error (BUFFER: SeE) on my Macs (10.8 and 10.9),
and also on FreeBSD 10.0.

But If I interactively type
word<Ctrl-W>line<Esc>ddiSE<Esc>P<Ctrl-P>
then I got the expected result, i.e., 'SwordE' with cursor on 'd'.

This suggests me that the problem is in the pty library of BSD's
(or zpty module of zsh). I suspect that '^W' is treated as the
'werase' special character by the tty of slave zsh.

The following patch is a simple workaround (use ^K instead of ^W),
and seems to work on both Mac and FreeBSD.
(It would be better if we could "fix" zpty so that ^W works as expected
on Mac/BSD, but I don't know wether it is possible or not.) 

Jun


diff --git a/Test/X02zlevi.ztst b/Test/X02zlevi.ztst
index bd3105d..9c92beb 100644
--- a/Test/X02zlevi.ztst
+++ b/Test/X02zlevi.ztst
@@ -116,29 +116,30 @@
 >BUFFER: stnwararart
 >CURSOR: 9
 
+  zpty_run 'bindkey "^K" vi-backward-kill-word'
   zpty_run 'bindkey -a "^P" yank-pop'
-  zletest $'word\C-wline\eddiSE\eP\C-P'
+  zletest $'word\C-kline\eddiSE\eP\C-P'
 0:line based put before followed by character based yank-pop
 >BUFFER: SwordE
 >CURSOR: 4
 
-  zletest $'line\eddiword\C-w\eiSE\eP\C-P'
+  zletest $'line\eddiword\C-k\eiSE\eP\C-P'
 0:character based put before followed by line based yank-pop
 >BUFFER: line
 >SE
 >CURSOR: 0
 
-  zletest $'one two three\C-w\C-w\C-wSE\e0p\C-P\C-P'
+  zletest $'one two three\C-k\C-k\C-kSE\e0p\C-P\C-P'
 0:put after cycled twice with yank-pop
 >BUFFER: SthreeE
 >CURSOR: 5
 
-  zletest $'word\C-wline\eddiSE\ehp\C-P'
+  zletest $'word\C-kline\eddiSE\ehp\C-P'
 0:line based put after followed by character based yank-pop
 >BUFFER: SwordE
 >CURSOR: 4
 
-  zletest $'line\eddiword\C-w\eiSE\ehp\C-P'
+  zletest $'line\eddiword\C-k\eiSE\ehp\C-P'
 0:character based after before followed by line based yank-pop
 >BUFFER: SE
 >line

Re: Test failures on MacOS with latest pull from git

[Bart Schaefer]

On Nov 1,  3:08am, Jun T. wrote:
}
} This suggests me that the problem is in the pty library of BSD's
} (or zpty module of zsh). I suspect that '^W' is treated as the
} 'werase' special character by the tty of slave zsh.

That's a good hypothesis, but a little strange because the zsh that
starts up on the pty ought to be resetting that (just as one starting
in a terminal window would).  It'd be nice to know what that's (not)
happening ... and then perhaps the setup that's sent to the pty before
the tests begin should include some other stty calls.

Re: Test failures on MacOS with latest pull from git

[Jun T.]

2014/11/04 03:18, Bart Schaefer <schaefer@brasslantern.com> wrote:
> That's a good hypothesis, but a little strange because the zsh that
> starts up on the pty ought to be resetting that (just as one starting
> in a terminal window would).

Yes, the slave zsh should set up the slave tty properly when it starts.
But Zle would change the tty mode between canonical and non-canonical
modes back and forth. It seems the slave tty is still in the canonical
mode when it should not be (i.e., when reading the command line character
by character). But I don't know why this happens only with BSD-based pty.

Currently I have no time to go into any more detail, but I must say
I would not be able to get any real fix even if I have enough time.

The following is another possible workaround (X02zlevi.ztst need not be
patched if this patch is used).

Jun


diff --git a/Test/comptest b/Test/comptest
index 4655f3b..b6256cc 100644
--- a/Test/comptest
+++ b/Test/comptest
@@ -34,7 +34,7 @@ comptestinit () {
 "fpath=( $fpath )" \
 "bindkey -$comptest_keymap" \
 'LISTMAX=10000000
-stty 38400 columns 80 rows 24
+stty 38400 columns 80 rows 24 werase undef
 TERM=vt100
 setopt zle
 autoload -U compinit

Re: Test failures on MacOS with latest pull from git

[Bart Schaefer]

On Nov 4, 11:59pm, Jun T. wrote:
}
} The following is another possible workaround (X02zlevi.ztst need not be
} patched if this patch is used).
} 
} -stty 38400 columns 80 rows 24
} +stty 38400 columns 80 rows 24 werase undef

Oliver, I'm going to let you (ask you to?) make the call on this one
because you're the one currently creating new vi-mode tests.

I'd actually consider applying both 33587 and 33599 unless the intent
is to test the default binding of backward-kill-word rather than just
the correct action of backward-kill-word.

Re: Test failures on MacOS with latest pull from git

[Oliver Kiddle]

Bart wrote:
> } -stty 38400 columns 80 rows 24
> } +stty 38400 columns 80 rows 24 werase undef
> 
> Oliver, I'm going to let you (ask you to?) make the call on this one
> because you're the one currently creating new vi-mode tests.

In that case, can anyone give me a tip on how to get a working zpty on
FreeBSD (fresh install of 10.0) so that I can reproduce this? I just get:
zpty: can't open pseudo terminal: bad file descriptor

I should have checked this first but 33587 (Ctrl-K) appears to break the
tests on Linux for me while 33599 (stty) doesn't. So I guess my call is
just 33599. It's probably closer to whatever a real fix would need to do
too. I wonder whether we should be setting more of them.

> I'd actually consider applying both 33587 and 33599 unless the intent
> is to test the default binding of backward-kill-word rather than just
> the correct action of backward-kill-word.

The intent was to test other things and those (or vi-backward-kill-word
to be more precise) get tested as a side-effect.

Oliver

Re: Test failures on MacOS with latest pull from git

[Jun T.]

Sorry for not responding quickly.

2014/11/09 07:11, Oliver Kiddle <okiddle@yahoo.co.uk> wrote:
> In that case, can anyone give me a tip on how to get a working zpty on
> FreeBSD (fresh install of 10.0) so that I can reproduce this? I just get:
> zpty: can't open pseudo terminal: bad file descriptor

On FreeBSD 10, pty kernel module is not loaded by default.
You need to load it manually:

$ su
# kldload pty


Jun