pcre test failures

pcre test failures

[Vin Shelton]

Phil et al,

I'm getting the following failure on the new pcre test:

--- zsh-2011-10-19/check.out
+++ zsh-2011-10-25/check.out
@@ -98,0 +99,12 @@
+/opt/src/zsh-2011-10-25/Test/V07pcre.ztst: starting.
+(eval):1: failed to load module `zsh/pcre':
/opt/build/zsh-2011-10-25/Test/Modules/zsh/pcre.so: cannot open shared
object file: No such file or directory
+Testing PCRE multibyte with locale en_US.UTF-8
+Test /opt/src/zsh-2011-10-25/Test/V07pcre.ztst failed: bad status 2,
expected 0 from:
+  [[ 'foo→bar' =~ .([^[:ascii:]]). ]]
+  print $MATCH
+  print $match[1]
+Error output:
+(eval):1: failed to load module `zsh/pcre':
/opt/build/zsh-2011-10-25/Test/Modules/zsh/pcre.so: cannot open shared
object file: No such file or directory
+(eval):1: -pcre-match not available for regex
+Was testing: Basic non-ASCII regexp matching
+/opt/src/zsh-2011-10-25/Test/V07pcre.ztst: test failed.
@@ -108 +120 @@
-40 successful test scripts, 0 failures, 0 skipped
+40 successful test scripts, 1 failure, 0 skipped
@@ -109,0 +122 @@
+make[1]: *** [check] Error 1
@@ -110,0 +124 @@
+make: *** [check] Error 2

Well, OK, I guess.  I suppose I have to enable-pcre to make this work....
Once I '--enable-pcre' and turn "load=no" to "load=yes" in
config.modules, I still get this failure:

./V07pcre.ztst: starting.
Testing PCRE multibyte with locale en_US.UTF-8
Test ./V07pcre.ztst failed: bad status 2, expected 0 from:
  [[ 'foo→bar' =~ .([^[:ascii:]]). ]]
  print $MATCH
  print $match[1]
Error output:
(eval):1: module `zsh/pcre' has no such feature: `C:pcre-match'
(eval):1: -pcre-match not available for regex
Was testing: Basic non-ASCII regexp matching

Hope this helps,
  Vin

Re: pcre test failures

[Phil Pennock]

On 2011-10-25 at 21:58 -0400, Vin Shelton wrote:
> I'm getting the following failure on the new pcre test:

> Once I '--enable-pcre' and turn "load=no" to "load=yes" in
> config.modules, I still get this failure:
> 
> ./V07pcre.ztst: starting.
> Testing PCRE multibyte with locale en_US.UTF-8
> Test ./V07pcre.ztst failed: bad status 2, expected 0 from:
>   [[ 'foo→bar' =~ .([^[:ascii:]]). ]]
>   print $MATCH
>   print $match[1]
> Error output:
> (eval):1: module `zsh/pcre' has no such feature: `C:pcre-match'
> (eval):1: -pcre-match not available for regex
> Was testing: Basic non-ASCII regexp matching

What version of zsh are you testing?  The most recent CVS head?

I think you might have a mismatch between the zsh you've built
(C:pcre-match feature reference) and the version from which you built
the zsh/pcre module.

-Phil

Re: pcre test failures

[Jun T.]

I got similar errors with the recent CVS head.

Vin Shelton has reported two types of errors from V07pcre.ztst:

[1] V07pcre.ztst is run even if the module pcre.so is not built
(i.e., if --enable-pcre is not specified for ./configure).
Then the test fails with an error like "failed to load module 'zsh/pcre'".
This is quite annoying.
To avoid this the "%prep" section of the V07pcre.ztst must be modified
as follows:

if zmodload zsh/pcre 2>/dev/null; then
  ....(current %prep section goes into here)
else
  ZTST_unimplemented="the zsh/pcre module is not available"
fi

[2] If --enable-pcre is specified for ./configure
but the pcre-config command is not installed on the host,
then virtually empty pcre.so is built.
I guess this is the second case in the Vin's post.
pcre-config (and pcre.h etc.) are in pcre-dev (or similar) package in
many Linux distros. Once I installed this package it worked fine
(tested on Fedora14).

I think it would be better to modify configure.ac so that enable-pcre
is reset to "no" if pcre-config command is not found. 

Re: pcre test failures

[Vin Shelton]

On Wed, Oct 26, 2011 at 2:27 AM, Phil Pennock
<zsh-workers+phil.pennock@spodhuis.org> wrote:
> What version of zsh are you testing?  The most recent CVS head?

Yes, a clean checkout from CVS head.

> I think you might have a mismatch between the zsh you've built
> (C:pcre-match feature reference) and the version from which you built
> the zsh/pcre module.

It's possible that the path for loading the module got confused
because I did several configure/"make test" cycles.  Are you saying
that you can't reproduce this result from a fresh
checkout/configure/"make test"?  I will investigate further.

  - Vin

Re: pcre test failures

[Peter Stephenson]

On Wed, 26 Oct 2011 19:49:17 +0900
Jun T. <takimoto-j@kba.biglobe.ne.jp> wrote:
> [2] If --enable-pcre is specified for ./configure
> but the pcre-config command is not installed on the host,
> then virtually empty pcre.so is built.
> I guess this is the second case in the Vin's post.
> pcre-config (and pcre.h etc.) are in pcre-dev (or similar) package in
> many Linux distros. Once I installed this package it worked fine
> (tested on Fedora14).
> 
> I think it would be better to modify configure.ac so that enable-pcre
> is reset to "no" if pcre-config command is not found. 

Yes, I've been trying to avoid problems like this.  In general, it's
much better for a module not to be built than to load and announce it
can't do anything, which is no use to anyone.

Index: Src/Modules/pcre.mdd
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Modules/pcre.mdd,v
retrieving revision 1.4
diff -p -u -r1.4 pcre.mdd
--- Src/Modules/pcre.mdd	20 Jun 2007 20:59:18 -0000	1.4
+++ Src/Modules/pcre.mdd	26 Oct 2011 12:45:49 -0000
@@ -1,5 +1,5 @@
 name=zsh/pcre
-link=`if test x$enable_pcre = xyes; then echo dynamic; else echo no; fi`
+link=`if test x$enable_pcre = xyes && (pcre-config --version >/dev/null 2>/dev/null); then echo dynamic; else echo no; fi`
 load=no
 
 autofeatures="b:pcre_compile b:pcre_study b:pcre_match"


-- 
Peter Stephenson <pws@csr.com>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog

Re: pcre test failures

[Vin Shelton]

Jun is right: after installing pcre-devel on my Fedora box,
--enable-pcre yields a shell which passes all tests.

May I also suggest this patch:

--- Test/V07pcre.ztst~  2011-10-24 07:31:25.000000000 -0400
+++ Test/V07pcre.ztst   2011-10-26 13:59:51.520399774 -0400
@@ -1,6 +1,6 @@
 %prep

-  zmodload zsh/pcre
+  zmodload zsh/pcre >& /dev/null || ZTST_unimplemented="PCRE module
does not exist"
   setopt rematch_pcre
 # Find a UTF-8 locale.
   setopt multibyte

This keeps V07 from reporting a false failure.

Thanks,
  Vin

Re: pcre test failures

[Bart Schaefer]

On Oct 26,  7:49pm, Jun T. wrote:
}
} To avoid this the "%prep" section of the V07pcre.ztst must be modified
} as follows:
} 
} if zmodload zsh/pcre 2>/dev/null; then
}   ....(current %prep section goes into here)
} else
}   ZTST_unimplemented="the zsh/pcre module is not available"
} fi

Or just this:

--- ../zsh-forge/current/Test/V07pcre.ztst      2011-10-24 04:31:25.000000000
-0700
+++ ./Test/V07pcre.ztst 2011-10-26 11:27:47.000000000 -0700
@@ -1,6 +1,10 @@
 %prep
 
-  zmodload zsh/pcre
+  if ! zmodload zsh/pcre 2>/dev/null
+  then
+    ZTST_unimplemented="the zsh/pcre module is not available"
+    return 0
+  fi
   setopt rematch_pcre
 # Find a UTF-8 locale.
   setopt multibyte

Re: pcre test failures

[Phil Pennock]

On 2011-10-26 at 11:30 -0700, Bart Schaefer wrote:
> Or just this:

> -  zmodload zsh/pcre
> +  if ! zmodload zsh/pcre 2>/dev/null
> +  then
> +    ZTST_unimplemented="the zsh/pcre module is not available"
> +    return 0
> +  fi

That looks by far the best; applied, thanks.

Vin: sorry, I wasn't aware that there was a third state of "compiled but
empty" for the PCRE module, thought it was either not built or fully
featured.  (And yes, I did test the test before submitting it, though I
didn't test Bart's change, it's Obviously Right By Inspection (famous
last words)).

Peter: thanks for the cleanup in the .mdd file.

Okay, I believe, once more, that _now_ we have cleaned up all my mess
from PCRE/=~/UTF-8 changes.  Is there anything outstanding?

Is there anyone with a UTF8-aware system regexp library that can report
on that?  If such things exist, we likely need to metafy/unmetafy in the
regex module too (which is my fault).  If not, we can ignore it or fix
it anyway, knowing that there's no way to test that all the changes are
making it through -- I'd rather not fix until there's a platform to
_test_ changes on.

-Phil