* PATCH: zsh/regex meta fixes for widechar (Re: Possible 4.3.18?)
2012-06-15 18:42 ` Peter Stephenson
2012-06-15 20:51 ` Danek Duvall
@ 2012-06-16 0:44 ` Phil Pennock
2012-06-16 0:45 ` Possible 4.3.18? Phil Pennock
2012-06-20 15:32 ` Peter Stephenson
3 siblings, 0 replies; 9+ messages in thread
From: Phil Pennock @ 2012-06-16 0:44 UTC (permalink / raw)
To: Peter Stephenson; +Cc: zsh-workers
-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160
On 2012-06-15 at 19:42 +0100, Peter Stephenson wrote:
> Phil was suggesting he might need to make a change before big release,
> is that still the case?
Oh right, sorry. zsh/regex module needed to support wide characters
without getting upset, so some unmeta/meta sprinkling needed.
Here's the patch, I'll commit shortly.
AFAICT, I'm already using the correct character length counting for
$mbegin $mend, and so this patch is sufficient. It seems suspiciously
simpler than the pcre.c revision 1.19 change I wrote.
With patch:
- ----------------------------8< cut here >8------------------------------
% unsetopt rematchpcre
% [[ 'aei→bx' =~ ^([aeiou]+)(.)(.) ]] && print -l $match === $mbegin === $mend === $MATCH
aei
→
b
===
1
4
5
===
3
4
5
===
aei→b
- ----------------------------8< cut here >8------------------------------
Without patch:
- ----------------------------8< cut here >8------------------------------
% [[ 'aei→bx' =~ ^([aeiou]+)(.)(.) ]] && print -l $match === $mbegin === $mend === $MATCH
ae
i
?
===
1
3
4
===
2
3
4
===
aei?
- ----------------------------8< cut here >8------------------------------
Index: Src/Modules/regex.c
===================================================================
RCS file: /home/cvsroot/remote-repos/zsh-repo/zsh/Src/Modules/regex.c,v
retrieving revision 1.7
diff -a -u -p -r1.7 regex.c
- --- Src/Modules/regex.c 20 Jan 2010 11:17:11 -0000 1.7
+++ Src/Modules/regex.c 16 Jun 2012 00:30:08 -0000
@@ -3,7 +3,7 @@
*
* This file is part of zsh, the Z shell.
*
- - * Copyright (c) 2007 Phil Pennock
+ * Copyright (c) 2007,2012 Phil Pennock
* All Rights Reserved.
*
* Permission is hereby granted, without written agreement and without
@@ -56,14 +56,19 @@ zcond_regex_match(char **a, int id)
regex_t re;
regmatch_t *m, *matches = NULL;
size_t matchessz = 0;
- - char *lhstr, *rhre, *s, **arr, **x;
+ char *lhstr, *lhstr_zshmeta, *rhre, *rhre_zshmeta, *s, **arr, **x;
int r, n, return_value, rcflags, reflags, nelem, start;
- - lhstr = cond_str(a,0,0);
- - rhre = cond_str(a,1,0);
+ lhstr_zshmeta = cond_str(a,0,0);
+ rhre_zshmeta = cond_str(a,1,0);
rcflags = reflags = 0;
return_value = 0; /* 1 => matched successfully */
+ lhstr = ztrdup(lhstr_zshmeta);
+ unmetafy(lhstr, NULL);
+ rhre = ztrdup(rhre_zshmeta);
+ unmetafy(rhre, NULL);
+
switch(id) {
case ZREGEX_EXTENDED:
rcflags |= REG_EXTENDED;
@@ -101,7 +106,7 @@ zcond_regex_match(char **a, int id)
if (nelem) {
arr = x = (char **) zalloc(sizeof(char *) * (nelem + 1));
for (m = matches + start, n = start; n <= (int)re.re_nsub; ++n, ++m, ++x) {
- - *x = ztrduppfx(lhstr + m->rm_so, m->rm_eo - m->rm_so);
+ *x = metafy(lhstr + m->rm_so, m->rm_eo - m->rm_so, META_DUP);
}
*x = NULL;
}
@@ -112,7 +117,7 @@ zcond_regex_match(char **a, int id)
char *ptr;
m = matches;
- - s = ztrduppfx(lhstr + m->rm_so, m->rm_eo - m->rm_so);
+ s = metafy(lhstr + m->rm_so, m->rm_eo - m->rm_so, META_DUP);
setsparam("MATCH", s);
/*
* Count the characters before the match.
@@ -174,12 +179,16 @@ zcond_regex_match(char **a, int id)
break;
default:
DPUTS(1, "bad regex option");
- - return 0; /* nothing to cleanup, especially not "re". */
+ return_value = 0;
+ goto CLEAN_BASEMETA;
}
if (matches)
zfree(matches, matchessz);
regfree(&re);
+CLEAN_BASEMETA:
+ free(lhstr);
+ free(rhre);
return return_value;
}
-----BEGIN PGP SIGNATURE-----
iEYEAREDAAYFAk/b1tIACgkQQDBDFTkDY39n9gCeLnbvIM3fpndE0GaNOGdN338s
u1cAn064nG9fOcKq80Zf2Dg5IwL/O5R7
=Q7/2
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: Possible 4.3.18?
2012-06-15 18:42 ` Peter Stephenson
` (2 preceding siblings ...)
2012-06-16 0:45 ` Possible 4.3.18? Phil Pennock
@ 2012-06-20 15:32 ` Peter Stephenson
3 siblings, 0 replies; 9+ messages in thread
From: Peter Stephenson @ 2012-06-20 15:32 UTC (permalink / raw)
To: zsh-workers
I've updated the MACHINES file by pushing a lot of rather ancient
looking stuff into the appendix. This doesn't mean everything left has
been recently tried.
Index: MACHINES
===================================================================
RCS file: /cvsroot/zsh/zsh/MACHINES,v
retrieving revision 1.10
diff -p -u -r1.10 MACHINES
--- MACHINES 23 Oct 2011 16:43:48 -0000 1.10
+++ MACHINES 20 Jun 2012 15:31:16 -0000
@@ -2,7 +2,7 @@
ZSH ON SPECIFIC ARCHITECTURES
-----------------------------
-These are the OSes that zsh has been tried on. If you succeed in getting
+These are the OSes that zsh has been tried on. If you succeed in getting
zsh to work on an OS not listed, let us know. The information in this
list may be out of date, as the developers do not have access to all
machines. In general, GNU/Linux distributions, Solaris and Cygwin are
@@ -35,13 +35,13 @@ Apple: MacOS X/Darwin 10.x
Reported to compile with no problems on 10.4.
- Multibyte support works, although (as on other architectures)
- Unicode combining characters are not properly handled.
+ Multibyte support works; you probably wish to set the
+ option COMBINING_CHARS, which is not enabled by default.
Problems have been noted when outputting multibyte characters
to the terminal from a "preexec" function.
Red Hat Inc.: Cygwin
- Should build `out-of-the-box'. The compilation directory should
+ Should build `out-of-the-box'. The compilation directory should
be on a file system mounted as binary (the mount command shows
`binmode'). There are various issues with Cygwin versions before
1.3.2 - you are adviced to update to the latest release.
@@ -51,7 +51,7 @@ Red Hat Inc.: Cygwin
a different mix of issues.
Problems handling subprocesses have been reported with Cygwin
- 1.7.5. It is not currently known how the problems split between
+ 1.7.5. It is not currently known how the problems split between
Cygwin and zsh.
Some of the tests in the Test subdirectory are known to fail:
@@ -62,31 +62,9 @@ Red Hat Inc.: Cygwin
Path completion will fail inside these mounts; make sure that
every mount point really exists.
-DEC: Ultrix (Alpha or DECstation)
-DEC: Mach 3.0 (DECstation 5000/25)
-DEC: OSF/1 1.2, 1.3, 2.0, 3.x, DEC Unix 4.x (Alpha)
- [Out of date.]
-
- In OSF/1 3.x, there is apparently a bug in the header file
- /usr/include/rpcsvc/ypclnt.h; the prototype for yp_all() has a
- struct ypall_callback as its final argument, which should be a
- pointer (struct ypall_callback *). This prevents compilation of
- one of zsh' files (zle_tricky.c). If you can't modify the header
- file, create a directory called `rpcsvc' in zsh's Src subdirectory
- and put a fixed version of the header file to it before compiling.
-
- The strip coming with gcc-2.7.2 seems to create unusable binaries.
- This problem is not related to zsh. If you have such problems,
- remove the bogus strip and use /bin/strip instead.
-
- On Digital UNIX 4.0, compilation with gcc and with --enable-dynamic
- apparently needs configuring with explicit flags when compiling
- with debugging enabled:
- DLLD=gcc LDFLAGS='-g -rpath <path-to-.so-files>' ./configure ...
-
-FreeBSD: FreeBSD 2.2.7, 3.x, 4.x
- Should build `out-of-the-box'. On FreeBSD 2.2, dynamic loading
- does not work, but it does with 3.x and 4.x.
+FreeBSD: FreeBSD 2.2.7, 3.x, 4.x, ... 7
+ Should build `out-of-the-box'. On FreeBSD 2.2, dynamic loading
+ does not work, but it does with 3.x and later.
HP: HP-UX 9, 10.20, 11.x (PA-RISC, Itanium)
Should build `out-of-the-box'.
@@ -94,29 +72,26 @@ HP: HP-UX 9, 10.20, 11.x (PA-RISC, Itani
Previous problems encountered on HP-UX 11.x:
Some of the special keys on the keyboard (backspace, delete)
- have been found to stop functioning. One suggested fix is
+ have been found to stop functioning. One suggested fix is
to alter the way the curses library is linked in the Makefile.
Replacing `-lcurses' with `-lHcurses -lcurses' in the libraries
is reported to fix this on 11.0, but is no longer necessary on
more recent versions of HP-UX 11, i.e. 11.11+.
Typical gcc installations on HP-UX use HP's linker rather than
- the GNU one. Configure will fail to set up dynamic linking in
+ the GNU one. Configure will fail to set up dynamic linking in
this situation. The following should allow building of modules:
DLLD=/usr/ccs/bin/ld DLLDFLAGS=-b DLCFLAGS=-fpic ./configure ...
Compiling with gcc 2.7.1 is known to fail with header file
conflicts. Use the HP ANSI C compiler.
-HP/Compaq: Tru64 4.x, 5.x
- Should build `out-of-the-box'.
-
IBM: AIX 3.2, 4.x, 5.x
Should build `out-of-the-box'.
Certain features will not work, in particular --enable-cap
and --enable-zsh-mem. (The feature enabled by --enable-cap
- is apparently present, however. Help getting this to work
+ is apparently present, however. Help getting this to work
would be appreciated.)
On 3.2, for 64-bit integer support you need to compile with gcc, as
@@ -132,20 +107,18 @@ IBM: AIX 3.2, 4.x, 5.x
very unhappy (GCC 3.0 apparently does not mind). Zsh now defaults
to termcap on AIX; any info about this problem is appreciated.
-Linux: Linux 2.x (various 32-bit and 64-bit processors)
+Linux: Linux 2.x, 3.x (various 32-bit and 64-bit processors)
Should build `out-of-the-box'.
+ The following problems should not occur with recent
+ distributions.
+
If you are using an early minor version of libc 5, then a bug
in the auto-configuration process may cause zsh to think that
- your system doesn't support the lstat function. If the configure
+ your system doesn't support the lstat function. If the configure
process reports that there is no lstat, edit config.h and change
HAVE_LSTAT to 1. libc-5.2.18 or later does not have this problem.
- Various problems have been reported when using optimisation
- with the experimental GNU compiler, egcs. In particular,
- on Linux Alpha with egcs 1.0.3a and 1.1.1 using -O1 or greater,
- the completion code is not correctly compiled.
-
Some versions of glibc2 have a conflict with <asm/resource.h>
which causes a redefinition warning on RLIM_INFINITY. This
causes configure to decide that <sys/resource.h> is not present,
@@ -162,65 +135,7 @@ OpenBSD: OpenBSD 2.x, 3.x
OpenIndiana: OpenIndiana 151a
Problems have been reported with awk when used to generate
prototype files for building zsh. Upgrading to gawk (GNU awk)
- version 4.0.0 fixes this.
-
-SIEMENS: Reliant UNIX
- [Out of date.]
-
- Builds `out-of-the-box'. Dynamic loading is supported.
- Large Files and 64-bit integers are supported as of version 5.44
- and CDS/CDS++ compiler.
-
-SIEMENS: SINIX
- [Out of date.]
-
- MX (Intel) platform: SINIX-L/M 5.41
- Builds out-of-the-box with EGCS. Neither dynamic loading nor
- 64-bit integers are supported. Native compiler was not tried
- mostly because GCC/EGCS builds out-of-the-box as well. If you
- succeed with native compiler, send a patch for this file
- to zsh-workers.
-
- RM (MIPS) platform: SINIX-N/Y 5.42
- Should build out-of-the-box but it was not tested. Neither
- dynamic loading nor 64-bit integers are supported.
- Note, that this version is obsolete and users are expected to
- update to Reliant UNIX.
-
-SGI: IRIX 6.2, 6.3
- [Out of date.]
-
- Should build `out-of-the-box'.
-
-SGI: IRIX 6.5
- Should build `out-of-the-box'; however, if using the native
- compiler, "cc" rather than "c99" is recommended. Compilation
- with gcc is also reported to work. Multibyte is supported.
-
- On 6.5.2, zsh malloc routines are reported not to work; also
- full optimization (cc -O3 -OPT:Olimit=0) causes problems.
-
- If using the SGI compiler, variable length arrays need to
- be turned off. configure can work this out for itself if it
- is passed the option --enable-cflags='-LANG:vla=off -O' (combined
- with other flags if necessary).
-
- The zpty module is not currently supported. This causes the
- tests starting `Y' in the Test directory to fail, even though
- the features to be tested are working.
-
-Sun: SunOS 4.1.x
- [Out of date.]
-
- Under 4.1.3 if yellow pages is used, username completion may cause
- segmentation violation. This is a bug in the shared library not
- in zsh. Some libc.so.1.9.* has this bug (it fails in yp_all).
- Statically linked binaries will work if linked with libc.so.1.8.1
- (which means that if you can get a statically linked binary
- compiled under 4.1.2 that it will probably work). An alternative
- but untested solution may be to undefine HAVE_NIS in config.h.
- This may work, but the first username completion will be _very_
- slow (as slow as in tcsh).
+ version 4.0.0 fixes this.
Sun: Solaris 2.x, 8, 9, ...
It is recommended that the system library version of iconv()
@@ -232,20 +147,21 @@ Sun: Solaris 2.x, 8, 9, ...
assumed by zsh). The symptom of this is that globbed filenames in
the compiled version of zsh will be missing the first two letters.
To avoid this, make sure you compile zsh without any reference
- to /usr/ucblib in your LD_LIBRARY_PATH. You can easily do this
+ to /usr/ucblib in your LD_LIBRARY_PATH. You can easily do this
by just unsetting LD_LIBRARY_PATH before building zsh.
Problems were once reported using --enable-largefile (the default) to
enable large file system and integer support on Solaris 2 with gcc
- before 2.95.2. Recent versions of gcc appear to be unproblematic.
+ before 2.95.2. Recent versions of gcc appear to be unproblematic.
Other machines
--------------
Zsh has previously been compiled on the following machines, but the
-developers do not have direct access to them and the reports may be out of
-date. We would be glad to receive any reports of success or failure on
-these OS's --- and, of course, any others not mentioned in this file.
+developers do not have direct access to them and the reports may be out
+of date. Some of these OS's are now very long in the tooth. We would
+be glad to receive any reports of success or failure on these OS's ---
+and, of course, any others not mentioned in this file.
Apple/NeXT OpenStep 4.2 for i386.
Reported to work at least with gcc 2.8.1 and gawk 2.15 patchlevel
@@ -257,9 +173,23 @@ Cray: Unicos (C90 and T90)
Data General: DG/UX 5.4R3.10 MU01 (various AViiONs)
Should build `out-of-the-box'.
+DEC: Ultrix (Alpha or DECstation)
+DEC: Mach 3.0 (DECstation 5000/25)
+DEC: OSF/1 1.2, 1.3, 2.0, 3.x, DEC Unix 4.x (Alpha)
+
+HP/Compaq: Tru64 4.x, 5.x
+
Next: NextStep 3.*
Should build `out-of-the-box', but the zsh malloc routines are
not recommended.
SCO: UnixWare 2.1.3
Builds `out-of-the-box'.
+
+SGI: IRIX 6.2, 6.3, 6.5
+
+SIEMENS: SINIX
+
+SIEMENS: Reliant UNIX
+
+Sun: SunOS 4.1.x
--
Peter Stephenson <pws@csr.com> Software Engineer
Tel: +44 (0)1223 692070 Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK
Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog
^ permalink raw reply [flat|nested] 9+ messages in thread