From: Stephane Chazelas <stephane@chazelas.org>
To: Zsh hackers list <zsh-workers@zsh.org>
Subject: [PATCH v6] regexp-replace and ^, word boundary or look-behind operators (and more).
Date: Sun, 10 Mar 2024 19:52:01 +0000 [thread overview]
Message-ID: <20240310195201.c53tzhgcyk5qgi27@chazelas.org> (raw)
In-Reply-To: <20240309130310.5zovit5jk6l4rnak@chazelas.org>
2024-03-09 13:03:10 +0000, Stephane Chazelas:
[...]
> I'll send a v6 likely using namespaced variables rather than
> going back to using positional parameters, once I understand the
> point of using .regexp_replace.myvar over _regexp_replace_myvar
[...]
So here it is. I ended up using none of the new features
(nameref and namespace) as they were not overly useful in this
instance and that means the code can be used as-is in older
versions. I'm using $_regexp_replace_localvarname for
namespacing. I compared performance with
${.regexp_replace.localvarname} and they were similar (the
latter about 2-3% slower in my limited tests).
diff --git a/Functions/Misc/regexp-replace b/Functions/Misc/regexp-replace
index d4408f0f7..630a5ceab 100644
--- a/Functions/Misc/regexp-replace
+++ b/Functions/Misc/regexp-replace
@@ -1,91 +1,99 @@
-# Replace all occurrences of a regular expression in a variable. The
-# variable is modified directly. Respects the setting of the
-# option RE_MATCH_PCRE.
+# Replace all occurrences of a regular expression in a scalar variable.
+# The variable is modified directly. Respects the setting of the option
+# RE_MATCH_PCRE, but otherwise sets the zsh emulation mode.
#
-# First argument: *name* (not contents) of variable.
-# Second argument: regular expression
-# Third argument: replacement string. This can contain all forms of
-# $ and backtick substitutions; in particular, $MATCH will be replaced
-# by the portion of the string matched by the regular expression.
-
-# we use positional parameters instead of variables to avoid
-# clashing with the user's variable. Make sure we start with 3 and only
-# 3 elements:
-argv=("$1" "$2" "$3")
-
-# $4 records whether pcre is enabled as that information would otherwise
-# be lost after emulate -L zsh
-4=0
-[[ -o re_match_pcre ]] && 4=1
+# Arguments:
+#
+# 1. *name* (not contents) of variable or more generally any lvalue;
+# expected to be scalar.
+#
+# 2. regular expression
+#
+# 3. replacement string. This can contain all forms of
+# $ and backtick substitutions; in particular, $MATCH will be
+# replaced by the portion of the string matched by the regular
+# expression. Parsing errors are fatal to the shell process.
+
+if (( $# < 2 || $# > 3 )); then
+ setopt localoptions functionargzero
+ print -ru2 "Usage: $0 <varname> <regexp> [<replacement>]"
+ return 2
+fi
+
+local _regexp_replace_use_pcre=0
+[[ -o re_match_pcre ]] && _regexp_replace_use_pcre=1
emulate -L zsh
+local _regexp_replace_subject=${(P)1} \
+ _regexp_replace_regexp=$2 \
+ _regexp_replace_replacement=$3 \
+ _regexp_replace_result \
+ MATCH MBEGIN MEND
-local MATCH MBEGIN MEND
local -a match mbegin mend
-if (( $4 )); then
+if (( _regexp_replace_use_pcre )); then
# if using pcre, we're using pcre_match and a running offset
# That's needed for ^, \A, \b, and look-behind operators to work
# properly.
zmodload zsh/pcre || return 2
- pcre_compile -- "$2" && pcre_study || return 2
+ pcre_compile -- "$_regexp_replace_regexp" && pcre_study || return 2
+
+ local _regexp_replace_offset=0 _regexp_replace_start _regexp_replace_stop _regexp_replace_new ZPCRE_OP
+ local -a _regexp_replace_finds
- # $4 is the current *byte* offset, $5, $6 reserved for later use
- 4=0 6=
+ while pcre_match -b -n $_regexp_replace_offset -- "$_regexp_replace_subject"; do
+ # we need to perform the evaluation in a scalar assignment so that
+ # if it generates an array, the elements are converted to string (by
+ # joining with the first chararacter of $IFS as usual)
+ _regexp_replace_new=${(Xe)_regexp_replace_replacement}
- local ZPCRE_OP
- while pcre_match -b -n $4 -- "${(P)1}"; do
- # append offsets and computed replacement to the array
- # we need to perform the evaluation in a scalar assignment so that if
- # it generates an array, the elements are converted to string (by
- # joining with the first character of $IFS as usual)
- 5=${(e)3}
- argv+=(${(s: :)ZPCRE_OP} "$5")
+ _regexp_replace_finds+=( ${(s[ ])ZPCRE_OP} "$_regexp_replace_new" )
# for 0-width matches, increase offset by 1 to avoid
# infinite loop
- 4=$((argv[-2] + (argv[-3] == argv[-2])))
+ (( _regexp_replace_offset = _regexp_replace_finds[-2] + (_regexp_replace_finds[-3] == _regexp_replace_finds[-2]) ))
done
- (($# > 6)) || return # no match
+ (( $#_regexp_replace_finds )) || return # no match
- set +o multibyte
+ unsetopt multibyte
- # $5 contains the result, $6 the current offset
- 5= 6=1
- for 2 3 4 in "$@[7,-1]"; do
- 5+=${(P)1[$6,$2]}$4
- 6=$(($3 + 1))
+ _regexp_replace_offset=1
+ for _regexp_replace_start _regexp_replace_stop _regexp_replace_new in "$_regexp_replace_finds[@]"; do
+ _regexp_replace_result+=${_regexp_replace_subject[_regexp_replace_offset,_regexp_replace_start]}$_regexp_replace_new
+ (( _regexp_replace_offset = _regexp_replace_stop + 1 ))
done
- 5+=${(P)1[$6,-1]}
-else
+ _regexp_replace_result+=${_regexp_replace_subject[_regexp_replace_offset,-1]}
+
+else # no PCRE
# in ERE, we can't use an offset so ^, (and \<, \b, \B, [[:<:]] where
# available) won't work properly.
- # $4 is the string to be matched
- 4=${(P)1}
-
- while [[ -n $4 ]]; do
- if [[ $4 =~ $2 ]]; then
- # append initial part and substituted match
- 5+=${4[1,MBEGIN-1]}${(e)3}
- # truncate remaining string
- if ((MEND < MBEGIN)); then
- # zero-width match, skip one character for the next match
- ((MEND++))
- 5+=${4[1]}
- fi
- 4=${4[MEND+1,-1]}
- # indicate we did something
- 6=1
- else
- break
+ local _regexp_replace_ok=0
+ while [[ $_regexp_replace_subject =~ $_regexp_replace_regexp ]]; do
+ # append initial part and substituted match
+ _regexp_replace_result+=$_regexp_replace_subject[1,MBEGIN-1]${(Xe)_regexp_replace_replacement}
+ # truncate remaining string
+ if (( MEND < MBEGIN )); then
+ # zero-width match, skip one character for the next match
+ (( MEND++ ))
+ _regexp_replace_result+=$_regexp_replace_subject[MBEGIN]
fi
+ _regexp_replace_subject=$_regexp_replace_subject[MEND+1,-1]
+ _regexp_replace_ok=1
+ [[ -z $_regexp_replace_subject ]] && break
done
- [[ -n $6 ]] || return # no match
- 5+=$4
+ (( _regexp_replace_ok )) || return
+ _regexp_replace_result+=$_regexp_replace_subject
fi
-eval $1=\$5
+# assign result to target variable if at least one substitution was
+# made. At this point, if the variable was originally array or assoc, it
+# is converted to scalar. If $1 doesn't contain a valid lvalue
+# specification, an exception is raised (exits the shell process if
+# non-interactive).
+: ${(P)1::="$_regexp_replace_result"}
+
prev parent reply other threads:[~2024-03-10 19:52 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-16 21:10 regexp-replace and ^, word boundary or look-behind operators Stephane Chazelas
2019-12-16 21:27 ` Stephane Chazelas
2019-12-17 7:38 ` Stephane Chazelas
2019-12-17 11:11 ` [PATCH] " Stephane Chazelas
2019-12-18 0:22 ` Daniel Shahaf
2019-12-18 8:31 ` Stephane Chazelas
2020-01-01 14:03 ` [PATCH v2] " Stephane Chazelas
2021-04-30 6:11 ` Stephane Chazelas
2021-04-30 23:13 ` Bart Schaefer
2021-05-05 11:45 ` [PATCH v3] regexp-replace and ^, word boundary or look-behind operators (and more) Stephane Chazelas
2021-05-31 0:58 ` Lawrence Velázquez
2021-05-31 18:18 ` Bart Schaefer
2021-05-31 21:37 ` [PATCH] (?) typeset array[position=index]=value Bart Schaefer
2021-06-01 5:32 ` Stephane Chazelas
2021-06-01 16:05 ` Bart Schaefer
2021-06-02 2:51 ` [PATCH] (take two?) typeset array[position=index]=value / unset hash[$stuff] Bart Schaefer
2021-06-02 10:06 ` Stephane Chazelas
2021-06-02 14:52 ` Bart Schaefer
2021-06-02 16:02 ` Stephane Chazelas
2021-06-02 9:11 ` [PATCH] (?) typeset array[position=index]=value Stephane Chazelas
2021-06-02 13:34 ` Daniel Shahaf
2021-06-02 14:20 ` Stephane Chazelas
2021-06-02 15:59 ` Bart Schaefer
2021-06-03 2:04 ` [PATCH (not final)] (take three?) unset "array[$anything]" Bart Schaefer
2021-06-03 2:42 ` Bart Schaefer
2021-06-03 6:12 ` Bart Schaefer
2021-06-03 8:54 ` Peter Stephenson
2021-06-03 13:13 ` Stephane Chazelas
2021-06-03 14:41 ` Peter Stephenson
2021-06-04 19:25 ` Bart Schaefer
2021-06-05 18:18 ` Peter Stephenson
2021-06-09 23:31 ` Bart Schaefer
2021-06-13 16:51 ` Peter Stephenson
2021-06-13 18:04 ` Bart Schaefer
2021-06-13 19:48 ` Peter Stephenson
2021-06-13 21:44 ` Bart Schaefer
2021-06-14 7:19 ` Stephane Chazelas
2021-06-03 18:12 ` Bart Schaefer
2021-06-04 8:02 ` Stephane Chazelas
2021-06-04 18:36 ` Bart Schaefer
2021-06-04 20:21 ` Stephane Chazelas
2021-06-05 0:20 ` Bart Schaefer
2021-06-05 17:05 ` Stephane Chazelas
2021-06-10 0:14 ` Square brackets in command position Bart Schaefer
2021-06-03 6:05 ` [PATCH (not final)] (take three?) unset "array[$anything]" Stephane Chazelas
2021-06-03 6:43 ` Bart Schaefer
2021-06-03 7:31 ` Stephane Chazelas
2021-06-10 0:21 ` [PATCH] (?) typeset array[position=index]=value Bart Schaefer
2021-06-05 4:29 ` Mikael Magnusson
2021-06-05 5:49 ` Bart Schaefer
2021-06-05 11:06 ` Mikael Magnusson
2021-06-05 16:22 ` Bart Schaefer
2021-06-18 10:53 ` Mikael Magnusson
2024-03-08 15:30 ` [PATCH v3] regexp-replace and ^, word boundary or look-behind operators (and more) Stephane Chazelas
2024-03-09 8:41 ` [PATCH v5] " Stephane Chazelas
2024-03-09 9:21 ` MBEGIN when =~ finds bytes inside characters (Was: [PATCH v5] regexp-replace and ^, word boundary or look-behind operators (and more).) Stephane Chazelas
2024-03-09 13:03 ` [PATCH v3] regexp-replace and ^, word boundary or look-behind operators (and more) Stephane Chazelas
2024-03-10 19:52 ` Stephane Chazelas [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240310195201.c53tzhgcyk5qgi27@chazelas.org \
--to=stephane@chazelas.org \
--cc=zsh-workers@zsh.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/zsh/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).