Modifier substitutions.

Modifier substitutions.

[Peter Stephenson]

This is all a little inconsistent.

% str="Think of a number"
% old=number
% new=word

% print ${str:s%number%$new}
Think of a word
% print ${str:s%$old%$new}
Think of a number
% print "${str:s%number%$new}"
Think of a $new

Substitutions with ${.../.../...} are more consistent, though there is
something a little odd with quoting in that case.  For example,

% str='#Think of a number'
% old='#Think'
% new='Despair'
% print ${str/$old/$new}
#Think of a number
% print ${str/\\$old/$new}
Despair of a number

The `#' is being interpreted as an anchor at the head of the string even
when buried inside $old.  I have a feeling that wasn't my intention.

pws

RE: Modifier substitutions.

[Andrej Borsenkow]

>
> The `#' is being interpreted as an anchor at the head of the
> string even
> when buried inside $old.  I have a feeling that wasn't my intention.
>

Erm ... it was done by Bart (do not have arcticle number handy) as a
result of my question. It was originally used when _configure called
_long_options (was it called so?) to make "--disable-*" out of
"--enable-*" (and vice versa). It gets pattern/replacement in the form

'#--disable- --enable-'
...

The intent was to not match string in the middle of option (it is
unlikely in case of configure, but for the sake of generality ...) But,
probably, now, with backreferences it can be rewrited as

'(#m)--disable-(*) --enable-$MATCH'

and explicitly anchor at the beginning.

I do not insist on this change. This was the sole reason for it.

-andrej

RE: Modifier substitutions.

[Andrej Borsenkow]

> >
>
> Erm ... it was done by Bart (do not have arcticle number handy)
>

6000, 6001, 6002, 6003

I was about to change it but then (from 6003) got impression, that bash
does the same. So, I let it to others to decide.

Personally, I agree now, it may be confusing.

-andrej

Should we backup this change? RE: Modifier substitutions.

[Andrej Borsenkow]

There was no reply ... so I ask once more: should we backup this change?
I found, it can really be confusing (because pattern contents may be
pretty random).

I will rewrite _arguments/_configure (and may be a  bit doc) in this
case.

-andrej

> -----Original Message-----
> From: Andrej Borsenkow [mailto:Andrej.Borsenkow@mow.siemens.ru]
> Sent: Thursday, April 06, 2000 1:33 PM
> To: Peter Stephenson; Zsh hackers list
> Subject: RE: Modifier substitutions.
>
>
>
> >
> > The `#' is being interpreted as an anchor at the head of the
> > string even
> > when buried inside $old.  I have a feeling that wasn't my intention.
> >
>
> Erm ... it was done by Bart (do not have arcticle number handy) as a
> result of my question. It was originally used when _configure called
> _long_options (was it called so?) to make "--disable-*" out of
> "--enable-*" (and vice versa). It gets pattern/replacement in the form
>
> '#--disable- --enable-'
> ...
>
> The intent was to not match string in the middle of option (it is
> unlikely in case of configure, but for the sake of generality
> ...) But,
> probably, now, with backreferences it can be rewrited as
>
> '(#m)--disable-(*) --enable-$MATCH'
>
> and explicitly anchor at the beginning.
>
> I do not insist on this change. This was the sole reason for it.
>
> -andrej
>
>

Re: Should we backup this change? RE: Modifier substitutions.

[Sven Wischnowsky]

Andrej Borsenkow wrote:

> There was no reply ... so I ask once more: should we backup this change?

I would be in favour of that. The possible confusion when
forgetting to check/quote the `#' and `%' at the beginning of
parameters used there is more important than the advantage, I think.

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: Should we backup this change? RE: Modifier substitutions.

[Bart Schaefer]

On Apr 12,  4:26pm, Andrej Borsenkow wrote:
} Subject: Should we backup this change? RE: Modifier substitutions.
}
} There was no reply ... so I ask once more: should we backup this change?
} 
} > -----Original Message-----
} > > The `#' is being interpreted as an anchor at the head of the
} > > string even
} > > when buried inside $old.  I have a feeling that wasn't my intention.
} > >
} >
} > Erm ... it was done by Bart (do not have arcticle number handy) as a
} > result of my question.

On Apr 12,  2:34pm, Sven Wischnowsky wrote:
}
} I would be in favour of that. The possible confusion when
} forgetting to check/quote the `#' and `%' at the beginning of
} parameters used there is more important than the advantage, I think.

The change went in in 6002.  It was followed in 6003 by a change to allow
a leading double backslash to escape the delimiter.  So all you need to
do is change uses of ${foo/$bar/...} into ${foo/\\$bar/...} to avoid the
confusion.  And without 6002, there's no way to build up an anchored
pattern in a variable (which was Andrej's original complaint).

Also, though I'm the last person to argue for keeping a change solely for
compatibility, bash recognizes the same syntaxes.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

RE: Should we backup this change? RE: Modifier substitutions.

[Andrej Borsenkow]

>
> The change went in in 6002.  It was followed in 6003 by a
> change to allow
> a leading double backslash to escape the delimiter.  So all
> you need to
> do is change uses of ${foo/$bar/...} into ${foo/\\$bar/...}
> to avoid the
> confusion.  And without 6002, there's no way to build up an anchored
> pattern in a variable (which was Andrej's original complaint).
>

The problem is (as I discovered later) it is not complete. There is no
way to anchor pattern to match full string. And my original problem is
better solved with back-references anyway (that were not available at
the time of my question).

-andrej

Re: Should we backup this change? RE: Modifier substitutions.

[Peter Stephenson]

"Andrej Borsenkow" wrote:
> > ${foo/$bar/...} into ${foo/\\$bar/...}
>
> The problem is (as I discovered later) it is not complete. There is no
> way to anchor pattern to match full string.

You should now be able to do ${foo/(#s)$bar(#e)/...}.  Unfortunately
there's a bug due to the implementation: (#s) always matches, because the
substitution only passes the tail end of the string.  (#e) works here, but
can fail when we are looking for the shortest match for ${...%...}.  In
principle this isn't hard to keep track of --- the calling code knows
whether the beginning or end of the string passed is really that --- but in
practice it's messy passing the status into pattry() from igetmatch() in
glob.c.  I'll think about this.  Are there any other areas apart from
parameter substitution where the string is longer than we pass to the
pattern matching routines?

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>
Work: pws@CambridgeSiliconRadio.com
Web: http://www.pwstephenson.fsnet.co.uk

RE: Should we backup this change? RE: Modifier substitutions.

[Andrej Borsenkow]

> > > ${foo/$bar/...} into ${foo/\\$bar/...}
> >
> > The problem is (as I discovered later) it is not complete.
> There is no
> > way to anchor pattern to match full string.
>
> You should now be able to do ${foo/(#s)$bar(#e)/...}.

In this particular case I was about to simply use ${...:/...}
(untested). And this is the form I was referred to - you cannot put this
anchor in pattern.

I also believe, that it should be possible to simulate left/right
anchoring by using ${...:/...} and (pat)* or *(pat). May be, it could
match too much, dunno.

BTW where are these (#s) and (#e) useful? The patterns in parameter
expansions are anchored by definition; the patterns in conditional
expressions/case statements are anchored to full word as well. Could you
provide example where you cannot do without (#s)/(#b)?

In any case, this is additional argument in favour of backing out this
change (assuming, that these (#s)/(#b) really work).

-andrej

Re: Should we backup this change? RE: Modifier substitutions.

[Peter Stephenson]

> BTW where are these (#s) and (#e) useful? The patterns in parameter
> expansions are anchored by definition; the patterns in conditional
> expressions/case statements are anchored to full word as well. Could you
> provide example where you cannot do without (#s)/(#b)?

In alternatives.  Try reading the manual entry.

> In any case, this is additional argument in favour of backing out this
> change (assuming, that these (#s)/(#b) really work).

If they worked here it would certainly simplify substitutions; although
there are ways of doing most things with the existing anchors, they are
even less readable and memorable (and that's saying something).

-- 
Peter Stephenson <pws@cambridgesiliconradio.com>
Cambridge Silicon Radio, Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

Let's do it RE: Should we backup this change? RE: Modifier substitutions.

[Andrej Borsenkow]

>
> The change went in in 6002.  It was followed in 6003 by a
> change to allow
> a leading double backslash to escape the delimiter.  So all
> you need to
> do is change uses of ${foo/$bar/...} into ${foo/\\$bar/...}
> to avoid the
> confusion.  And without 6002, there's no way to build up an anchored
> pattern in a variable (which was Andrej's original complaint).
>
> Also, though I'm the last person to argue for keeping a
> change solely for
> compatibility, bash recognizes the same syntaxes.
>


Unfortunately, it is so inconsistent, that I really suggest to backup
it. Look here:

from Zsh manual (example for ${.../.../...})

          foo="twinkle twinkle little star" sub="t*e" rep="spy"
          print ${foo//${~sub}/$rep}
          print ${(S)foo//${~sub}/$rep}

but

bor@itsrm2% foo=foo
bor@itsrm2% foopat='#foo'
bor@itsrm2% print ${foo/$~foopat/bar}
zsh: bad pattern: #foo
bor@itsrm2% print ${foo/$foopat/bar}
bar
bor@itsrm2% if [[ $foo == $~foopat ]]; then print yes; else print no; fi
zsh: bad pattern: #foo

Looking at the _arguments, there is no ~ flag either:

    # ... and add "same" options

    while (( $#sopts )); do
      lopts=( $lopts ${lopts/$sopts[1]/$sopts[2]} )
      shift 2 sopts
    done


So, currently

- example in manual is wrong. Or, better said, it _is_ correct because
#foo is invalid pattern - but, then, how _arguments work currently?
Well, they work because there is no metachars in pattern (the only use
is _configure). So, this is wrong anyway.

- any way to fix it will lead to inconsistency - #foo will match `foo'
in replacement, but not anywhere else.

- nobody really expects run native bash scripts with zsh.

- this "anchoring in pattern" is needed in probably 1% of all cases -
forcing use of `\\' in 99%. That is weird. And if anybody needs
anchoring - (#s) is there.

So, my strong suggestion - let's backup it, and for the time being use
(#s) in _configure (I actually do not like these "same" arguments
anyway - it does not work reliably, sigh)

-andrej

Re: Let's do it RE: Should we backup this change? RE: Modifier substitutions.

[Peter Stephenson]

> > The change went in in 6002.  It was followed in 6003 by a
> > change to allow
> > a leading double backslash to escape the delimiter.  So all
> > you need to
> > do is change uses of ${foo/$bar/...} into ${foo/\\$bar/...}
> > to avoid the
> > confusion.
> 
> Unfortunately, it is so inconsistent, that I really suggest to backup
> it.

I'm quite happy to remove it which is easy.  Currently there's no simple
replacement in the case of _configure, however, because (#s) doesn't get
tokenized (the `#' didn't need to be, it was always recognised, hence the
need for quoting).  This would require the -s option to _arguments to do
some kind of globsubst thing, which may or may not be desirable.  I haven't
looked for any other uses of this feature.

-- 
Peter Stephenson <pws@cambridgesiliconradio.com>
Cambridge Silicon Radio, Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

PATCH:RE: Let's do it RE: Should we backup this change? RE: Modifiersubstitutions.

[Andrej Borsenkow]

[-- Attachment #1: Type: text/plain, Size: 621 bytes --]

>
> I'm quite happy to remove it which is easy.  Currently
> there's no simple
> replacement in the case of _configure, however, because (#s)
> doesn't get
> tokenized (the `#' didn't need to be, it was always
> recognised, hence the
> need for quoting).  This would require the -s option to
> _arguments to do
> some kind of globsubst thing, which may or may not be
> desirable.  I haven't
> looked for any other uses of this feature.
>

That's because I used wrong syntax in _arguments (_long_options at the
time of writing). This patch gets it right and replaces # with (#s),
that also works. Patch attached.

-andrej

[-- Attachment #2: anchor.patch --]
[-- Type: application/octet-stream, Size: 999 bytes --]

--- Completion/Base/_arguments	Wed May  3 16:16:06 2000
+++ /home/bor/.zsh.d/completion/_arguments	Fri May  5 17:26:55 2000
@@ -82,7 +82,7 @@
     # ... and add "same" options
 
     while (( $#sopts )); do
-      lopts=( $lopts ${lopts/$sopts[1]/$sopts[2]} )
+      lopts=( $lopts ${lopts/$~sopts[1]/$sopts[2]} )
       shift 2 sopts
     done
 
--- Completion/User/_configure	Mon Aug 30 14:08:26 1999
+++ /home/bor/.zsh.d/completion/_configure	Fri May  5 17:27:49 2000
@@ -1,9 +1,9 @@
 #compdef configure
 
 _arguments -- -i '(--(disable|enable)-FEATURE* --(with|without)-PACKAGE*)' \
-              -s '(#--disable- --enable-
-	           #--enable- --disable-
-		   #--with- --without-
-		   #--without- --with-)' \
+              -s '((#s)--disable- --enable-
+	           (#s)--enable- --disable-
+		   (#s)--with- --without-
+		   (#s)--without- --with-)' \
 	      '*=(E|)PREFIX*:prefix directory:_files -/' \
               '*=PROGRAM*:program:_command_names -e'

PATCH: substitution anchors

[Peter Stephenson]

> That's because I used wrong syntax in _arguments (_long_options at the
> time of writing). This patch gets it right and replaces # with (#s),
> that also works. Patch attached.

OK, this would remove the effect of anchors in substituted parameters, and
makes it a bit clearer what's going on.  I don't if Bart is still keen on
keeping this effect.  This needs Andrej's patch (the only effect at the
moment is whether `configure' shows all --enable/--disable pairs when the
help text contains one of them, as is usual) and I will commit the two
together or not at all.

Index: Doc/Zsh/expn.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/expn.yo,v
retrieving revision 1.11
diff -u -r1.11 expn.yo
--- Doc/Zsh/expn.yo	2000/05/04 06:21:18	1.11
+++ Doc/Zsh/expn.yo	2000/05/05 14:20:23
@@ -456,13 +456,21 @@
 Replace the longest possible match of var(pattern) in the expansion of
 parameter var(name) by string var(repl).  The first form
 replaces just the first occurrence, the second form all occurrences.
+Both var(pattern) and var(repl) are subject to double-quoted substitution,
+so that expressions like tt(${name/$opat/$npat}) will work, but note the
+usual rule that pattern characters in tt($opat) are not treated specially
+unless either the option tt(GLOB_SUBST) is set, or tt($opat) is instead
+substituted as tt(${~opat}).
+
 The var(pattern) may begin with a `tt(#)', in which case the
 var(pattern) must match at the start of the string, or `tt(%)', in
 which case it must match at the end of the string.  The var(repl) may
 be an empty string, in which case the final `tt(/)' may also be omitted.
 To quote the final `tt(/)' in other cases it should be preceded by two
 backslashes (i.e., a quoted backslash); this is not necessary if the
-`tt(/)' occurs inside a substituted parameter.
+`tt(/)' occurs inside a substituted parameter.  Note also that the `tt(#)'
+and `tt(%)' are not active if they occur inside a substituted parameter,
+even at the start.
 
 The first `tt(/)' may be preceded by a `tt(:)', in which case the match
 will only succeed if it matches the entire word.  Note also the
Index: Src/subst.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/subst.c,v
retrieving revision 1.3
diff -u -r1.3 subst.c
--- Src/subst.c	2000/05/03 12:21:56	1.3
+++ Src/subst.c	2000/05/05 14:20:23
@@ -1466,6 +1466,13 @@
 		char t = s[-1];
 
 		singsub(&s);
+#if 0
+		/*
+		 * This allows # and % to be at the start of
+		 * a parameter in the substitution, which is
+		 * a bit nasty, and can be done (although
+		 * less efficiently) with anchors.
+		 */
 		if (t == '/' && (flags & SUB_SUBSTR)) {
 		    if ((c = *s) == '#' || c == '%') {
 			flags &= ~SUB_SUBSTR;
@@ -1476,6 +1483,7 @@
 			s++;
 		    }
 		}
+#endif
 	    }
 
 	    if (!vunset && isarr) {

-- 
Peter Stephenson <pws@cambridgesiliconradio.com>
Cambridge Silicon Radio, Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

Re: PATCH: substitution anchors

[Bart Schaefer]

On May 5,  3:26pm, Peter Stephenson wrote:
} Subject: PATCH: substitution anchors
}
} OK, this would remove the effect of anchors in substituted parameters, and
} makes it a bit clearer what's going on.  I don't if Bart is still keen on
} keeping this effect.

I would never have said I was "keen" on it; go ahead and commit.

But you ought to mention the difference WRT bash, in the FAQ.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

RE: PATCH: substitution anchors

[Andrej Borsenkow]

>
> I would never have said I was "keen" on it; go ahead and commit.
>
> But you ought to mention the difference WRT bash, in the FAQ.
>

I suspect, bash does not have `pat#' syntax (that makes `#foo' invalid).
Better if FAQ also explains why we cannot preserve compatibility in this
case.

-andrej