Re: a='foo"'; echo ${a/foo"/"bar} outputs bar

[Bart Schaefer <schaefer@brasslantern.com>]

Circling back to this ...

On Mon, Dec 12, 2022 at 10:21 PM Bart Schaefer
<schaefer@brasslantern.com> wrote:
>
> On Sun, Dec 11, 2022 at 9:50 AM Stephane Chazelas <stephane@chazelas.org> wrote:
> >
> > $ a='foo"'
> > $ echo ${a/foo"/"bar}
> > bar
> >
> > Whether quotes should escape the "/" is not clearly documented,
> > though the doc does tell us to use backslash for that.
>
> The whole expression only works because the quotes are balanced:
>
> % echo ${x/foo"/bar}
> braceparam dquote>
>
> Clearly something a little wacky is going on here.

With an increased understanding of lex.c gained by stepping through
variations of ${|...} a thousand times, I think I have identified two
possible ways to address this.

> > However the fact that the first " is taken as being part of the
> > pattern while the second one is removed doesn't make much sense.
>
> This is the fault of singsub() called from line 2670 of
> compgetmatch(), which eventually calls remnulargs() via prefork(),
> which deletes the Dnull representing the double-quote.  This is only a
> problem because an unbalanced quote was able to sneak through.

One possible change is for the first quote to be taken as part of the
pattern but the second quote to NOT be removed.  This results in

$ echo ${a/foo"/"bar}
braceparam dquote>

It's always been possible to do quoted substrings in the replacement
rather than in the pattern, so this is consistent.  But it also means
that

$ a='foo"'
$ x= bar
$ echo ${a/foo"/"$x"}
$ bar

Something similar would have to be done with single quotes, because
handling double quotes doesn't fix this oddity:

% a="foo'"
% x=bar
% echo ${a/foo'/$'${x}y}
$bary

That is, the single quote after the slash is simply ignored in the
replacement.  That happens in 5.9, 5.8, and earlier.

The second possible change is to report "bad substitution" for the
original example.  As mentioned in the previous post,

> This was actually caught at paramsubst() line 3118:
>                 haserr = parse_subst_string(s);
> This returned haserr == 1, but the only effect of that is that the
> string is retokenized at line 3124.

An option I haven't pursued so far is to allow double-quoted
substrings to appear in the pattern.  This would make some sense given
that we allow parameter expansions in the pattern and can protect them
with single quotes, but using single quotes in the pattern has other
unexpected side-effects on the replacement:

% echo ${a/'foo"'/"$x"y}
"bar"y

Incidentally, it's sort-of possible in 5.8 to use $'...' quoting in
patterns, but the same odd stuff is going on there as well.

Thoughts?