The ~ parameter expansion flag: bug or misunderstanding

The ~ parameter expansion flag: bug or misunderstanding

[Clint Hepner]

[-- Attachment #1: Type: text/plain, Size: 355 bytes --]

I understand that you can split a parameter value on a fixed string:

% print -l ${(s.1.):-a1b1c}
a
b
c

My reading of the ~ flag leads me to believe that you can replace the
literal string with a pattern, so that

% print -l ${(~s.[12].):-a1b2c}
a
b
c

However, the ~ flag seems to have no effect, with the parameter string
remaining unsplit.

-- 
Clint

Re: The ~ parameter expansion flag: bug or misunderstanding

[Peter Stephenson]

On Wed, 03 Sep 2014 10:58:23 -0400
Clint Hepner <clint.hepner@gmail.com> wrote:

> I understand that you can split a parameter value on a fixed string:
> 
> % print -l ${(s.1.):-a1b1c}
> a
> b
> c
> 
> My reading of the ~ flag leads me to believe that you can replace the
> literal string with a pattern, so that
> 
> % print -l ${(~s.[12].):-a1b2c}
> a
> b
> c
> 
> However, the ~ flag seems to have no effect, with the parameter string
> remaining unsplit.

No, splitting is with (s.X.) is always on a string.

You can do something like

print -l ${(s.1.)${${:-a1b2c}//[12]/1}}

The documentation isn't very clear:

~      Force string arguments to any of the  flags  below  that  follow
       within  the parentheses to be treated as patterns.  Compare with
       a ~ outside parentheses, which  forces  the  entire  substituted
       string to be treated as a pattern.  Hence, for example,
       [[ "?" = ${(~j.|.)array} ]]

actually means string arguments remaining after substitution are treated
as a pattern subseqeuently on the command line --- which means that it
has no effect on splitting, despite "s" being one of the flags below.
So it doesn't really say what it does in this case.

-- 
Peter Stephenson <p.stephenson@samsung.com>  Principal Software Engineer
Tel: +44 (0)1223 434724                Samsung Cambridge Solution Centre
St John's House, St John's Innovation Park, Cowley Road,
Cambridge, CB4 0DS, UK

Re: The ~ parameter expansion flag: bug or misunderstanding

[Bart Schaefer]

On Sep 3,  4:14pm, Peter Stephenson wrote:
}
} The documentation isn't very clear

While we're sort of on the subject, the (l) and (r) padding flags say:

     The arguments :STRING1: and :STRING2: are optional; neither, the
     first, or both may be given.  Note that the same pairs of
     delimiters must be used for each of the three arguments.  The
     space to the left will be filled with STRING1 (concatenated as
     often as needed) or spaces if STRING1 is not given.  If both
     STRING1 and STRING2 are given, string2 is inserted once directly
     to the left of each word, truncated if necessary, before STRING1
     is used to produce any remaining padding.

What this doesn't say is that if STRING2 is the empty string, a space
is inserted instead:

torch% f=f
torch% print X${(l:2::.:)f}X
X.fX
torch% print X${(l:2::.:::)f}X
X fX
torch% print X${(l:5::.:::)f}X
X... fX
torch% print X${(l:5::.::+:)f}X
X...+fX

I'm not sure if that's a bug, or intentional.  Of course one might ask
why one would bother with the empty :: when it could simply be left out.

Re: The ~ parameter expansion flag: bug or misunderstanding

[Peter Stephenson]

On Wed, 03 Sep 2014 20:18:24 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> What this doesn't say is that if STRING2 is the empty string, a space
> is inserted instead:

Apparently the same for STRING1.  Looking at the code, this appears to
be the following intentional but undocumented feature.

diff --git a/Doc/Zsh/expn.yo b/Doc/Zsh/expn.yo
index d01d804..b4aa9c7 100644
--- a/Doc/Zsh/expn.yo
+++ b/Doc/Zsh/expn.yo
@@ -1155,6 +1155,10 @@ var(string1) and var(string2) are given, tt(string2) is inserted once
 directly to the left of each word, truncated if necessary, before
 var(string1) is used to produce any remaining padding.
 
+If either of var(string1) or tt(string2) is present but empty,
+i.e. there are two delimiters together at that point, the first
+character of tt($IFS) is used instead.
+
 If the tt(MULTIBYTE) option is in effect, the flag tt(m) may also
 be given, in which case widths will be used for the calculation of
 padding; otherwise individual multibyte characters are treated as occupying

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: The ~ parameter expansion flag: bug or misunderstanding

[Bart Schaefer]

On Sep 3, 10:58am, Clint Hepner wrote:
}
} My reading of the ~ flag leads me to believe that you can replace the
} literal string with a pattern, so that
} 
} % print -l ${(~s.[12].):-a1b2c}
} a
} b
} c

No, that's not what the flag means.  It means that after the expansion
of the parameter has been completed, any pattern characters inserted
into the result are active (i.e., not considered quoted).  It doesn't
otherwise change the operation of the other flags themselves.

This doesn't have any effect for the (s) flag because (s) removes
characters from the result, rather than inserting them.

The purpose of the (~) flag is really to work with the (j) flag, so
that you can create a pattern from an array without having any other
pattern characters in the values of the array also become active.
It also works with (l) and (r), though the uses for treating padding
as pattern characters are pretty obscure.

To use a pattern for processing the value during expansion, you need
to use the trailing // notation, e.g.

    % print -l ${=${:-a1b2c}//[12]/ }

or

    % print -l ${(s:1:)${:-a1b2c}//[12]/1}

Re: The ~ parameter expansion flag: bug or misunderstanding

[Clint Hepner]

[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]

Thanks. One thing I still can't quite wrap my mind around is a scenario
where

    ${(j.|.)~array}

and

    ${~j.|.)array}

would behave differently. I assume it would involve some additional
modifications made to $array; I tried something like

    array=(foo bar)
    [[ "?" = ${(~j.|.)array:s/foo/?/} ]]   # match
    [[ "?" = ${(j.|.)~array:s/foo/?/} ]]   # match

but that was not sufficient.

On Wed, Sep 3, 2014 at 11:44 AM, Bart Schaefer <schaefer@brasslantern.com>
wrote:

> On Sep 3, 10:58am, Clint Hepner wrote:
> }
> } My reading of the ~ flag leads me to believe that you can replace the
> } literal string with a pattern, so that
> }
> } % print -l ${(~s.[12].):-a1b2c}
> } a
> } b
> } c
>
> No, that's not what the flag means.  It means that after the expansion
> of the parameter has been completed, any pattern characters inserted
> into the result are active (i.e., not considered quoted).  It doesn't
> otherwise change the operation of the other flags themselves.
>
> This doesn't have any effect for the (s) flag because (s) removes
> characters from the result, rather than inserting them.
>
> The purpose of the (~) flag is really to work with the (j) flag, so
> that you can create a pattern from an array without having any other
> pattern characters in the values of the array also become active.
> It also works with (l) and (r), though the uses for treating padding
> as pattern characters are pretty obscure.
>
> To use a pattern for processing the value during expansion, you need
> to use the trailing // notation, e.g.
>
>     % print -l ${=${:-a1b2c}//[12]/ }
>
> or
>
>     % print -l ${(s:1:)${:-a1b2c}//[12]/1}
>

Re: The ~ parameter expansion flag: bug or misunderstanding

[Peter Stephenson]

On Tue, 09 Sep 2014 12:21:16 -0400
Clint Hepner <clint.hepner@gmail.com> wrote:
> Thanks. One thing I still can't quite wrap my mind around is a scenario
> where
> 
>     ${(j.|.)~array}
> 
> and
> 
>     ${~j.|.)array}
> 
> would behave differently.

It depends on the contents of array.  Here's the sort of scenario that
caused the ~ inside the parentheses to be added.

Supposing someone has helpfully created a file "a|b", and your array is
a list of files in the current directory,

array=(*)

so it contains an alement "a|b".  zsh with its default settings usually
handles that fine because it makes a point of not expanding the results
of variable substitutions as patterns for exactly this reason.

However, when you do ${(j.|.)~array} it looks like two alternatives, a
or b, because the "|" in the value of $array is expanded.

If you do ${(~j.|.)array}, on the other hand, the "a|b" is kept with the
| as an ordinary character, so it still works.

pws

Re: The ~ parameter expansion flag: bug or misunderstanding

[Clint Hepner]

[-- Attachment #1: Type: text/plain, Size: 1268 bytes --]

Ah, that makes sense: ${(j.|.)~array} treats *all* | as part of the
pattern, but `${(~j.|.)array}` treats only those added as part of the join
operation. Thank you.

On Tue, Sep 9, 2014 at 12:41 PM, Peter Stephenson <p.stephenson@samsung.com>
wrote:

> On Tue, 09 Sep 2014 12:21:16 -0400
> Clint Hepner <clint.hepner@gmail.com> wrote:
> > Thanks. One thing I still can't quite wrap my mind around is a scenario
> > where
> >
> >     ${(j.|.)~array}
> >
> > and
> >
> >     ${~j.|.)array}
> >
> > would behave differently.
>
> It depends on the contents of array.  Here's the sort of scenario that
> caused the ~ inside the parentheses to be added.
>
> Supposing someone has helpfully created a file "a|b", and your array is
> a list of files in the current directory,
>
> array=(*)
>
> so it contains an alement "a|b".  zsh with its default settings usually
> handles that fine because it makes a point of not expanding the results
> of variable substitutions as patterns for exactly this reason.
>
> However, when you do ${(j.|.)~array} it looks like two alternatives, a
> or b, because the "|" in the value of $array is expanded.
>
> If you do ${(~j.|.)array}, on the other hand, the "a|b" is kept with the
> | as an ordinary character, so it still works.
>
> pws
>