(s) splitting – is there any way to provide «dynamic» separator

(s) splitting – is there any way to provide «dynamic» separator

[Vasiliy Ivanov]

Hi,

Sorry if I (maybe) missed something obvious, but I failed to find a way to use separator from
parameter (e.g. a='1:2:3'; sep=':'; print -l ${(s.$sep.)a}).
I think this is because expansion/substitution are not available in flags? If so, is there any
another way for such a «dynamic» splitting?

-- 
Regards,
  Vasiliy Ivanov <beelzebubbie.logs@gmail.com>

Re: (s) splitting – is there any way to provide «dynamic» separator

[Peter Stephenson]

On Thu, 09 Oct 2014 21:00:32 +0600
Vasiliy Ivanov <beelzebubbie.logs@gmail.com> wrote:
> Sorry if I (maybe) missed something obvious, but I failed to find a way to use separator from
> parameter (e.g. a='1:2:3'; sep=':'; print -l ${(s.$sep.)a}).
> I think this is because expansion/substitution are not available in
> flags? If so, is there any  another way for such a «dynamic» splitting?

I don't think you've missed anything obvious or elegant.

One way is

(){
  local IFS=$sep
  print -l ${=a}
}

Or if you can rely on not having whitespace

print -l ${=${a//:/ }}

Or, of course,

eval print -l '${(s.'$sep'.)a}'

as long as you can sanity check $sep.


Of course there's nothing to stop us adding a simple parameter
substitution as a special case at this point (or any similarly delimited
parameter argument) ...  Can't see how this can do much harm since it
sure as heck *looks* like a parameter expansion.

diff --git a/Src/subst.c b/Src/subst.c
index 1aa9b98..5727c12 100644
--- a/Src/subst.c
+++ b/Src/subst.c
@@ -1385,12 +1385,23 @@ static char *
 untok_and_escape(char *s, int escapes, int tok_arg)
 {
     int klen;
-    char *dst;
+    char *dst = NULL;
 
-    untokenize(dst = dupstring(s));
-    if (escapes) {
-	dst = getkeystring(dst, &klen, GETKEYS_SEP, NULL);
-	dst = metafy(dst, klen, META_HREALLOC);
+    if ((*s == String || *s == Qstring) && s[1]) {
+	char *pstart = s+1, *pend;
+	for (pend = pstart; *pend; pend++)
+	    if (!iident(*pend))
+		break;
+	if (!*pend) {
+	    dst = dupstring(getsparam(pstart));
+	}
+    }
+    if (dst == NULL) {
+	untokenize(dst = dupstring(s));
+	if (escapes) {
+	    dst = getkeystring(dst, &klen, GETKEYS_SEP, NULL);
+	    dst = metafy(dst, klen, META_HREALLOC);
+	}
     }
     if (tok_arg)
 	shtokenize(dst);

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: (s) splitting – is there any way to provide «dynamic» separator

[Mikael Magnusson]

On 9 October 2014 23:14, Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> On Thu, 09 Oct 2014 21:00:32 +0600
> Vasiliy Ivanov <beelzebubbie.logs@gmail.com> wrote:
>> Sorry if I (maybe) missed something obvious, but I failed to find a way to use separator from
>> parameter (e.g. a='1:2:3'; sep=':'; print -l ${(s.$sep.)a}).
>> I think this is because expansion/substitution are not available in
>> flags? If so, is there any  another way for such a «dynamic» splitting?
>
> I don't think you've missed anything obvious or elegant.
>
> One way is
>
> (){
>   local IFS=$sep
>   print -l ${=a}
> }
>
> Or if you can rely on not having whitespace
>
> print -l ${=${a//:/ }}
>
> Or, of course,
>
> eval print -l '${(s.'$sep'.)a}'
>
> as long as you can sanity check $sep.
>
>
> Of course there's nothing to stop us adding a simple parameter
> substitution as a special case at this point (or any similarly delimited
> parameter argument) ...  Can't see how this can do much harm since it
> sure as heck *looks* like a parameter expansion.

I checked to make sure you can still do (s:$:) and that does still
work. Moving into extremely unlikely corner-case land, (s:$b:) would
split on the literal string $b before. With the patch, it actually
still does, but only if b is unset. If b is set, then it is expanded
and the expansion is split on instead. So the only problem here would
be if someone used to split on a literal string $foo and then foo is
set, their code would stop working.

I don't know if it's worth it but we could possibly elect to only do
this expansion if the p flag is active? Right now it means to expand
\t into a literal tab, but it makes sense to have it more generally
mean "expand stuff" in the string I think?

eg, I think this would do it?
+    if (escapes && (*s == String || *s == Qstring) && s[1]) {


-- 
Mikael Magnusson

Re: (s) splitting - is there any way to provid e "dynamic" separator

[Bart Schaefer]

On Oct 9,  9:00pm, Vasiliy Ivanov wrote:
}
} Sorry if I (maybe) missed something obvious, but I failed to find a
} way to use separator from parameter
} (e.g. a='1:2:3'; sep=':'; print -l ${(s.$sep.)a}).

Something like this:

    print -l ${(ps.\0.)a//$sep/$'\0'}

(Assuming there are no nul-bytes in the value of $a to begin with.)

Re: (s) splitting - is there any way to provide "dynamic" separator

[Vasiliy Ivanov]

On 10.10.2014 07:45, Bart Schaefer wrote:
> On Oct 9,  9:00pm, Vasiliy Ivanov wrote:
> }
> } Sorry if I (maybe) missed something obvious, but I failed to find a
> } way to use separator from parameter
> } (e.g. a='1:2:3'; sep=':'; print -l ${(s.$sep.)a}).
> 
> Something like this:
> 
>     print -l ${(ps.\0.)a//$sep/$'\0'}
> 
> (Assuming there are no nul-bytes in the value of $a to begin with.)
> 

Thanks, this seems more elegant than «eval» way, but I failed to understand this:

% a='11::22:33'; b=("${(@s.:.)a}"); print $#b
4 (as expected)

but (I expected same result)

% a='11::22:33'; sep=':'; b=("${(@ps.\0.)a//$sep/$'\0'}"); print $#b
1

while

% a='11::22:33'; sep=':'; b=(${(@ps.\0.)a//$sep/$'\0'}); print $#b
3

-- 
Regards,
  Vasiliy Ivanov <beelzebubbie.logs@gmail.com>

Re: (s) splitting – is there any way to provide «dynamic» separator

[Peter Stephenson]

On Thu, 9 Oct 2014 23:48:37 +0200
Mikael Magnusson <mikachu@gmail.com> wrote:
> I don't know if it's worth it but we could possibly elect to only do
> this expansion if the p flag is active? Right now it means to expand
> \t into a literal tab, but it makes sense to have it more generally
> mean "expand stuff" in the string I think?

Yes, that works very well because anything with an escape is not a
parameter expansion.  So you're only going to come a cropper if you use
the (p) flag for no reason.  It's also easy and gives me a single place
to document the effect.

We could in principle do a full singsub() on the string argument when
the flag is present, but I think that's asking for trouble in terms of
complexity.

diff --git a/Doc/Zsh/expn.yo b/Doc/Zsh/expn.yo
index 5aab259..a0478e7 100644
--- a/Doc/Zsh/expn.yo
+++ b/Doc/Zsh/expn.yo
@@ -1124,6 +1124,19 @@ item(tt(p))(
 Recognize the same escape sequences as the tt(print) builtin
 in string arguments to any of the flags described below that
 follow this argument.
+
+Alternatively, with this option string arguments may be in the form
+tt($)var(var) in which case the value of the variable is substituted.
+Note this form is strict; the string argument does not undergo general
+parameter expansion.
+
+For example,
+
+example(sep=:
+val=a:b:c
+print ${+LPAR()ps.$sep.+RPAR()val})
+
+splits the variable on a tt(:).
 )
 item(tt(~))(
 Strings inserted into the expansion by any of the flags below are to
diff --git a/Src/subst.c b/Src/subst.c
index 1aa9b98..61aa1c1 100644
--- a/Src/subst.c
+++ b/Src/subst.c
@@ -1385,12 +1385,23 @@ static char *
 untok_and_escape(char *s, int escapes, int tok_arg)
 {
     int klen;
-    char *dst;
+    char *dst = NULL;
 
-    untokenize(dst = dupstring(s));
-    if (escapes) {
-	dst = getkeystring(dst, &klen, GETKEYS_SEP, NULL);
-	dst = metafy(dst, klen, META_HREALLOC);
+    if (escapes && (*s == String || *s == Qstring) && s[1]) {
+	char *pstart = s+1, *pend;
+	for (pend = pstart; *pend; pend++)
+	    if (!iident(*pend))
+		break;
+	if (!*pend) {
+	    dst = dupstring(getsparam(pstart));
+	}
+    }
+    if (dst == NULL) {
+	untokenize(dst = dupstring(s));
+	if (escapes) {
+	    dst = getkeystring(dst, &klen, GETKEYS_SEP, NULL);
+	    dst = metafy(dst, klen, META_HREALLOC);
+	}
     }
     if (tok_arg)
 	shtokenize(dst);

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: (s) splitting - is there any way to provide "dynamic" separator

[Han Pingtian]

On Fri, Oct 10, 2014 at 01:06:21PM +0600, Vasiliy Ivanov wrote:
> On 10.10.2014 07:45, Bart Schaefer wrote:
> > On Oct 9,  9:00pm, Vasiliy Ivanov wrote:
> > }
> > } Sorry if I (maybe) missed something obvious, but I failed to find a
> > } way to use separator from parameter
> > } (e.g. a='1:2:3'; sep=':'; print -l ${(s.$sep.)a}).
> > 
> > Something like this:
> > 
> >     print -l ${(ps.\0.)a//$sep/$'\0'}
> > 
> > (Assuming there are no nul-bytes in the value of $a to begin with.)
> > 
> 
> Thanks, this seems more elegant than «eval» way, but I failed to understand this:
> 
> % a='11::22:33'; b=("${(@s.:.)a}"); print $#b
> 4 (as expected)
> 
> but (I expected same result)
> 
> % a='11::22:33'; sep=':'; b=("${(@ps.\0.)a//$sep/$'\0'}"); print $#b
> 1
> 

% a=11::22:33;print -l ${(ps.\0.)a//:/$'\0'}
11
22
33
% a=11::22:33;print -l "${(ps.\0.)a//:/$'\0'}"
11$''$''22$''33
%

Looks like $'\0' in double-quotes is converted to $'' and it doesn't equal
to \0 ?

Re: (s) splitting - is there any way to provide "dynamic" separator

[Bart Schaefer]

On Oct 11,  8:38am, Han Pingtian wrote:
}
} % a=11::22:33;print -l ${(ps.\0.)a//:/$'\0'}
} 11
} 22
} 33
} % a=11::22:33;print -l "${(ps.\0.)a//:/$'\0'}"
} 11$''$''22$''33
} %
} 
} Looks like $'\0' in double-quotes is converted to $'' and it doesn't equal
} to \0 ?

Although it looks like an expansion, $'...' is actually a form of quotes,
and therefore cannot be used inside a double-quoted string.

The \0 is actually being removed by "print -l".  Add the -R option:

torch% a=11::22:33;print -l -R "${(ps.\0.)a//:/$'\0'}" 
11$'\0'$'\0'22$'\0'33
torch% 

Re: (s) splitting - is there any way to provide "dynamic" separator

[Han Pingtian]

On Fri, Oct 10, 2014 at 08:31:35PM -0700, Bart Schaefer wrote:
> On Oct 11,  8:38am, Han Pingtian wrote:
> }
> } % a=11::22:33;print -l ${(ps.\0.)a//:/$'\0'}
> } 11
> } 22
> } 33
> } % a=11::22:33;print -l "${(ps.\0.)a//:/$'\0'}"
> } 11$''$''22$''33
> } %
> } 
> } Looks like $'\0' in double-quotes is converted to $'' and it doesn't equal
> } to \0 ?
> 
> Although it looks like an expansion, $'...' is actually a form of quotes,
> and therefore cannot be used inside a double-quoted string.
> 
Thanks.
> The \0 is actually being removed by "print -l".  Add the -R option:
> 
> torch% a=11::22:33;print -l -R "${(ps.\0.)a//:/$'\0'}" 
> 11$'\0'$'\0'22$'\0'33
> torch% 

Re: (s) splitting – is there any way to provide «dynamic» separator

[Peter Stephenson]

Here's a test.

diff --git a/Test/D04parameter.ztst b/Test/D04parameter.ztst
index 49dcea9..d7f39cb 100644
--- a/Test/D04parameter.ztst
+++ b/Test/D04parameter.ztst
@@ -1636,3 +1636,23 @@
   print ${noexist:^foo})
 1:Zipping arrays, NO_UNSET part 2
 ?(eval):2: noexist: parameter not set
+
+  expr="a@b,c@d:e@f,g@h:i@j,k@l"
+  for sep in : , @; do
+    print -l ${(ps.$sep.)expr}
+  done
+0:Use of variable to get separator when splitting parameter
+>a@b,c@d
+>e@f,g@h
+>i@j,k@l
+>a@b
+>c@d:e@f
+>g@h:i@j
+>k@l
+>a
+>b,c
+>d:e
+>f,g
+>h:i
+>j,k
+>l

pws