zsh-users
 help / color / mirror / code / Atom feed
* coloring substitution seems to eat next line.
@ 2022-11-10  0:43 Ray Andrews
  2022-11-10  8:02 ` Roman Perepelitsa
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-10  0:43 UTC (permalink / raw)
  To: Zsh Users

So I'm trying to replace a sed with native zsh code.  This stuff in 
inside a 'for' loop with 'aa' as the counter and  '$filter' is 'zsh':

     cc[$aa]=$( print -r $cc[$aa] | sed -rn 
"s/($filter)/\x1b\[$color;1m\1\x1b\[0m/${sed_case}gp" )

... that's the sed version working fine.  But this:

     print "going in: $cc[aa]"

     cc[aa]=( 
${${cc[aa]}//(#b)((#i)${filter})/$'\e[31;1m'${match[1]}$'\e[0m'} )

     print "coming out: $cc[aa]"

... almost works but it seems to eat the next element of the array every 
now and then with no pattern that I can discern.

/aWorking/garbageZSH              colored
/usr/share/zsh                            colored
/aWorking/Zsh                            disappears going in but seems 
to be there going out , but not colored
/aWorking/Backup/Zsh                ditto
/aWorking/Zsh-55555                  colored
/aWorking/Zsh/Zsh-5.8                no color as with above
/usr/share/doc/zsh-common        ditto

... however if I make the output array different from the input array 
everything is fine.  But I've always had success with that sort of array 
being overwritten by some manipulation of itself, tho it seems 
dangerous.  sed has no issue with it.   Anyway, the expansion seems to 
interfere with the the clean separation of the elements that I'm hoping 
for.  Using a separate output array is fine, still I'm curious as to 
what's going on.  In particular the way some lines seem to disappear 
going 'in' -- I print the array element -- and get nothing at all, but 
there is output ... which seems impossible (tho it's never colorized).  
It seems the expansion/substitution is ignoring line endings or some 
such.   I don't doubt there's some logic to what's happening but I sure 
don't get it.  It hasta be line separated.






^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10  0:43 coloring substitution seems to eat next line Ray Andrews
@ 2022-11-10  8:02 ` Roman Perepelitsa
  2022-11-10 18:25   ` Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Roman Perepelitsa @ 2022-11-10  8:02 UTC (permalink / raw)
  To: Ray Andrews; +Cc: Zsh Users

On Thu, Nov 10, 2022 at 1:44 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>      cc[$aa]=$( print -r $cc[$aa] | sed -rn
> "s/($filter)/\x1b\[$color;1m\1\x1b\[0m/${sed_case}gp" )
>
> ... that's the sed version working fine.  But this:
>
>      cc[aa]=(
> ${${cc[aa]}//(#b)((#i)${filter})/$'\e[31;1m'${match[1]}$'\e[0m'} )

Differences in the second version compared to the first:

- aa instead of $aa
- slice assignment instead of scalar (almost certainly unintended)
- hardcoded (#i) instead of $sed_case
- hardcoded 31 instead of $color
- $filter treated as a literal instead of regex

Given that the second version works without errors, I assume that cc
must be a regular (not associative) array. This means you can replace
the whole loop with this:

  local MATCH MBEGIN MEND
  cc=(${cc//(#m)$~zsh_case$filter/$'\e[31;1m'$MATCH$'\e[0m'})

Here's a test:

  () {
    emulate -L zsh -o extended_glob
    local cc=(/foo/Zsh/bar /baz /ZSHzsh /zsh-zsh)
    local filter=zsh
    local zsh_case='(#i)'
    local MATCH MBEGIN MEND
    cc=(${cc//(#m)$~zsh_case$filter/$'\e[31;1m'$MATCH$'\e[0m'})
    print -rC1 -- ${(qqqq)cc}
    print -rC1 -- $cc
  }

> /aWorking/Zsh                            disappears going in but seems
> to be there going out , but not colored

You can debug this by printing all relevant parameters.

    print -r -- "going in: $cc[aa]"
    typeset -p cc aa filter
    ...
    typeset -p cc
    print -r -- "coming out: $cc[aa]"

After that you can reproduce and dissect the issue in isolation. You
can also ask other people for help and give them everything they need
to reproduce the problem.

Roman.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10  8:02 ` Roman Perepelitsa
@ 2022-11-10 18:25   ` Ray Andrews
  2022-11-10 18:36     ` Roman Perepelitsa
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-10 18:25 UTC (permalink / raw)
  To: zsh-users


On 2022-11-10 00:02, Roman Perepelitsa wrote:

> Differences in the second version compared to the first: 
Sorry about the sloppy code Roman,  the differences there don't matter 
but you have no way of knowing that.
> - aa instead of $aa
Does that matter?  I thought it was one of those situations where the 
dollar sign is optional.
> - slice assignment instead of scalar (almost certainly unintended)
Indeed.  I don't even know what the difference is.  I've never heard of 
slice assignment :(
> Given that the second version works without errors
The first version using sed.
> This means you can replace
> the whole loop with this:
>
>    local MATCH MBEGIN MEND
>    cc=(${cc//(#m)$~zsh_case$filter/$'\e[31;1m'$MATCH$'\e[0m'})

Unfortunately not.  That works as you intend, but I need to filter not 
just colorize.  Any line without all matches must be deleted.  The 'for' 
loop runs the lines thru each filter in turn and must zero any line that 
does not match.

You can debug this by printing all relevant parameters.
>      print -r -- "going in: $cc[aa]"
>      typeset -p cc aa filter

Good idea!  That's going to be useful.




^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 18:25   ` Ray Andrews
@ 2022-11-10 18:36     ` Roman Perepelitsa
  2022-11-10 18:45       ` Ray Andrews
  2022-11-10 18:54       ` Bart Schaefer
  0 siblings, 2 replies; 39+ messages in thread
From: Roman Perepelitsa @ 2022-11-10 18:36 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Thu, Nov 10, 2022 at 7:25 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> On 2022-11-10 00:02, Roman Perepelitsa wrote:
>
> > Differences in the second version compared to the first:
> Sorry about the sloppy code Roman,  the differences there don't matter
> but you have no way of knowing that.
> > - aa instead of $aa
> Does that matter?  I thought it was one of those situations where the
> dollar sign is optional.

It matters if cc is an associative array. Given that you've
(accidentally) used slice assignment with cc, it cannot be an
associative array.

> > - slice assignment instead of scalar (almost certainly unintended)
> I've never heard of slice assignment :(

These two aren't the same:

  cc[42]="foo bar"
  cc[42]=(foo bar)

The second is a slice assignment.

> > This means you can replace
> > the whole loop with this:
> >
> >    local MATCH MBEGIN MEND
> >    cc=(${cc//(#m)$~zsh_case$filter/$'\e[31;1m'$MATCH$'\e[0m'})
>
> Unfortunately not.  That works as you intend, but I need to filter not
> just colorize. Any line without all matches must be deleted

You can filter beforehand like this:

  cc=(${(M)cc:#*$filter*})

This assumes that $filter doesn't have metacharacters. If it does and
you want to treat them as such, use $~filter.

Roman.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 18:36     ` Roman Perepelitsa
@ 2022-11-10 18:45       ` Ray Andrews
  2022-11-10 18:54       ` Bart Schaefer
  1 sibling, 0 replies; 39+ messages in thread
From: Ray Andrews @ 2022-11-10 18:45 UTC (permalink / raw)
  To: zsh-users


On 2022-11-10 10:36, Roman Perepelitsa wrote:
> Does that matter? I thought it was one of those situations where the
>> dollar sign is optional.
> It matters if cc is an associative array. Given that you've
> (accidentally) used slice assignment with cc, it cannot be an
> associative array.
Correct.  Anyway that's good to know, I think I've only used an 
associative array once and not here.

> I've never heard of slice assignment :(
> These two aren't the same:
>
>    cc[42]="foo bar"
>    cc[42]=(foo bar)
>
> The second is a slice assignment.
Ok, I'll play with that.  Sheesh, the things I don't even know that I 
don't know.  Is there somewhere I can read up on that?
> You can filter beforehand like this:
>    cc=(${(M)cc:#*$filter*})
>
> This assumes that $filter doesn't have metacharacters. If it does and
> you want to treat them as such, use $~filter.
Thanks, I'll tinker with that.  It will be satisfying to be able to 
avoid sed ... come to that I just went back to grep for the filter and 
colorization because sed's separation character might just be part of a 
directory name.  These little things! grep is faster too. But native 
code will be best.
>
> Roman.
>


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 18:36     ` Roman Perepelitsa
  2022-11-10 18:45       ` Ray Andrews
@ 2022-11-10 18:54       ` Bart Schaefer
  2022-11-10 19:28         ` Ray Andrews
  1 sibling, 1 reply; 39+ messages in thread
From: Bart Schaefer @ 2022-11-10 18:54 UTC (permalink / raw)
  To: zsh-users; +Cc: Ray Andrews

On Thu, Nov 10, 2022 at 10:36 AM Roman Perepelitsa
<roman.perepelitsa@gmail.com> wrote:
>
> These two aren't the same:
>
>   cc[42]="foo bar"
>   cc[42]=(foo bar)

The first one resets cc[42].  The second one both updates cc[42] and
inserts a new cc[43] after it.  There is also the form

cc[42,44]=(foo bar)

which removes three elements (42,43,44) and replaces them with two new
ones (42 and 43).  Hence a "slice".


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 18:54       ` Bart Schaefer
@ 2022-11-10 19:28         ` Ray Andrews
  2022-11-10 20:22           ` Bart Schaefer
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-10 19:28 UTC (permalink / raw)
  To: zsh-users


On 2022-11-10 10:54, Bart Schaefer wrote:
>
>>    cc[42]="foo bar"
>>    cc[42]=(foo bar)
> The first one resets cc[42].  The second one both updates cc[42] and
> inserts a new cc[43] after it.

Now there's a surprise.  Nuts, that explains the weird output and the 
seemingly empty next element.  It bleeding creates a new empty next 
element!!  I'll leave it to more adept people to understand why that's 
ever wanted.

Anyway, I stopped slicing (or did I start slicing? ...) anyway:

cc[$aa]=( ${cc[$aa]/(#b)((#i)$filter)/$'\e['$color;1m${match[1]}$'\e[0m'} )

... is the now slightly understandable strange output and ...

cc[$aa]=${cc[$aa]/(#b)((#i)$filter)/$'\e['$color;1m${match[1]}$'\e[0m'}

...  works just fine :-)  I've gotten into the habit of always using the 
parentheses just because.


> There is also the form
>
> cc[42,44]=(foo bar)
>
> which removes three elements (42,43,44) and replaces them with two new
> ones (42 and 43).  Hence a "slice".

Sheesh.  So it looks like I stopped slicing.  But that just above is 
sorta understandable.

BTW I'm not using Roman's '(#M)' syntax because it seems to want to 
print all sorts of values to the terminal.  Like '$MATCH' is some 
internal variable that always prints itself or something.

One last issue, tinkering with the code even as I compose this:


cc[$aa]=${cc[$aa]/(#b)(${zsh_case}${filter})/$'\e['$color;1m${match[1]}$'\e[0m'}

... the one thing left is to insert '$zsh_case' in there, which so far 
is not working.  But Roman has the answer:

cc[$aa]=${cc[$aa]/(#b)($~zsh_case${filter})/$'\e['$color;1m${match[1]}$'\e[0m'}

... I have not idea what '$~' means, but it works.

BTW, to my surprise, using native code gives the same performance as 
using grep, 40 seconds on a stress test here.  sed wanted 43 seconds.





^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 19:28         ` Ray Andrews
@ 2022-11-10 20:22           ` Bart Schaefer
  2022-11-10 21:42             ` Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Bart Schaefer @ 2022-11-10 20:22 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Thu, Nov 10, 2022 at 11:28 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I'll leave it to more adept people to understand why that's
> ever wanted.

It's just a quick way to lengthen or shorten arrays when not merely
adding elements at the end.

> Anyway, I stopped slicing (or did I start slicing? ...) anyway:
>
> cc[$aa]=( ${cc[$aa]/(#b)((#i)$filter)/$'\e['$color;1m${match[1]}$'\e[0m'} )
>
> ... is the now slightly understandable strange output

That would only be "strange" if the result in the parens somehow came
out as either nothing, or as two or more strings with a space between,
but ordinarily the fact that you're doing a single ${cc[$aa]}
(regardless of the replacements) would mean you get exactly one
element.

> I've gotten into the habit of always using the
> parentheses just because.

Most of the time you want the parens when assigning all the elements
of an array at once.  You should not use the parens for anything
that's meant to be a single string (or number).

> BTW I'm not using Roman's '(#M)' syntax because it seems to want to
> print all sorts of values to the terminal.

That's almost certainly because of
> > local MATCH MBEGIN MEND

which should never appear except inside a function, and should only
appear at the beginning of the function and not inside a loop.

> ... I have not idea what '$~' means, but it works.

$~foo (or ${~foo} means that the value of $foo should be interpreted
as a glob pattern rather than as a literal string.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 20:22           ` Bart Schaefer
@ 2022-11-10 21:42             ` Ray Andrews
  2022-11-10 21:51               ` Roman Perepelitsa
  2022-11-10 22:47               ` Bart Schaefer
  0 siblings, 2 replies; 39+ messages in thread
From: Ray Andrews @ 2022-11-10 21:42 UTC (permalink / raw)
  To: zsh-users


On 2022-11-10 12:22, Bart Schaefer wrote:
>
> Most of the time you want the parens when assigning all the elements
> of an array at once.  You should not use the parens for anything
> that's meant to be a single string (or number).
Slowly I figure out what's really going on and don't have to rely on 
rote copying of syntax.
> local MATCH MBEGIN MEND
> which should never appear except inside a function, and should only
> appear at the beginning of the function and not inside a loop.
Heavy duty diagnostic stuff it seems.
> $~foo (or ${~foo} means that the value of $foo should be interpreted
> as a glob pattern rather than as a literal string.

Ah!  When you type it in there verbatim it seems to always be the 
pattern, but as a variable who's to say that the intention is?  So some 
way of making it explicit avoids semantic confusion.  That's robust, I 
like it.  One more little thing:


string="${cc[$aa]/*(#i)$filter*/}"
if [[ "$string" ]]; then
cc[aa]=''
else
cc[$aa]=${cc[$aa]/(#b)($~zsh_case${filter})/$'\e['$color;1m${match[1]}$'\e[0m'}

... to delete non-matching lines, but I'll bet there's a better way than 
this failed substitution.




^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 21:42             ` Ray Andrews
@ 2022-11-10 21:51               ` Roman Perepelitsa
  2022-11-11 17:24                 ` Ray Andrews
  2022-11-10 22:47               ` Bart Schaefer
  1 sibling, 1 reply; 39+ messages in thread
From: Roman Perepelitsa @ 2022-11-10 21:51 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Thu, Nov 10, 2022 at 10:42 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>
> On 2022-11-10 12:22, Bart Schaefer wrote:
> >
> > Most of the time you want the parens when assigning all the elements
> > of an array at once.  You should not use the parens for anything
> > that's meant to be a single string (or number).
> Slowly I figure out what's really going on and don't have to rely on
> rote copying of syntax.
> > local MATCH MBEGIN MEND
> > which should never appear except inside a function, and should only
> > appear at the beginning of the function and not inside a loop.
> Heavy duty diagnostic stuff it seems.
> > $~foo (or ${~foo} means that the value of $foo should be interpreted
> > as a glob pattern rather than as a literal string.
>
> Ah!  When you type it in there verbatim it seems to always be the
> pattern, but as a variable who's to say that the intention is?  So some
> way of making it explicit avoids semantic confusion.  That's robust, I
> like it.  One more little thing:
>
>
> string="${cc[$aa]/*(#i)$filter*/}"
> if [[ "$string" ]]; then
> cc[aa]=''
> else
> cc[$aa]=${cc[$aa]/(#b)($~zsh_case${filter})/$'\e['$color;1m${match[1]}$'\e[0m'}

If you look back, you can find in my answers a way to:

1. Remove all elements of an array that don't match a pattern. It has this form:

  array=(${(M)array:#pattern})

2. Perform a replacement in all elements of an array. Like this:

  array=(${array//pattern/substitute})

Perhaps now you have enough familiarity with the syntax to see how it works.

You can save a tremendous amount of time by reading the official guide
and the reference. Set aside an evening or two and read them from
start to end. Exploring a language as a science project is a great way
to get started but at some point you'll become more efficient by
reading the blueprint. After that the language becomes an artifact
rather than a blackbox.

Roman.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 21:42             ` Ray Andrews
  2022-11-10 21:51               ` Roman Perepelitsa
@ 2022-11-10 22:47               ` Bart Schaefer
  2022-11-10 23:07                 ` Ray Andrews
  1 sibling, 1 reply; 39+ messages in thread
From: Bart Schaefer @ 2022-11-10 22:47 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Thu, Nov 10, 2022 at 1:42 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> > local MATCH MBEGIN MEND
> > which should never appear except inside a function, and should only
> > appear at the beginning of the function and not inside a loop.
> Heavy duty diagnostic stuff it seems.

Well, no.  "local" is a variable scope declaration.  But "local" is a
synonym for "typeset" and we just had that long discussion about how
"typeset" both declares variables and prints their values ... so you
don't want to execute it multiple times, because the second+ time it
is going to start printing instead of declaring.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 22:47               ` Bart Schaefer
@ 2022-11-10 23:07                 ` Ray Andrews
  2022-11-10 23:27                   ` Bart Schaefer
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-10 23:07 UTC (permalink / raw)
  To: zsh-users


On 2022-11-10 14:47, Bart Schaefer wrote:
> Heavy duty diagnostic stuff it seems.
> Well, no.  "local" is a variable scope declaration.

Yabut MBEGIN and MEND are showing values that are never explicitly set, 
they are showing some internal workings surely?

Roman:

> If you look back, you can find in my answers a way to:

... there's stuff pending.  I had to sort out some other issues and now I'll be attending to your other suggestions.  I sometimes have more information than I can digest at once.  And I'm an old dog.





^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 23:07                 ` Ray Andrews
@ 2022-11-10 23:27                   ` Bart Schaefer
  2022-11-11 15:00                     ` Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Bart Schaefer @ 2022-11-10 23:27 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Thu, Nov 10, 2022 at 3:07 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Yabut MBEGIN and MEND are showing values that are never explicitly set,
> they are showing some internal workings surely?

They are set by the (#m) operator in ${cc//(#m).../...} ... try
looking those names up in the manual page, hmm?


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 23:27                   ` Bart Schaefer
@ 2022-11-11 15:00                     ` Ray Andrews
  2022-11-11 18:15                       ` Bart Schaefer
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-11 15:00 UTC (permalink / raw)
  To: zsh-users


On 2022-11-10 15:27, Bart Schaefer wrote:
> On Thu, Nov 10, 2022 at 3:07 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>> Yabut MBEGIN and MEND are showing values that are never explicitly set,
>> they are showing some internal workings surely?
> They are set by the (#m) operator in ${cc//(#m).../...} ... try
> looking those names up in the manual page, hmm?
If I ever cross paths with them again, then yeah.  Worth noting tho is 
that one might actually search for 'MBEGIN' on the internet or even as a 
raw text search in the manual and find something.  As I'm always 
whining, with most of zsh syntax issues that's impossible you have to 
already know the 'name of the family' of the sort of syntactical element 
that you are looking for.  Is it a glob expansion or a parameter flag or 
a pattern match or ... what?  You guys won't sympathize.  Mind, this 
problem is sorta unavoidable since the syntax is so terse.    What I 
wouldn't give for a glossary or a sort of overview of zsh syntactical 
elements  '(#i)' -- so I've learned -- is a regex not a glob not a 
pattern match not any other sort of animal.  Knowing that it's a regex, 
I know where to look for information. but '/**/' is a glob and '(N)' is 
a glob qualifier. Can't learn it from the inside out, only from the 
outside in.  But my little immediate problems are always from the inside 
out.  Slowly but surely tho.
>


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-10 21:51               ` Roman Perepelitsa
@ 2022-11-11 17:24                 ` Ray Andrews
  0 siblings, 0 replies; 39+ messages in thread
From: Ray Andrews @ 2022-11-11 17:24 UTC (permalink / raw)
  To: zsh-users


On 2022-11-10 13:51, Roman Perepelitsa wrote:
> If you look back, you can find in my answers a way to:
> 1. Remove all elements of an array that don't match a pattern. It has this form:
>
>    array=(${(M)array:#pattern})
>
> 2. Perform a replacement in all elements of an array. Like this:
>
>    array=(${array//pattern/substitute})
>
Fabulous:

         for filter in "$@"; do

             cc=( ${(M)cc:#*$~zsh_case${filter}*} )

             cc=( 
${cc//(#b)($~zsh_case${filter})/$'\e['$color;1m${match[1]}$'\e[0m'} )

         done

... no need for a a loop processing each line one at a time. Once the 
non-matching lines are removed, the whole array can be colorized in one 
gulp.  I knew there'd be an elegant way.

> You can save a tremendous amount of time by reading the official guide
> and the reference.
IF you know what you are looking for! I can't look for '(M)' before I 
know the thing exists to be looked for.
> Set aside an evening or two and read them from
> start to end.

This mortal would need to devote a year to it.  I have tried but one 
needs to understand the jargon before it's even intelligible. But yes, 
it needs to be learned from the outside in.  You experts don't realize 
how steep the learning curve is.  Oh, for a glossary!




^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-11 15:00                     ` Ray Andrews
@ 2022-11-11 18:15                       ` Bart Schaefer
  2022-11-11 18:50                         ` Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Bart Schaefer @ 2022-11-11 18:15 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Fri, Nov 11, 2022 at 7:00 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> On 2022-11-10 15:27, Bart Schaefer wrote:
> > On Thu, Nov 10, 2022 at 3:07 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
> >> Yabut MBEGIN and MEND are showing values that are never explicitly set,
> >> they are showing some internal workings surely?
> > They are set by the (#m) operator in ${cc//(#m).../...} ... try
> > looking those names up in the manual page, hmm?
> If I ever cross paths with them again, then yeah.

Why wouldn't you do this as soon as they began exhibiting behavior you
found mysterious?

> What I
> wouldn't give for a glossary or a sort of overview of zsh syntactical
> elements

You mean like maybe the six indexes at the bottom of the page here?
https://zsh.sourceforge.io/Doc/

> '(#i)' -- so I've learned -- is a regex not a glob not a
> pattern match not any other sort of animal.

That actually is (part of) a glob.  The (?i) thing for PCRE is not
going to be in the zsh doc, of course.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-11 18:15                       ` Bart Schaefer
@ 2022-11-11 18:50                         ` Ray Andrews
  2022-11-11 19:25                           ` Bart Schaefer
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-11 18:50 UTC (permalink / raw)
  To: zsh-users


On 2022-11-11 10:15, Bart Schaefer wrote:
> Why wouldn't you do this as soon as they began exhibiting behavior you
> found mysterious?
Because I'd never seen anything like that before and I had no idea where 
to even begin looking for an answer.  God knows how many tools are out 
there that I've never even heard of.
>
> You mean like maybe the six indexes at the bottom of the page here?
> https://zsh.sourceforge.io/Doc/
>
> Not really.  Again you'd have to know ahead of time what  you're 
> looking for.  But if I don't flame-out I think it's time to read 
> Peter's User's Guide.  Then scan the manual end to end. Understandable 
> there's not more resources available tho, I think genuine neophytes to 
> zsh are very rare, almost everyone is already a shell adept even if 
> moving from, say, bash.  I'll bet that even after several years I'm 
> still the junior man here.





^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-11 18:50                         ` Ray Andrews
@ 2022-11-11 19:25                           ` Bart Schaefer
  2022-11-11 21:26                             ` Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Bart Schaefer @ 2022-11-11 19:25 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Fri, Nov 11, 2022 at 10:50 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> On 2022-11-11 10:15, Bart Schaefer wrote:
> > Why wouldn't you do this as soon as they began exhibiting behavior you
> > found mysterious?
> Because I'd never seen anything like that before and I had no idea where
> to even begin looking for an answer.  God knows how many tools are out
> there that I've never even heard of.

I find this response baffling.  Roman wrote "local MATCH ..." so
obviously it's a zsh thing.  Even if it might be another tool, at
least try checking the zsh manual first?

https://zsh.sourceforge.io/Doc/Release/zsh_6.html#index_split-5_vr_letter-M

> > You mean like maybe the six indexes at the bottom of the page here?
> > https://zsh.sourceforge.io/Doc/
> >
> Not really.  Again you'd have to know ahead of time what  you're
> looking for.

Isn't that also true of a glossary?

I mean, go back to the mention of how you've never heard of $foo:t
before.  Well, gosh, look up "colon" in the concept index:

https://zsh.sourceforge.io/Doc/Release/Concept-Index.html#Concept-Index-1_cp_letter-C

Takes you right here:

https://zsh.sourceforge.io/Doc/Release/Expansion.html#index-colon-modifiers

Now you know it's called a "modifier" so when you see $foo:A you know
where to look.

Or you could look up "substitution"

https://zsh.sourceforge.io/Doc/Release/zsh_4.html#index_split-3_cp_letter-S

where you find that for parameters it's usually called "expansion" and
expansion has flags and oh by the way there's even a set of expansion
"rules" that will tell you all about the procedure zsh follows to
perform one.

Or you could just scan through the concept index to get an idea of
what terminology you're likely to encounter on this list, without
having to read the whole manual.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-11 19:25                           ` Bart Schaefer
@ 2022-11-11 21:26                             ` Ray Andrews
  2022-11-12  4:24                               ` Bart Schaefer
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-11 21:26 UTC (permalink / raw)
  To: zsh-users


On 2022-11-11 11:25, Bart Schaefer wrote:
> Because I'd never seen anything like that before and I had no idea where
>> to even begin looking for an answer.  God knows how many tools are out
>> there that I've never even heard of.
> I find this response baffling.  Roman wrote "local MATCH ..." so
> obviously it's a zsh thing.  Even if it might be another tool, at
> least try checking the zsh manual first?
>
Obviously but he showed it to me before I knew there was any such thing 
to look for.  If I have need of it again then I'll be sure to read up on 
it.  For now, it's water under the bridge.
> https://zsh.sourceforge.io/Doc/Release/zsh_6.html#index_split-5_vr_letter-M
Tx, now that I know there is such a thing, I'll read up on it.
> Isn't that also true of a glossary?
Not exactly.  The sort of thing I have in mind would be purpose built to 
explain zsh anatomy.  It would show you sample code and then 'dissect' 
it for you, explaining what all the bits and pieces are called and how 
they function.  You'd get an understanding of the structure of the thing 
at the most zoomed out level.  Anatomy 101.
>
> I mean, go back to the mention of how you've never heard of $foo:t
> before.  Well, gosh, look up "colon" in the concept index:
You have me there.  I was pleased to find out you could search for 
'colon'.  Searching for ':' was obviously not much use.  Sorta like just 
recently the ':#' construction.  You won't have much luck googling for 
'zsh'  ':#' you hafta know ahead of time the name of those sorts of 
manipulations.  'parameter expansions' no?  Once I know what to call it, 
I can zoom in on the correct part of the manual.
> Now you know it's called a "modifier" so when you see $foo:A you know
> where to look.
>
> Or you could look up "substitution"
Nuts, I thought I  had it right with 'expansion' :(  But yes, that's the 
thing, '//'  ':#'  '%%' ... these are substitutions.  So  yeah, that's 
the magic word that takes me to the relevant information. Without the 
terminology one is quite lost.
>
> https://zsh.sourceforge.io/Doc/Release/zsh_4.html#index_split-3_cp_letter-S
>
> where you find that for parameters it's usually called "expansion" and
> expansion has flags and oh by the way there's even a set of expansion
> "rules" that will tell you all about the procedure zsh follows to
> perform one.
Ah! So I was right the first time. Yes 'expansions' even when they are 
contractions -- doesn't matter what the word is, just so long as I know it.
>
> Or you could just scan through the concept index to get an idea of
> what terminology you're likely to encounter on this list, without
> having to read the whole manual.

My imaginary glossary would be specifically designed to get you up to 
speed on terminology.  Anyway it IS time to do some reading.  Anecdote: 
when I first tried Linux, coming from DOS, one of the first things I 
wanted to find out was how you write a batch-file.  Googled, nada. 
What?  No batch-files in Linux?  Well yes, but they're called scripts.  
And switches are called options.  And variables are called parameters.  
And so on. Getting the lingo straight should be done up front with a 
purpose built document for that.


>


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-11 21:26                             ` Ray Andrews
@ 2022-11-12  4:24                               ` Bart Schaefer
  2022-11-12 14:03                                 ` Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Bart Schaefer @ 2022-11-12  4:24 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Fri, Nov 11, 2022 at 1:26 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> On 2022-11-11 11:25, Bart Schaefer wrote:
> > Isn't that also true of a glossary?
> Not exactly.  The sort of thing I have in mind would be purpose built to
> explain zsh anatomy.  It would show you sample code and then 'dissect'

I've never heard the term "glossary" applied to something like that.

This is a glossary:
https://homepages.uc.edu/~thomam/Intro_Unix_Text/Glossary.html

But you seem to want something more like the whole book:
How about https://homepages.uc.edu/~thomam/Intro_Unix_Text/TOC.html

(Or at least everything from Chapter 9 and later.)

> recently the ':#' construction.  You won't have much luck googling for
> 'zsh'  ':#'

Indeed, search engine text indexing is lousy at short strings and punctuation.

> > Or you could look up "substitution"
> > where you find that for parameters it's usually called "expansion"
>
> Ah! So I was right the first time.

Substitution is a special case of expansion.  Globbing is also a
special case of expansion.  I was just trying to illustrate that the
index will get you to the general case from the specific.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-12  4:24                               ` Bart Schaefer
@ 2022-11-12 14:03                                 ` Ray Andrews
  2022-11-13 15:09                                   ` Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-12 14:03 UTC (permalink / raw)
  To: zsh-users


On 2022-11-11 20:24, Bart Schaefer wrote:
> I've never heard the term "glossary" applied to something like that.


True, as I muse I expand the idea from a glossary to a primer to a grammar.

> This is a glossary:
> https://homepages.uc.edu/~thomam/Intro_Unix_Text/Glossary.html

That looks really good, I'm going to scan it end to end.   Even if it 
was nothing  more, something like that focused on zsh -- or I guess just 
'shell' -- terminology would be invaluable to someone like me.  Probably 
was something like that at one time but its fallen by the wayside since, 
as I said, raw beginners are now so rare.  Really, how many people at my 
level are there now?   I just decided to dive in at the deep end and 
learn to swim by not drowning.  Inside out -- solve one problem at a 
time, and hope that the grammar would reveal itself eventually.  Nope, 
the grammar is too terse, too powerful.  Latin and Greek are 'C' by 
comparison.

> But you seem to want something more like the whole book:
> How about https://homepages.uc.edu/~thomam/Intro_Unix_Text/TOC.html
>
> (Or at least everything from Chapter 9 and later.)
Nope, we already have 'the whole book' that's just the thing, I want the 
language to be able to read the whole book and understand it. Peter's 
'From Bash to Z Shell' was good except that it discussed Bash as much as 
Zsh so it confused and diluted as much as it clarified and explained.
> Indeed, search engine text indexing is lousy at short strings and punctuation.

And that's the unavoidable dilemma.  Zsh uses one or two characters as 
syntactic entities and those same characters might have three or four 
meanings in different contexts so the whole paradigm of the google 
search is just useless.  Nope, hafta read the book.  But manuals are not 
even designed to be helpful they are repositories of information that 
you consult when you know what you are looking for.  The right book will 
have a glossary in it, helpful hints, anatomy lessons, and a 'phrase 
book' -- even if you can't speak German, you can learn to order a beer.  
Build up from simple examples.   Mean time, my only salvation is asking 
you guys for help -- which I always get :-)







^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-12 14:03                                 ` Ray Andrews
@ 2022-11-13 15:09                                   ` Ray Andrews
  2022-11-14 14:12                                     ` Roman Perepelitsa
  2022-11-14 17:08                                     ` Ray Andrews
  0 siblings, 2 replies; 39+ messages in thread
From: Ray Andrews @ 2022-11-13 15:09 UTC (permalink / raw)
  To: zsh-users

Roman:

             cc=( ${(M)cc:#*$~zsh_case${filter}*} )

I've been applying that in some other functions but I need a variation 
that will erase non-matching lines but leave them as blanks so that 
array indexes do not change.  Reason being that a subsequent merge with 
another array will happen and indexes must mesh.  I'm doing it right now 
by using a dummy string as replacement for a non-match and then 
combining the arrays and then deleting the dummies but that's obviously 
clumsy.





^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-13 15:09                                   ` Ray Andrews
@ 2022-11-14 14:12                                     ` Roman Perepelitsa
  2022-11-14 17:08                                     ` Ray Andrews
  1 sibling, 0 replies; 39+ messages in thread
From: Roman Perepelitsa @ 2022-11-14 14:12 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Sun, Nov 13, 2022 at 4:10 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Roman:
>
>              cc=( ${(M)cc:#*$~zsh_case${filter}*} )
>
> I've been applying that in some other functions but I need a variation
> that will erase non-matching lines but leave them as blanks so that
> array indexes do not change.

This should do it:

  cc=( "${(@)cc/#%$~zsh_case^(*$filter*)}" )

This uses ${name/pattern/repl} with the pattern starting with %# to
signify full match. Within the pattern ^ is negation. Double quotes
with (@) are to retain empty elements. The rest you've already used,
so it should be familiar.

Roman.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-13 15:09                                   ` Ray Andrews
  2022-11-14 14:12                                     ` Roman Perepelitsa
@ 2022-11-14 17:08                                     ` Ray Andrews
  2022-11-14 17:12                                       ` Roman Perepelitsa
  1 sibling, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-14 17:08 UTC (permalink / raw)
  To: zsh-users

[-- Attachment #1: Type: text/plain, Size: 983 bytes --]


On 2022-11-13 07:09, Ray Andrews wrote:
> Roman:
>
>             cc=( ${(M)cc:#*$~zsh_case${filter}*} )
>
> I've been applying that in some other functions but I need a variation 
> that will erase non-matching lines but leave them as blanks so that 
> array indexes do not change.  Reason being that a subsequent merge 
> with another array will happen and indexes must mesh.  I'm doing it 
> right now by using a dummy string as replacement for a non-match and 
> then combining the arrays and then deleting the dummies but that's 
> obviously clumsy.

Stéphane Chazelas 
<https://unix.stackexchange.com/users/22565/st%c3%a9phane-chazelas> over 
at StackExchange offers this:

         cc=( "${cc[@]/#%^*$~zsh_case${filter}*}" )

... and it seems to do exactly what I want.  I understand the negation 
and the way the '[@]' and the outer quotes protect the length of the 
array, now I just need to understand the '#%' anchors -- but they are 
essential.


>
>
>

[-- Attachment #2: Type: text/html, Size: 1764 bytes --]

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-14 17:08                                     ` Ray Andrews
@ 2022-11-14 17:12                                       ` Roman Perepelitsa
  2022-11-14 18:58                                         ` Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Roman Perepelitsa @ 2022-11-14 17:12 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Mon, Nov 14, 2022 at 6:09 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Stéphane Chazelas over at StackExchange offers this:
>
>         cc=( "${cc[@]/#%^*$~zsh_case${filter}*}" )
>

This is the same as what I posted.

> now I just need to understand the '#%' anchors

See my previous post.

Roman.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-14 17:12                                       ` Roman Perepelitsa
@ 2022-11-14 18:58                                         ` Ray Andrews
  2022-11-14 20:00                                           ` Bart Schaefer
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-14 18:58 UTC (permalink / raw)
  To: zsh-users


On 2022-11-14 09:12, Roman Perepelitsa wrote:
>
> This is the same as what I posted.
Things get overlooked in the confusion, I often have ten things to learn 
all at the same time.  I'll sometimes take some code on faith and forget 
that I've forgotten to really understand it.  Or that an explanation was 
there previously.
>> now I just need to understand the '#%' anchors

I think I might:  Because it's a negation, you have to force the match 
to include the entire element from beginning to end.  You're sorta 
forcing a greedy match.  " ^$foo " searched for in " xxxfoobarxxx " will 
match " xxxfo " but will not match the whole string -- which is to say 
that it matches and therefore is rejected by the caret forcing 
negation.  It's not a simple construction.  Oh, and I would never have 
figured it out by myself, too many things have to work together just right.



> See my previous post.
>
> Roman.
>


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-14 18:58                                         ` Ray Andrews
@ 2022-11-14 20:00                                           ` Bart Schaefer
  2022-11-14 23:25                                             ` Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Bart Schaefer @ 2022-11-14 20:00 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Mon, Nov 14, 2022 at 10:58 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> >> now I just need to understand the '#%' anchors
>
> I think I might:  Because it's a negation, you have to force the match
> to include the entire element from beginning to end.  You're sorta
> forcing a greedy match.

Yes, it forces a match of the entire string.  No, that's not the same
as "greedy".

Given a pattern like "#%(*)foo(*)" and a string like
"xxxfooyyyfoozzz", a "greedy" match would have the first (*) match
"xxxfooyyy".  A non-greedy match would have the second (*) match
"yyyfoozzz" because the non-greedy first (*) matched only "xxx" before
"foo".

This isn't particularly important in the context here, but if you were
using (#b) to activate backreferences it could be crucial.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: coloring substitution seems to eat next line.
  2022-11-14 20:00                                           ` Bart Schaefer
@ 2022-11-14 23:25                                             ` Ray Andrews
  2022-11-15 14:17                                               ` Belaboring substitution syntax Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-14 23:25 UTC (permalink / raw)
  To: zsh-users


On 2022-11-14 12:00, Bart Schaefer wrote:
>
> Yes, it forces a match of the entire string.  No, that's not the same
> as "greedy".

Right, it seems to me that 'greedy' is perfectly defined and 'whole 
string' isn't it.  It's a rather subtle thing getting these pattern 
matches to do what a casual intuition might think they do.  Throw in a 
negation and it's not simple.  That line would be a good candidate for 
dissection in one of my imagined anatomy lessons.  Really, it's half an 
hour to completely understand it.






^ permalink raw reply	[flat|nested] 39+ messages in thread

* Belaboring substitution syntax
  2022-11-14 23:25                                             ` Ray Andrews
@ 2022-11-15 14:17                                               ` Ray Andrews
  2022-11-16  1:49                                                 ` Bart Schaefer
  0 siblings, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-15 14:17 UTC (permalink / raw)
  To: zsh-users


Ignore this post unless you're the sort of person who loved grammar in 
school:

    cc=( "${cc[@]/#%^*$~zsh_case${filter}*}" )

... working fine, and approved by Stephane and Roman.  But, trying to 'feel' the rightness of it, I sorta get that '[@]' is saying: "process this array line by line" thus preserving array indexes.  I'm not sure about the quotes.  Quotes seem to mostly combine words into lines -- or should I say that they mostly combine what would be word splitting into bigger elements: array=( "this array" "has" "three elements" ) ... and sometimes the quotes seem to protect stuff from expansion or interpretation: " $ echo "Yar 'en ign'rint fool!" ".  In my case, leaving the quotes out doesn't *seem* to matter but I know it would bite me eventually.  When would it bite me?

And:  " /#%^ "  ... it seems to me this is belaboring something simple.  Do we not have an operator that says (in English): If string A is a substring of B, then return B, else return nothing.  No anchors, no negation.  Inventing '!!' as my operator (yes I know it won't work but just for discussion):

    cc=( "${cc[@]!!$~zsh_case${filter}}" )

One operator instead of four. Actually, in such a construction the anchors would say: ... furthermore the substring must start at the beginning of the element:

    cc=( "${cc[@]!!#$~zsh_case${filter}}" )

... or must end at the end of the element:

    cc=( "${cc[@]!!%$~zsh_case${filter}}" )

... This wouldn't be a fancy substitution so much as a truth test with nulling of 'false'.  Dunno, but it seems like a simple thing and one of the first things a shell might be able to do.  Instead what we seem to have is a powerful operator that must then be dumbed down.

Not important, but if anyone is interested.




^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Belaboring substitution syntax
  2022-11-15 14:17                                               ` Belaboring substitution syntax Ray Andrews
@ 2022-11-16  1:49                                                 ` Bart Schaefer
  2022-11-16  2:54                                                   ` Ray Andrews
  2022-11-16 10:32                                                   ` Roman Perepelitsa
  0 siblings, 2 replies; 39+ messages in thread
From: Bart Schaefer @ 2022-11-16  1:49 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Tue, Nov 15, 2022 at 6:18 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I sorta get that '[@]' is saying: "process this array line by line" thus preserving array indexes.  I'm not sure about the quotes.

This follows the same rules as $@.  If you write "${array[@]}" with
the quotes, you get each element of the array separately quoted.  This
often doesn't matter in zsh but always matters in other shells or when
SH_WORD_SPLIT is enabled in zsh.

BTW you need to stop thinking of these as "lines" -- line breaks have
nothing to do with it.

Conversely if you write "${array}" or ${array[*]}" either one with the
quotes, you get the entire array joined into a single string.

> and sometimes the quotes seem to protect stuff from expansion or interpretation:

Quotes ALWAYS protect SOMETHING from expansion or interpretation,
unless you start throwing other things in there such as eval.  WHAT is
protected differs:  Double quotes allow ${param} and $(process) and
$((math)) replacements but protect syntax tokens and glob patterns and
whitespace; single quotes protect pretty much everything.

> In my case, leaving the quotes out doesn't *seem* to matter but I know it would bite me eventually.  When would it bite me?

In the specific example in this thread, zsh's default array behavior
(no_SH_WORD_SPLIT) is sufficient, but if you later enclose that or a
similar construct in a deeper context, yes, you might be bitten.

> Do we not have an operator that says (in English): If string A is a substring of B, then return B, else return nothing.

Your trouble is the definition of "nothing."  ${B:#*A*} does that, but
in your example here, you don't actually want "nothing", you want "the
empty string" which is still something.

In fact the example in question is "if A is NOT a substring of B then
return B, else return the empty string."

So we have this:
Do a replacement on every element of $cc is  ${cc[@]/...}
Replacement must start at beginning of element ${cc[@]/#...}
Replacement must also end at end of element ${cc[@]/#%...}
Use $zsh_case as a pattern is $~zsh_case
Match that pattern anywhere with $filter is *$~zsh_case${filter}*
Match NOT that whole pattern is ^*$~zsh_case${filter}*
Put it all together:  ${cc[@]/#%^*$~zsh_case${filter}*
Replace all that with the empty string is just close the brace
${cc[@]/#%^*$~zsh_case${filter}*}
... and then the double-quotes for safety.

About the only thing your magical operator could remove is putting
*...* around the pattern to mean "is a substring".


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Belaboring substitution syntax
  2022-11-16  1:49                                                 ` Bart Schaefer
@ 2022-11-16  2:54                                                   ` Ray Andrews
  2022-11-16  6:26                                                     ` Bart Schaefer
  2022-11-16 10:32                                                   ` Roman Perepelitsa
  1 sibling, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-16  2:54 UTC (permalink / raw)
  To: zsh-users


On 2022-11-15 17:49, Bart Schaefer wrote:
> BTW you need to stop thinking of these as "lines" -- line breaks have
> nothing to do with it.
Because I've never had an element that was more than one line I tend to 
say 'lines' but I know abstractly what you're saying.  I'll crash into 
the issue eventually and I'll be ready for it.
> Conversely if you write "${array}" or ${array[*]}" either one with the
> quotes, you get the entire array joined into a single string.
Right, that's quotes doing joining which I expect.
> Quotes ALWAYS protect SOMETHING from expansion or interpretation,
> unless you start throwing other things in there such as eval.  WHAT is
> protected differs:  Double quotes allow ${param} and $(process) and
> $((math)) replacements but protect syntax tokens and glob patterns and
> whitespace; single quotes protect pretty much everything.
Yeah, I get all that.  There are times when I want to protect but not 
join tho and it gets confusing.
> In the specific example in this thread, zsh's default array behavior
> (no_SH_WORD_SPLIT) is sufficient, but if you later enclose that or a
> similar construct in a deeper context, yes, you might be bitten.
I'll stay safe.
> Your trouble is the definition of "nothing."  ${B:#*A*} does that, but
> in your example here, you don't actually want "nothing", you want "the
> empty string" which is still something.
Quite so, rather sloppy of me to say 'nothing', I know better.
> About the only thing your magical operator could remove is putting
> *...* around the pattern to mean "is a substring".
>
Well no, I *define* my operator to do just as I said (except that I 
should have said 'empty string').  All that puzzles me is that it's not 
already available.  Mind, functionality is added as needed so it's 
sufficient to just say that nobody has much wanted such a thing -- which 
is not debatable.  It's a fact on the ground that my operator doesn't 
exist ergo it's not been in demand and that's that.   So long is my 
functionality is achievable the rest is just musing.  Still I'd have 
expected it to be one of the earlier operators because it's the simplest 
case.  Nevermind.




^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Belaboring substitution syntax
  2022-11-16  2:54                                                   ` Ray Andrews
@ 2022-11-16  6:26                                                     ` Bart Schaefer
  2022-11-16 14:08                                                       ` Ray Andrews
  2022-11-16 20:46                                                       ` Ray Andrews
  0 siblings, 2 replies; 39+ messages in thread
From: Bart Schaefer @ 2022-11-16  6:26 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Tue, Nov 15, 2022 at 6:54 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> There are times when I want to protect but not
> join tho and it gets confusing.

That's almost always the instance in which you want to use
"${array[@]}" or the almost-equivalent "${(@)array}".  The difference
is that you can [sometimes must] use (@) on a nested expansion
"${(@)${somethingthatcreatesanarray}}".


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Belaboring substitution syntax
  2022-11-16  1:49                                                 ` Bart Schaefer
  2022-11-16  2:54                                                   ` Ray Andrews
@ 2022-11-16 10:32                                                   ` Roman Perepelitsa
  1 sibling, 0 replies; 39+ messages in thread
From: Roman Perepelitsa @ 2022-11-16 10:32 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Ray Andrews, zsh-users

On Wed, Nov 16, 2022 at 2:50 AM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Tue, Nov 15, 2022 at 6:18 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
> >
> > In my case, leaving the quotes out doesn't *seem* to matter but I know it would bite me eventually.  When would it bite me?
>
> In the specific example in this thread, zsh's default array behavior
> (no_SH_WORD_SPLIT) is sufficient, but if you later enclose that or a
> similar construct in a deeper context, yes, you might be bitten.

Without the quotes empty elements will be dropped. Ray wants ${#cc} to
stay unchanged, so quotes are required.

Roman.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Belaboring substitution syntax
  2022-11-16  6:26                                                     ` Bart Schaefer
@ 2022-11-16 14:08                                                       ` Ray Andrews
  2022-11-16 14:13                                                         ` Roman Perepelitsa
  2022-11-16 20:46                                                       ` Ray Andrews
  1 sibling, 1 reply; 39+ messages in thread
From: Ray Andrews @ 2022-11-16 14:08 UTC (permalink / raw)
  To: zsh-users


On 2022-11-15 22:26, Bart Schaefer wrote:
>
> That's almost always the instance in which you want to use
> "${array[@]}" or the almost-equivalent "${(@)array}".  The difference
> is that you can [sometimes must] use (@) on a nested expansion
> "${(@)${somethingthatcreatesanarray}}".

Sheesh, that's a new one again.  I'll add that to my list of potential 
expand/protect/join/split/slice/don't-slice/compact/don't-compact 
candidate syntaxes.  How you guys can possibly parse all this is too 
scary to even think about.  Still:

"${array[@]}" ... Read: protect the elements of this array, do any expansions within them, but do NOT combine them ??



^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Belaboring substitution syntax
  2022-11-16 14:08                                                       ` Ray Andrews
@ 2022-11-16 14:13                                                         ` Roman Perepelitsa
  2022-11-17  2:31                                                           ` Bart Schaefer
  0 siblings, 1 reply; 39+ messages in thread
From: Roman Perepelitsa @ 2022-11-16 14:13 UTC (permalink / raw)
  To: Ray Andrews; +Cc: zsh-users

On Wed, Nov 16, 2022 at 3:09 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> "${array[@]}" ... Read: protect the elements of this array, do any expansions within them, but do NOT combine them ??

No, this just expands the array to all its elements. If you drop the
quotes, then it's a more complex thing.

Roman.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Belaboring substitution syntax
  2022-11-16  6:26                                                     ` Bart Schaefer
  2022-11-16 14:08                                                       ` Ray Andrews
@ 2022-11-16 20:46                                                       ` Ray Andrews
  1 sibling, 0 replies; 39+ messages in thread
From: Ray Andrews @ 2022-11-16 20:46 UTC (permalink / raw)
  To: zsh-users


On 2022-11-15 22:26, Bart Schaefer wrote:
> "${(@)${somethingthatcreatesanarray}}". 

Speak of the devil and he appears:  a guy on StackExchange just offered 
this for my 'keep erased lines' issue:

     remnant=( "${(@M)remnant##*$~zsh_case${searchstring[ee]}*}" )

... (name off the array was 'cc' previously) but this seems to be about 
approximately exactly my imaginary operator, no?  '##', which I 
understand, throw in the '(@M)' which is probably not that hard to 
understand, so unless I hear different, it seems to produce identical 
results to the more belabored form:

     remnant=( "${remnant[@]/#%^*$~zsh_case${searchstring[ee]}*}" )

... so thereyago.  Two operators shorter.







^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Belaboring substitution syntax
  2022-11-16 14:13                                                         ` Roman Perepelitsa
@ 2022-11-17  2:31                                                           ` Bart Schaefer
  2022-11-17  8:59                                                             ` Roman Perepelitsa
  0 siblings, 1 reply; 39+ messages in thread
From: Bart Schaefer @ 2022-11-17  2:31 UTC (permalink / raw)
  To: Roman Perepelitsa; +Cc: Ray Andrews, zsh-users

On Wed, Nov 16, 2022 at 6:15 AM Roman Perepelitsa
<roman.perepelitsa@gmail.com> wrote:
>
> On Wed, Nov 16, 2022 at 3:09 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
> >
> > "${array[@]}" ... Read: protect the elements of this array, do any expansions within them, but do NOT combine them ??
>
> No, this just expands the array to all its elements.

In the base cases, you can think of it as "if I can't see it, nothing
happens to it."  The exceptions only occur when you start adding
parameter flags like ${(e)array[@]}.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Belaboring substitution syntax
  2022-11-17  2:31                                                           ` Bart Schaefer
@ 2022-11-17  8:59                                                             ` Roman Perepelitsa
  2022-11-17 16:02                                                               ` Ray Andrews
  0 siblings, 1 reply; 39+ messages in thread
From: Roman Perepelitsa @ 2022-11-17  8:59 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Ray Andrews, zsh-users

On Thu, Nov 17, 2022 at 3:32 AM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Wed, Nov 16, 2022 at 6:15 AM Roman Perepelitsa
> <roman.perepelitsa@gmail.com> wrote:
> >
> > On Wed, Nov 16, 2022 at 3:09 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
> > >
> > > "${array[@]}" ... Read: protect the elements of this array, do any expansions within them, but do NOT combine them ??
> >
> > No, this just expands the array to all its elements.
>
> In the base cases, you can think of it as "if I can't see it, nothing
> happens to it."  The exceptions only occur when you start adding
> parameter flags like ${(e)array[@]}.

If I didn't know that ${array[@]} without quotes does not in general
expand to the array's elements, I could be confused by this exchange,
so I'll post some examples for Ray's benefit.

This does nothing (the array's content is unchanged):

    array=("${array[@]}")

This may change the content of the array:

    array=(${array[@]})

The content of the array is changed by this statement in two ways:

1. All empty elements are removed.
2. If sh_word_split is set, elements of the array are split on $IFS.

To demonstrate (1):

    % () {
      emulate -L zsh
      local array=('' 'foo bar')
      typeset -p array
      array=(${array[@]})
      typeset -p array
    }

    typeset -a array=( '' 'foo bar' )
    typeset -a array=( 'foo bar' )

To demonstrate (2):

    % () {
      emulate -L zsh -o sh_word_split
      local array=('' 'foo bar')
      typeset -a array=( 'foo bar' )
      array=(${array[@]})
      typeset -p array
    }

    typeset -a array=( '' 'foo bar' )
    typeset -a array=( foo bar )

In short, "${array[@]}" expands to array elements just like "$scalar"
expands to the scalar's value. Versions without quotes are more
complex, for they may transform the value in various ways.

Additionally, unless ksh_arrays is set (it's unset by default), these
two are equivalent:

    "${array[@]}"
    "${(@)array}"

And these four are equivalent:

    ${array[@]}
    ${(@)array}
    ${array}
    $array

Roman.


^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: Belaboring substitution syntax
  2022-11-17  8:59                                                             ` Roman Perepelitsa
@ 2022-11-17 16:02                                                               ` Ray Andrews
  0 siblings, 0 replies; 39+ messages in thread
From: Ray Andrews @ 2022-11-17 16:02 UTC (permalink / raw)
  To: zsh-users


On 2022-11-17 00:59, Roman Perepelitsa wrote:
> ...
> And these four are equivalent:
>
>      ${array[@]}
>      ${(@)array}
>      ${array}
>      $array
>
> Roman.

Thanks, that's what I call quality explanation, I'm going to save that 
whole post to my cheat sheet and work it over at length.  2/3 of my own 
difficulties here involve all this splitting/joining stuff. Mostly 
because I use Sebastian's n_list() quite a bit and it demands 
lines/elements for input, with blank lines and spaces in filenames 
protected.  The construction that seems robust is like this:

n_list "${array[@]}"

... for the reasons you mention.  A useful document would be the 
Unabridged Guide to Zsh Arrays (their splitting, splicing, slicing and 
dicing, with protections and without, with all flags explained and 
specimens of every possible syntax and when you'd want to use them).

I know it's far too late in the game to even think about fundamental 
changes, but if we had to do it all over again I'd advocate for a single 
universal expansion grammar:

${(CONTROL) FIELD  /  FIELD / FIELD }

No '//' '##' '%%' or any of that.  '/' is the universal separator, and 
what is done to the fields is entirely explained within the 
parenthesis.  No more worries about special characters except '/' and 
maybe " " and ' ' and backslash cuz those expansion controls are 
robust.  Within the parenthesis there are zero worries about literals, 
it's 100% operators so the syntax-space is unlimited.  Within the fields 
there are zero operators (except quote expansions) so parsing becomes 
vastly simpler.

${(S:a!$%&@N/(xx^)**/*<>) ${array} / ${filter} }

... what's in the () explains exactly what is to be done vis a vis the 
two fields ... or more.

${(VERBS) NOUNS / NOUNS / NOUNS }

${(PREDICATES) SUBJECTS } ... that's the way a grammar should look.  
Basically 'flags' do everything and verbs and nouns are never 
intermingled.  That's the reason glob qualifiers are so stinking 
powerful, cuz within the parenthesis their is nothing one might not 
achieve.

Nevermind.  :-)







^ permalink raw reply	[flat|nested] 39+ messages in thread

end of thread, other threads:[~2022-11-17 16:03 UTC | newest]

Thread overview: 39+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-10  0:43 coloring substitution seems to eat next line Ray Andrews
2022-11-10  8:02 ` Roman Perepelitsa
2022-11-10 18:25   ` Ray Andrews
2022-11-10 18:36     ` Roman Perepelitsa
2022-11-10 18:45       ` Ray Andrews
2022-11-10 18:54       ` Bart Schaefer
2022-11-10 19:28         ` Ray Andrews
2022-11-10 20:22           ` Bart Schaefer
2022-11-10 21:42             ` Ray Andrews
2022-11-10 21:51               ` Roman Perepelitsa
2022-11-11 17:24                 ` Ray Andrews
2022-11-10 22:47               ` Bart Schaefer
2022-11-10 23:07                 ` Ray Andrews
2022-11-10 23:27                   ` Bart Schaefer
2022-11-11 15:00                     ` Ray Andrews
2022-11-11 18:15                       ` Bart Schaefer
2022-11-11 18:50                         ` Ray Andrews
2022-11-11 19:25                           ` Bart Schaefer
2022-11-11 21:26                             ` Ray Andrews
2022-11-12  4:24                               ` Bart Schaefer
2022-11-12 14:03                                 ` Ray Andrews
2022-11-13 15:09                                   ` Ray Andrews
2022-11-14 14:12                                     ` Roman Perepelitsa
2022-11-14 17:08                                     ` Ray Andrews
2022-11-14 17:12                                       ` Roman Perepelitsa
2022-11-14 18:58                                         ` Ray Andrews
2022-11-14 20:00                                           ` Bart Schaefer
2022-11-14 23:25                                             ` Ray Andrews
2022-11-15 14:17                                               ` Belaboring substitution syntax Ray Andrews
2022-11-16  1:49                                                 ` Bart Schaefer
2022-11-16  2:54                                                   ` Ray Andrews
2022-11-16  6:26                                                     ` Bart Schaefer
2022-11-16 14:08                                                       ` Ray Andrews
2022-11-16 14:13                                                         ` Roman Perepelitsa
2022-11-17  2:31                                                           ` Bart Schaefer
2022-11-17  8:59                                                             ` Roman Perepelitsa
2022-11-17 16:02                                                               ` Ray Andrews
2022-11-16 20:46                                                       ` Ray Andrews
2022-11-16 10:32                                                   ` Roman Perepelitsa

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).