zsh-users
 help / Atom feed
* Match to the end of string when using (S) flag
@ 2016-11-01 14:25 Sebastian Gniazdowski
  2016-11-01 16:27 ` Sebastian Gniazdowski
  0 siblings, 1 reply; 10+ messages in thread
From: Sebastian Gniazdowski @ 2016-11-01 14:25 UTC (permalink / raw)
  To: zsh-users

Hello,
I do:

% a="AXbcAXdeAXfg"; echo ${(S)a//*(AX)/x}
xxxfg

to generate region_highlight entries. As it can be seen, there is
trailing data, which Zsh converts to "-1 -1 none". I would want to
generate clean region_highlight entries. Came up with this:

% a="AXbcAXdeAXfg"; echo ${a//(*(AX)*(#e)~*AX*AX*|*AX~*AX*AX*)/x}
xxx

Surprisingly, it is faster (e.g. 170ms vs 780 ms) than (S) flag IF AX is
not a complex pattern like AX|BX. Then it is slow, so I cannot use this.
Also, I'm not sure if it isn't possible that exclusion ~*AX*AX* will
lead to acceptance of X*AX*, but this doesn't seem to occur.

Is there way out of this? Thought about using (R) flag out of the set of
BEMNR "... include in result" flags, but I cannot provoke anything
distinct with them.

PS. The actual region_highlight generating code is:

region_highlight+=( "${(f)${(S)text//*(#bi)(${~colsearch_pattern})/$((
offset + mbegin[1] - 1 )) $(( offset + mend[1] ))
${__hsmw_hl_color}${nl}}%$nl*}" )

offset is length of a preamble (all for $POSTDISPLAY) that is not
colored.

-- 
  Sebastian Gniazdowski
  psprint@fastmail.com


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Match to the end of string when using (S) flag
  2016-11-01 14:25 Match to the end of string when using (S) flag Sebastian Gniazdowski
@ 2016-11-01 16:27 ` Sebastian Gniazdowski
  2016-11-01 18:29   ` Sebastian Gniazdowski
  0 siblings, 1 reply; 10+ messages in thread
From: Sebastian Gniazdowski @ 2016-11-01 16:27 UTC (permalink / raw)
  To: zsh-users

One solution can be:

local nul=$'\0'
region_highlight+=( "${(0)${(S)text//*(#bi)(${~colsearch_pattern})/$((
offset + mbegin[1] - 1 )) $(( offset + mend[1] ))
${__hsmw_hl_color}${nul}}%$nul*}" )

however user probably can enter or somehow have in history a null byte?
That said, the solution is rather acceptable. Generates clean
region_highlight entries and in emergency case of a null byte in $text
after last match – relies on Zshell's robust region_highlight
implementation converting improper entries to "-1 -1 none". Provided
there is no other way, this is fine. But maybe there is other way?

-- 
  Sebastian Gniazdowski
  psprint@fastmail.com


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Match to the end of string when using (S) flag
  2016-11-01 16:27 ` Sebastian Gniazdowski
@ 2016-11-01 18:29   ` Sebastian Gniazdowski
  2016-11-01 21:23     ` Bart Schaefer
  0 siblings, 1 reply; 10+ messages in thread
From: Sebastian Gniazdowski @ 2016-11-01 18:29 UTC (permalink / raw)
  To: zsh-users

A miracle optimization – via mathematical function!

    __hsmw_region_highlight_data=( )
    : "${text//(#mi)(${~colsearch_pattern})/$(( append(MBEGIN,MEND) ))}"
    region_highlight+=( $__hsmw_region_highlight_data )

    shappend() {
        __hsmw_region_highlight_data+=( "$(( offset + $1 - 1 )) $((
        offset + $2 )) ${__hsmw_hl_color}" )
    }
    functions -M append 2 2 shappend

This gives total time (time of anonymous function wrapping first three
lines) for a complex – three-word A|B|C – input pattern: 44 ms (17 ms in
shappend).

When I do:
    : "${(S)text//*(#bi)(${~colsearch_pattern})/$((
    append(mbegin[1],mend[1]) ))}"

then the time is: 528 ms ! After removing leading star *: 52 ms (18 ms
in shappend).

Time of original one-liner without math-function call: 502 ms.

-- 
  Sebastian Gniazdowski
  psprint@fastmail.com


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Match to the end of string when using (S) flag
  2016-11-01 18:29   ` Sebastian Gniazdowski
@ 2016-11-01 21:23     ` Bart Schaefer
  2016-11-02  6:53       ` Sebastian Gniazdowski
                         ` (2 more replies)
  0 siblings, 3 replies; 10+ messages in thread
From: Bart Schaefer @ 2016-11-01 21:23 UTC (permalink / raw)
  To: zsh-users

On Nov 1, 11:29am, Sebastian Gniazdowski wrote:
}
} A miracle optimization - via mathematical function!

You can do this with just a counter, you don't need a math function:

  i=$#region_highlight
  : ${text//(#mi)(${~colsearch_pattern})/
      ${region_highlight[++i]=$((offset + MBEGIN))}
      ${region_highlight[++i]=$((offset + MEND))}
      ${region_highlight[++i]=${__hsmw_hl_color}}}


} When I do:
}     : "${(S)text//*(#bi)(${~colsearch_pattern})/$((
}     append(mbegin[1],mend[1]) ))}"
} 
} then the time is: 528 ms ! After removing leading star *: 52 ms (18 ms
} in shappend).

With the leading "*" the algorithm has to check after each character
whether the following characters match the sub-pattern in the parens,
so it's checking every position multiple times.

I have to give you credit for coming up with the idea of using a text
replacement in the first place.  I had to re-read your first message
on this thread about four times before I understood what it meant.


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Match to the end of string when using (S) flag
  2016-11-01 21:23     ` Bart Schaefer
@ 2016-11-02  6:53       ` Sebastian Gniazdowski
  2016-11-02 15:50         ` Bart Schaefer
  2016-11-02  9:46       ` Peter Stephenson
  2018-11-07 12:30       ` Sebastian Gniazdowski
  2 siblings, 1 reply; 10+ messages in thread
From: Sebastian Gniazdowski @ 2016-11-02  6:53 UTC (permalink / raw)
  To: Bart Schaefer, zsh-users

On Tue, Nov 1, 2016, at 02:23 PM, Bart Schaefer wrote:
> You can do this with just a counter, you don't need a math function:
> 
>   i=$#region_highlight
>   : ${text//(#mi)(${~colsearch_pattern})/
>       ${region_highlight[++i]=$((offset + MBEGIN))}
>       ${region_highlight[++i]=$((offset + MEND))}
>       ${region_highlight[++i]=${__hsmw_hl_color}}}

Couldn't get this to work, although $i before and after differs much,
however looked at contents and there are lines like:

193 -1 none
-1 -1 none
-1 -1 bg=17
-1 -1 none

so something doesn't append well.

> I have to give you credit for coming up with the idea of using a text
> replacement in the first place.  I had to re-read your first message
> on this thread about four times before I understood what it meant.

Well it comes directly from zsh-users/Zaw, I've even added Zaw license
to the project because of that single line, now I dropped it.

-- 
  Sebastian Gniazdowski
  psprint@fastmail.com


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Match to the end of string when using (S) flag
  2016-11-01 21:23     ` Bart Schaefer
  2016-11-02  6:53       ` Sebastian Gniazdowski
@ 2016-11-02  9:46       ` Peter Stephenson
  2018-11-07 12:30       ` Sebastian Gniazdowski
  2 siblings, 0 replies; 10+ messages in thread
From: Peter Stephenson @ 2016-11-02  9:46 UTC (permalink / raw)
  To: zsh-users

On Tue, 1 Nov 2016 14:23:01 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> With the leading "*" the algorithm has to check after each character
> whether the following characters match the sub-pattern in the parens,
> so it's checking every position multiple times.

Basically, the only optimised case in the matching code within the
shell is looking for a single must-match string, which doesn't apply
here.

Unless someone chooses to dedicate their lives to this --- and it is at
least a fairly well delimited part of the code --- that'll be the way it
stays.

pws


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Match to the end of string when using (S) flag
  2016-11-02  6:53       ` Sebastian Gniazdowski
@ 2016-11-02 15:50         ` Bart Schaefer
  0 siblings, 0 replies; 10+ messages in thread
From: Bart Schaefer @ 2016-11-02 15:50 UTC (permalink / raw)
  To: zsh-users

On Nov 1, 11:53pm, Sebastian Gniazdowski wrote:
} Subject: Re: Match to the end of string when using (S) flag
}
} On Tue, Nov 1, 2016, at 02:23 PM, Bart Schaefer wrote:
} > You can do this with just a counter, you don't need a math function:
} > 
} >   i=$#region_highlight
} >   : ${text//(#mi)(${~colsearch_pattern})/
} >       ${region_highlight[++i]=$((offset + MBEGIN))}
} >       ${region_highlight[++i]=$((offset + MEND))}
} >       ${region_highlight[++i]=${__hsmw_hl_color}}}
} 
} Couldn't get this to work, although $i before and after differs much,
} however looked at contents and there are lines like:
} 
} 193 -1 none
} -1 -1 none
} -1 -1 bg=17
} -1 -1 none
} 
} so something doesn't append well.

Here was my dummy test case:

setopt extendedglob
text="123 Abc 456 Ade 789 Afg 0"
colsearch_pattern="A??"
() {
  local -ah region_highlight=( 1 1 xy=z )
  local i=$#region_highlight offset=3 __hsmw_hl_color="pd=q"
  : ${text//(#mi)${~colsearch_pattern}/
      ${region_highlight[++i]=$((offset + MBEGIN))}
      ${region_highlight[++i]=$((offset + MEND))}
      ${region_highlight[++i]=${__hsmw_hl_color}}}
  print -r -- $region_highlight
}

With the result

1 1 xy=z 8 10 pd=q 16 18 pd=q 24 26 pd=q

(note offset=3 so 8 10 means $text[5,7] which is Abc, etc.)


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Match to the end of string when using (S) flag
  2016-11-01 21:23     ` Bart Schaefer
  2016-11-02  6:53       ` Sebastian Gniazdowski
  2016-11-02  9:46       ` Peter Stephenson
@ 2018-11-07 12:30       ` Sebastian Gniazdowski
  2018-11-07 15:25         ` Bart Schaefer
  2 siblings, 1 reply; 10+ messages in thread
From: Sebastian Gniazdowski @ 2018-11-07 12:30 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh Users

On Tue, 1 Nov 2016 at 22:29, Bart Schaefer <schaefer@brasslantern.com> wrote:
> You can do this with just a counter, you don't need a math function:
>
>   i=$#region_highlight
>   : ${text//(#mi)(${~colsearch_pattern})/
>       ${region_highlight[++i]=$((offset + MBEGIN))}
>       ${region_highlight[++i]=$((offset + MEND))}
>       ${region_highlight[++i]=${__hsmw_hl_color}}}

% text='# This is an example code that is diverse and allows to test a'
% local last=""; : ${text//(#b)([a-d])/${last=${match[1]}}}; print -rl $last
<empty result>

With #m:
% local last=""; : ${text//(#m)([a-d])/${last=$MATCH}}; print -rl $last
<empty result>

How come it works for you?
-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin
Blog: http://zdharma.org

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Match to the end of string when using (S) flag
  2018-11-07 12:30       ` Sebastian Gniazdowski
@ 2018-11-07 15:25         ` Bart Schaefer
  2018-11-07 17:16           ` Sebastian Gniazdowski
  0 siblings, 1 reply; 10+ messages in thread
From: Bart Schaefer @ 2018-11-07 15:25 UTC (permalink / raw)
  To: Sebastian Gniazdowski; +Cc: Zsh Users

On Wed, Nov 7, 2018 at 4:30 AM Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
>
> % text='# This is an example code that is diverse and allows to test a'
> % local last=""; : ${text//(#b)([a-d])/${last=${match[1]}}}; print -rl $last
> <empty result>
>
> With #m:
> % local last=""; : ${text//(#m)([a-d])/${last=$MATCH}}; print -rl $last
> <empty result>
>
> How come it works for you?

In your expression, $last already has a value (empty string), so you
need ${last::=$match[1]} or ${last::=$MATCH} to assign it.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Match to the end of string when using (S) flag
  2018-11-07 15:25         ` Bart Schaefer
@ 2018-11-07 17:16           ` Sebastian Gniazdowski
  0 siblings, 0 replies; 10+ messages in thread
From: Sebastian Gniazdowski @ 2018-11-07 17:16 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh Users

On Wed, 7 Nov 2018 at 16:25, Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Wed, Nov 7, 2018 at 4:30 AM Sebastian Gniazdowski
> <sgniazdowski@gmail.com> wrote:
> >
> > % text='# This is an example code that is diverse and allows to test a'
> > % local last=""; : ${text//(#b)([a-d])/${last=${match[1]}}}; print -rl $last
> > <empty result>
> >
> > With #m:
> > % local last=""; : ${text//(#m)([a-d])/${last=$MATCH}}; print -rl $last
> > <empty result>
> >
> > How come it works for you?
>
> In your expression, $last already has a value (empty string), so you
> need ${last::=$match[1]} or ${last::=$MATCH} to assign it.

It now works, thanks!

-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin
Blog: http://zdharma.org

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, back to index

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-01 14:25 Match to the end of string when using (S) flag Sebastian Gniazdowski
2016-11-01 16:27 ` Sebastian Gniazdowski
2016-11-01 18:29   ` Sebastian Gniazdowski
2016-11-01 21:23     ` Bart Schaefer
2016-11-02  6:53       ` Sebastian Gniazdowski
2016-11-02 15:50         ` Bart Schaefer
2016-11-02  9:46       ` Peter Stephenson
2018-11-07 12:30       ` Sebastian Gniazdowski
2018-11-07 15:25         ` Bart Schaefer
2018-11-07 17:16           ` Sebastian Gniazdowski

zsh-users

Archives are clonable: git clone --mirror http://inbox.vuxu.org/zsh-users

Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.zsh.users


AGPL code for this site: git clone https://public-inbox.org/ public-inbox