"drop-in replacement" and transpose-words-match

"drop-in replacement" and transpose-words-match

[Bart Schaefer]

Of the eight match-words-by-style functions, transpose-words-match is the
only one that is not really a drop-in, because it adopts the paradigm of
match-words-by-style where a cursor in the middle of a word splits that
word (treating the LBUFFER part as "word before the cursor" and RBUFFER
part as "word after the cursor").

There are some useful things about this behavior, e.g., "skip-chars 1" 
where you can make "fooXbar" swap as "barXfoo", but there are also some
drawbacks -- cases where transpose-words would do something but -match
does nothing or does something very different.  So how about this:  if
the skip-chars style is set (even to zero), then keep the cursor where
it is and split the word, otherwise act like the builtin.  The subword
flavor of word-style can similarly be checked.

Also, it looks to me as though transpose-words-match assumes zsh emulation
but (unlike e.g. select-word-style) never declares it.  Also "skip" is a
local in match-words-by-style, it doesn't help to declare it here.

This still isn't perfect -- if you are on or immediately after a space
in the middle of a quoted string, for example, you'll swap around that
space.  I haven't decided on the best way to deal with that.  But this
works much better if you are at the end of a line and invoke transpose:
it swaps the two words to the left, like the builtin.


diff --git a/Functions/Zle/transpose-words-match b/Functions/Zle/transpose-words-match
index c1db310..4d2ac71 100644
--- a/Functions/Zle/transpose-words-match
+++ b/Functions/Zle/transpose-words-match
@@ -11,14 +11,23 @@
 # on X would be turned into `barXfoo' with the cursor still on the X,
 # regardless of what the character X is.
 
+emulate -L zsh
 autoload -Uz match-words-by-style
 
-local curcontext=":zle:$WIDGET" skip
+local curcontext=":zle:$WIDGET"
 local -a matched_words
 integer count=${NUMERIC:-1} neg
 
 (( count < 0 )) && (( count = -count, neg = 1 ))
 
+if [[ $WIDGET == transpose-words ]]; then
+  # default is to be a drop-in replacement, check styles for change
+  zstyle -m $curcontext skip-chars \* ||
+  zstyle -m $curcontext word-style '*subword*' ||
+  { [[ $LBUFFER[-1] != [[:space:]] && $RBUFFER[1] != [[:space:]] ||
+       -z ${RBUFFER//[[:space:]]/} ]] && zle backward-word }
+fi
+
 while (( count-- > 0 )); do
     match-words-by-style
 

Re: "drop-in replacement" and transpose-words-match

[Daniel Shahaf]

[ Sorry for the late reply, I missed this mail last week. ]

Bart Schaefer wrote on Mon, Jan 11, 2016 at 23:32:59 -0800:
> Of the eight match-words-by-style functions, transpose-words-match is the
> only one that is not really a drop-in, because it adopts the paradigm of
> match-words-by-style where a cursor in the middle of a word splits that
> word (treating the LBUFFER part as "word before the cursor" and RBUFFER
> part as "word after the cursor").
> 
> There are some useful things about this behavior, e.g., "skip-chars 1" 
> where you can make "fooXbar" swap as "barXfoo", but there are also some
> drawbacks -- cases where transpose-words would do something but -match
> does nothing or does something very different.  So how about this:  if
> the skip-chars style is set (even to zero), then keep the cursor where
> it is and split the word, otherwise act like the builtin.  The subword
> flavor of word-style can similarly be checked.
> 

That wouldn't let a user have the skip_chars=0 beahaviour for a wider
context and the "act like the builtin" behaviour for a more specific
subcontext.  (It's not possible to unset for a subcontext a style that's
set on a wider context.)

How about changing the pattern from «*» to «<->»?  Then a setting on
a wider context can be hidden for a subcontext by setting the style to
'' on the subcontext.

Cheers,

Daniel


> +++ b/Functions/Zle/transpose-words-match
> @@ -11,14 +11,23 @@
> +if [[ $WIDGET == transpose-words ]]; then
> +  # default is to be a drop-in replacement, check styles for change
> +  zstyle -m $curcontext skip-chars \* ||
> +  zstyle -m $curcontext word-style '*subword*' ||
> +  { [[ $LBUFFER[-1] != [[:space:]] && $RBUFFER[1] != [[:space:]] ||
> +       -z ${RBUFFER//[[:space:]]/} ]] && zle backward-word }
> +fi

Re: "drop-in replacement" and transpose-words-match

[Bart Schaefer]

On Jan 18,  2:25am, Daniel Shahaf wrote:
} 
} Bart Schaefer wrote on Mon, Jan 11, 2016 at 23:32:59 -0800:
} > [if] skip-chars style is set (even to zero), then keep the cursor where
} > it is and split the word, otherwise act like the builtin.  The subword
} > flavor of word-style can similarly be checked.
} 
} That wouldn't let a user have the skip_chars=0 beahaviour for a wider
} context and the "act like the builtin" behaviour for a more specific
} subcontext.

I don't think that's a problem here because the context is ALWAYS
:zle:$WIDGET.  There is no wider or narrower context to consider.

Re: "drop-in replacement" and transpose-words-match

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 1410 bytes --]

On 12 January 2016 at 08:32, Bart Schaefer <schaefer@brasslantern.com> wrote:
> This still isn't perfect -- if you are on or immediately after a space
> in the middle of a quoted string, for example, you'll swap around that
> space.  I haven't decided on the best way to deal with that.  But this
> works much better if you are at the end of a line and invoke transpose:
> it swaps the two words to the left, like the builtin.

I was looking at various sources in recent months and earlier, and was
getting an impression of how important LBUFFER is to them. Your
implementation of widen_for_history is recent example. I am in general
quite repelled by LBUFFER and can even see it as not actually working,
how it was with the widen_for_history – only after a while I saw that
it was working perfectly. However in my code I choose to use the
function I've attached, available also here:

https://github.com/psprint/zsh-editing-workbench/blob/master/zew-process-buffer

My point is: isn't it that sticking to LBUFFER is the source of
various problems? Its use is always more or less hackish and has
effects like "you should position cursor on beginning of a word to
swap it". Implementation generates features while it should be the
opposite.

It's hard for me to provide more evidence to what I'm stating, but
maybe you agree with me in general

Best regards,
Sebastian Gniazdowski

[-- Attachment #2: zew-process-buffer --]
[-- Type: application/octet-stream, Size: 3056 bytes --]

# Input:
# $1 - buffer to process
#
# Output:
# ZEW_PB_WORDS - split of "$1" into shell words; array
# ZEW_PB_WORDS_BEGINNINGS - indexes of first letters of corresponding words in ZEW_PB_WORDS
# ZEW_PB_SPACES - white spaces before corresponding words in ZEW_PB_WORDS
# ZEW_PB_SELECTED_WORD - index in ZEW_PB_WORDS pointing to word activated by CURSOR position
# ZEW_PB_LEFT - left part of active word
# ZEW_PB_RIGHT - right part of active word
#

emulate -LR zsh
setopt typesetsilent extendedglob noshortloops

local MBEGIN MEND MATCH mbegin mend match

local buf="$1"
ZEW_PB_WORDS=( "${(Z+n+)BUFFER}" )
ZEW_PB_SPACES=( )
ZEW_PB_WORDS_BEGINNINGS=( )
ZEW_PB_SELECTED_WORD="-1"

integer nwords="${#ZEW_PB_WORDS}"

# Remove ZEW_PB_WORDS one by one, counting characters,
# computing beginning of each word, to find
# place to break the word into 2 halves (for
# complete_in_word option)

local i word
integer char_count=0

# (Z) handles spaces nicely, but we need them for the user
# Also compute words beginnings and the selected word
for (( i=1; i<=nwords; i++ )); do
    # Remove spurious space generated by Z-flag when
    # input is an unbound '$(' (happens with zsh < 5.1)
    # and also real spaces gathered by an unbound '$(',
    # to handle them in a way normal to this loop
    ZEW_PB_WORDS[i]="${ZEW_PB_WORDS[i]%% ##}"
    word="${ZEW_PB_WORDS[i]}"

    # In general, $buf can start with white spaces
    # We will not search for them, but instead for
    # leading character of current shell word,
    # negated. This is an ambition to completely
    # avoid character classes

    # Remove white spaces
    buf="${buf##(#m)[^$word[1]]#}"
    # Count them
    char_count=char_count+"$#MATCH"
    # This is the beginning of current word
    ZEW_PB_WORDS_BEGINNINGS[i]=$(( char_count + 1 ))
    # Remember the spaces
    ZEW_PB_SPACES[i]="$MATCH"

    # Remove the word
    MATCH=""
    buf="${buf#(#m)$word}"

    # If shell word not found, return. This shoudln't happen
    [ -z "$MATCH" ] && return 0

    # Spaces point to previous shell word
    # Visual cursor right after spaces (-ge) -> not enough to select previous word (-gt required)
    [[ "$ZEW_PB_SELECTED_WORD" -eq "-1" && "$char_count" -gt "$CURSOR" ]] && ZEW_PB_SELECTED_WORD=$(( i-1 ))

    # Actual characters point to current shell word
    # Visual cursor right after letters (-ge) -> enough to select current word
    char_count=char_count+"$#word"
    [[ "$ZEW_PB_SELECTED_WORD" -eq "-1" && "$char_count" -ge "$CURSOR" ]] && ZEW_PB_SELECTED_WORD="$i"
done 

# What's left in $buf can be only white spaces
char_count=char_count+"$#buf"
ZEW_PB_SPACES[i]="$buf"

# Visual cursor right after spaces (-ge) -> enough to select last word
[[ "$ZEW_PB_SELECTED_WORD" -eq "-1" && "$char_count" -ge "$CURSOR" ]] && ZEW_PB_SELECTED_WORD=$(( i-1 ))

# Divide active word into two halves
integer diff=$(( CURSOR - ZEW_PB_WORDS_BEGINNINGS[ZEW_PB_SELECTED_WORD] + 1 ))
word="${ZEW_PB_WORDS[ZEW_PB_SELECTED_WORD]}"
ZEW_PB_LEFT="${word[1,diff]}"
ZEW_PB_RIGHT="${word[diff+1,-1]}"

# vim:ft=zsh

Re: "drop-in replacement" and transpose-words-match

[Bart Schaefer]

On Jan 18,  6:04pm, Sebastian Gniazdowski wrote:
}
} My point is: isn't it that sticking to LBUFFER is the source of
} various problems? Its use is always more or less hackish and has
} effects like "you should position cursor on beginning of a word to
} swap it". Implementation generates features while it should be the
} opposite.

Sorry, I can't agree.  By *FAR* the most common editing operation is
to either either insert or delete to the left of the cursor position,
which means at the end of LBUFFER.

Furthermore, zew-process-buffer always manipulates whatever the (z)
flag's idea of a "word" is, which [as the "bufferwords()" thread has
demonstrated] is somewhat tenuously defined, and may not correspond
to what the user wants to consider a "word" (see e.g. $WORDCHARS).

[I would probably save CURSOR and MARK, set CURSOR to zero, then loop
on "zle forward-word" until I reached or passed the old CURSOR, rather
than try to match up the results of (Z:n:) to $BUFFER.  Then restore
CURSOR and MARK, of course.]

Re: "drop-in replacement" and transpose-words-match

[Sebastian Gniazdowski]

On 19 January 2016 at 07:31, Bart Schaefer <schaefer@brasslantern.com> wrote:
> [I would probably save CURSOR and MARK, set CURSOR to zero, then loop
> on "zle forward-word" until I reached or passed the old CURSOR, rather
> than try to match up the results of (Z:n:) to $BUFFER.  Then restore
> CURSOR and MARK, of course.]

This would miss spaces. zew-pb remembers white space before each word.
This allows to preserve them when rebuilding buffer after e.g. word
swapping

I'm now writing a shell division function that would be used instead
of pure (Z+n+), in zew-pb. The point is to have a meaningful word
division in case if user enters '$(( 0 + 1 abc $(( 0 + 2' and tries to
complete after '$(( 0 + 1' or '$(( 0 + 2'. A tedious task, will call
(z) in number slightly greater than number of letters. My point is
that Zsh could provide more information (besides LBUFFER) to Zle and
compsys, and also zsh-syntax-highlighting. Not completely sure yet
what the information should be.

Zsh-syntax-highlighting can be slow, which can be seen by editing ~400
lines function with zed -f. Maybe it's because it's doing various
things to compensate not that rich information provided by Zsh.

Best regards,
Sebastian Gniazdowski

Re: "drop-in replacement" and transpose-words-match

[Bart Schaefer]

On Jan 19,  9:28am, Sebastian Gniazdowski wrote:
} Subject: Re: "drop-in replacement" and transpose-words-match
}
} On 19 January 2016 at 07:31, Bart Schaefer <schaefer@brasslantern.com> wrote:
} > [I would probably save CURSOR and MARK, set CURSOR to zero, then loop
} > on "zle forward-word" until I reached or passed the old CURSOR, rather
} > than try to match up the results of (Z:n:) to $BUFFER.  Then restore
} > CURSOR and MARK, of course.]
} 
} This would miss spaces. zew-pb remembers white space before each word.

Forward-word advances past the whitespace, so you just have to split it
off the end of each region as you advance.

} My point is
} that Zsh could provide more information (besides LBUFFER) to Zle and
} compsys, and also zsh-syntax-highlighting.

Of course the $words array etc. is supplied to compsys for exactly this
reason; you are merely asking for a different scope and definition of
"word".  In fact a huge amount of information is supplied to compsys,
way more than most completion functions need to use.

Beyond that, the information you're looking for doesn't exist.  The
parser is not activated until accept-line or the equivalent occurs.
To get what you seem to want requires an entire separate-but-lesser
implementation of the parser, kept in sync with the real one.

Which I guess is what zsh-syntax-highlighting has attempted to do,
except by writing the second parser in the shell language itself.

} Zsh-syntax-highlighting can be slow, which can be seen by editing ~400
} lines function with zed -f. Maybe it's because it's doing various
} things to compensate not that rich information provided by Zsh.

It's mostly because it's re-analyzing the entire buffer every time any
change is made, if I recall correctly from the few times I've tried it.

Re: "drop-in replacement" and transpose-words-match

[Daniel Shahaf]

Sebastian Gniazdowski wrote on Tue, Jan 19, 2016 at 09:28:37 +0100:
> Zsh-syntax-highlighting can be slow, which can be seen by editing ~400
> lines function with zed -f. Maybe it's because it's doing various
> things to compensate not that rich information provided by Zsh.

If you get to the bottom of zsh-syntax-highlighting's slowness on fned
buffers please let me know.  I'm aware of the issue but some quality
time with zprof hasn't produced an actionable diagnosis yet.

Re: "drop-in replacement" and transpose-words-match

[Daniel Shahaf]

Bart Schaefer wrote on Tue, Jan 19, 2016 at 19:56:08 -0800:
> On Jan 19,  9:28am, Sebastian Gniazdowski wrote:
> } Zsh-syntax-highlighting can be slow, which can be seen by editing ~400
> } lines function with zed -f. Maybe it's because it's doing various
> } things to compensate not that rich information provided by Zsh.
> 
> It's mostly because it's re-analyzing the entire buffer every time any
> change is made, if I recall correctly from the few times I've tried it.

That's basically how it works, but it doesn't explain the slowness on
fned buffers: the initial highlight of an fned buffer involves exactly
one _zsh_highlight call, and yet is slow.

Cheers,

Daniel

P.S. There are two workarounds for the slowness: invoke <edit-command-line>
within fned, or set the undocumented $ZSH_HIGHLIGHT_MAXLENGTH parameter.

Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Bart Schaefer]

On Jan 23, 11:53pm, Daniel Shahaf wrote:
}
} [Re-analyzing on every keystroke] doesn't explain the slowness on
} fned buffers: the initial highlight of an fned buffer involves exactly
} one _zsh_highlight call, and yet is slow.

Is it similarly slow on any use of "vared"?  What about with a history
line containing a function definition?

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Daniel Shahaf]

Bart Schaefer wrote on Sat, Jan 23, 2016 at 22:20:57 -0800:
> On Jan 23, 11:53pm, Daniel Shahaf wrote:
> } [Re-analyzing on every keystroke] doesn't explain the slowness on
> } fned buffers: the initial highlight of an fned buffer involves exactly
> } one _zsh_highlight call, and yet is slow.
> 
> Is it similarly slow on any use of "vared"?

fned with z-sy-h is slow.

fned without z-sy-h is fast.

vared with z-sy-h is slow.

I tested this with
.
    autoload +X vcs_info
    autoload zed
    zed -f vcs_info
.
compared to
.
    autoload +X vcs_info
    s=$(which vcs_info); vared s
.
.

> What about with a history line containing a function definition?

Also slow.  Tested with:

    # Copy the function's definition to the clipboard, then paste it back
    autoload +X vcs_info
    which vcs_info | xsel
    <shift+insert>

I also tested it as a single line, and that was still slow:

    autoload +X vcs_info
    which vcs_info | perl -pe 's/\n/;/' | xsel
    <shift+insert>

Thanks for the help!

Daniel

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Bart Schaefer]

On Jan 26, 10:50pm, Daniel Shahaf wrote:
} Subject: Re: Slow highlighting (Re: "drop-in replacement" and transpose-wo
}
} fned with z-sy-h is slow.
} 
} vared with z-sy-h is slow.
} 
} > What about with a history line containing a function definition?
} 
} Also slow.
} 
} I also tested it as a single line, and that was still slow:

So, it's not "zed", it's any case where you're doing syntax analysis of
a function definition.

Time to start eliminating the set of highlighters one by one until you
find one that seems to be the culprit?  Glancing at the github pages,
there seem to be several hightlighters on by default.  My bet would be
on the one that looks for balanced braces, because there is always an
unmatched open brace lurking at the start of the function.

Of course it a combination of highlighters may be involved, which would
be more difficult to isolate.

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Mikael Magnusson]

On Wed, Jan 27, 2016 at 5:31 AM, Bart Schaefer
<schaefer@brasslantern.com> wrote:
> On Jan 26, 10:50pm, Daniel Shahaf wrote:
> } Subject: Re: Slow highlighting (Re: "drop-in replacement" and transpose-wo
> }
> } fned with z-sy-h is slow.
> }
> } vared with z-sy-h is slow.
> }
> } > What about with a history line containing a function definition?
> }
> } Also slow.
> }
> } I also tested it as a single line, and that was still slow:
>
> So, it's not "zed", it's any case where you're doing syntax analysis of
> a function definition.
>
> Time to start eliminating the set of highlighters one by one until you
> find one that seems to be the culprit?  Glancing at the github pages,
> there seem to be several hightlighters on by default.  My bet would be
> on the one that looks for balanced braces, because there is always an
> unmatched open brace lurking at the start of the function.
>
> Of course it a combination of highlighters may be involved, which would
> be more difficult to isolate.

If it is the brace matching code, does it use shell code to loop over
the buffer, or use the zle widget (vi-match-bracket / match-bracket (i
don't think i've committed the latter, the former is a bit finicky to
use but it is possible)? I found using shell code is way too slow on
anything over a couple hundred characters, whereas the widget is fast
enough for anything i've thrown at it. I just tried zed -f
zsh_directory_name which is 200 lines long, and everything works fine
here with my highlighting code (i don't use z-sy-h).

-- 
Mikael Magnusson

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Daniel Shahaf]

Bart Schaefer wrote on Tue, Jan 26, 2016 at 20:31:05 -0800:
> On Jan 26, 10:50pm, Daniel Shahaf wrote:
> } Subject: Re: Slow highlighting (Re: "drop-in replacement" and transpose-wo
> }
> } fned with z-sy-h is slow.
> } 
> } vared with z-sy-h is slow.
> } 
> } > What about with a history line containing a function definition?
> } 
> } Also slow.
> } 
> } I also tested it as a single line, and that was still slow:
> 
> So, it's not "zed", it's any case where you're doing syntax analysis of
> a function definition.
> 
> Time to start eliminating the set of highlighters one by one until you
> find one that seems to be the culprit?

By default, and when reproducing the problem, only the 'main'
highlighter is enabled.  The problem doesn't occur with only the
'cursor' highlighter enabled («ZSH_HIGHLIGHT_HIGHLIGHTERS=(cursor)»),
nor with no highlighters enabled («ZSH_HIGHLIGHT_HIGHLIGHTERS=()»).

Thanks again,

Daniel

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Sebastian Gniazdowski]

On 24 January 2016 at 07:20, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Jan 23, 11:53pm, Daniel Shahaf wrote:
> }
> } [Re-analyzing on every keystroke] doesn't explain the slowness on
> } fned buffers: the initial highlight of an fned buffer involves exactly
> } one _zsh_highlight call, and yet is slow.
>
> Is it similarly slow on any use of "vared"?  What about with a history
> line containing a function definition?

Isn't it that zsyh does a fork for each token it analyzes?

# NOTE: This runs 'setopt', but that should be safe since it'll only ever be
# called inside a $(...) subshell, so the effects will be local.
_zsh_highlight_main__type() {
  if (( $#options_to_set )); then
    setopt $options_to_set;
  fi
  LC_ALL=C builtin type -w -- $1 2>/dev/null
}

...
      local res="$(_zsh_highlight_main__type ${expanded_arg})"
...
      case $res in
        *': reserved')  style=$ZSH_HIGHLIGHT_STYLES[reserved-word];;
        *': suffix alias')
                        style=$ZSH_HIGHLIGHT_STYLES[suffix-alias]
                        ;;
        *': alias')     () {
...

Best regards,
Sebastian Gniazdowski

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Bart Schaefer]

On Feb 10,  5:32pm, Sebastian Gniazdowski wrote:
}
} Isn't it that zsyh does a fork for each token it analyzes?

_zsh_highlight_main__type does appear to be called slightly more often
than once per line on a sample function body.  However it doesn't take
much time:

1.602947000e+00 local res="$(_zsh_highlight_main__type ${expanded_arg})"
1.603546000e+00 _zsh_highlight_main__type ${expanded_arg}
1.603900000e+00 if (( $#options_to_set )) ; then ; setopt $options_to_set ; fi
1.603988000e+00 (( $#options_to_set ))
1.604063000e+00 LC_ALL=C builtin type -w -- $1 2> /dev/null
1.604682000e+00

That's 0.001735 seconds per call on my system.  Not enough to account by
itself for the noticeable delay, I think.  It's true that most of that
time is managing the fork -- each call could be be 0.0012 or so faster
without forking -- but I'd still be surprised if that clears it all up.

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Sebastian Gniazdowski]

Did a following test and indeed zed on 323 lines long function is still slow

--- a/highlighters/main/main-highlighter.zsh
+++ b/highlighters/main/main-highlighter.zsh
@@ -87,10 +87,10 @@ _zsh_highlight_main_add_region_highlight() {
 # NOTE: This runs 'setopt', but that should be safe since it'll only ever be
 # called inside a $(...) subshell, so the effects will be local.
 _zsh_highlight_main__type() {
-  if (( $#options_to_set )); then
-    setopt $options_to_set;
-  fi
-  LC_ALL=C builtin type -w -- $1 2>/dev/null
+  typeset -a arr
+  arr=( ': reserved' ': suffix alias' ': alias' ': builtin' ':
function' ': command'  ': hashed' )
+  integer len=${#arr}
+  REPLY=${arr[ (RANDOM % len) + 1 ]}
 }

 # Main syntax highlighting function.
@@ -283,7 +283,8 @@ _zsh_highlight_main_highlighter()
      else
       _zsh_highlight_main_highlighter_expand_path $arg
       local expanded_arg="$REPLY"
-      local res="$(_zsh_highlight_main__type ${expanded_arg})"
+      _zsh_highlight_main__type ${expanded_arg}
+      local res="$REPLY"
       () {
         # Special-case: command word is '$foo', like that, without
braces or anything.
         #
@@ -295,7 +296,8 @@ _zsh_highlight_main_highlighter()
         local MATCH; integer MBEGIN MEND
         if [[ $res == *': none' ]] && (( ${+parameters} )) &&
            [[ ${arg[1]} == \$ ]] && [[ ${arg:1} =~
^([A-Za-z_][A-Za-z0-9_]*|[0-9]+)$ ]]; then
-          res="$(_zsh_highlight_main__type ${(P)MATCH})"
+          _zsh_highlight_main__type ${(P)MATCH}
+          res="$REPLY"
         fi
       }

Best regards,
Sebastian Gniazdowski

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Sebastian Gniazdowski]

Hello,

a=""; a=${(r:100000::_:)a}; zshstyle() { repeat 100; do
b=${a[-5000,-1]}; done }; shstyle() { repeat 100; do b=${a:
-5000:5000}; done }; time ( zshstyle ); time ( shstyle )

( zshstyle; )  0,66s user 0,01s system 99% cpu 0,673 total
( shstyle; )  0,40s user 0,00s system 99% cpu 0,402 total

The point of this tangent example is: every indexing works by
iterating over buffer and counting characters. By using ":5000" one
pass of finding where an index points to is skipped, as it says "5000
characters from now on". Index -1 iterates from the beginning again to
find end of string.

Here Zsyh indexes buffer, uses -1 and also (i):

    if [[ $arg == ';' ]] ; then
      local needle=$'[;\n]'
      integer offset=${${buf[start_pos+1,-1]}[(i)$needle]}
      (( start_pos += offset - 1 ))
      (( end_pos = start_pos + $#arg ))
    else
      ((start_pos+=${#buf[$start_pos+1,-1]}-${#${buf[$start_pos+1,-1]##([[:space:]]|\\[[:space:]])#}}))
      ((end_pos=$start_pos+${#arg}))
    fi

The longer the buffer the more time -1 consumes. Of course any
indexing is slow, not only -1, but this is nice emphasis of the issue.
The only solution is apparently making Zsh storing strings as real
arrays, of wint_t type.

As for the (i), as far as I remember from the time I worked on C
source, reverse indexing uses one additional "iterate counting
characters" block:

https://github.com/zsh-users/zsh/blob/master/Src/params.c#L1360-L1396

That said, zsyh could be somewhat optimized if the ":howmany" syntax
would be utilized.

Best regards,
Sebastian Gniazdowski

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 2071 bytes --]

I attach the zsyh parser extracted, it's probably easier to profile.
Output is range, style name and the body of the range. Few worries:

10937 10958 double-quoted-argument | "$NLIST_IS_UNIQ_MODE"
10938 10957 dollar-double-quoted-argument | $NLIST_IS_UNIQ_MODE

930 945 single-hyphen-option | -(#c0,1)[0-9]##

Best regards,
Sebastian Gniazdowski

On 11 February 2016 at 11:43, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> Hello,
>
> a=""; a=${(r:100000::_:)a}; zshstyle() { repeat 100; do
> b=${a[-5000,-1]}; done }; shstyle() { repeat 100; do b=${a:
> -5000:5000}; done }; time ( zshstyle ); time ( shstyle )
>
> ( zshstyle; )  0,66s user 0,01s system 99% cpu 0,673 total
> ( shstyle; )  0,40s user 0,00s system 99% cpu 0,402 total
>
> The point of this tangent example is: every indexing works by
> iterating over buffer and counting characters. By using ":5000" one
> pass of finding where an index points to is skipped, as it says "5000
> characters from now on". Index -1 iterates from the beginning again to
> find end of string.
>
> Here Zsyh indexes buffer, uses -1 and also (i):
>
>     if [[ $arg == ';' ]] ; then
>       local needle=$'[;\n]'
>       integer offset=${${buf[start_pos+1,-1]}[(i)$needle]}
>       (( start_pos += offset - 1 ))
>       (( end_pos = start_pos + $#arg ))
>     else
>       ((start_pos+=${#buf[$start_pos+1,-1]}-${#${buf[$start_pos+1,-1]##([[:space:]]|\\[[:space:]])#}}))
>       ((end_pos=$start_pos+${#arg}))
>     fi
>
> The longer the buffer the more time -1 consumes. Of course any
> indexing is slow, not only -1, but this is nice emphasis of the issue.
> The only solution is apparently making Zsh storing strings as real
> arrays, of wint_t type.
>
> As for the (i), as far as I remember from the time I worked on C
> source, reverse indexing uses one additional "iterate counting
> characters" block:
>
> https://github.com/zsh-users/zsh/blob/master/Src/params.c#L1360-L1396
>
> That said, zsyh could be somewhat optimized if the ":howmany" syntax
> would be utilized.
>
> Best regards,
> Sebastian Gniazdowski

[-- Attachment #2: out.txt --]
[-- Type: text/plain, Size: 58533 bytes --]

0 36 unknown-token | # $1, $2, ... - elements of the list
36 37 commandseparator | 

37 121 unknown-token | # $NLIST_NONSELECTABLE_ELEMENTS - array of indexes (1-based) that cannot be selected
121 122 commandseparator | 

122 204 unknown-token | # $REPLY is the output variable - contains index (1-based) or -1 when no selection
204 205 commandseparator | 

205 302 unknown-token | # $reply (array) is the second part of the output - use the index (REPLY) to get selected element
302 303 commandseparator | 

303 304 unknown-token | #
304 305 commandseparator | 

305 357 unknown-token | # Copy this file into /usr/share/zsh/site-functions/
357 358 commandseparator | 

358 395 unknown-token | # and add 'autoload n-list` to .zshrc
395 396 commandseparator | 

396 397 unknown-token | #
397 398 commandseparator | 

398 452 unknown-token | # This function outputs a list of elements that can be
452 453 commandseparator | 

453 499 unknown-token | # navigated with keyboard. Uses curses library
499 500 commandseparator | 

500 501 unknown-token | 

501 508 builtin | emulate
509 512 single-hyphen-option | -LR
513 516 default | zsh
516 517 commandseparator | 

517 518 unknown-token | 

518 524 builtin | setopt
525 538 default | typesetsilent
539 551 default | extendedglob
552 564 default | noshortloops
564 565 commandseparator | 

565 566 unknown-token | 

566 587 assign | _nlist_has_terminfo=0
587 588 commandseparator | 

588 589 unknown-token | 

589 597 builtin | zmodload
598 608 default | zsh/curses
608 609 commandseparator | 

609 617 builtin | zmodload
618 630 default | zsh/terminfo
631 633 redirection | 2>
633 642 path | /dev/null
643 645 commandseparator | &&
646 667 assign | _nlist_has_terminfo=1
667 668 commandseparator | 

668 669 unknown-token | 

669 691 alias | want_to_call_something
691 692 commandseparator | 

692 693 unknown-token | 

693 697 builtin | trap
698 726 double-quoted-argument | "REPLY=-2; reply=(); return"
727 731 default | TERM
732 735 default | INT
736 740 default | QUIT
740 741 commandseparator | 

741 745 builtin | trap
746 759 double-quoted-argument | "_nlist_exit"
760 764 default | EXIT
764 765 commandseparator | 

765 766 unknown-token | 

766 785 unknown-token | # Drawing and input
785 786 commandseparator | 

786 794 builtin | autoload
795 806 default | n-list-draw
807 819 default | n-list-input
819 820 commandseparator | 

820 821 unknown-token | 

821 846 unknown-token | # Cleanup before any exit
846 847 commandseparator | 

847 858 unknown-token | _nlist_exit
858 860 default | ()
861 862 default | {
862 863 commandseparator | 

867 873 builtin | setopt
874 886 default | localoptions
886 887 commandseparator | 

891 897 builtin | setopt
898 910 default | extendedglob
910 911 commandseparator | 

911 912 unknown-token | 

916 918 reserved-word | [[
919 927 double-quoted-argument | "$REPLY"
920 926 dollar-double-quoted-argument | $REPLY
928 929 default | =
930 945 single-hyphen-option | -(#c0,1)[0-9]##
946 948 default | ]]
949 951 commandseparator | ||
952 962 assign | REPLY="-1"
962 963 commandseparator | 

967 974 unknown-token | zcurses
975 977 redirection | 2>
977 986 path | /dev/null
987 993 default | delwin
994 999 default | inner
999 1000 commandseparator | 

1004 1011 unknown-token | zcurses
1012 1014 redirection | 2>
1014 1023 path | /dev/null
1024 1030 default | delwin
1031 1035 default | main
1035 1036 commandseparator | 

1040 1047 unknown-token | zcurses
1048 1050 redirection | 2>
1050 1059 path | /dev/null
1060 1067 default | refresh
1067 1068 commandseparator | 

1072 1079 unknown-token | zcurses
1080 1083 default | end
1083 1084 commandseparator | 

1088 1111 unknown-token | _nlist_alternate_screen
1112 1113 default | 0
1113 1114 commandseparator | 

1118 1142 unknown-token | _nlist_cursor_visibility
1143 1144 default | 1
1144 1145 commandseparator | 

1149 1154 builtin | unset
1155 1174 default | _nlist_has_terminfo
1174 1175 commandseparator | 

1175 1176 reserved-word | }
1176 1177 commandseparator | 

1177 1178 unknown-token | 

1178 1225 unknown-token | # Outputs a message in the bottom of the screen
1225 1226 commandseparator | 

1226 1243 unknown-token | _nlist_status_msg
1243 1245 default | ()
1246 1247 default | {
1247 1248 commandseparator | 

1252 1292 unknown-token | # -1 for border, -1 for 0-based indexing
1292 1293 commandseparator | 

1297 1304 unknown-token | zcurses
1305 1309 default | move
1310 1314 default | main
1315 1341 default | $(( term_height - 1 - 1 ))
1342 1343 default | 2
1343 1344 commandseparator | 

1348 1355 unknown-token | zcurses
1356 1361 default | clear
1362 1366 default | main
1367 1370 default | eol
1370 1371 commandseparator | 

1375 1382 unknown-token | zcurses
1383 1389 default | string
1390 1394 default | main
1395 1399 double-quoted-argument | "$1"
1396 1398 dollar-double-quoted-argument | $1
1399 1400 commandseparator | 

1404 1445 unknown-token | #status_msg_strlen is localized in caller
1445 1446 commandseparator | 

1450 1471 assign | status_msg_strlen=$#1
1471 1472 commandseparator | 

1472 1473 reserved-word | }
1473 1474 commandseparator | 

1474 1475 unknown-token | 

1475 1510 unknown-token | # Prefer tput, then module terminfo
1510 1511 commandseparator | 

1511 1535 unknown-token | _nlist_cursor_visibility
1535 1537 default | ()
1538 1539 default | {
1539 1540 commandseparator | 

1544 1546 reserved-word | if
1547 1551 builtin | type
1552 1556 default | tput
1557 1559 redirection | 2>
1559 1568 path | /dev/null
1569 1572 redirection | 1>&
1572 1573 default | 2
1573 1574 commandseparator | ;
1575 1579 reserved-word | then
1579 1580 unknown-token | 

1588 1589 builtin | [
1590 1594 double-quoted-argument | "$1"
1591 1593 dollar-double-quoted-argument | $1
1595 1596 default | =
1597 1600 double-quoted-argument | "1"
1601 1602 default | ]
1603 1605 commandseparator | &&
1606 1607 reserved-word | {
1608 1612 command | tput
1613 1618 default | cvvis
1618 1619 commandseparator | ;
1620 1624 command | tput
1625 1630 default | cnorm
1631 1632 reserved-word | }
1632 1633 commandseparator | 

1641 1642 builtin | [
1643 1647 double-quoted-argument | "$1"
1644 1646 dollar-double-quoted-argument | $1
1648 1649 default | =
1650 1653 double-quoted-argument | "0"
1654 1655 default | ]
1656 1658 commandseparator | &&
1659 1663 command | tput
1664 1669 default | civis
1669 1670 commandseparator | 

1674 1678 reserved-word | elif
1679 1680 builtin | [
1681 1703 double-quoted-argument | "$_nlist_has_terminfo"
1682 1702 dollar-double-quoted-argument | $_nlist_has_terminfo
1704 1705 default | =
1706 1709 double-quoted-argument | "1"
1710 1711 default | ]
1711 1712 commandseparator | ;
1713 1717 reserved-word | then
1717 1718 unknown-token | 

1726 1727 builtin | [
1728 1732 double-quoted-argument | "$1"
1729 1731 dollar-double-quoted-argument | $1
1733 1734 default | =
1735 1738 double-quoted-argument | "1"
1739 1740 default | ]
1741 1743 commandseparator | &&
1744 1745 reserved-word | {
1746 1747 builtin | [
1748 1750 single-hyphen-option | -n
1751 1767 default | $terminfo[cvvis]
1768 1769 default | ]
1770 1772 commandseparator | &&
1773 1777 builtin | echo
1778 1780 single-hyphen-option | -n
1781 1797 default | $terminfo[cvvis]
1797 1798 commandseparator | ;
1798 1799 unknown-token | 

1826 1827 builtin | [
1828 1830 single-hyphen-option | -n
1831 1847 default | $terminfo[cnorm]
1848 1849 default | ]
1850 1852 commandseparator | &&
1853 1857 builtin | echo
1858 1860 single-hyphen-option | -n
1861 1877 default | $terminfo[cnorm]
1878 1879 reserved-word | }
1879 1880 commandseparator | 

1888 1889 builtin | [
1890 1894 double-quoted-argument | "$1"
1891 1893 dollar-double-quoted-argument | $1
1895 1896 default | =
1897 1900 double-quoted-argument | "0"
1901 1902 default | ]
1903 1905 commandseparator | &&
1906 1907 builtin | [
1908 1910 single-hyphen-option | -n
1911 1927 default | $terminfo[civis]
1928 1929 default | ]
1930 1932 commandseparator | &&
1933 1937 builtin | echo
1938 1940 single-hyphen-option | -n
1941 1957 default | $terminfo[civis]
1957 1958 commandseparator | 

1962 1964 reserved-word | fi
1965 1966 commandseparator | 

1966 1967 reserved-word | }
1967 1968 commandseparator | 

1968 1969 unknown-token | 

1969 2019 unknown-token | # Reason for this function is that on some systems
2019 2020 commandseparator | 

2020 2068 unknown-token | # smcup and rmcup are not knowing why left empty
2068 2069 commandseparator | 

2069 2092 unknown-token | _nlist_alternate_screen
2092 2094 default | ()
2095 2096 default | {
2096 2097 commandseparator | 

2101 2102 builtin | [
2103 2125 double-quoted-argument | "$_nlist_has_terminfo"
2104 2124 dollar-double-quoted-argument | $_nlist_has_terminfo
2126 2129 single-hyphen-option | -ne
2130 2133 double-quoted-argument | "1"
2134 2135 default | ]
2136 2138 commandseparator | &&
2139 2145 builtin | return
2145 2146 commandseparator | 

2150 2152 reserved-word | [[
2153 2157 double-quoted-argument | "$1"
2154 2156 dollar-double-quoted-argument | $1
2158 2159 default | =
2160 2163 double-quoted-argument | "1"
2164 2166 commandseparator | &&
2167 2169 unknown-token | -n
2170 2188 double-quoted-argument | "$terminfo[smcup]"
2171 2180 dollar-double-quoted-argument | $terminfo
2189 2191 default | ]]
2192 2194 commandseparator | &&
2195 2201 builtin | return
2201 2202 commandseparator | 

2206 2208 reserved-word | [[
2209 2213 double-quoted-argument | "$1"
2210 2212 dollar-double-quoted-argument | $1
2214 2215 default | =
2216 2219 double-quoted-argument | "0"
2220 2222 commandseparator | &&
2223 2225 unknown-token | -n
2226 2244 double-quoted-argument | "$terminfo[rmcup]"
2227 2236 dollar-double-quoted-argument | $terminfo
2245 2247 default | ]]
2248 2250 commandseparator | &&
2251 2257 builtin | return
2257 2258 commandseparator | 

2258 2259 unknown-token | 

2263 2267 reserved-word | case
2268 2275 double-quoted-argument | "$TERM"
2269 2274 dollar-double-quoted-argument | $TERM
2276 2278 default | in
2278 2279 commandseparator | 

2287 2293 unknown-token | *rxvt*
2293 2294 reserved-word | )
2294 2295 commandseparator | 

2307 2308 builtin | [
2309 2313 double-quoted-argument | "$1"
2310 2312 dollar-double-quoted-argument | $1
2314 2315 default | =
2316 2319 double-quoted-argument | "1"
2320 2321 default | ]
2322 2324 commandseparator | &&
2325 2329 builtin | echo
2330 2332 single-hyphen-option | -n
2333 2350 dollar-quoted-argument | $'\x1b7\x1b[?47h'
2335 2339 back-dollar-quoted-argument | \x1b
2340 2344 back-dollar-quoted-argument | \x1b
2350 2351 commandseparator | 

2363 2364 builtin | [
2365 2369 double-quoted-argument | "$1"
2366 2368 dollar-double-quoted-argument | $1
2370 2371 default | =
2372 2375 double-quoted-argument | "0"
2376 2377 default | ]
2378 2380 commandseparator | &&
2381 2385 builtin | echo
2386 2388 single-hyphen-option | -n
2389 2413 dollar-quoted-argument | $'\x1b[2J\x1b[?47l\x1b8'
2391 2395 back-dollar-quoted-argument | \x1b
2398 2402 back-dollar-quoted-argument | \x1b
2407 2411 back-dollar-quoted-argument | \x1b
2413 2414 commandseparator | 

2426 2428 unknown-token | ;;
2428 2429 commandseparator | 

2437 2438 unknown-token | *
2438 2439 reserved-word | )
2439 2440 commandseparator | 

2452 2453 builtin | [
2454 2458 double-quoted-argument | "$1"
2455 2457 dollar-double-quoted-argument | $1
2459 2460 default | =
2461 2464 double-quoted-argument | "1"
2465 2466 default | ]
2467 2469 commandseparator | &&
2470 2474 builtin | echo
2475 2477 single-hyphen-option | -n
2478 2492 dollar-quoted-argument | $'\x1b[?1049h'
2480 2484 back-dollar-quoted-argument | \x1b
2492 2493 commandseparator | 

2505 2506 builtin | [
2507 2511 double-quoted-argument | "$1"
2508 2510 dollar-double-quoted-argument | $1
2512 2513 default | =
2514 2517 double-quoted-argument | "0"
2518 2519 default | ]
2520 2522 commandseparator | &&
2523 2527 builtin | echo
2528 2530 single-hyphen-option | -n
2531 2545 dollar-quoted-argument | $'\x1b[?1049l'
2533 2537 back-dollar-quoted-argument | \x1b
2545 2546 commandseparator | 

2558 2640 unknown-token | # just to remember two other that work: $'\x1b7\x1b[r\x1b[?47h', $'\x1b[?47l\x1b8'
2640 2641 commandseparator | 

2653 2655 unknown-token | ;;
2655 2656 commandseparator | 

2660 2664 reserved-word | esac
2664 2665 commandseparator | 

2665 2666 reserved-word | }
2666 2667 commandseparator | 

2667 2668 unknown-token | 

2668 2703 unknown-token | _nlist_compute_user_vars_difference
2703 2705 default | ()
2706 2707 default | {
2707 2708 commandseparator | 

2716 2718 reserved-word | if
2719 2721 reserved-word | [[
2722 2758 double-quoted-argument | "${(t)NLIST_NONSELECTABLE_ELEMENTS}"
2759 2761 history-expansion | !=
2762 2769 double-quoted-argument | "array"
2770 2772 commandseparator | &&
2772 2773 unknown-token | 

2789 2825 unknown-token | "${(t)NLIST_NONSELECTABLE_ELEMENTS}"
2826 2828 history-expansion | !=
2829 2842 double-quoted-argument | "array-local"
2843 2845 default | ]]
2845 2846 commandseparator | 

2854 2858 reserved-word | then
2858 2859 unknown-token | 

2871 2896 assign | last_element_difference=0
2896 2897 commandseparator | 

2909 2929 assign | current_difference=0
2929 2930 commandseparator | 

2938 2942 reserved-word | else
2942 2943 unknown-token | 

2955 3009 assign | last_element_difference=$#NLIST_NONSELECTABLE_ELEMENTS
3009 3010 commandseparator | 

3022 3042 assign | current_difference=0
3042 3043 commandseparator | 

3055 3060 builtin | local
3061 3064 default | idx
3064 3065 commandseparator | 

3077 3080 reserved-word | for
3081 3084 default | idx
3085 3087 default | in
3088 3127 double-quoted-argument | "${(n)NLIST_NONSELECTABLE_ELEMENTS[@]}"
3127 3128 commandseparator | ;
3129 3131 reserved-word | do
3131 3132 unknown-token | 

3148 3149 builtin | [
3150 3156 double-quoted-argument | "$idx"
3151 3155 dollar-double-quoted-argument | $idx
3157 3160 single-hyphen-option | -le
3161 3181 double-quoted-argument | "$NLIST_CURRENT_IDX"
3162 3180 dollar-double-quoted-argument | $NLIST_CURRENT_IDX
3182 3183 default | ]
3184 3186 commandseparator | &&
3187 3208 assign | current_difference+=1
3209 3211 commandseparator | ||
3212 3217 builtin | break
3217 3218 commandseparator | 

3230 3234 reserved-word | done
3234 3235 commandseparator | 

3243 3245 reserved-word | fi
3245 3246 commandseparator | 

3246 3247 reserved-word | }
3247 3248 commandseparator | 

3248 3249 unknown-token | 

3249 3306 unknown-token | # List was processed, check if variables aren't off range
3306 3307 commandseparator | 

3307 3325 unknown-token | _nlist_verify_vars
3325 3327 default | ()
3328 3329 default | {
3329 3330 commandseparator | 

3334 3335 builtin | [
3336 3356 double-quoted-argument | "$NLIST_CURRENT_IDX"
3337 3355 dollar-double-quoted-argument | $NLIST_CURRENT_IDX
3357 3360 single-hyphen-option | -gt
3361 3376 double-quoted-argument | "$last_element"
3362 3375 dollar-double-quoted-argument | $last_element
3377 3378 default | ]
3379 3381 commandseparator | &&
3382 3415 assign | NLIST_CURRENT_IDX="$last_element"
3415 3416 commandseparator | 

3420 3422 reserved-word | [[
3423 3443 double-quoted-argument | "$NLIST_CURRENT_IDX"
3424 3442 dollar-double-quoted-argument | $NLIST_CURRENT_IDX
3444 3447 single-hyphen-option | -eq
3448 3449 default | 0
3450 3452 commandseparator | &&
3453 3468 unknown-token | "$last_element"
3469 3472 single-hyphen-option | -ne
3473 3474 default | 0
3475 3477 default | ]]
3478 3480 commandseparator | &&
3481 3500 assign | NLIST_CURRENT_IDX=1
3500 3501 commandseparator | 

3505 3507 reserved-word | ((
3594 3596 reserved-word | ))
3596 3597 commandseparator | 

3597 3598 reserved-word | }
3598 3599 commandseparator | 

3599 3600 unknown-token | 

3600 3651 unknown-token | # Compute the variables which are shown to the user
3651 3652 commandseparator | 

3652 3674 unknown-token | _nlist_setup_user_vars
3674 3676 default | ()
3677 3678 default | {
3678 3679 commandseparator | 

3683 3685 reserved-word | if
3686 3687 builtin | [
3688 3692 double-quoted-argument | "$1"
3689 3691 dollar-double-quoted-argument | $1
3693 3694 default | =
3695 3698 double-quoted-argument | "1"
3699 3700 default | ]
3700 3701 commandseparator | ;
3702 3706 reserved-word | then
3706 3707 unknown-token | 

3715 3763 unknown-token | # Basic values when there are no non-selectables
3763 3764 commandseparator | 

3772 3815 assign | NLIST_USER_CURRENT_IDX="$NLIST_CURRENT_IDX"
3815 3816 commandseparator | 

3824 3863 assign | NLIST_USER_LAST_ELEMENT="$last_element"
3863 3864 commandseparator | 

3868 3872 reserved-word | else
3872 3873 unknown-token | 

3881 3916 unknown-token | _nlist_compute_user_vars_difference
3916 3917 commandseparator | 

3925 3993 assign | NLIST_USER_CURRENT_IDX=$(( NLIST_CURRENT_IDX - current_difference ))
3993 3994 commandseparator | 

4002 4071 assign | NLIST_USER_LAST_ELEMENT=$(( last_element - last_element_difference ))
4071 4072 commandseparator | 

4076 4078 reserved-word | fi
4078 4079 commandseparator | 

4079 4080 reserved-word | }
4080 4081 commandseparator | 

4081 4082 unknown-token | 

4082 4107 unknown-token | _nlist_colorify_disp_list
4107 4109 default | ()
4110 4111 default | {
4111 4112 commandseparator | 

4116 4121 builtin | local
4122 4140 default | col=$'\x1b[00;34m'
4141 4157 default | reset=$'\x1b[0m'
4157 4158 commandseparator | 

4162 4163 builtin | [
4164 4166 single-hyphen-option | -n
4167 4190 double-quoted-argument | "$NLIST_COLORING_COLOR"
4168 4189 dollar-double-quoted-argument | $NLIST_COLORING_COLOR
4191 4192 default | ]
4193 4195 commandseparator | &&
4196 4223 assign | col="$NLIST_COLORING_COLOR"
4223 4224 commandseparator | 

4228 4229 builtin | [
4230 4232 single-hyphen-option | -n
4233 4260 double-quoted-argument | "$NLIST_COLORING_END_COLOR"
4234 4259 dollar-double-quoted-argument | $NLIST_COLORING_END_COLOR
4261 4262 default | ]
4263 4265 commandseparator | &&
4266 4299 assign | reset="$NLIST_COLORING_END_COLOR"
4299 4300 commandseparator | 

4300 4301 unknown-token | 

4305 4307 reserved-word | if
4308 4309 builtin | [
4310 4342 double-quoted-argument | "$NLIST_COLORING_MATCH_MULTIPLE"
4311 4341 dollar-double-quoted-argument | $NLIST_COLORING_MATCH_MULTIPLE
4343 4346 single-hyphen-option | -eq
4347 4348 default | 1
4349 4350 default | ]
4350 4351 commandseparator | ;
4352 4356 reserved-word | then
4356 4357 unknown-token | 

4365 4376 assign | disp_list=(
4377 4444 double-quoted-argument | "${(@)disp_list//(#mi)$~NLIST_COLORING_PATTERN/$col${MATCH}$reset}"
4424 4428 dollar-double-quoted-argument | $col
4428 4436 dollar-double-quoted-argument | ${MATCH}
4436 4442 dollar-double-quoted-argument | $reset
4445 4446 assign | )
4446 4447 commandseparator | 

4451 4455 reserved-word | else
4455 4456 unknown-token | 

4464 4475 assign | disp_list=(
4476 4542 double-quoted-argument | "${(@)disp_list/(#mi)$~NLIST_COLORING_PATTERN/$col${MATCH}$reset}"
4522 4526 dollar-double-quoted-argument | $col
4526 4534 dollar-double-quoted-argument | ${MATCH}
4534 4540 dollar-double-quoted-argument | $reset
4543 4544 assign | )
4544 4545 commandseparator | 

4549 4551 reserved-word | fi
4551 4552 commandseparator | 

4552 4553 reserved-word | }
4553 4554 commandseparator | 

4554 4555 unknown-token | 

4555 4556 unknown-token | #
4556 4557 commandseparator | 

4557 4568 unknown-token | # Main code
4568 4569 commandseparator | 

4569 4570 unknown-token | #
4570 4571 commandseparator | 

4571 4572 unknown-token | 

4572 4604 unknown-token | # Check if there is proper input
4604 4605 commandseparator | 

4605 4607 reserved-word | if
4608 4609 builtin | [
4610 4614 double-quoted-argument | "$#"
4615 4618 single-hyphen-option | -lt
4619 4620 default | 1
4621 4622 default | ]
4622 4623 commandseparator | ;
4624 4628 reserved-word | then
4628 4629 unknown-token | 

4633 4637 builtin | echo
4638 4667 double-quoted-argument | "Usage: n-list element_1 ..."
4667 4668 commandseparator | 

4672 4678 builtin | return
4679 4680 default | 1
4680 4681 commandseparator | 

4681 4683 reserved-word | fi
4683 4684 commandseparator | 

4684 4685 unknown-token | 

4685 4695 assign | REPLY="-1"
4695 4696 commandseparator | 

4696 4703 builtin | typeset
4704 4707 single-hyphen-option | -ga
4708 4713 default | reply
4713 4714 commandseparator | 

4714 4721 assign | reply=(
4721 4722 assign | )
4722 4723 commandseparator | 

4723 4724 unknown-token | 

4724 4731 builtin | integer
4732 4752 default | term_height="$LINES"
4752 4753 commandseparator | 

4753 4760 builtin | integer
4761 4782 default | term_width="$COLUMNS"
4782 4783 commandseparator | 

4783 4785 reserved-word | if
4786 4788 reserved-word | [[
4789 4803 double-quoted-argument | "$term_height"
4790 4802 dollar-double-quoted-argument | $term_height
4804 4807 single-hyphen-option | -lt
4808 4809 default | 1
4810 4812 commandseparator | ||
4813 4826 unknown-token | "$term_width"
4827 4830 single-hyphen-option | -lt
4831 4832 default | 1
4833 4835 default | ]]
4835 4836 commandseparator | ;
4837 4841 reserved-word | then
4841 4842 unknown-token | 

4846 4851 builtin | local
4852 4875 default | stty_out=$( stty size )
4875 4876 commandseparator | 

4880 4908 assign | term_height="${stty_out% *}"
4908 4909 commandseparator | 

4913 4940 assign | term_width="${stty_out#* }"
4940 4941 commandseparator | 

4941 4943 reserved-word | fi
4943 4944 commandseparator | 

4944 4951 builtin | integer
4952 4978 default | inner_height=term_height-3
4978 4979 commandseparator | 

4979 4986 builtin | integer
4987 5011 default | inner_width=term_width-3
5011 5012 commandseparator | 

5012 5019 builtin | integer
5020 5044 default | page_height=inner_height
5044 5045 commandseparator | 

5045 5052 builtin | integer
5053 5075 default | page_width=inner_width
5075 5076 commandseparator | 

5076 5077 unknown-token | 

5077 5084 builtin | typeset
5085 5087 single-hyphen-option | -a
5088 5092 default | list
5093 5102 default | disp_list
5102 5103 commandseparator | 

5103 5110 builtin | integer
5111 5126 default | last_element=$#
5126 5127 commandseparator | 

5127 5132 builtin | local
5133 5139 default | action
5139 5140 commandseparator | 

5140 5145 builtin | local
5146 5155 default | final_key
5155 5156 commandseparator | 

5156 5163 builtin | integer
5164 5173 default | selection
5173 5174 commandseparator | 

5174 5181 builtin | integer
5182 5207 default | last_element_difference=0
5207 5208 commandseparator | 

5208 5215 builtin | integer
5216 5236 default | current_difference=0
5236 5237 commandseparator | 

5237 5242 builtin | local
5243 5264 default | prev_search_buffer=""
5264 5265 commandseparator | 

5265 5272 builtin | integer
5273 5289 default | prev_uniq_mode=0
5289 5290 commandseparator | 

5290 5297 builtin | integer
5298 5315 default | prev_start_idx=-1
5315 5316 commandseparator | 

5316 5321 builtin | local
5322 5328 default | MBEGIN
5329 5333 default | MEND
5334 5339 default | MATCH
5340 5346 default | mbegin
5347 5351 default | mend
5352 5357 default | match
5357 5358 commandseparator | 

5358 5359 unknown-token | 

5359 5403 unknown-token | # Ability to remember the list between calls
5403 5404 commandseparator | 

5404 5406 reserved-word | if
5407 5409 reserved-word | [[
5410 5412 single-hyphen-option | -z
5413 5436 double-quoted-argument | "$NLIST_REMEMBER_STATE"
5414 5435 dollar-double-quoted-argument | $NLIST_REMEMBER_STATE
5437 5439 commandseparator | ||
5440 5463 unknown-token | "$NLIST_REMEMBER_STATE"
5464 5467 single-hyphen-option | -eq
5468 5469 default | 0
5470 5472 commandseparator | ||
5473 5496 unknown-token | "$NLIST_REMEMBER_STATE"
5497 5500 single-hyphen-option | -eq
5501 5502 default | 2
5503 5505 default | ]]
5505 5506 commandseparator | ;
5507 5511 reserved-word | then
5511 5512 unknown-token | 

5516 5551 assign | NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN=1
5551 5552 commandseparator | 

5556 5575 assign | NLIST_CURRENT_IDX=1
5575 5576 commandseparator | 

5580 5602 assign | NLIST_IS_SEARCH_MODE=0
5602 5603 commandseparator | 

5607 5629 assign | NLIST_SEARCH_BUFFER=""
5629 5630 commandseparator | 

5634 5653 assign | NLIST_TEXT_OFFSET=0
5653 5654 commandseparator | 

5658 5678 assign | NLIST_IS_UNIQ_MODE=0
5678 5679 commandseparator | 

5679 5680 unknown-token | 

5684 5726 unknown-token | # Zero - because it isn't known, unless we
5726 5727 commandseparator | 

5731 5773 unknown-token | # confirm that first element is selectable
5773 5774 commandseparator | 

5778 5802 assign | NLIST_USER_CURRENT_IDX=0
5802 5803 commandseparator | 

5807 5809 reserved-word | [[
5810 5847 default | ${NLIST_NONSELECTABLE_ELEMENTS[(r)1]}
5848 5850 history-expansion | !=
5851 5852 default | 1
5853 5855 default | ]]
5856 5858 commandseparator | &&
5859 5883 assign | NLIST_USER_CURRENT_IDX=1
5883 5884 commandseparator | 

5888 5964 assign | NLIST_USER_LAST_ELEMENT=$(( last_element - $#NLIST_NONSELECTABLE_ELEMENTS ))
5964 5965 commandseparator | 

5965 5966 unknown-token | 

5970 6001 unknown-token | # 2 is init once, then remember
6001 6002 commandseparator | 

6006 6007 builtin | [
6008 6031 double-quoted-argument | "$NLIST_REMEMBER_STATE"
6009 6030 dollar-double-quoted-argument | $NLIST_REMEMBER_STATE
6032 6035 single-hyphen-option | -eq
6036 6037 default | 2
6038 6039 default | ]
6040 6042 commandseparator | &&
6043 6065 assign | NLIST_REMEMBER_STATE=1
6065 6066 commandseparator | 

6066 6068 reserved-word | fi
6068 6069 commandseparator | 

6069 6070 unknown-token | 

6070 6072 reserved-word | if
6073 6074 builtin | [
6075 6104 double-quoted-argument | "$NLIST_START_IN_SEARCH_MODE"
6076 6103 dollar-double-quoted-argument | $NLIST_START_IN_SEARCH_MODE
6105 6108 single-hyphen-option | -eq
6109 6110 default | 1
6111 6112 default | ]
6112 6113 commandseparator | ;
6114 6118 reserved-word | then
6118 6119 unknown-token | 

6123 6151 assign | NLIST_START_IN_SEARCH_MODE=0
6151 6152 commandseparator | 

6156 6178 assign | NLIST_IS_SEARCH_MODE=1
6178 6179 commandseparator | 

6179 6181 reserved-word | fi
6181 6182 commandseparator | 

6182 6183 unknown-token | 

6183 6185 reserved-word | if
6186 6187 builtin | [
6188 6190 single-hyphen-option | -n
6191 6213 double-quoted-argument | "$NLIST_SET_SEARCH_TO"
6192 6212 dollar-double-quoted-argument | $NLIST_SET_SEARCH_TO
6214 6215 default | ]
6215 6216 commandseparator | ;
6217 6221 reserved-word | then
6221 6222 unknown-token | 

6226 6268 assign | NLIST_SEARCH_BUFFER="$NLIST_SET_SEARCH_TO"
6268 6269 commandseparator | 

6273 6295 assign | NLIST_SET_SEARCH_TO=""
6295 6296 commandseparator | 

6296 6298 reserved-word | fi
6298 6299 commandseparator | 

6299 6300 unknown-token | 

6300 6302 reserved-word | if
6303 6304 builtin | [
6305 6332 double-quoted-argument | "$NLIST_START_IN_UNIQ_MODE"
6306 6331 dollar-double-quoted-argument | $NLIST_START_IN_UNIQ_MODE
6333 6336 single-hyphen-option | -eq
6337 6338 default | 1
6339 6340 default | ]
6340 6341 commandseparator | ;
6342 6346 reserved-word | then
6346 6347 unknown-token | 

6351 6377 assign | NLIST_START_IN_UNIQ_MODE=0
6377 6378 commandseparator | 

6382 6402 assign | NLIST_IS_UNIQ_MODE=1
6402 6403 commandseparator | 

6403 6405 reserved-word | fi
6405 6406 commandseparator | 

6406 6407 unknown-token | 

6407 6430 unknown-token | _nlist_alternate_screen
6431 6432 default | 1
6432 6433 commandseparator | 

6433 6440 unknown-token | zcurses
6441 6445 default | init
6445 6446 commandseparator | 

6446 6453 unknown-token | zcurses
6454 6460 default | delwin
6461 6465 default | main
6466 6468 redirection | 2>
6468 6477 path | /dev/null
6477 6478 commandseparator | 

6478 6485 unknown-token | zcurses
6486 6492 default | delwin
6493 6498 default | inner
6499 6501 redirection | 2>
6501 6510 path | /dev/null
6510 6511 commandseparator | 

6511 6518 unknown-token | zcurses
6519 6525 default | addwin
6526 6530 default | main
6531 6545 double-quoted-argument | "$term_height"
6532 6544 dollar-double-quoted-argument | $term_height
6546 6559 double-quoted-argument | "$term_width"
6547 6558 dollar-double-quoted-argument | $term_width
6560 6561 default | 0
6562 6563 default | 0
6563 6564 commandseparator | 

6564 6571 unknown-token | zcurses
6572 6578 default | addwin
6579 6584 default | inner
6585 6600 double-quoted-argument | "$inner_height"
6586 6599 dollar-double-quoted-argument | $inner_height
6601 6615 double-quoted-argument | "$inner_width"
6602 6614 dollar-double-quoted-argument | $inner_width
6616 6617 default | 1
6618 6619 default | 2
6619 6620 commandseparator | 

6620 6627 unknown-token | zcurses
6628 6630 default | bg
6631 6635 default | main
6636 6647 default | white/black
6647 6648 commandseparator | 

6648 6655 unknown-token | zcurses
6656 6658 default | bg
6659 6664 default | inner
6665 6676 default | white/black
6676 6677 commandseparator | 

6677 6679 reserved-word | if
6680 6681 builtin | [
6682 6705 double-quoted-argument | "$NLIST_IS_SEARCH_MODE"
6683 6704 dollar-double-quoted-argument | $NLIST_IS_SEARCH_MODE
6706 6709 single-hyphen-option | -ne
6710 6711 default | 1
6712 6713 default | ]
6713 6714 commandseparator | ;
6715 6719 reserved-word | then
6719 6720 unknown-token | 

6724 6748 unknown-token | _nlist_cursor_visibility
6749 6750 default | 0
6750 6751 commandseparator | 

6751 6753 reserved-word | fi
6753 6754 commandseparator | 

6754 6755 unknown-token | 

6755 6756 unknown-token | #
6756 6757 commandseparator | 

6757 6778 unknown-token | # Listening for input
6778 6779 commandseparator | 

6779 6780 unknown-token | #
6780 6781 commandseparator | 

6781 6782 unknown-token | 

6782 6787 builtin | local
6788 6791 default | key
6792 6798 default | keypad
6798 6799 commandseparator | 

6799 6800 unknown-token | 

6800 6820 unknown-token | # Clear input buffer
6820 6821 commandseparator | 

6821 6828 unknown-token | zcurses
6829 6836 default | timeout
6837 6841 default | main
6842 6843 default | 0
6843 6844 commandseparator | 

6844 6851 unknown-token | zcurses
6852 6857 default | input
6858 6862 default | main
6863 6866 default | key
6867 6873 default | keypad
6873 6874 commandseparator | 

6874 6881 unknown-token | zcurses
6882 6889 default | timeout
6890 6894 default | main
6895 6897 single-hyphen-option | -1
6897 6898 commandseparator | 

6898 6904 assign | key=""
6904 6905 commandseparator | 

6905 6914 assign | keypad=""
6914 6915 commandseparator | 

6915 6916 unknown-token | 

6916 6974 unknown-token | # This loop makes script faster on some Zsh's (e.g. 5.0.8)
6974 6975 commandseparator | 

6975 6981 reserved-word | repeat
6982 6983 default | 1
6983 6984 unknown-token | ;
6985 6987 reserved-word | do
6987 6988 unknown-token | 

6992 6998 assign | list=(
6999 7003 double-quoted-argument | "$@"
7004 7005 assign | )
7005 7006 commandseparator | 

7006 7010 reserved-word | done
7010 7011 commandseparator | 

7011 7012 unknown-token | 

7012 7033 assign | last_element="$#list"
7033 7034 commandseparator | 

7034 7035 unknown-token | 

7035 7040 reserved-word | while
7041 7043 reserved-word | ((
7046 7048 reserved-word | ))
7048 7049 commandseparator | ;
7050 7052 reserved-word | do
7052 7053 unknown-token | 

7057 7095 unknown-token | # Do searching (filtering with string)
7095 7096 commandseparator | 

7100 7102 reserved-word | if
7103 7104 builtin | [
7105 7107 single-hyphen-option | -n
7108 7130 double-quoted-argument | "$NLIST_SEARCH_BUFFER"
7109 7129 dollar-double-quoted-argument | $NLIST_SEARCH_BUFFER
7131 7132 default | ]
7132 7133 commandseparator | ;
7134 7138 reserved-word | then
7138 7139 unknown-token | 

7147 7166 unknown-token | # Compute new list?
7166 7167 commandseparator | 

7175 7177 reserved-word | if
7178 7180 reserved-word | [[
7181 7203 double-quoted-argument | "$NLIST_SEARCH_BUFFER"
7182 7202 dollar-double-quoted-argument | $NLIST_SEARCH_BUFFER
7204 7206 history-expansion | !=
7207 7228 double-quoted-argument | "$prev_search_buffer"
7208 7227 dollar-double-quoted-argument | $prev_search_buffer
7229 7231 commandseparator | ||
7232 7253 unknown-token | "$NLIST_IS_UNIQ_MODE"
7254 7257 single-hyphen-option | -ne
7258 7275 double-quoted-argument | "$prev_uniq_mode"
7259 7274 dollar-double-quoted-argument | $prev_uniq_mode
7276 7278 default | ]]
7278 7279 commandseparator | ;
7280 7284 reserved-word | then
7284 7285 unknown-token | 

7297 7338 assign | prev_search_buffer="$NLIST_SEARCH_BUFFER"
7338 7339 commandseparator | 

7351 7387 assign | prev_uniq_mode="$NLIST_IS_UNIQ_MODE"
7387 7388 commandseparator | 

7400 7445 unknown-token | # regenerating list -> regenerating disp_list
7445 7446 commandseparator | 

7458 7475 assign | prev_start_idx=-1
7475 7476 commandseparator | 

7476 7477 unknown-token | 

7489 7550 unknown-token | # Take all elements, including duplicates and non-selectables
7550 7551 commandseparator | 

7563 7570 builtin | typeset
7571 7573 default | +U
7574 7578 default | list
7578 7579 commandseparator | 

7591 7597 reserved-word | repeat
7598 7599 default | 1
7599 7600 unknown-token | ;
7601 7603 reserved-word | do
7603 7604 unknown-token | 

7620 7626 assign | list=(
7627 7631 double-quoted-argument | "$@"
7632 7633 assign | )
7633 7634 commandseparator | 

7646 7650 reserved-word | done
7650 7651 commandseparator | 

7651 7652 unknown-token | 

7664 7696 unknown-token | # Remove non-selectable elements
7696 7697 commandseparator | 

7709 7710 builtin | [
7711 7743 double-quoted-argument | "$#NLIST_NONSELECTABLE_ELEMENTS"
7744 7747 single-hyphen-option | -gt
7748 7749 default | 0
7750 7751 default | ]
7752 7754 commandseparator | &&
7755 7758 reserved-word | for
7759 7760 default | i
7761 7763 default | in
7764 7804 double-quoted-argument | "${(nO)NLIST_NONSELECTABLE_ELEMENTS[@]}"
7804 7805 commandseparator | ;
7806 7808 reserved-word | do
7808 7809 unknown-token | 

7825 7835 assign | list[$i]=(
7835 7836 assign | )
7836 7837 commandseparator | 

7849 7853 reserved-word | done
7853 7854 commandseparator | 

7854 7855 unknown-token | 

7867 7886 unknown-token | # Remove duplicates
7886 7887 commandseparator | 

7899 7900 builtin | [
7901 7922 double-quoted-argument | "$NLIST_IS_UNIQ_MODE"
7902 7921 dollar-double-quoted-argument | $NLIST_IS_UNIQ_MODE
7923 7926 single-hyphen-option | -eq
7927 7928 default | 1
7929 7930 default | ]
7931 7933 commandseparator | &&
7934 7941 builtin | typeset
7942 7944 single-hyphen-option | -U
7945 7949 default | list
7949 7950 commandseparator | 

7950 7951 unknown-token | 

7963 7984 assign | last_element="$#list"
7984 7985 commandseparator | 

7985 7986 unknown-token | 

7998 8021 unknown-token | # Next do the filtering
8021 8022 commandseparator | 

8034 8039 builtin | local
8040 8083 default | search_buffer="${NLIST_SEARCH_BUFFER%% ##}"
8083 8084 commandseparator | 

8096 8133 assign | search_buffer="${search_buffer## ##}"
8133 8134 commandseparator | 

8146 8209 assign | search_buffer="${search_buffer//(#m)[][*?|#~^()><\\]/\\$MATCH}"
8209 8210 commandseparator | 

8222 8227 builtin | local
8228 8245 default | search_pattern=""
8245 8246 commandseparator | 

8258 8263 builtin | local
8264 8284 default | colsearch_pattern=""
8284 8285 commandseparator | 

8297 8299 reserved-word | if
8300 8301 builtin | [
8302 8304 single-hyphen-option | -n
8305 8321 double-quoted-argument | "$search_buffer"
8306 8320 dollar-double-quoted-argument | $search_buffer
8322 8323 default | ]
8323 8324 commandseparator | ;
8325 8329 reserved-word | then
8329 8330 unknown-token | 

8346 8391 unknown-token | # Patterns will be *foo*~^*bar* and (foo|bar)
8391 8392 commandseparator | 

8408 8451 assign | search_pattern="${search_buffer// ##/*~^*}"
8451 8452 commandseparator | 

8468 8511 assign | colsearch_pattern="${search_buffer// ##/|}"
8511 8512 commandseparator | 

8512 8513 unknown-token | 

8529 8585 unknown-token | # The repeat will make the matching work on a fresh heap
8585 8586 commandseparator | 

8602 8608 reserved-word | repeat
8609 8610 default | 1
8610 8611 unknown-token | ;
8612 8614 reserved-word | do
8614 8615 unknown-token | 

8635 8641 assign | list=(
8642 8679 double-quoted-argument | "${(@M)list:#(#i)*$~search_pattern*}"
8680 8681 assign | )
8681 8682 commandseparator | 

8698 8702 reserved-word | done
8702 8703 commandseparator | 

8703 8704 unknown-token | 

8720 8741 assign | last_element="$#list"
8741 8742 commandseparator | 

8754 8756 reserved-word | fi
8756 8757 commandseparator | 

8757 8758 unknown-token | 

8770 8800 unknown-token | # Called after processing list
8800 8801 commandseparator | 

8813 8831 unknown-token | _nlist_verify_vars
8831 8832 commandseparator | 

8840 8842 reserved-word | fi
8842 8843 commandseparator | 

8843 8844 unknown-token | 

8852 8874 unknown-token | _nlist_setup_user_vars
8875 8876 default | 1
8876 8877 commandseparator | 

8877 8878 unknown-token | 

8886 8893 builtin | integer
8894 8960 default | end_idx=$(( NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN + page_height - 1 ))
8960 8961 commandseparator | 

8969 8970 builtin | [
8971 8981 double-quoted-argument | "$end_idx"
8972 8980 dollar-double-quoted-argument | $end_idx
8982 8985 single-hyphen-option | -gt
8986 9001 double-quoted-argument | "$last_element"
8987 9000 dollar-double-quoted-argument | $last_element
9002 9003 default | ]
9004 9006 commandseparator | &&
9007 9027 assign | end_idx=last_element
9027 9028 commandseparator | 

9028 9029 unknown-token | 

9037 9039 reserved-word | if
9040 9041 builtin | [
9042 9059 double-quoted-argument | "$prev_start_idx"
9043 9058 dollar-double-quoted-argument | $prev_start_idx
9060 9063 single-hyphen-option | -ne
9064 9100 double-quoted-argument | "$NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN"
9065 9099 dollar-double-quoted-argument | $NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN
9101 9102 default | ]
9102 9103 commandseparator | ;
9104 9108 reserved-word | then
9108 9109 unknown-token | 

9121 9172 assign | prev_start_idx="$NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN"
9172 9173 commandseparator | 

9185 9196 assign | disp_list=(
9197 9253 double-quoted-argument | "${(@)list[NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN, end_idx]}"
9254 9255 assign | )
9255 9256 commandseparator | 

9256 9257 unknown-token | 

9269 9271 reserved-word | if
9272 9273 builtin | [
9274 9276 single-hyphen-option | -n
9277 9297 double-quoted-argument | "$colsearch_pattern"
9278 9296 dollar-double-quoted-argument | $colsearch_pattern
9298 9299 default | ]
9299 9300 commandseparator | ;
9301 9305 reserved-word | then
9305 9306 unknown-token | 

9322 9327 builtin | local
9328 9346 default | red=$'\x1b[00;31m'
9347 9367 default | reset=$'\x1b[00;00m'
9367 9368 commandseparator | 

9384 9440 unknown-token | # The repeat will make the matching work on a fresh heap
9440 9441 commandseparator | 

9457 9463 reserved-word | repeat
9464 9465 default | 1
9465 9466 unknown-token | ;
9467 9469 reserved-word | do
9469 9470 unknown-token | 

9490 9501 assign | disp_list=(
9502 9566 double-quoted-argument | "${(@)disp_list//(#mi)($~colsearch_pattern)/$red${MATCH}$reset}"
9546 9550 dollar-double-quoted-argument | $red
9550 9558 dollar-double-quoted-argument | ${MATCH}
9558 9564 dollar-double-quoted-argument | $reset
9567 9568 assign | )
9568 9569 commandseparator | 

9585 9589 reserved-word | done
9589 9590 commandseparator | 

9602 9604 reserved-word | fi
9604 9605 commandseparator | 

9605 9606 unknown-token | 

9618 9691 unknown-token | # We have display list, lets replace newlines with "\n" when needed (1/2)
9691 9692 commandseparator | 

9704 9705 builtin | [
9706 9731 double-quoted-argument | "$NLIST_REPLACE_NEWLINES"
9707 9730 dollar-double-quoted-argument | $NLIST_REPLACE_NEWLINES
9732 9735 single-hyphen-option | -eq
9736 9737 default | 1
9738 9739 default | ]
9740 9742 commandseparator | &&
9743 9754 assign | disp_list=(
9755 9783 double-quoted-argument | "${(@)disp_list//$'\n'/\\n}"
9778 9780 back-double-quoted-argument | \\
9784 9785 assign | )
9785 9786 commandseparator | 

9794 9796 reserved-word | fi
9796 9797 commandseparator | 

9797 9798 unknown-token | 

9806 9827 unknown-token | # Output colored list
9827 9828 commandseparator | 

9836 9847 unknown-token | n-list-draw
9848 9896 double-quoted-argument | "$(( (NLIST_CURRENT_IDX-1) % page_height + 1 ))"
9911 9925 double-quoted-argument | "$page_height"
9912 9924 dollar-double-quoted-argument | $page_height
9926 9939 double-quoted-argument | "$page_width"
9927 9938 dollar-double-quoted-argument | $page_width
9940 9941 default | 0
9942 9943 default | 0
9944 9964 double-quoted-argument | "$NLIST_TEXT_OFFSET"
9945 9963 dollar-double-quoted-argument | $NLIST_TEXT_OFFSET
9965 9970 default | inner
9985 10000 double-quoted-argument | "$disp_list[@]"
9986 9996 dollar-double-quoted-argument | $disp_list
10000 10001 commandseparator | 

10005 10009 reserved-word | else
10009 10010 unknown-token | 

10018 10070 unknown-token | # There is no search, but there was in previous loop
10070 10071 commandseparator | 

10079 10083 unknown-token | # OR
10083 10084 commandseparator | 

10092 10127 unknown-token | # Uniq mode was entered or left out
10127 10128 commandseparator | 

10136 10157 unknown-token | # -> compute new list
10157 10158 commandseparator | 

10166 10168 reserved-word | if
10169 10171 reserved-word | [[
10172 10174 single-hyphen-option | -n
10175 10196 double-quoted-argument | "$prev_search_buffer"
10176 10195 dollar-double-quoted-argument | $prev_search_buffer
10197 10199 commandseparator | ||
10200 10221 unknown-token | "$NLIST_IS_UNIQ_MODE"
10222 10225 single-hyphen-option | -ne
10226 10243 double-quoted-argument | "$prev_uniq_mode"
10227 10242 dollar-double-quoted-argument | $prev_uniq_mode
10244 10246 default | ]]
10246 10247 commandseparator | ;
10248 10252 reserved-word | then
10252 10253 unknown-token | 

10265 10286 assign | prev_search_buffer=""
10286 10287 commandseparator | 

10299 10335 assign | prev_uniq_mode="$NLIST_IS_UNIQ_MODE"
10335 10336 commandseparator | 

10348 10393 unknown-token | # regenerating list -> regenerating disp_list
10393 10394 commandseparator | 

10406 10423 assign | prev_start_idx=-1
10423 10424 commandseparator | 

10424 10425 unknown-token | 

10437 10498 unknown-token | # Take all elements, including duplicates and non-selectables
10498 10499 commandseparator | 

10511 10518 builtin | typeset
10519 10521 default | +U
10522 10526 default | list
10526 10527 commandseparator | 

10539 10545 reserved-word | repeat
10546 10547 default | 1
10547 10548 unknown-token | ;
10549 10551 reserved-word | do
10551 10552 unknown-token | 

10568 10574 assign | list=(
10575 10579 double-quoted-argument | "$@"
10580 10581 assign | )
10581 10582 commandseparator | 

10594 10598 reserved-word | done
10598 10599 commandseparator | 

10599 10600 unknown-token | 

10612 10667 unknown-token | # Remove non-selectable elements only when in uniq mode
10667 10668 commandseparator | 

10680 10681 builtin | [
10682 10703 double-quoted-argument | "$NLIST_IS_UNIQ_MODE"
10683 10702 dollar-double-quoted-argument | $NLIST_IS_UNIQ_MODE
10704 10707 single-hyphen-option | -eq
10708 10709 default | 1
10710 10711 default | ]
10712 10714 commandseparator | &&
10715 10716 builtin | [
10717 10749 double-quoted-argument | "$#NLIST_NONSELECTABLE_ELEMENTS"
10750 10753 single-hyphen-option | -gt
10754 10755 default | 0
10756 10757 default | ]
10758 10760 commandseparator | &&
10760 10761 unknown-token | 

10773 10776 reserved-word | for
10777 10778 default | i
10779 10781 default | in
10782 10822 double-quoted-argument | "${(nO)NLIST_NONSELECTABLE_ELEMENTS[@]}"
10822 10823 commandseparator | ;
10824 10826 reserved-word | do
10826 10827 unknown-token | 

10843 10853 assign | list[$i]=(
10853 10854 assign | )
10854 10855 commandseparator | 

10867 10871 reserved-word | done
10871 10872 commandseparator | 

10872 10873 unknown-token | 

10885 10922 unknown-token | # Remove duplicates when in uniq mode
10922 10923 commandseparator | 

10935 10936 builtin | [
10937 10958 double-quoted-argument | "$NLIST_IS_UNIQ_MODE"
10938 10957 dollar-double-quoted-argument | $NLIST_IS_UNIQ_MODE
10959 10962 single-hyphen-option | -eq
10963 10964 default | 1
10965 10966 default | ]
10967 10969 commandseparator | &&
10970 10977 builtin | typeset
10978 10980 single-hyphen-option | -U
10981 10985 default | list
10985 10986 commandseparator | 

10986 10987 unknown-token | 

10999 11020 assign | last_element="$#list"
11020 11021 commandseparator | 

11033 11063 unknown-token | # Called after processing list
11063 11064 commandseparator | 

11076 11094 unknown-token | _nlist_verify_vars
11094 11095 commandseparator | 

11103 11105 reserved-word | fi
11105 11106 commandseparator | 

11106 11107 unknown-token | 

11115 11160 unknown-token | # "1" - shouldn't bother with non-selectables
11160 11161 commandseparator | 

11169 11191 unknown-token | _nlist_setup_user_vars
11192 11213 double-quoted-argument | "$NLIST_IS_UNIQ_MODE"
11193 11212 dollar-double-quoted-argument | $NLIST_IS_UNIQ_MODE
11213 11214 commandseparator | 

11214 11215 unknown-token | 

11223 11230 builtin | integer
11231 11297 default | end_idx=$(( NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN + page_height - 1 ))
11297 11298 commandseparator | 

11306 11307 builtin | [
11308 11318 double-quoted-argument | "$end_idx"
11309 11317 dollar-double-quoted-argument | $end_idx
11319 11322 single-hyphen-option | -gt
11323 11338 double-quoted-argument | "$last_element"
11324 11337 dollar-double-quoted-argument | $last_element
11339 11340 default | ]
11341 11343 commandseparator | &&
11344 11364 assign | end_idx=last_element
11364 11365 commandseparator | 

11365 11366 unknown-token | 

11374 11376 reserved-word | if
11377 11378 builtin | [
11379 11396 double-quoted-argument | "$prev_start_idx"
11380 11395 dollar-double-quoted-argument | $prev_start_idx
11397 11400 single-hyphen-option | -ne
11401 11437 double-quoted-argument | "$NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN"
11402 11436 dollar-double-quoted-argument | $NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN
11438 11439 default | ]
11439 11440 commandseparator | ;
11441 11445 reserved-word | then
11445 11446 unknown-token | 

11458 11509 assign | prev_start_idx="$NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN"
11509 11510 commandseparator | 

11522 11533 assign | disp_list=(
11534 11590 double-quoted-argument | "${(@)list[NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN, end_idx]}"
11591 11592 assign | )
11592 11593 commandseparator | 

11593 11594 unknown-token | 

11606 11607 builtin | [
11608 11610 single-hyphen-option | -n
11611 11636 double-quoted-argument | "$NLIST_COLORING_PATTERN"
11612 11635 dollar-double-quoted-argument | $NLIST_COLORING_PATTERN
11637 11638 default | ]
11639 11641 commandseparator | &&
11642 11667 unknown-token | _nlist_colorify_disp_list
11667 11668 commandseparator | 

11668 11669 unknown-token | 

11681 11754 unknown-token | # We have display list, lets replace newlines with "\n" when needed (2/2)
11754 11755 commandseparator | 

11767 11768 builtin | [
11769 11794 double-quoted-argument | "$NLIST_REPLACE_NEWLINES"
11770 11793 dollar-double-quoted-argument | $NLIST_REPLACE_NEWLINES
11795 11798 single-hyphen-option | -eq
11799 11800 default | 1
11801 11802 default | ]
11803 11805 commandseparator | &&
11806 11817 assign | disp_list=(
11818 11846 double-quoted-argument | "${(@)disp_list//$'\n'/\\n}"
11841 11843 back-double-quoted-argument | \\
11847 11848 assign | )
11848 11849 commandseparator | 

11857 11859 reserved-word | fi
11859 11860 commandseparator | 

11860 11861 unknown-token | 

11869 11886 unknown-token | # Output the list
11886 11887 commandseparator | 

11895 11906 unknown-token | n-list-draw
11907 11955 double-quoted-argument | "$(( (NLIST_CURRENT_IDX-1) % page_height + 1 ))"
11970 11984 double-quoted-argument | "$page_height"
11971 11983 dollar-double-quoted-argument | $page_height
11985 11998 double-quoted-argument | "$page_width"
11986 11997 dollar-double-quoted-argument | $page_width
11999 12000 default | 0
12001 12002 default | 0
12003 12023 double-quoted-argument | "$NLIST_TEXT_OFFSET"
12004 12022 dollar-double-quoted-argument | $NLIST_TEXT_OFFSET
12024 12029 default | inner
12044 12059 double-quoted-argument | "$disp_list[@]"
12045 12055 dollar-double-quoted-argument | $disp_list
12059 12060 commandseparator | 

12064 12066 reserved-word | fi
12066 12067 commandseparator | 

12067 12068 unknown-token | 

12072 12077 builtin | local
12078 12095 default | status_msg_strlen
12095 12096 commandseparator | 

12100 12102 reserved-word | if
12103 12104 builtin | [
12105 12128 double-quoted-argument | "$NLIST_IS_SEARCH_MODE"
12106 12127 dollar-double-quoted-argument | $NLIST_IS_SEARCH_MODE
12129 12130 default | =
12131 12134 double-quoted-argument | "1"
12135 12136 default | ]
12136 12137 commandseparator | ;
12138 12142 reserved-word | then
12142 12143 unknown-token | 

12151 12156 builtin | local
12157 12165 default | _txt2=""
12165 12166 commandseparator | 

12174 12175 builtin | [
12176 12197 double-quoted-argument | "$NLIST_IS_UNIQ_MODE"
12177 12196 dollar-double-quoted-argument | $NLIST_IS_UNIQ_MODE
12198 12201 single-hyphen-option | -eq
12202 12203 default | 1
12204 12205 default | ]
12206 12208 commandseparator | &&
12209 12226 assign | _txt2="[-UNIQ-] "
12226 12227 commandseparator | 

12235 12252 unknown-token | _nlist_status_msg
12253 12306 double-quoted-argument | "${_txt2}Filtering with: ${NLIST_SEARCH_BUFFER// /+}"
12254 12262 dollar-double-quoted-argument | ${_txt2}
12306 12307 commandseparator | 

12311 12315 reserved-word | elif
12316 12318 reserved-word | [[
12319 12373 default | ${NLIST_NONSELECTABLE_ELEMENTS[(r)$NLIST_CURRENT_IDX]}
12374 12376 history-expansion | !=
12377 12395 default | $NLIST_CURRENT_IDX
12396 12398 commandseparator | ||
12398 12399 unknown-token | 

12411 12413 unknown-token | -n
12414 12436 double-quoted-argument | "$NLIST_SEARCH_BUFFER"
12415 12435 dollar-double-quoted-argument | $NLIST_SEARCH_BUFFER
12437 12439 commandseparator | ||
12440 12461 unknown-token | "$NLIST_IS_UNIQ_MODE"
12462 12465 single-hyphen-option | -eq
12466 12467 default | 1
12468 12470 default | ]]
12470 12471 commandseparator | ;
12472 12476 reserved-word | then
12476 12477 unknown-token | 

12485 12490 builtin | local
12491 12498 default | _txt=""
12499 12507 default | _txt2=""
12507 12508 commandseparator | 

12516 12517 builtin | [
12518 12520 single-hyphen-option | -n
12521 12541 double-quoted-argument | "$NLIST_GREP_STRING"
12522 12540 dollar-double-quoted-argument | $NLIST_GREP_STRING
12542 12543 default | ]
12544 12546 commandseparator | &&
12547 12575 assign | _txt=" [$NLIST_GREP_STRING]"
12575 12576 commandseparator | 

12584 12585 builtin | [
12586 12607 double-quoted-argument | "$NLIST_IS_UNIQ_MODE"
12587 12606 dollar-double-quoted-argument | $NLIST_IS_UNIQ_MODE
12608 12611 single-hyphen-option | -eq
12612 12613 default | 1
12614 12615 default | ]
12616 12618 commandseparator | &&
12619 12636 assign | _txt2="[-UNIQ-] "
12636 12637 commandseparator | 

12645 12662 unknown-token | _nlist_status_msg
12663 12749 double-quoted-argument | "${_txt2}Current #$NLIST_USER_CURRENT_IDX (of #$NLIST_USER_LAST_ELEMENT entries)$_txt"
12664 12672 dollar-double-quoted-argument | ${_txt2}
12681 12704 dollar-double-quoted-argument | $NLIST_USER_CURRENT_IDX
12710 12734 dollar-double-quoted-argument | $NLIST_USER_LAST_ELEMENT
12743 12748 dollar-double-quoted-argument | $_txt
12749 12750 commandseparator | 

12754 12758 reserved-word | else
12758 12759 unknown-token | 

12767 12784 unknown-token | _nlist_status_msg
12785 12787 double-quoted-argument | ""
12787 12788 commandseparator | 

12792 12794 reserved-word | fi
12794 12795 commandseparator | 

12795 12796 unknown-token | 

12800 12807 unknown-token | zcurses
12808 12814 default | border
12815 12819 default | main
12819 12820 commandseparator | 

12820 12821 unknown-token | 

12825 12830 builtin | local
12831 12895 default | top_msg="${(C)ZSH_NAME} $ZSH_VERSION, shell level $SHLVL, $USER"
12895 12896 commandseparator | 

12900 12907 unknown-token | zcurses
12908 12912 default | move
12913 12917 default | main
12918 12919 default | 0
12920 12957 default | $(( term_width / 2 - $#top_msg / 2 ))
12957 12958 commandseparator | 

12962 12969 unknown-token | zcurses
12970 12976 default | string
12977 12981 default | main
12982 12990 default | $top_msg
12990 12991 commandseparator | 

12991 12992 unknown-token | 

12996 13003 unknown-token | zcurses
13004 13011 default | refresh
13012 13016 default | main
13017 13022 default | inner
13022 13023 commandseparator | 

13027 13034 unknown-token | zcurses
13035 13039 default | move
13040 13044 default | main
13045 13071 default | $(( term_height - 1 - 1 ))
13072 13100 default | $(( status_msg_strlen + 2 ))
13100 13101 commandseparator | 

13101 13102 unknown-token | 

13106 13122 unknown-token | # Wait for input
13122 13123 commandseparator | 

13127 13134 unknown-token | zcurses
13135 13140 default | input
13141 13145 default | main
13146 13149 default | key
13150 13156 default | keypad
13156 13157 commandseparator | 

13157 13158 unknown-token | 

13162 13214 unknown-token | # Get the special (i.e. "keypad") key or regular key
13214 13215 commandseparator | 

13219 13221 reserved-word | if
13222 13223 builtin | [
13224 13226 single-hyphen-option | -n
13227 13233 double-quoted-argument | "$key"
13228 13232 dollar-double-quoted-argument | $key
13234 13235 default | ]
13235 13236 commandseparator | ;
13237 13241 reserved-word | then
13241 13242 unknown-token | 

13250 13266 assign | final_key="$key"
13266 13267 commandseparator | 

13271 13275 reserved-word | elif
13276 13277 builtin | [
13278 13280 single-hyphen-option | -n
13281 13290 double-quoted-argument | "$keypad"
13282 13289 dollar-double-quoted-argument | $keypad
13291 13292 default | ]
13292 13293 commandseparator | ;
13294 13298 reserved-word | then
13298 13299 unknown-token | 

13307 13326 assign | final_key="$keypad"
13326 13327 commandseparator | 

13331 13335 reserved-word | else
13335 13336 unknown-token | 

13344 13361 unknown-token | _nlist_status_msg
13362 13387 double-quoted-argument | "Inproper input detected"
13387 13388 commandseparator | 

13396 13403 unknown-token | zcurses
13404 13411 default | refresh
13412 13416 default | main
13417 13422 default | inner
13422 13423 commandseparator | 

13427 13429 reserved-word | fi
13429 13430 commandseparator | 

13430 13431 unknown-token | 

13435 13447 unknown-token | n-list-input
13448 13468 double-quoted-argument | "$NLIST_CURRENT_IDX"
13449 13467 dollar-double-quoted-argument | $NLIST_CURRENT_IDX
13469 13505 double-quoted-argument | "$NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN"
13470 13504 dollar-double-quoted-argument | $NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN
13528 13542 double-quoted-argument | "$page_height"
13529 13541 dollar-double-quoted-argument | $page_height
13543 13556 double-quoted-argument | "$page_width"
13544 13555 dollar-double-quoted-argument | $page_width
13557 13572 double-quoted-argument | "$last_element"
13558 13571 dollar-double-quoted-argument | $last_element
13573 13593 double-quoted-argument | "$NLIST_TEXT_OFFSET"
13574 13592 dollar-double-quoted-argument | $NLIST_TEXT_OFFSET
13616 13628 double-quoted-argument | "$final_key"
13617 13627 dollar-double-quoted-argument | $final_key
13629 13652 double-quoted-argument | "$NLIST_IS_SEARCH_MODE"
13630 13651 dollar-double-quoted-argument | $NLIST_IS_SEARCH_MODE
13653 13675 double-quoted-argument | "$NLIST_SEARCH_BUFFER"
13654 13674 dollar-double-quoted-argument | $NLIST_SEARCH_BUFFER
13698 13719 double-quoted-argument | "$NLIST_IS_UNIQ_MODE"
13699 13718 dollar-double-quoted-argument | $NLIST_IS_UNIQ_MODE
13719 13720 commandseparator | 

13720 13721 unknown-token | 

13725 13746 assign | selection="$reply[1]"
13746 13747 commandseparator | 

13751 13769 assign | action="$reply[2]"
13769 13770 commandseparator | 

13774 13803 assign | NLIST_CURRENT_IDX="$reply[3]"
13803 13804 commandseparator | 

13808 13853 assign | NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN="$reply[4]"
13853 13854 commandseparator | 

13858 13887 assign | NLIST_TEXT_OFFSET="$reply[5]"
13887 13888 commandseparator | 

13892 13924 assign | NLIST_IS_SEARCH_MODE="$reply[6]"
13924 13925 commandseparator | 

13929 13960 assign | NLIST_SEARCH_BUFFER="$reply[7]"
13960 13961 commandseparator | 

13965 13995 assign | NLIST_IS_UNIQ_MODE="$reply[8]"
13995 13996 commandseparator | 

13996 13997 unknown-token | 

14001 14003 reserved-word | if
14004 14005 builtin | [
14006 14015 double-quoted-argument | "$action"
14007 14014 dollar-double-quoted-argument | $action
14016 14017 default | =
14018 14026 double-quoted-argument | "SELECT"
14027 14028 default | ]
14028 14029 commandseparator | ;
14030 14034 reserved-word | then
14034 14035 unknown-token | 

14043 14061 assign | REPLY="$selection"
14061 14062 commandseparator | 

14070 14077 assign | reply=(
14078 14088 double-quoted-argument | "$list[@]"
14079 14084 dollar-double-quoted-argument | $list
14089 14090 assign | )
14090 14091 commandseparator | 

14099 14104 builtin | break
14104 14105 commandseparator | 

14109 14113 reserved-word | elif
14114 14115 builtin | [
14116 14125 double-quoted-argument | "$action"
14117 14124 dollar-double-quoted-argument | $action
14126 14127 default | =
14128 14134 double-quoted-argument | "QUIT"
14135 14136 default | ]
14136 14137 commandseparator | ;
14138 14142 reserved-word | then
14142 14143 unknown-token | 

14151 14159 assign | REPLY=-1
14159 14160 commandseparator | 

14168 14175 assign | reply=(
14176 14186 double-quoted-argument | "$list[@]"
14177 14182 dollar-double-quoted-argument | $list
14187 14188 assign | )
14188 14189 commandseparator | 

14197 14202 builtin | break
14202 14203 commandseparator | 

14207 14211 reserved-word | elif
14212 14213 builtin | [
14214 14223 double-quoted-argument | "$action"
14215 14222 dollar-double-quoted-argument | $action
14224 14225 default | =
14226 14234 double-quoted-argument | "REDRAW"
14235 14236 default | ]
14236 14237 commandseparator | ;
14238 14242 reserved-word | then
14242 14243 unknown-token | 

14251 14258 unknown-token | zcurses
14259 14264 default | clear
14265 14269 default | main
14270 14276 default | redraw
14276 14277 commandseparator | 

14285 14292 unknown-token | zcurses
14293 14298 default | clear
14299 14304 default | inner
14305 14311 default | redraw
14311 14312 commandseparator | 

14316 14318 reserved-word | fi
14318 14319 commandseparator | 

14319 14323 reserved-word | done
14323 14324 commandseparator | 

14324 14325 unknown-token | 

14325 14349 unknown-token | # vim: set filetype=zsh:

[-- Attachment #3: parse.zsh --]
[-- Type: application/octet-stream, Size: 20500 bytes --]

#!/usr/bin/env zsh

# -------------------------------------------------------------------------------------------------
# Copyright (c) 2010-2015 zsh-syntax-highlighting contributors
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification, are permitted
# provided that the following conditions are met:
#
#  * Redistributions of source code must retain the above copyright notice, this list of conditions
#    and the following disclaimer.
#  * Redistributions in binary form must reproduce the above copyright notice, this list of
#    conditions and the following disclaimer in the documentation and/or other materials provided
#    with the distribution.
#  * Neither the name of the zsh-syntax-highlighting contributors nor the names of its contributors
#    may be used to endorse or promote products derived from this software without specific prior
#    written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
# IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# -------------------------------------------------------------------------------------------------
# -*- mode: zsh; sh-indentation: 2; indent-tabs-mode: nil; sh-basic-offset: 2; -*-
# vim: ft=zsh sw=2 ts=2 et
# -------------------------------------------------------------------------------------------------

zmodload zsh/zprof

alias want_to_call_something=":"

# Possible sorts are:
# default
# unknown-token
# reserved-word
# alias
# suffix-alias
# builtin
# function
# command
# precommand
# commandseparator
# hashed-command
# path
# path_prefix
# globbing
# history-expansion
# single-hyphen-option
# double-hyphen-option
# back-quoted-argument
# single-quoted-argument
# double-quoted-argument
# dollar-quoted-argument
# dollar-double-quoted-argument
# back-double-quoted-argument
# back-dollar-quoted-argument
# assign
# redirection
# comment
 
# Helper to deal with tokens crossing line boundaries.
-zplg-statica-save() {
  integer start=$1 end=$2
  local sort=$3

  # Having end<0 would be a bug
  (( end < 0 )) && return 
  # Having start<0 is normal with e.g. multiline strings
  (( start < 0 )) && start=0

  region_highlight+=("$start $end $sort | ${buf[start+1,end]}")
}

# Wrapper around 'type -w'.
#
# Takes a single argument and outputs the output of 'type -w $1'.
#
# NOTE: This runs 'setopt', but that should be safe since it'll only ever be
# called inside a $(...) subshell, so the effects will be local.
-zplg-statica-type() {
  if (( $#options_to_set )); then
    setopt $options_to_set;
  fi
  LC_ALL=C builtin type -w -- $1 2>/dev/null
}

# Main syntax highlighting function.
-zplg-statica()
{
  if [[ -o path_dirs ]]; then
    integer path_dirs_was_set=1
  else
    integer path_dirs_was_set=0
  fi

  emulate -L zsh
  setopt localoptions extendedglob bareglobqual

  ## Variable declarations and initializations
  local start_pos=0 end_pos highlight_glob=true arg sort
  local in_array_assignment=false # true between 'a=(' and the matching ')'
  typeset -a ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR
  typeset -a ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS
  typeset -a ZSH_HIGHLIGHT_TOKENS_CONTROL_FLOW
  local -a options_to_set # used in callees
  local buf="$(<$1)"
  region_highlight=()

  if (( path_dirs_was_set )); then
    options_to_set+=( PATH_DIRS )
  fi
  unset path_dirs_was_set

  ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR=(
    '|' '||' ';' '&' '&&'
    '|&'
    '&!' '&|'
    # ### 'case' syntax, but followed by a pattern, not by a command
    # ';;' ';&' ';|'
  )
  ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS=(
    'builtin' 'command' 'exec' 'nocorrect' 'noglob'
    'pkexec' # immune to #121 because it's usually not passed --option flags
  )

  # Tokens that, at (naively-determined) "command position", are followed by
  # a de jure command position.  All of these are reserved words.
  ZSH_HIGHLIGHT_TOKENS_CONTROL_FLOW=(
    $'\x7b' # block
    $'\x28' # subshell
    '()' # anonymous function
    'while'
    'until'
    'if'
    'then'
    'elif'
    'else'
    'do'
    'time'
    'coproc'
    '!' # reserved word; unrelated to $histchars[1]
  )

  local this_word=':start:' next_word
  integer in_redirection
  for arg in ${(zZ+c+)buf}; do
    if (( in_redirection )); then
      (( --in_redirection ))
    fi
    if (( in_redirection == 0 )); then
      # Initialize $next_word to its default value.
      next_word=':regular:'
    else
      # Stall $next_word.
      :
    fi
    # $already_added is set to 1 to disable adding an entry to region_highlight
    # for this iteration.  Currently, that is done for "" and $'' strings,
    # which add the entry early so escape sequences within the string override
    # the string's color.
    integer already_added=0
    local style_override=""
    if [[ $this_word == *':start:'* ]]; then
      in_array_assignment=false
      if [[ $arg == 'noglob' ]]; then
        highlight_glob=false
      fi
    fi

    # advance $start_pos, skipping over whitespace in $buf.
    if [[ $arg == ';' ]] ; then
      # We're looking for either a semicolon or a newline, whichever comes
      # first.  Both of these are rendered as a ";" (SEPER) by the ${(z)..}
      # flag.
      #
      # We can't use the (Z+n+) flag because that elides the end-of-command
      # token altogether, so 'echo foo\necho bar' (two commands) becomes
      # indistinguishable from 'echo foo echo bar' (one command with three
      # words for arguments).
      local needle=$'[;\n]'
      integer offset=${${buf[start_pos+1,-1]}[(i)$needle]}
      (( start_pos += offset - 1 ))
      (( end_pos = start_pos + $#arg ))
    else
      ((start_pos+=${#buf[$start_pos+1,-1]}-${#${buf[$start_pos+1,-1]##([[:space:]]|\\[[:space:]])#}}))
      ((end_pos=$start_pos+${#arg}))
    fi

    if [[ -n ${interactive_comments+'set'} && $arg[1] == $histchars[3] ]]; then
      if [[ $this_word == *(':regular:'|':start:')* ]]; then
        sort="comment"
      else
        sort="unknown-token" # prematurely terminated
      fi
      -zplg-statica-save $start_pos $end_pos $sort
      already_added=1
      continue
    fi

    # Parse the sudo command line
    if (( ! in_redirection )); then
      if [[ $this_word == *':sudo_opt:'* ]]; then
        case "$arg" in
          # Flag that requires an argument
          '-'[Cgprtu]) this_word=${this_word//:start:/};
                       next_word=':sudo_arg:';;
          # This prevents misbehavior with sudo -u -otherargument
          '-'*)        this_word=${this_word//:start:/};
                       next_word+=':start:';
                       next_word+=':sudo_opt:';;
          *)           ;;
        esac
      elif [[ $this_word == *':sudo_arg:'* ]]; then
        next_word+=':sudo_opt:'
        next_word+=':start:'
      fi
    fi

    if [[ $this_word == *':start:'* ]] && (( in_redirection == 0 )); then # $arg is the command word
     if [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS:#"$arg"} ]]; then
      sort="precommand"
     elif [[ "$arg" = "sudo" ]]; then
      sort="precommand"
      next_word=${next_word//:regular:/}
      next_word+=':sudo_opt:'
      next_word+=':start:'
     else
      -zplg-statica-expand-path $arg
      local expanded_arg="$REPLY"
      local res="$(-zplg-statica-type ${expanded_arg})"
      () {
        # Special-case: command word is '$foo', like that, without braces or anything.
        #
        # That's not entirely correct --- if the parameter's value happens to be a reserved
        # word, the parameter expansion will be highlighted as a reserved word --- but that
        # incorrectness is outweighed by the usability improvement of permitting the use of
        # parameters that refer to commands, functions, and builtins.
        local -a match mbegin mend
        local MATCH; integer MBEGIN MEND
        if [[ $res == *': none' ]] && (( ${+parameters} )) &&
           [[ ${arg[1]} == \$ ]] && [[ ${arg:1} =~ ^([A-Za-z_][A-Za-z0-9_]*|[0-9]+)$ ]]; then
          res="$(-zplg-statica-type ${(P)MATCH})"
        fi
      }
      case $res in
        *': reserved')  sort="reserved-word";;
        *': suffix alias')
                        sort="suffix-alias"
                        ;;
        *': alias')     () {
                          integer insane_alias
                          case $arg in 
                            # Issue #263: aliases with '=' on their LHS.
                            #
                            # There are three cases:
                            #
                            # - Unsupported, breaks 'alias -L' output, but invokable:
                            ('='*) :;;
                            # - Unsupported, not invokable:
                            (*'='*) insane_alias=1;;
                            # - The common case:
                            (*) :;;
                          esac
                          if (( insane_alias )); then
                            sort="unknown-token"
                          else
                            sort="alias"
                            local aliased_command="${"$(alias -- $arg)"#*=}"
                            [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS:#"$aliased_command"} && -z ${(M)ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS:#"$arg"} ]] && ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS+=($arg)
                          fi
                        }
                        ;;
        *': builtin')   sort="builtin";;
        *': function')  sort="function";;
        *': command')   sort="command";;
        *': hashed')    sort="hashed-command";;
        *)              if _zsh_highlight_main_highlighter_check_assign; then
                          sort="assign"
                          if [[ $arg[-1] == '(' ]]; then
                            in_array_assignment=true
                          else
                            # assignment to a scalar parameter.
                            # (For array assignments, the command doesn't start until the ")" token.)
                            next_word+=':start:'
                          fi
                        elif [[ $arg[0,1] == $histchars[0,1] || $arg[0,1] == $histchars[2,2] ]]; then
                          sort="history-expansion"
                        elif [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR:#"$arg"} ]]; then
                          if [[ $this_word == *':regular:'* ]]; then
                            # This highlights empty commands (semicolon follows nothing) as an error.
                            # Zsh accepts them, though.
                            sort="commandseparator"
                          else
                            sort="unknown-token"
                          fi
                        elif [[ $arg == (<0-9>|)(\<|\>)* ]]; then
                          # A '<' or '>', possibly followed by a digit
                          sort="redirection"
                          (( in_redirection=2 ))
                        elif [[ $arg[1,2] == '((' ]]; then
                          # Arithmetic evaluation.
                          #
                          # Note: prior to zsh-5.1.1-52-g4bed2cf (workers/36669), the ${(z)...}
                          # splitter would only output the '((' token if the matching '))' had
                          # been typed.  Therefore, under those versions of zsh, BUFFER="(( 42"
                          # would be highlighted as an error until the matching "))" are typed.
                          #
                          # We highlight just the opening parentheses, as a reserved word; this
                          # is how [[ ... ]] is highlighted, too.
                          sort="reserved-word"
                          -zplg-statica-save $start_pos $((start_pos + 2)) $sort
                          already_added=1
                          if [[ $arg[-2,-1] == '))' ]]; then
                            -zplg-statica-save $((end_pos - 2)) $end_pos $sort
                            already_added=1
                          fi
                        elif [[ $arg == '()' || $arg == $'\x28' ]]; then
                          # anonymous function
                          # subshell
                          sort="reserved-word"
                        else
                          if _zsh_highlight_main_highlighter_check_path; then
                            sort="path"
                          else
                            sort="unknown-token"
                          fi
                        fi
                        ;;
      esac
     fi
    else # $arg is a non-command word
      case $arg in
        $'\x29') # subshell or end of array assignment
                 if $in_array_assignment; then
                   sort="assign"
                   in_array_assignment=false
                 else
                   sort="reserved-word"
                 fi;;
        $'\x7d') sort="reserved-word";; # block
        '--'*)   sort="double-hyphen-option";;
        '-'*)    sort="single-hyphen-option";;
        "'"*)    sort="single-quoted-argument";;
        '"'*)    sort="double-quoted-argument"
                 -zplg-statica-save $start_pos $end_pos $sort
                 _zsh_highlight_main_highlighter_highlight_string
                 already_added=1
                 ;;
        \$\'*)   sort="dollar-quoted-argument"
                 -zplg-statica-save $start_pos $end_pos $sort
                 _zsh_highlight_main_highlighter_highlight_dollar_string
                 already_added=1
                 ;;
        '`'*)    sort="back-quoted-argument";;
        [*?]*|*[^\\][*?]*)
                 $highlight_glob && sort="globbing" || sort="default";;
        *)       if false; then
                 elif [[ $arg[0,1] = $histchars[0,1] ]]; then
                   sort="history-expansion"
                 elif [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR:#"$arg"} ]]; then
                   if [[ $this_word == *':regular:'* ]]; then
                     sort="commandseparator"
                   else
                     sort="unknown-token"
                   fi
                 elif [[ $arg == (<0-9>|)(\<|\>)* ]]; then
                   sort="redirection"
                   (( in_redirection=2 ))
                 else
                   if _zsh_highlight_main_highlighter_check_path; then
                     sort="path"
                   else
                     sort="default"
                   fi
                 fi
                 ;;
      esac
    fi
    # if a style_override was set (eg in _zsh_highlight_main_highlighter_check_path), use it
    [[ -n $style_override ]] && sort=$ZSH_HIGHLIGHT_STYLES[$style_override]
    (( already_added )) || -zplg-statica-save $start_pos $end_pos $sort
    if [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_COMMANDSEPARATOR:#"$arg"} ]]; then
      next_word=':start:'
      highlight_glob=true
    elif
       [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_CONTROL_FLOW:#"$arg"} && $this_word == *':start:'* ]] ||
       [[ -n ${(M)ZSH_HIGHLIGHT_TOKENS_PRECOMMANDS:#"$arg"} && $this_word == *':start:'* ]]; then
      next_word=':start:'
    elif [[ $arg == "repeat" && $this_word == *':start:'* ]]; then
      # skip the repeat-count word
      in_redirection=2
      # The redirection mechanism assumes $this_word describes the word
      # following the redirection.  Make it so.
      #
      # The repeat-count word will be handled like a redirection target.
      this_word=':start:'
    fi
    start_pos=$end_pos
    (( in_redirection == 0 )) && this_word=$next_word
  done
}

# Check if $arg is variable assignment
_zsh_highlight_main_highlighter_check_assign()
{
    setopt localoptions extended_glob
    [[ $arg == [[:alpha:]_][[:alnum:]_]#(|\[*\])(|[+])=* ]]
}

# Check if $arg is a path.
_zsh_highlight_main_highlighter_check_path()
{
  -zplg-statica-expand-path $arg;
  local expanded_path="$REPLY"

  [[ -z $expanded_path ]] && return 1
  [[ -e $expanded_path ]] && return 0

  # Search the path in CDPATH
  local cdpath_dir
  for cdpath_dir in $cdpath ; do
    [[ -e "$cdpath_dir/$expanded_path" ]] && return 0
  done

  # If dirname($arg) doesn't exist, neither does $arg.
  [[ ! -e ${expanded_path:h} ]] && return 1

  # If this word ends the buffer, check if it's the prefix of a valid path.
  if [[ ${BUFFER[1]} != "-" && ${#BUFFER} == $end_pos ]] &&
     [[ $WIDGET != accept-* ]]; then
    local -a tmp
    tmp=( ${expanded_path}*(N) )
    (( $#tmp > 0 )) && style_override=path_prefix && return 0
  fi

  # It's not a path.
  return 1
}

# Highlight special chars inside double-quoted strings
_zsh_highlight_main_highlighter_highlight_string()
{
  setopt localoptions noksharrays
  local -a match mbegin mend
  local MATCH; integer MBEGIN MEND
  local i j k sort
  # Starting quote is at 1, so start parsing at offset 2 in the string.
  for (( i = 2 ; i < end_pos - start_pos ; i += 1 )) ; do
    (( j = i + start_pos - 1 ))
    (( k = j + 1 ))
    case "$arg[$i]" in
      '$' ) sort="dollar-double-quoted-argument"
            # Look for an alphanumeric parameter name.
            if [[ ${arg:$i} =~ ^([A-Za-z_][A-Za-z0-9_]*|[0-9]+) ]] ; then
              (( k += $#MATCH )) # highlight the parameter name
              (( i += $#MATCH )) # skip past it
            elif [[ ${arg:$i} =~ ^[{]([A-Za-z_][A-Za-z0-9_]*|[0-9]+)[}] ]] ; then
              (( k += $#MATCH )) # highlight the parameter name and braces
              (( i += $#MATCH )) # skip past it
            else
              continue
            fi
            ;;
      "\\") sort="back-double-quoted-argument"
            if [[ \\\`\"\$ == *$arg[$i+1]* ]]; then
              (( k += 1 )) # Color following char too.
              (( i += 1 )) # Skip parsing the escaped char.
            else
              continue
            fi
            ;;
      *) continue ;;

    esac
    -zplg-statica-save $j $k $sort
  done
}

# Highlight special chars inside dollar-quoted strings
_zsh_highlight_main_highlighter_highlight_dollar_string()
{
  setopt localoptions noksharrays
  local -a match mbegin mend
  local MATCH; integer MBEGIN MEND
  local i j k sort
  local AA
  integer c
  # Starting dollar-quote is at 1:2, so start parsing at offset 3 in the string.
  for (( i = 3 ; i < end_pos - start_pos ; i += 1 )) ; do
    (( j = i + start_pos - 1 ))
    (( k = j + 1 ))
    case "$arg[$i]" in
      "\\") sort="back-dollar-quoted-argument"
            for (( c = i + 1 ; c <= end_pos - start_pos ; c += 1 )); do
              [[ "$arg[$c]" != ([0-9xXuUa-fA-F]) ]] && break
            done
            AA=$arg[$i+1,$c-1]
            # Matching for HEX and OCT values like \0xA6, \xA6 or \012
            if [[    "$AA" =~ "^(x|X)[0-9a-fA-F]{1,2}"
                  || "$AA" =~ "^[0-7]{1,3}"
                  || "$AA" =~ "^u[0-9a-fA-F]{1,4}"
                  || "$AA" =~ "^U[0-9a-fA-F]{1,8}"
               ]]; then
              (( k += $#MATCH ))
              (( i += $#MATCH ))
            else
              if (( $#arg > $i+1 )) && [[ $arg[$i+1] == [xXuU] ]]; then
                # \x not followed by hex digits is probably an error
                sort="unknown-token"
              fi
              (( k += 1 )) # Color following char too.
              (( i += 1 )) # Skip parsing the escaped char.
            fi
            ;;
      *) continue ;;

    esac
    -zplg-statica-save $j $k $sort
  done
}

# Called with a single positional argument.
# Perform filename expansion (tilde expansion) on the argument and set $REPLY to the expanded value.
#
# Does not perform filename generation (globbing).
-zplg-statica-expand-path()
{
  # The $~1 syntax normally performs filename generation, but not when it's on the right-hand side of ${x:=y}.
  setopt localoptions nonomatch
  unset REPLY
  : ${REPLY:=${(Q)~1}}
}

-zplg-statica "$1"

print -rl "${region_highlight[@]}"

#zprof | head

[-- Attachment #4: n-list --]
[-- Type: application/octet-stream, Size: 14350 bytes --]

# $1, $2, ... - elements of the list
# $NLIST_NONSELECTABLE_ELEMENTS - array of indexes (1-based) that cannot be selected
# $REPLY is the output variable - contains index (1-based) or -1 when no selection
# $reply (array) is the second part of the output - use the index (REPLY) to get selected element
#
# Copy this file into /usr/share/zsh/site-functions/
# and add 'autoload n-list` to .zshrc
#
# This function outputs a list of elements that can be
# navigated with keyboard. Uses curses library

emulate -LR zsh

setopt typesetsilent extendedglob noshortloops

_nlist_has_terminfo=0

zmodload zsh/curses
zmodload zsh/terminfo 2>/dev/null && _nlist_has_terminfo=1

want_to_call_something

trap "REPLY=-2; reply=(); return" TERM INT QUIT
trap "_nlist_exit" EXIT

# Drawing and input
autoload n-list-draw n-list-input

# Cleanup before any exit
_nlist_exit() {
    setopt localoptions
    setopt extendedglob

    [[ "$REPLY" = -(#c0,1)[0-9]## ]] || REPLY="-1"
    zcurses 2>/dev/null delwin inner
    zcurses 2>/dev/null delwin main
    zcurses 2>/dev/null refresh
    zcurses end
    _nlist_alternate_screen 0
    _nlist_cursor_visibility 1
    unset _nlist_has_terminfo
}

# Outputs a message in the bottom of the screen
_nlist_status_msg() {
    # -1 for border, -1 for 0-based indexing
    zcurses move main $(( term_height - 1 - 1 )) 2
    zcurses clear main eol
    zcurses string main "$1"
    #status_msg_strlen is localized in caller
    status_msg_strlen=$#1
}

# Prefer tput, then module terminfo
_nlist_cursor_visibility() {
    if type tput 2>/dev/null 1>&2; then
        [ "$1" = "1" ] && { tput cvvis; tput cnorm }
        [ "$1" = "0" ] && tput civis
    elif [ "$_nlist_has_terminfo" = "1" ]; then
        [ "$1" = "1" ] && { [ -n $terminfo[cvvis] ] && echo -n $terminfo[cvvis];
                           [ -n $terminfo[cnorm] ] && echo -n $terminfo[cnorm] }
        [ "$1" = "0" ] && [ -n $terminfo[civis] ] && echo -n $terminfo[civis]
    fi 
}

# Reason for this function is that on some systems
# smcup and rmcup are not knowing why left empty
_nlist_alternate_screen() {
    [ "$_nlist_has_terminfo" -ne "1" ] && return
    [[ "$1" = "1" && -n "$terminfo[smcup]" ]] && return
    [[ "$1" = "0" && -n "$terminfo[rmcup]" ]] && return

    case "$TERM" in
        *rxvt*)
            [ "$1" = "1" ] && echo -n $'\x1b7\x1b[?47h'
            [ "$1" = "0" ] && echo -n $'\x1b[2J\x1b[?47l\x1b8'
            ;;
        *)
            [ "$1" = "1" ] && echo -n $'\x1b[?1049h'
            [ "$1" = "0" ] && echo -n $'\x1b[?1049l'
            # just to remember two other that work: $'\x1b7\x1b[r\x1b[?47h', $'\x1b[?47l\x1b8'
            ;;
    esac
}

_nlist_compute_user_vars_difference() {
        if [[ "${(t)NLIST_NONSELECTABLE_ELEMENTS}" != "array" &&
                "${(t)NLIST_NONSELECTABLE_ELEMENTS}" != "array-local" ]]
        then
            last_element_difference=0
            current_difference=0
        else
            last_element_difference=$#NLIST_NONSELECTABLE_ELEMENTS
            current_difference=0
            local idx
            for idx in "${(n)NLIST_NONSELECTABLE_ELEMENTS[@]}"; do
                [ "$idx" -le "$NLIST_CURRENT_IDX" ] && current_difference+=1 || break
            done
        fi
}

# List was processed, check if variables aren't off range
_nlist_verify_vars() {
    [ "$NLIST_CURRENT_IDX" -gt "$last_element" ] && NLIST_CURRENT_IDX="$last_element"
    [[ "$NLIST_CURRENT_IDX" -eq 0 && "$last_element" -ne 0 ]] && NLIST_CURRENT_IDX=1
    (( NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN=0+((NLIST_CURRENT_IDX-1)/page_height)*page_height+1 ))
}

# Compute the variables which are shown to the user
_nlist_setup_user_vars() {
    if [ "$1" = "1" ]; then
        # Basic values when there are no non-selectables
        NLIST_USER_CURRENT_IDX="$NLIST_CURRENT_IDX"
        NLIST_USER_LAST_ELEMENT="$last_element"
    else
        _nlist_compute_user_vars_difference
        NLIST_USER_CURRENT_IDX=$(( NLIST_CURRENT_IDX - current_difference ))
        NLIST_USER_LAST_ELEMENT=$(( last_element - last_element_difference ))
    fi
}

_nlist_colorify_disp_list() {
    local col=$'\x1b[00;34m' reset=$'\x1b[0m'
    [ -n "$NLIST_COLORING_COLOR" ] && col="$NLIST_COLORING_COLOR"
    [ -n "$NLIST_COLORING_END_COLOR" ] && reset="$NLIST_COLORING_END_COLOR"

    if [ "$NLIST_COLORING_MATCH_MULTIPLE" -eq 1 ]; then
        disp_list=( "${(@)disp_list//(#mi)$~NLIST_COLORING_PATTERN/$col${MATCH}$reset}" )
    else
        disp_list=( "${(@)disp_list/(#mi)$~NLIST_COLORING_PATTERN/$col${MATCH}$reset}" )
    fi
}

#
# Main code
#

# Check if there is proper input
if [ "$#" -lt 1 ]; then
    echo "Usage: n-list element_1 ..."
    return 1
fi

REPLY="-1"
typeset -ga reply
reply=()

integer term_height="$LINES"
integer term_width="$COLUMNS"
if [[ "$term_height" -lt 1 || "$term_width" -lt 1 ]]; then
    local stty_out=$( stty size )
    term_height="${stty_out% *}"
    term_width="${stty_out#* }"
fi
integer inner_height=term_height-3
integer inner_width=term_width-3
integer page_height=inner_height
integer page_width=inner_width

typeset -a list disp_list
integer last_element=$#
local action
local final_key
integer selection
integer last_element_difference=0
integer current_difference=0
local prev_search_buffer=""
integer prev_uniq_mode=0
integer prev_start_idx=-1
local MBEGIN MEND MATCH mbegin mend match

# Ability to remember the list between calls
if [[ -z "$NLIST_REMEMBER_STATE" || "$NLIST_REMEMBER_STATE" -eq 0 || "$NLIST_REMEMBER_STATE" -eq 2 ]]; then
    NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN=1
    NLIST_CURRENT_IDX=1
    NLIST_IS_SEARCH_MODE=0
    NLIST_SEARCH_BUFFER=""
    NLIST_TEXT_OFFSET=0
    NLIST_IS_UNIQ_MODE=0

    # Zero - because it isn't known, unless we
    # confirm that first element is selectable
    NLIST_USER_CURRENT_IDX=0
    [[ ${NLIST_NONSELECTABLE_ELEMENTS[(r)1]} != 1 ]] && NLIST_USER_CURRENT_IDX=1
    NLIST_USER_LAST_ELEMENT=$(( last_element - $#NLIST_NONSELECTABLE_ELEMENTS ))

    # 2 is init once, then remember
    [ "$NLIST_REMEMBER_STATE" -eq 2 ] && NLIST_REMEMBER_STATE=1
fi

if [ "$NLIST_START_IN_SEARCH_MODE" -eq 1 ]; then
    NLIST_START_IN_SEARCH_MODE=0
    NLIST_IS_SEARCH_MODE=1
fi

if [ -n "$NLIST_SET_SEARCH_TO" ]; then
    NLIST_SEARCH_BUFFER="$NLIST_SET_SEARCH_TO"
    NLIST_SET_SEARCH_TO=""
fi

if [ "$NLIST_START_IN_UNIQ_MODE" -eq 1 ]; then
    NLIST_START_IN_UNIQ_MODE=0
    NLIST_IS_UNIQ_MODE=1
fi

_nlist_alternate_screen 1
zcurses init
zcurses delwin main 2>/dev/null
zcurses delwin inner 2>/dev/null
zcurses addwin main "$term_height" "$term_width" 0 0
zcurses addwin inner "$inner_height" "$inner_width" 1 2
zcurses bg main white/black
zcurses bg inner white/black
if [ "$NLIST_IS_SEARCH_MODE" -ne 1 ]; then
    _nlist_cursor_visibility 0
fi

#
# Listening for input
#

local key keypad

# Clear input buffer
zcurses timeout main 0
zcurses input main key keypad
zcurses timeout main -1
key=""
keypad=""

# This loop makes script faster on some Zsh's (e.g. 5.0.8)
repeat 1; do
    list=( "$@" )
done

last_element="$#list"

while (( 1 )); do
    # Do searching (filtering with string)
    if [ -n "$NLIST_SEARCH_BUFFER" ]; then
        # Compute new list?
        if [[ "$NLIST_SEARCH_BUFFER" != "$prev_search_buffer" || "$NLIST_IS_UNIQ_MODE" -ne "$prev_uniq_mode" ]]; then
            prev_search_buffer="$NLIST_SEARCH_BUFFER"
            prev_uniq_mode="$NLIST_IS_UNIQ_MODE"
            # regenerating list -> regenerating disp_list
            prev_start_idx=-1

            # Take all elements, including duplicates and non-selectables
            typeset +U list
            repeat 1; do
                list=( "$@" )
            done

            # Remove non-selectable elements
            [ "$#NLIST_NONSELECTABLE_ELEMENTS" -gt 0 ] && for i in "${(nO)NLIST_NONSELECTABLE_ELEMENTS[@]}"; do
                list[$i]=()
            done

            # Remove duplicates
            [ "$NLIST_IS_UNIQ_MODE" -eq 1 ] && typeset -U list

            last_element="$#list"

            # Next do the filtering
            local search_buffer="${NLIST_SEARCH_BUFFER%% ##}"
            search_buffer="${search_buffer## ##}"
            search_buffer="${search_buffer//(#m)[][*?|#~^()><\\]/\\$MATCH}"
            local search_pattern=""
            local colsearch_pattern=""
            if [ -n "$search_buffer" ]; then
                # Patterns will be *foo*~^*bar* and (foo|bar)
                search_pattern="${search_buffer// ##/*~^*}"
                colsearch_pattern="${search_buffer// ##/|}"

                # The repeat will make the matching work on a fresh heap
                repeat 1; do
                    list=( "${(@M)list:#(#i)*$~search_pattern*}" )
                done

                last_element="$#list"
            fi

            # Called after processing list
            _nlist_verify_vars
        fi

        _nlist_setup_user_vars 1

        integer end_idx=$(( NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN + page_height - 1 ))
        [ "$end_idx" -gt "$last_element" ] && end_idx=last_element

        if [ "$prev_start_idx" -ne "$NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN" ]; then
            prev_start_idx="$NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN"
            disp_list=( "${(@)list[NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN, end_idx]}" )

            if [ -n "$colsearch_pattern" ]; then
                local red=$'\x1b[00;31m' reset=$'\x1b[00;00m'
                # The repeat will make the matching work on a fresh heap
                repeat 1; do
                    disp_list=( "${(@)disp_list//(#mi)($~colsearch_pattern)/$red${MATCH}$reset}" )
                done
            fi

            # We have display list, lets replace newlines with "\n" when needed (1/2)
            [ "$NLIST_REPLACE_NEWLINES" -eq 1 ] && disp_list=( "${(@)disp_list//$'\n'/\\n}" )
        fi

        # Output colored list
        n-list-draw "$(( (NLIST_CURRENT_IDX-1) % page_height + 1 ))" \
            "$page_height" "$page_width" 0 0 "$NLIST_TEXT_OFFSET" inner \
            "$disp_list[@]"
    else
        # There is no search, but there was in previous loop
        # OR
        # Uniq mode was entered or left out
        # -> compute new list
        if [[ -n "$prev_search_buffer" || "$NLIST_IS_UNIQ_MODE" -ne "$prev_uniq_mode" ]]; then
            prev_search_buffer=""
            prev_uniq_mode="$NLIST_IS_UNIQ_MODE"
            # regenerating list -> regenerating disp_list
            prev_start_idx=-1

            # Take all elements, including duplicates and non-selectables
            typeset +U list
            repeat 1; do
                list=( "$@" )
            done

            # Remove non-selectable elements only when in uniq mode
            [ "$NLIST_IS_UNIQ_MODE" -eq 1 ] && [ "$#NLIST_NONSELECTABLE_ELEMENTS" -gt 0 ] &&
            for i in "${(nO)NLIST_NONSELECTABLE_ELEMENTS[@]}"; do
                list[$i]=()
            done

            # Remove duplicates when in uniq mode
            [ "$NLIST_IS_UNIQ_MODE" -eq 1 ] && typeset -U list

            last_element="$#list"
            # Called after processing list
            _nlist_verify_vars
        fi

        # "1" - shouldn't bother with non-selectables
        _nlist_setup_user_vars "$NLIST_IS_UNIQ_MODE"

        integer end_idx=$(( NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN + page_height - 1 ))
        [ "$end_idx" -gt "$last_element" ] && end_idx=last_element

        if [ "$prev_start_idx" -ne "$NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN" ]; then
            prev_start_idx="$NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN"
            disp_list=( "${(@)list[NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN, end_idx]}" )

            [ -n "$NLIST_COLORING_PATTERN" ] && _nlist_colorify_disp_list

            # We have display list, lets replace newlines with "\n" when needed (2/2)
            [ "$NLIST_REPLACE_NEWLINES" -eq 1 ] && disp_list=( "${(@)disp_list//$'\n'/\\n}" )
        fi

        # Output the list
        n-list-draw "$(( (NLIST_CURRENT_IDX-1) % page_height + 1 ))" \
            "$page_height" "$page_width" 0 0 "$NLIST_TEXT_OFFSET" inner \
            "$disp_list[@]"
    fi

    local status_msg_strlen
    if [ "$NLIST_IS_SEARCH_MODE" = "1" ]; then
        local _txt2=""
        [ "$NLIST_IS_UNIQ_MODE" -eq 1 ] && _txt2="[-UNIQ-] "
        _nlist_status_msg "${_txt2}Filtering with: ${NLIST_SEARCH_BUFFER// /+}"
    elif [[ ${NLIST_NONSELECTABLE_ELEMENTS[(r)$NLIST_CURRENT_IDX]} != $NLIST_CURRENT_IDX ||
            -n "$NLIST_SEARCH_BUFFER" || "$NLIST_IS_UNIQ_MODE" -eq 1 ]]; then
        local _txt="" _txt2=""
        [ -n "$NLIST_GREP_STRING" ] && _txt=" [$NLIST_GREP_STRING]"
        [ "$NLIST_IS_UNIQ_MODE" -eq 1 ] && _txt2="[-UNIQ-] "
        _nlist_status_msg "${_txt2}Current #$NLIST_USER_CURRENT_IDX (of #$NLIST_USER_LAST_ELEMENT entries)$_txt"
    else
        _nlist_status_msg ""
    fi

    zcurses border main

    local top_msg="${(C)ZSH_NAME} $ZSH_VERSION, shell level $SHLVL, $USER"
    zcurses move main 0 $(( term_width / 2 - $#top_msg / 2 ))
    zcurses string main $top_msg

    zcurses refresh main inner
    zcurses move main $(( term_height - 1 - 1 )) $(( status_msg_strlen + 2 ))

    # Wait for input
    zcurses input main key keypad

    # Get the special (i.e. "keypad") key or regular key
    if [ -n "$key" ]; then
        final_key="$key"
    elif [ -n "$keypad" ]; then
        final_key="$keypad"
    else
        _nlist_status_msg "Inproper input detected"
        zcurses refresh main inner
    fi

    n-list-input "$NLIST_CURRENT_IDX" "$NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN" \
                    "$page_height" "$page_width" "$last_element" "$NLIST_TEXT_OFFSET" \
                    "$final_key" "$NLIST_IS_SEARCH_MODE" "$NLIST_SEARCH_BUFFER" \
                    "$NLIST_IS_UNIQ_MODE"

    selection="$reply[1]"
    action="$reply[2]"
    NLIST_CURRENT_IDX="$reply[3]"
    NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN="$reply[4]"
    NLIST_TEXT_OFFSET="$reply[5]"
    NLIST_IS_SEARCH_MODE="$reply[6]"
    NLIST_SEARCH_BUFFER="$reply[7]"
    NLIST_IS_UNIQ_MODE="$reply[8]"

    if [ "$action" = "SELECT" ]; then
        REPLY="$selection"
        reply=( "$list[@]" )
        break
    elif [ "$action" = "QUIT" ]; then
        REPLY=-1
        reply=( "$list[@]" )
        break
    elif [ "$action" = "REDRAW" ]; then
        zcurses clear main redraw
        zcurses clear inner redraw
    fi
done

# vim: set filetype=zsh:

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Sebastian Gniazdowski]

On 11 February 2016 at 11:43, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> That said, zsyh could be somewhat optimized if the ":howmany" syntax
> would be utilized.

Could came up with this:
./parse.zsh "zplugin.zsh" > out1.txt 128,78s user 8,61s system 99% cpu
2:17,85 total
./parse.zsh "zplugin.zsh" > out2.txt 74,34s user 8,38s system 99% cpu
1:23,12 total

https://github.com/zsh-users/zsh-syntax-highlighting/pull/272

--- a/highlighters/main/main-highlighter.zsh
+++ b/highlighters/main/main-highlighter.zsh
@@ -125,6 +125,7 @@ _zsh_highlight_main_highlighter()
   typeset -a ZSH_HIGHLIGHT_TOKENS_CONTROL_FLOW
   local -a options_to_set # used in callees
   local buf="$PREBUFFER$BUFFER"
+  integer len="${#buf}"
   region_highlight=()

   if (( path_dirs_was_set )); then
@@ -234,11 +235,13 @@ _zsh_highlight_main_highlighter()
       # indistinguishable from 'echo foo echo bar' (one command with three
       # words for arguments).
       local needle=$'[;\n]'
-      integer offset=${${buf[start_pos+1,-1]}[(i)$needle]}
+      # Len-start_pos drops one character, but it should do it, as start_pos
+      # starts from next, not from "start_pos", character
+      integer offset=${${buf: start_pos: len-start_pos}[(i)$needle]}
       (( start_pos += offset - 1 ))
       (( end_pos = start_pos + $#arg ))
     else
-      ((start_pos+=${#buf[$start_pos+1,-1]}-${#${buf[$start_pos+1,-1]##([[:space:]]|\\[[:space:]])#}}))
+      ((start_pos+=(len-start_pos)-${#${${buf: start_pos:
len-start_pos}##([[:space:]]|\\[[:space:]])#}}))
       ((end_pos=$start_pos+${#arg}))

Best regards,
Sebastian Gniazdowski

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Sebastian Gniazdowski]

Just realized that I was doing:

    region_highlight+=("$start $end $sort | ${buf[start+1,end]}")

instead of:

    region_highlight+=("$start $end $sort")

in the parse.zsh. With this corrected the result is closer to 50%:

./parse.zsh "zplugin.zsh" > out_norm.txt 106,17s user 7,84s system 98%
cpu 1:55,85 total

./parse.zsh "zplugin.zsh" > out_opt.txt 51,91s user 7,44s system 99%
cpu 59,912 total

Best regards,
Sebastian Gniazdowski

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Bart Schaefer]

On Feb 11, 11:43am, Sebastian Gniazdowski wrote:
}
} The point of this tangent example is: every indexing works by
} iterating over buffer and counting characters.

That's only true for indexing from the end of the string (negative indices).
Normal indexing should be direct and very fast.

} By using ":5000" one pass of finding where an index points to is
} skipped, as it says "5000 characters from now on". Index -1 iterates
} from the beginning again to find end of string.

True, but again only relevant if indexing right to left at the end of
a subscript range.

} The only solution is apparently making Zsh storing strings as real
} arrays, of wint_t type.

I'm not sure what kind of arrays you think are "real" as compared to
what zsh uses now, but vast tracts of the shell would have to be
entirely rewritten if we were to change the C string paradigm.

} As for the (i), as far as I remember from the time I worked on C
} source, reverse indexing uses one additional "iterate counting
} characters" block:

IIRC this is unavoidable given that we're counting characters vs.
bytes.  Perhaps that's what you mean by "only solution is wint_t",
but I think that would be discarding a lot of other optimizations.

-- 
Barton E. Schaefer

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Bart Schaefer]

On Feb 11,  5:11pm, Sebastian Gniazdowski wrote:
} Subject: Re: Slow highlighting (Re: "drop-in replacement" and transpose-wo
}
} -      integer offset=${${buf[start_pos+1,-1]}[(i)$needle]}
} +      # Len-start_pos drops one character, but it should do it, as start_pos
} +      # starts from next, not from "start_pos", character
} +      integer offset=${${buf: start_pos: len-start_pos}[(i)$needle]}

Hrm.  This goes back to the discussion from several months ago about
$string[left,right] using the same generic code as $array[left,right],
if I'm recalling that thread correctly.  The ${string:left:len} form
on the other hand uses a scalar-specific approach.

Avoiding github link bitrot Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Daniel Shahaf]

Sebastian Gniazdowski wrote on Thu, Feb 11, 2016 at 11:43:50 +0100:
> https://github.com/zsh-users/zsh/blob/master/Src/params.c#L1360-L1396

Links that name a ref (such as "master") rather than a hash will
eventually point to different contents than they do now.  You can avoid
this by typing 'y' in the browser (which pegs the link at the hash
'master' currently points to) before copying the address.

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Daniel Shahaf]

Sebastian Gniazdowski wrote on Thu, Feb 11, 2016 at 13:07:14 +0100:
> I attach the zsyh parser extracted, it's probably easier to profile.
> Output is range, style name and the body of the range. Few worries:
> 
> 10937 10958 double-quoted-argument | "$NLIST_IS_UNIQ_MODE"
> 10938 10957 dollar-double-quoted-argument | $NLIST_IS_UNIQ_MODE
> 
> 930 945 single-hyphen-option | -(#c0,1)[0-9]##

Can you explain what's worrying about these?

The only issue I see is that single-hyphen-option should perhaps not
have been selected in this case.

Thanks,

Daniel

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Daniel Shahaf]

Sebastian Gniazdowski wrote on Thu, Feb 11, 2016 at 17:11:57 +0100:
> On 11 February 2016 at 11:43, Sebastian Gniazdowski
> <sgniazdowski@gmail.com> wrote:
> > That said, zsyh could be somewhat optimized if the ":howmany" syntax
> > would be utilized.
> 
> Could came up with this:
> ./parse.zsh "zplugin.zsh" > out1.txt 128,78s user 8,61s system 99% cpu
> 2:17,85 total
> ./parse.zsh "zplugin.zsh" > out2.txt 74,34s user 8,38s system 99% cpu
> 1:23,12 total
> 
> https://github.com/zsh-users/zsh-syntax-highlighting/pull/272
> 

Thanks!  Merged, will be in 0.5.0.

However, I think avoiding the strlen() isn't the main benefit here:
I suspect the expression $buf[start_pos+1,-1], which is evaluated in
every iteration of the loop, might have triggered an O(N²) allocations
pattern, even before considering the implicit strlen for the -1 (or the
explicit strlen in the "else" branch).

The next step might be to see whether the two remaining instances of
that expression (one in the 'if' branch and one in the 'else' branch)
can also be simplified: for example, by precomputing the indices of all
newlines, semicolons, and starts-of-words (places where the regexp /\</
matches) in $buf.

Many thanks for your investigation!  I'd also like to thank Bart for his
help (some of which was offlist).

Daniel

Re: Slow highlighting (Re: "drop-in replacement" and transpose-words-match)

[Sebastian Gniazdowski]

On 12 February 2016 at 10:41, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Feb 11, 11:43am, Sebastian Gniazdowski wrote:
> }
> } The point of this tangent example is: every indexing works by
> } iterating over buffer and counting characters.
>
> That's only true for indexing from the end of the string (negative indices).
> Normal indexing should be direct and very fast.

I was on this some time ago. I remember, and now it also looks like
being so, that following block of code does the iteration for indices
>, =, < 0:

https://github.com/zsh-users/zsh/blob/master/Src/params.c#L1348-L1394

It comes down to following loop for indices > 0:

for (t = s; nchars && *t; nchars--) t += (lastcharlen = MB_METACHARLEN(t));

It counts characters (nchars--), skips bytes (lastcharlen).

> } The only solution is apparently making Zsh storing strings as real
> } arrays, of wint_t type.
>
> I'm not sure what kind of arrays you think are "real" as compared to
> what zsh uses now, but vast tracts of the shell would have to be
> entirely rewritten if we were to change the C string paradigm.

In C one can optimize the paradigm. Carefully skip multibyte
characters once, then keep and pass pointer. That optimization comes
to my mind. In Zsh there are no pointers. One keeps index but it
causes full reiteration each time it's used (if that's confirmed).

However maybe there could be added something similar? To get special
pointer: byte offset – via some flag. Then, to index with byte
pointer, when connected with some flag.

byteoffset=$long_string[(x)2500]
character=$long_string[(X)$byteoffset]

Best regards,
Sebastian Gniazdowski