New widget "transpose-segments"

New widget "transpose-segments"

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 364 bytes --]

Hello
I've attached a new widget "transpose-segments". Works like
"transpose-words", but the "words" are what I call "segments" – parts
of a command obtained by (z) flag. This allows to comfortably swap
arbitrarily quoted things at prompt,

http://asciinema.org/a/5tkda9lr7dju4lh3vt5sj35wi

Can this be commited?

Best regards,
Sebastian Gniazdowski

[-- Attachment #2: transpose-segments.diff --]
[-- Type: text/plain, Size: 2481 bytes --]

diff --git a/Functions/Zle/transpose-segments b/Functions/Zle/transpose-segments
new file mode 100644
index 0000000..313febf
--- /dev/null
+++ b/Functions/Zle/transpose-segments
@@ -0,0 +1,82 @@
+# Transpose segments, i.e. parts of line obtained by (z) flag, i.e.
+# as if zsh parsed the line.
+#
+# Code to activate the functionality with Alt-t:
+# autoload transpose-segments
+# zle -N transpose-segments
+# bindkey "^[t" transpose-segments
+
+local curcontext=":zle:$WIDGET" skip
+local MATCH MBEGIN MEND
+
+# Will remember white spaces before each segment
+typeset -a spaces
+spaces=()
+
+# Working variable for $BUFFER
+local buf="$BUFFER"
+
+# Split the buffer
+typeset -a bufarr
+bufarr=( "${(z)buf}" )
+
+# (z) handles spaces nicely, but we need them for the user
+integer size="$#bufarr"
+integer char_count=0
+integer selected_segment=0
+
+for (( i=1; i<=size; i++ )); do
+    local segment="$bufarr[i]"
+
+    # In general, $buf can start with white spaces
+    # We will not search for them, but instead
+    # search for any char that's current segment's
+    # leading character. So this is an approach that
+    # doesn't trust [::space::]
+
+    local leadchar="$segment[1]"
+    buf="${buf##(#m)[^$leadchar]#}"
+
+    # Remember the spaces
+    spaces[$i]="$MATCH"
+
+    # Count all characters being processed, remove the spaces, then remove segment
+    char_count=char_count+"$#MATCH"+"$#segment"
+    buf="${buf#$MATCH}"
+    MATCH=""
+    buf="${buf#(#m)$segment}"
+
+    # If segment not found, return from the function doing nothing
+    # This of course shoudln't happen
+    [ -z "$MATCH" ] && return 0
+
+    # Detect which segment is active
+    [[ "$selected_segment" -eq 0 && "$char_count" -ge "$CURSOR" ]] && selected_segment=i
+done
+
+
+# What's left in $buf can be only white spaces
+spaces[i]="$buf"
+char_count=char_count+"$#buf"
+
+[[ "$selected_segment" -eq 0 && "$char_count" -ge "$CURSOR" ]] && selected_segment=i-1
+# No active segment found, or it's a first segment? Return. (this shouldn't happen)
+[[ "$selected_segment" -eq "0" || "$selected_segment" -eq "1" ]] && return 0
+
+# Swap segments
+local tmp="$bufarr[selected_segment]"
+bufarr[selected_segment]="$bufarr[selected_segment-1]"
+bufarr[selected_segment-1]="$tmp"
+
+# Build BUFFER
+integer curbkp="$CURSOR"
+BUFFER=""
+for (( i=1; i<=size; i++ )); do
+    BUFFER+="$spaces[i]$bufarr[i]"
+done
+CURSOR="$curbkp"
+
+# Append final white spaces
+BUFFER+="$spaces[i]"
+
+return 0

[-- Attachment #3: transpose-segments --]
[-- Type: application/octet-stream, Size: 2204 bytes --]

# Transpose segments, i.e. parts of line obtained by (z) flag, i.e.
# as if zsh parsed the line.
#
# Code to activate the functionality with Alt-t:
# autoload transpose-segments
# zle -N transpose-segments
# bindkey "^[t" transpose-segments

local curcontext=":zle:$WIDGET" skip
local MATCH MBEGIN MEND

# Will remember white spaces before each segment
typeset -a spaces
spaces=()

# Working variable for $BUFFER
local buf="$BUFFER"

# Split the buffer
typeset -a bufarr
bufarr=( "${(z)buf}" )

# (z) handles spaces nicely, but we need them for the user
integer size="$#bufarr"
integer char_count=0
integer selected_segment=0

for (( i=1; i<=size; i++ )); do
    local segment="$bufarr[i]"

    # In general, $buf can start with white spaces
    # We will not search for them, but instead
    # search for any char that's current segment's
    # leading character. So this is an approach that
    # doesn't trust [::space::]

    local leadchar="$segment[1]"
    buf="${buf##(#m)[^$leadchar]#}"

    # Remember the spaces
    spaces[$i]="$MATCH"

    # Count all characters being processed, remove the spaces, then remove segment
    char_count=char_count+"$#MATCH"+"$#segment"
    buf="${buf#$MATCH}"
    MATCH=""
    buf="${buf#(#m)$segment}"

    # If segment not found, return from the function doing nothing
    # This of course shoudln't happen
    [ -z "$MATCH" ] && return 0

    # Detect which segment is active
    [[ "$selected_segment" -eq 0 && "$char_count" -ge "$CURSOR" ]] && selected_segment=i
done


# What's left in $buf can be only white spaces
spaces[i]="$buf"
char_count=char_count+"$#buf"

[[ "$selected_segment" -eq 0 && "$char_count" -ge "$CURSOR" ]] && selected_segment=i-1
# No active segment found, or it's a first segment? Return. (this shouldn't happen)
[[ "$selected_segment" -eq "0" || "$selected_segment" -eq "1" ]] && return 0

# Swap segments
local tmp="$bufarr[selected_segment]"
bufarr[selected_segment]="$bufarr[selected_segment-1]"
bufarr[selected_segment-1]="$tmp"

# Build BUFFER
integer curbkp="$CURSOR"
BUFFER=""
for (( i=1; i<=size; i++ )); do
    BUFFER+="$spaces[i]$bufarr[i]"
done
CURSOR="$curbkp"

# Append final white spaces
BUFFER+="$spaces[i]"

return 0

Re: New widget "transpose-segments"

[Jérémie Roquet]

Hi,

2016-01-09 12:03 GMT+01:00 Sebastian Gniazdowski <sgniazdowski@gmail.com>:
> I've attached a new widget "transpose-segments". Works like
> "transpose-words", but the "words" are what I call "segments" – parts
> of a command obtained by (z) flag. This allows to comfortably swap
> arbitrarily quoted things at prompt,
>
> http://asciinema.org/a/5tkda9lr7dju4lh3vt5sj35wi

This looks awesome. I'll probably use it much, much more often than
transpose-words. Thanks!

This made me wonder if there was something similar for Emacs. It
appears that it's builtin and called transpose-sexps.

Best regards,

-- 
Jérémie

Re: New widget "transpose-segments"

[Sebastian Gniazdowski]

I've updated the code to support swapping segments across new lines:

https://asciinema.org/a/1rg6zqgzsduenjnuuc1dvhz50

Will this be commited?

Best regards,
Sebastian Gniazdowski


On 9 January 2016 at 12:03, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> Hello
> I've attached a new widget "transpose-segments". Works like
> "transpose-words", but the "words" are what I call "segments" – parts
> of a command obtained by (z) flag. This allows to comfortably swap
> arbitrarily quoted things at prompt,
>
> http://asciinema.org/a/5tkda9lr7dju4lh3vt5sj35wi
>
> Can this be commited?
>
> Best regards,
> Sebastian Gniazdowski

Re: New widget "transpose-segments"

[Sebastian Gniazdowski]

On 9 January 2016 at 13:19, Jérémie Roquet <jroquet@arkanosis.net> wrote:
> Hi,
>
> 2016-01-09 12:03 GMT+01:00 Sebastian Gniazdowski <sgniazdowski@gmail.com>:
>> I've attached a new widget "transpose-segments". Works like
>> "transpose-words", but the "words" are what I call "segments" – parts
>> of a command obtained by (z) flag. This allows to comfortably swap
>> arbitrarily quoted things at prompt,
>>
>> http://asciinema.org/a/5tkda9lr7dju4lh3vt5sj35wi
>
> This looks awesome. I'll probably use it much, much more often than
> transpose-words. Thanks!

No problem! I'm happy you like it. I've created a new version that
supports new lines:

https://asciinema.org/a/1rg6zqgzsduenjnuuc1dvhz50

Best regards,
Sebastian Gniazdowski

Re: New widget "transpose-segments"

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 66 bytes --]

The forgotten patch and file

Best regards,
Sebastian Gniazdowski

[-- Attachment #2: transpose-segments.diff --]
[-- Type: text/plain, Size: 4491 bytes --]

diff --git a/Functions/Zle/transpose-segments b/Functions/Zle/transpose-segments
new file mode 100644
index 0000000..796ac18
--- /dev/null
+++ b/Functions/Zle/transpose-segments
@@ -0,0 +1,134 @@
+# Transpose segments, i.e. parts of lines obtained by (z) flag, i.e.
+# as if zsh parsed the line.
+#
+# Code to activate the functionality with binding to Alt-t:
+# autoload transpose-segments
+# zle -N transpose-segments
+# bindkey "^[t" transpose-segments
+
+local curcontext=":zle:$WIDGET"
+local MATCH MBEGIN MEND i
+
+# Will remember white spaces before each segment
+typeset -a spaces
+spaces=()
+
+# Working variable for $BUFFER
+local buf="$BUFFER"
+
+# Split the buffer
+typeset -a bufarr
+bufarr=( "${(z)buf}" )
+
+# Output array, needed to skip (z)-generated semicolons
+typeset -a outarr
+outarr=()
+
+integer size="$#bufarr"
+integer char_count=0
+integer selected_segment=0
+local newbuf newspaces
+
+# (z) handles spaces nicely, but we need them for the user
+for (( i=1; i<=size; i++ )); do
+    local segment="$bufarr[i]"
+
+    # In general, $buf can start with white spaces
+    # We will not search for them, but instead segment's
+    # leading character, negated. This is an ambition
+    # to completely avoid character classes
+
+    newbuf="${buf##(#m)[^$segment[1]]#}"
+    newspaces="$MATCH"
+
+    # Is this a last segment?
+    if [ "$i" -lt "$size" ]; then
+        # If there was (z)-added ";" then $newspaces will contain char $nextsegment[1]
+        # because the "${buf##..}" above will move onward in $buf
+        # Also, correct detection of spaces implies that $newbuf[1] = $segment[1]
+        local nextsegment="$bufarr[i+1]"
+        if [[ ("${newspaces/$nextsegment[1]/}" = "$newspaces")
+                && "$newbuf[1]" = "$segment[1]" ]]; then
+            # Normal situation, white space before $segment
+            spaces[i]="$newspaces"
+            outarr[i]="$segment"
+            buf="$newbuf"
+            MATCH=""
+            buf="${buf#(#m)$segment}"
+
+            # If segment not found, return from the function doing nothing
+            # This shoudln't happen
+            [ -z "$MATCH" ] && return 0;
+
+            char_count=char_count+"$#newspaces"+"$#segment"
+        else
+            # Special situation: we searched for [^;], but ";" was added
+            # by (z) flag, and we ended matching not only white spaces
+            # but also something from the next segment
+            # Or, if (z)-added ";" was one of last ones, then $segment[1]
+            # != $newbuf[1]
+            #
+            # This not-actual segment should be ignored
+            spaces[i]=""
+            outarr[i]=""
+        fi
+    else
+        # Last segment
+        if [[ "$newbuf[1]" = "$segment[1]" ]]; then
+            spaces[i]="$newspaces"
+            outarr[i]="$segment"
+
+            buf="$newbuf"
+            MATCH=""
+            buf="${buf#(#m)$segment}"
+
+            # If segment not found, return from the function doing nothing
+            # This shoudln't happen
+            [ -z "$MATCH" ] && return 0
+
+            char_count=char_count+"$#segment"+"$#newspaces"
+        else
+            # This handles multiple (z)-added ";" at the end (i.e. newlines)
+            outarr[i]=""
+            spaces[i]="$newspaces"
+        fi
+    fi
+
+    # Detect which segment is active
+    [[ "$selected_segment" -eq 0 && "$char_count" -ge "$CURSOR" ]] && selected_segment=i
+done
+
+# What's left in $buf can be only white spaces
+spaces[i]="$buf"
+char_count=char_count+"$#buf"
+
+# Detect which segment is active - this takes into account possible trailing white spaces
+[[ "$selected_segment" -eq 0 && "$char_count" -ge "$CURSOR" ]] && selected_segment=i-1
+
+# No active segment found (shouldn't happen)? Return
+[ "$selected_segment" -eq "0" ] && return 0
+
+# First find actual previous segment
+integer prev_segment=0
+for (( i=selected_segment-1; i>0; i-- )) [ -n "$outarr[i]" ] && { prev_segment=i; break }
+# No actual previous segment? Return doing nothing
+[ "$prev_segment" = "0" ] && return 0
+
+# Swap segments
+local tmp="$outarr[selected_segment]"
+outarr[selected_segment]="$outarr[prev_segment]"
+outarr[prev_segment]="$tmp"
+
+# Build BUFFER
+integer newcursor
+BUFFER=""
+for (( i=1; i<=size; i++ )); do
+    BUFFER+="$spaces[i]$outarr[i]"
+    [ "$i" = "$selected_segment" ] && newcursor="$#BUFFER"
+done
+CURSOR="$newcursor"
+
+# Append final white spaces
+BUFFER+="$spaces[i]"
+
+return 0

[-- Attachment #3: transpose-segments --]
[-- Type: application/octet-stream, Size: 4161 bytes --]

# Transpose segments, i.e. parts of lines obtained by (z) flag, i.e.
# as if zsh parsed the line.
#
# Code to activate the functionality with binding to Alt-t:
# autoload transpose-segments
# zle -N transpose-segments
# bindkey "^[t" transpose-segments

local curcontext=":zle:$WIDGET"
local MATCH MBEGIN MEND i

# Will remember white spaces before each segment
typeset -a spaces
spaces=()

# Working variable for $BUFFER
local buf="$BUFFER"

# Split the buffer
typeset -a bufarr
bufarr=( "${(z)buf}" )

# Output array, needed to skip (z)-generated semicolons
typeset -a outarr
outarr=()

integer size="$#bufarr"
integer char_count=0
integer selected_segment=0
local newbuf newspaces

# (z) handles spaces nicely, but we need them for the user
for (( i=1; i<=size; i++ )); do
    local segment="$bufarr[i]"

    # In general, $buf can start with white spaces
    # We will not search for them, but instead segment's
    # leading character, negated. This is an ambition
    # to completely avoid character classes

    newbuf="${buf##(#m)[^$segment[1]]#}"
    newspaces="$MATCH"

    # Is this a last segment?
    if [ "$i" -lt "$size" ]; then
        # If there was (z)-added ";" then $newspaces will contain char $nextsegment[1]
        # because the "${buf##..}" above will move onward in $buf
        # Also, correct detection of spaces implies that $newbuf[1] = $segment[1]
        local nextsegment="$bufarr[i+1]"
        if [[ ("${newspaces/$nextsegment[1]/}" = "$newspaces")
                && "$newbuf[1]" = "$segment[1]" ]]; then
            # Normal situation, white space before $segment
            spaces[i]="$newspaces"
            outarr[i]="$segment"
            buf="$newbuf"
            MATCH=""
            buf="${buf#(#m)$segment}"

            # If segment not found, return from the function doing nothing
            # This shoudln't happen
            [ -z "$MATCH" ] && return 0;

            char_count=char_count+"$#newspaces"+"$#segment"
        else
            # Special situation: we searched for [^;], but ";" was added
            # by (z) flag, and we ended matching not only white spaces
            # but also something from the next segment
            # Or, if (z)-added ";" was one of last ones, then $segment[1]
            # != $newbuf[1]
            #
            # This not-actual segment should be ignored
            spaces[i]=""
            outarr[i]=""
        fi
    else
        # Last segment
        if [[ "$newbuf[1]" = "$segment[1]" ]]; then
            spaces[i]="$newspaces"
            outarr[i]="$segment"

            buf="$newbuf"
            MATCH=""
            buf="${buf#(#m)$segment}"

            # If segment not found, return from the function doing nothing
            # This shoudln't happen
            [ -z "$MATCH" ] && return 0

            char_count=char_count+"$#segment"+"$#newspaces"
        else
            # This handles multiple (z)-added ";" at the end (i.e. newlines)
            outarr[i]=""
            spaces[i]="$newspaces"
        fi
    fi

    # Detect which segment is active
    [[ "$selected_segment" -eq 0 && "$char_count" -ge "$CURSOR" ]] && selected_segment=i
done

# What's left in $buf can be only white spaces
spaces[i]="$buf"
char_count=char_count+"$#buf"

# Detect which segment is active - this takes into account possible trailing white spaces
[[ "$selected_segment" -eq 0 && "$char_count" -ge "$CURSOR" ]] && selected_segment=i-1

# No active segment found (shouldn't happen)? Return
[ "$selected_segment" -eq "0" ] && return 0

# First find actual previous segment
integer prev_segment=0
for (( i=selected_segment-1; i>0; i-- )) [ -n "$outarr[i]" ] && { prev_segment=i; break }
# No actual previous segment? Return doing nothing
[ "$prev_segment" = "0" ] && return 0

# Swap segments
local tmp="$outarr[selected_segment]"
outarr[selected_segment]="$outarr[prev_segment]"
outarr[prev_segment]="$tmp"

# Build BUFFER
integer newcursor
BUFFER=""
for (( i=1; i<=size; i++ )); do
    BUFFER+="$spaces[i]$outarr[i]"
    [ "$i" = "$selected_segment" ] && newcursor="$#BUFFER"
done
CURSOR="$newcursor"

# Append final white spaces
BUFFER+="$spaces[i]"

return 0

Re: New widget "transpose-segments"

[Bart Schaefer]

On Jan 9, 12:03pm, Sebastian Gniazdowski wrote:
} Subject: New widget "transpose-segments"
} 
} I've attached a new widget "transpose-segments". Works like
} "transpose-words", but the "words" are what I call "segments"

How does this differ from

    autoload select-word-style
    select-word-style shell
    zle -N transpose-words transpose-words-match

??

transpose-words-match (Re: New widget "transpose-segments")

[Bart Schaefer]

On Jan 9,  9:22am, I asked:
}
} How does this differ from
} 
}     zle -N transpose-words transpose-words-match

One difference is that transpose-words-match uses a little more context,
so

    % echo two words<alt-t>

does not transpose because the cursor is not between two words, and

    % echo "two <alt-t>words"
    % echo words" "very 

swaps around the space even though it is in the middle of a "shell word".

(In fact it rather appears as though "-subword" matching has been turned
on implicitly even though I did not name the widget that way; there may
very well be a bug.)

Re: New widget "transpose-segments"

[Sebastian Gniazdowski]

On 9 January 2016 at 18:22, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Jan 9, 12:03pm, Sebastian Gniazdowski wrote:
> } Subject: New widget "transpose-segments"
> }
> } I've attached a new widget "transpose-segments". Works like
> } "transpose-words", but the "words" are what I call "segments"
>
> How does this differ from
>
>     autoload select-word-style
>     select-word-style shell
>     zle -N transpose-words transpose-words-match
>
> ??

I now tested:

autoload select-word-style
select-word-style shell
autoload transpose-words-match
zle -N transpose-words-match
bindkey "^[t" transpose-words-match

The main difference is working across newlines. Also, the requirement
to position cursor at beginning of word to transpose is absent. I
think those two are important. Also, setting word style is not needed.
I have to admit that my functionality isn't new and practically exists
in Zsh already, however, why not have alternative approach available,
with freedom of where to position cursor and with support for
newlines. And with freedom for word style.

BTW. I asked about transpose-words on IRC and here, and then nobody
told me about set-word-style shell. That's maybe not bad if the
alternative (z)-based approach will be recognized as valuable.

Best regards,
Sebastian Gniazdowski

Re: transpose-words-match (Re: New widget "transpose-segments")

[Sebastian Gniazdowski]

On 9 January 2016 at 18:48, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Jan 9,  9:22am, I asked:
> }
> } How does this differ from
> }
> }     zle -N transpose-words transpose-words-match
>
> One difference is that transpose-words-match uses a little more context,
> so
>
>     % echo two words<alt-t>
>
> does not transpose because the cursor is not between two words, and
>
>     % echo "two <alt-t>words"
>     % echo words" "very
>
> swaps around the space even though it is in the middle of a "shell word".

That's a nice catch, finally a de facto difference between
transpose-words-match and transpose-segments. I tried lines like:

$(( i+1 )) a="${(u)a}" 試句相當長

and they worked the same. Now it can be seen that what you described
can also occur inside $(( )).

Best regards,
Sebastian Gniazdowski

Re: transpose-words-match (Re: New widget "transpose-segments")

[Bart Schaefer]

On Jan 9,  7:03pm, Sebastian Gniazdowski wrote:
}
} That's a nice catch, finally a de facto difference between
} transpose-words-match and transpose-segments.

Yes, I'm just not sure it's an *intentional* difference.

Re: New widget "transpose-segments"

[Sebastian Gniazdowski]

One other mini-difference is a feature of t-seg – the way it detects
current segment (the one to swap with a previous one). Given a line:

"$ a b c d     "

t-seg allows to place cursor arbitrarily after "d", and it will not
loose the trailing spaces, and swap "c" and "d". t-w-match will swap
"d" and spaces after "d".

Best regards,
Sebastian Gniazdowski

Re: transpose-words-match (Re: New widget "transpose-segments")

[Sebastian Gniazdowski]

On 9 January 2016 at 19:18, Bart Schaefer <schaefer@brasslantern.com> wrote:
> Yes, I'm just not sure it's an *intentional* difference.

Bart there already is "copy-prev-shell-word". It's good to not be
bound to global select-word-style setting. My widget named
"transpose-shell-words" and your simple "backward-kill-shell-word"
(the one for zsh >= 5.0.8) together with "copy-prev-shell-word" would
constitute a nice feature set. And I'm using Daniel's
"backward-kill-shell-word" in my Zew so that all Zsh users, even
before zsh-5.0.8 can be happy

That said I tested your bksw and it doesn't fully work for following
multi-line command:

a=${(j:,:)a}
a="${(r:100000:: _:)a}"
 試句相當長
""
a\ b
c\ b

There is a space after "". When cursor is positioned after this space,
invoking bksw joins lines "" and a\ b instead of deleting "". Daniel's
code works on this. It's not the first time that (z) flags proves its
robustness, but maybe your simpler bksw can be updated to handle this
case?

Best regards,
Sebastian Gniazdowski

Re: transpose-words-match (Re: New widget "transpose-segments")

[Bart Schaefer]

On Jan 11,  2:13pm, Sebastian Gniazdowski wrote:
}
} Bart there already is "copy-prev-shell-word". It's good to not be
} bound to global select-word-style setting.

Just for the record, although there is a global select-word-style
setting for convenience, you can also select different word styles
for different contexts.  There are a few examples in the manual.

The word styles can also handle things like swapping CapsLikeThis
to CapsThisLike where the only "word break" is a case difference.

Further, copy-prev-shell-word is a built-in widget; and it appears to
have a bug when the previous shell word is a single character with no
whitespace between it and the cursor.  (The transpose-words builtin
has a similar difficulty with one-character words, I think.)

} That said I tested your bksw and it doesn't fully work for following
[...]
} There is a space after "". When cursor is positioned after this space,
} invoking bksw joins lines "" and a\ b instead of deleting "".

select-in-shell-word is intended to emulate some vim functionality that
excludes quotes (only selects what's inside them) so empty quotes might
be expected to confuse it.

I have to thank you for this example, because it allowed me to realize
why newlines confuse match-words-by-style.  It needs to be using the
(Z:n:) flag rather than the (z) flag; the latter converts newlines into
semicolons, but this wants newlines as whitespace.

I see Daniel has also done something for interactive_comments, I'm not
sure whether that's needed in match-words-by-style.  [By the way, Daniel,
(zZ:c:) is redundant, you only need (Z:c:).]

diff --git a/Functions/Zle/match-words-by-style b/Functions/Zle/match-words-by-style
index b387828..7ba6157 100644
--- a/Functions/Zle/match-words-by-style
+++ b/Functions/Zle/match-words-by-style
@@ -111,20 +111,20 @@ done
 case $wordstyle in
   (*shell*) local bufwords
 	  # This splits the line into words as the shell understands them.
-	  bufwords=(${(z)LBUFFER})
+	  bufwords=(${(Z:n:)LBUFFER})
 	  nwords=${#bufwords}
 	  wordpat1="${(q)bufwords[-1]}"
 
 	  # Take substring of RBUFFER to skip over $skip characters
 	  # from the cursor position.
-	  bufwords=(${(z)RBUFFER[1+$skip,-1]})
+	  bufwords=(${(Z:n:)RBUFFER[1+$skip,-1]})
 	  wordpat2="${(q)bufwords[1]}"
 	  spacepat='[[:space:]]#'
 
 	  # Assume the words are at the top level, i.e. if we are inside
 	  # 'something with spaces' then we need to ignore the embedded
 	  # spaces and consider the whole word.
-	  bufwords=(${(z)BUFFER})
+	  bufwords=(${(Z:n:)BUFFER})
 	  if (( ${#bufwords[$nwords]} > ${#wordpat1} )); then
 	    # Yes, we're in the middle of a shell word.
 	    # Find out what's in front.

Re: transpose-words-match (Re: New widget "transpose-segments")

[Bart Schaefer]

On Jan 11, 10:26pm, Bart Schaefer wrote:
}
} Further, copy-prev-shell-word is a built-in widget; and it appears to
} have a bug when the previous shell word is a single character with no
} whitespace between it and the cursor.  (The transpose-words builtin
} has a similar difficulty with one-character words, I think.)

Sorry, no; the builtin transpose-words does not have that problem, I
had forgotten that I ran select-words-by-style earlier in that shell,
so I wasn't getting the builtin from ESC t.

Re: New widget "transpose-segments"

[Bart Schaefer]

On Jan 9,  6:59pm, Sebastian Gniazdowski wrote:
}
} The main difference is working across newlines. Also, the requirement
} to position cursor at beginning of word to transpose is absent. I
} think those two are important.

Patches for both of these sent a little while ago.

} Also, setting word style is not needed.

"select-word-style" is a shortcut for loading the entire set of widgets
based on match-words-by-style.  The style itself defaults properly if
not set.  So this is similar to saying transpose-words-match is better
because "zle -N transpose-segments" would not be needed.

I have no real objection to including either your widget or Daniel's,
but the number of different ways to do the almost but not quote the
same thing begins to be overwhelming (e.g., I would have preferred
that Oliver not make the "vim text objects" into built-ins because
that's also duplicating parts of match-words-by-style, but it was
necesary to be able to have default key bindings).

However, I should note that I would rather that we don't introduce yet
another term ("segments") to mean "shell words".

Re: New widget "transpose-segments"

[Bart Schaefer]

On Jan 9,  8:06pm, Sebastian Gniazdowski wrote:
}
} "$ a b c d     "
} 
} t-seg allows to place cursor arbitrarily after "d", and it will not
} loose the trailing spaces, and swap "c" and "d". t-w-match will swap
} "d" and spaces after "d".

I believe this works as desired following the patch in 37570, as long
as you have not set skip-chars or *-subword styles in the context
of transpose-words.

Re: transpose-words-match (Re: New widget "transpose-segments")

[Sebastian Gniazdowski]

For me it does fix transpose words running on multi line command, with
shell word style.

Best regards,
Sebastian Gniazdowski

Re: transpose-words-match (Re: New widget "transpose-segments")

[Sebastian Gniazdowski]

On 12 January 2016 at 07:26, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Jan 11,  2:13pm, Sebastian Gniazdowski wrote:
> }
> } Bart there already is "copy-prev-shell-word". It's good to not be
> } bound to global select-word-style setting.
>
> Just for the record, although there is a global select-word-style
> setting for convenience, you can also select different word styles
> for different contexts.  There are a few examples in the manual.
>
> The word styles can also handle things like swapping CapsLikeThis
> to CapsThisLike where the only "word break" is a case difference.

Tried googling but no success. Could you give name of the man page
and what to look for?

> Further, copy-prev-shell-word is a built-in widget; and it appears to
> have a bug when the previous shell word is a single character with no
> whitespace between it and the cursor.  (The transpose-words builtin
> has a similar difficulty with one-character words, I think.)

Really a bug, it reproduces here. Yesterday I switched to
copy-earlier-word (still bind copy-prev-shell-word to ^[M). It
integrates with insert-last-word and allows to copy arbitrary word
from arbitrary previous history event. For the record:

bindkey "^[." insert-last-word
autoload -Uz copy-earlier-word
zle -N copy-earlier-word
bindkey "^[m" copy-earlier-word

It's interesting that insert-last-word operates on shell words
although this is not stated in manual.

> } That said I tested your bksw and it doesn't fully work for following
> [...]
> } There is a space after "". When cursor is positioned after this space,
> } invoking bksw joins lines "" and a\ b instead of deleting "".
>
> select-in-shell-word is intended to emulate some vim functionality that
> excludes quotes (only selects what's inside them) so empty quotes might
> be expected to confuse it.

Other lines, when ended with space, will also cause the unexpected
behavior. I should have stated this that it's about space at the end.

> I have to thank you for this example, because it allowed me to realize
> why newlines confuse match-words-by-style.  It needs to be using the
> (Z:n:) flag rather than the (z) flag; the latter converts newlines into
> semicolons, but this wants newlines as whitespace.

Cool. I tested transpose-words with shell word style and now it works
on the multi line command I gave, e.g. I can transpose 'a\ b' with
'""' when I position cursor at 'a'.

I've missed the (Z) flag, wrote much code in transpose-shell-words to
track which arguments correspond to which spaces to be able to glue
the command back, and had to handle the "\n" -> ";" semicolons which
complicated things. Seems that (Z:n:) would discard the new lines like
any other white spaces and the code would be simpler. And this works
even in zsh 4.3.17.

BTW. I discovered that following multi line command can make various
versions of zsh hang on it:

a=${(j:,:)a}
a="${(r:100000:: _:)a}"
試句相當長
""
a\ b
c\ b

Does it reproduce to you? It was fully reproducible on my setup
however first it started to work for a second - and I thought that
it's connected to $SHLVL>1, then it worked again, and then pasting
broke up in my terminal, and I thought it might be connected to zle -N
self-insert url-quote-magic. Now pasting works and again it hangs
zsh-5.2-dev-0 and 5.0.8, even when I provide it with history
operations (cursor up). However, $SHLVL has to be 2 or more.

Best regards,
Sebastian Gniazdowski

Re: transpose-words-match (Re: New widget "transpose-segments")

[Sebastian Gniazdowski]

On 12 January 2016 at 10:23, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> Cool. I tested transpose-words with shell word style and now it works
> on the multi line command I gave, e.g. I can transpose 'a\ b' with
> '""' when I position cursor at 'a'.

Sorry, I was testing transpose-words-match, and it's that widget
that's fixed by match-words-by-style.

Best regards,
Sebastian Gniazdowski

Re: transpose-words-match (Re: New widget "transpose-segments")

[Sebastian Gniazdowski]

On 12 January 2016 at 10:09, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> For me it does fix transpose words running on multi line command, with
> shell word style.

Sorry, I was testing transpose-words-match, and it's that widget
that's fixed by match-words-by-style.

Best regards,
Sebastian Gniazdowski

Re: transpose-words-match (Re: New widget "transpose-segments")

[Peter Stephenson]

On Tue, 12 Jan 2016 10:23:36 +0100
Sebastian Gniazdowski <sgniazdowski@gmail.com> wrote:
> > The word styles can also handle things like swapping CapsLikeThis
> > to CapsThisLike where the only "word break" is a case difference.
> 
> Tried googling but no success. Could you give name of the man page
> and what to look for?

Search "zshcontrib" for "subword".

pws

Re: New widget "transpose-segments"

[Sebastian Gniazdowski]

On 12 January 2016 at 09:09, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Jan 9,  6:59pm, Sebastian Gniazdowski wrote:
> }
> } The main difference is working across newlines. Also, the requirement
> } to position cursor at beginning of word to transpose is absent. I
> } think those two are important.
>
> Patches for both of these sent a little while ago.
>
> } Also, setting word style is not needed.
>
> I have no real objection to including either your widget or Daniel's,
> but the number of different ways to do the almost but not quote the
> same thing begins to be overwhelming

In my project Zew I quite track what's going on here and it should
there clarify if transpose-segments and other widgets have real reason
to exist (besides providing functionality for zsh < 5.2-dev-0)

> However, I should note that I would rather that we don't introduce yet
> another term ("segments") to mean "shell words".

Yeah I already renamed "transpose-segments" to "transpose-shell-words" in Zew.

Best regards,
Sebastian Gniazdowski

Re: transpose-words-match (Re: New widget "transpose-segments")

[Bart Schaefer]

On Jan 12, 10:23am, Sebastian Gniazdowski wrote:
}
} It's interesting that insert-last-word operates on shell words
} although this is not stated in manual.

insert-last-word operates on elements of the $historywords array, so
if you setopt histlexwords then insert-last-word uses shell words,
and if you don't have that setopt then it uses space-separated words.

} > select-in-shell-word
} 
} Other lines, when ended with space, will also cause the unexpected
} behavior. I should have stated this that it's about space at the end.

Yes, sorry; that's the "in" part of select-in-shell-word.  It selects
the word under the cursor, or (what I hadn't noticed when I created
that little three-line widget) the word to the *right* of the cursor
if between words; and because of vim-ism, it does not cross newlines.
So if you're at the end of a line with whitespace it selects nothing.

} Seems that (Z:n:) would discard the new lines like
} any other white spaces and the code would be simpler.

Actually it doesn't discard them -- they are treated as a word if in
context they act as a separator.  They just aren't converted into
semicolons.

} BTW. I discovered that following multi line command can make various
} versions of zsh hang on it:
} 
[...]
} 
} Does it reproduce to you?

Hangs how?  In ZLE, or when attempting to actually execute the commands?

If the latter, can you "set -vx" to see if it's possible to tell which
line causes the problem?

If the former, you'll have to explain in more detail the steps to
reproduce, including whether it's possible to do so from "zsh -f".