_history-complete-older problems with $(

_history-complete-older problems with $(

[Sebastian Gniazdowski]

Hello
$(<Alt-/> doesn't complete $(( 0+1 )) correctly. When I empty my
~/.zhistory and do:

set HISTSIZE=3
a=""
a="${(r:100000:: _:)a}"
echo $(( 0+1 ))

then $<Alt-/> works, but $(<Alt-/> doesn't. The completing is setup with:

setopt hist_lex_words
bindkey "^[/" _history-complete-older
zstyle ':completion:*:history-words' remove-all-dups true
zstyle ':completion:*' range 50000:10000

Also, the remove-all-dups zstyle doesn't work which may be verified by doing:

set HISTSIZE=3
echo $(( 0+1 ))
a="${(r:100000:: _:)a}"
echo $(( 0+1 ))

and doing echo $(<Alt-/> – there will be echo shown twice, so as $(( 0+1 ))

(If I don't empty ~/.zhistory, then Alt-/ sometimes wants me to
propose hundreds of words, apparently taken from ~/.zhistory, so this
ignores HISTSIZE).

Best regards,
Sebastian Gniazdowski

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

Hello
on IRC it came up that $( is a start of a subshell and that it makes
completion start over at this place. Seems that $(( works in the same
way. Is there any possibility to change that? Even at level of C code?
So that completion sees $( as a string? I know chances are low but who
knows, maybe it's a really basic change.

Best regards,
Sebastian Gniazdowski

Re: _history-complete-older problems with $(

[Bart Schaefer]

On Jan 11,  5:21pm, Sebastian Gniazdowski wrote:
} Subject: Re: _history-complete-older problems with $(
}
} on IRC it came up that $( is a start of a subshell and that it makes
} completion start over at this place. Seems that $(( works in the same
} way. Is there any possibility to change that? Even at level of C code?

The following produces the correct set of completion matches (pardon the
unimaginative name) but is a bit of a hack (it breaks _complete_debug
and _complete_help):

  _widen_for_history () {
    local -a left=( "${(z)LBUFFER}" )
    local right="${words[CURRENT]#${left[-1]}}"
    words=( "${(z)BUFFER}" )
    CURRENT=${#left}
    PREFIX="${left[-1]}"
    IPREFIX=''
    SUFFIX="${right}"
    ISUFFIX=''
    {
      compadd () {
	local -a found
	builtin compadd -O found "$@"
	builtin compadd -Q -U "${(@)${(@)found#$PREFIX}%$SUFFIX}"
      }
      _history
    } always {
      unfunction compadd
    }
  }
  zstyle ':completion:*' completer _widen_for_history _complete

This needs some work around the issue of empty words, i.e., when
completing in empty parens such as $(<TAB>) then $words[CURRENT] is
empty and the trailing paren must be copied from $RBUFFER to SUFFIX.
I haven't dived into that detail.

It could be further refined (with a lot of work) to widen only to the
next "outward" command, e.g., in

  foo $(bar $(<TAB>

to extend back only as far as "bar" rather than to "foo", which might
turn out to be important for some multi-line buffers, I'm not sure.

Another approach would be to create a custom key binding for a widget
which

- inserts an open quote before the current word
- invokes completion with completer style set to _history
- removes quoting again

but that has its own complications if the word on the line is already
quoted, etc.

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

On 12 January 2016 at 01:15, Bart Schaefer <schaefer@brasslantern.com> wrote:
> The following produces the correct set of completion matches (pardon the
> unimaginative name) but is a bit of a hack (it breaks _complete_debug
> and _complete_help):

I think the hacks are acceptable if the code will be used only by
Alt-/ (_history-complete-older). How to accomplish this, btw? Probably
the context should be changed:

>   zstyle ':completion:*' completer _widen_for_history _complete

but I don't know how to do this exactly. Can you help?

>   _widen_for_history () {
>     local -a left=( "${(z)LBUFFER}" )
>     local right="${words[CURRENT]#${left[-1]}}"
>     words=( "${(z)BUFFER}" )
>     CURRENT=${#left}
>     *
>     PREFIX="${left[-1]}"

I've added following line where * is:

echo -e "\nCURRENT: $CURRENT, NEWCUR: $CURRENT, CURSR: $CURSOR, lft:
|$left|, rght: |$left|, words: >""$words[@]""<"

Example outputs are:
# ls<TAB>
CURRENT: 1, NEWCUR: 1, CURSR: 2, lft: |ls|, rght: |ls|, words: >ls<
# ls <TAB>
CURRENT: 1, NEWCUR: 1, CURSR: 3, lft: |ls|, rght: |ls|, words: >ls<
# $(<TAB>
CURRENT: 1, NEWCUR: 1, CURSR: 2, lft: |$(|, rght: |$(|, words: >$(<
# $( TAB
CURRENT: 1, NEWCUR: 1, CURSR: 3, lft: |$( |, rght: |$( |, words: >$( <

Are this expected values? Because despite proposing $(( 0 + 1 ))
correctly after $(<TAB>  (btw., not after $(<Alt-/>), the completer
behaves weird. For "ls <TAB>" it doesn't propose any files.

> This needs some work around the issue of empty words, i.e., when
> completing in empty parens such as $(<TAB>) then $words[CURRENT] is
> empty and the trailing paren must be copied from $RBUFFER to SUFFIX.
> I haven't dived into that detail.

I would gladly perform the work but the code is quite difficult. Could
you state in one sentence what it is doing? Variables $CURRENT, $word
aren't documented in zshzle.1. What does the {} always {} block of
code do? Next, I would guess that "builtin compadd -O found "$@""
performs one run of matching, is this correct? Why second comadd, and
what does "${(@)${(@)found#$PREFIX}%$SUFFIX}" do?

> Another approach would be to create a custom key binding for a widget
> which
>
> - inserts an open quote before the current word
> - invokes completion with completer style set to _history
> - removes quoting again
>
> but that has its own complications if the word on the line is already
> quoted, etc.

Maybe I could get the complications resolved for Zew. The separate key
binding seems close to what I want for Alt-/

Best regards,
Sebastian Gniazdowski

Re: _history-complete-older problems with $(

[Bart Schaefer]

On Jan 12,  2:09pm, Sebastian Gniazdowski wrote:
} Subject: Re: _history-complete-older problems with $(
}
} I think the hacks are acceptable if the code will be used only by
} Alt-/ (_history-complete-older). How to accomplish this, btw?

You would need to change the _history_complete_word_gen_matches
function in the _history_complete_word file to replace

    _main_complete _history

with

    _main_complete _widen_for_history

} Are this expected values? Because despite proposing $(( 0 + 1 ))
} correctly after $(<TAB>  (btw., not after $(<Alt-/>), the completer
} behaves weird. For "ls <TAB>" it doesn't propose any files.

I can't reproduce this.  There may need to be some tests for success
of the calls to "builtin compadd" in the substitute compadd function;
given

    zstyle ':completion:*' completer _widen_for_history _complete

(the _complete at the end is important) there will only be completions
generated by _complete when _widen_for_history returns nonzero.  It is
possible you've found some circumstance in which it returns 0 but does
not add any actual matches.

I'm not trivally able to reproduce that.

} > This needs some work around the issue of empty words, i.e., when
} > completing in empty parens such as $(<TAB>) then $words[CURRENT] is
} > empty and the trailing paren must be copied from $RBUFFER to SUFFIX.
} > I haven't dived into that detail.
} 
} I would gladly perform the work but the code is quite difficult. Could
} you state in one sentence what it is doing? Variables $CURRENT, $word
} aren't documented in zshzle.1.

They're documented in "man zshcompwid".

} What does the {} always {} block of code do?

 { TRY-LIST } always { ALWAYS-LIST }
     First execute TRY-LIST.  Regardless of errors, or break, continue,
     or return commands encountered within TRY-LIST, execute
     ALWAYS-LIST.  Execution then continues from the result of the
     execution of TRY-LIST; in other words, any error, or break,
     continue, or return command is treated in the normal way, as if
     ALWAYS-LIST were not present.  The two chunks of code are referred
     to as the `try block' and the `always block'.

} Next, I would guess that "builtin compadd -O found "$@""
} performs one run of matching, is this correct?

    -O ARRAY
          If this option is given, the WORDS are _not_ added to the set
          of possible completions.  Instead, matching is done as usual
          and all of the WORDS given as arguments that match the string
          on the command line will be stored in the array parameter
          whose name is given as ARRAY.

} Why second comadd, and
} what does "${(@)${(@)found#$PREFIX}%$SUFFIX}" do?

Suppose you have "$(" on the line, and the completion is $((1+2)).  The
$found array will contain "$((1+2))" (and possibly other elements).  In
the setup to at the beginning of _widen_for_history, PREFIX="$(" was
assigned (that $left[-1]).  So now we need to remove "$(" from the
values in $found to get the actual string to insert to finish the word,
i.e., "(1+2))".  Similarly we would need to remove $SUFFIX (taken from
$right) from the end of the values if there were one.

${...#$PREFIX} removes from the left and ${...%$SUFFIX} removes from
the right.

Normally "builtin compadd" would handle this for us, but internally it
still thinks it is populating command words that must follow the open
paren; so we have to fix things up before we make the final call.

(That's not an entirely accurate description of the "internally" part
but it suffices to explain why we must trim the prefix/suffix.)

Re: _history-complete-older problems with $(

[Daniel Shahaf]

Bart Schaefer wrote on Tue, Jan 12, 2016 at 11:39:28 -0800:
> On Jan 12,  2:09pm, Sebastian Gniazdowski wrote:
> } Why second comadd, and
> } what does "${(@)${(@)found#$PREFIX}%$SUFFIX}" do?
> 
> Suppose you have "$(" on the line, and the completion is $((1+2)).  The
> $found array will contain "$((1+2))" (and possibly other elements).  In
> the setup to at the beginning of _widen_for_history, PREFIX="$(" was
> assigned (that $left[-1]).  So now we need to remove "$(" from the
> values in $found to get the actual string to insert to finish the word,
> i.e., "(1+2))".  Similarly we would need to remove $SUFFIX (taken from
> $right) from the end of the values if there were one.
> 

That seems to always behave as though COMPLETE_IN_WORD is set,
regardless of the value of the option.

Re: _history-complete-older problems with $(

[Bart Schaefer]

On Jan 13,  1:01am, Daniel Shahaf wrote:
} Subject: Re: _history-complete-older problems with $(
}
} Bart Schaefer wrote on Tue, Jan 12, 2016 at 11:39:28 -0800:
} > So now we need to remove "$(" from the
} > values in $found to get the actual string to insert to finish the word,
} > i.e., "(1+2))".  Similarly we would need to remove $SUFFIX (taken from
} > $right) from the end of the values if there were one.
} 
} That seems to always behave as though COMPLETE_IN_WORD is set,
} regardless of the value of the option.

True.  Anyone who wants to fix these issues and post an update is more
than welcome.  I already spent way more time on it than I intended.

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 2837 bytes --]

On 13 January 2016 at 03:01, Bart Schaefer <schaefer@brasslantern.com> wrote:
> True.  Anyone who wants to fix these issues and post an update is more
> than welcome.  I already spent way more time on it than I intended.

Thanks for the explanations and pointing the manual, this allowed me
to move onward with the code. I didn't use $LBUFFER, but say manually
detected active shell word, and point of break to left and right part
in that word. This will allow to nicely handle COMPLETE_IN_WORD. I
must say it's quite exhausting to handle which word is active. Setting
cursor to "|" in terminal application clarifies few things. Block
cursor highlights a letter, but $CURSOR points before it, where "|"
cursor would stand. Zsh doesn't make following word active, i.e. "ls
|/Users/user_na" with the cursor where "|" is will not complete the
"user_name". I have chosen different approach, in such case I treat
what's after "|" as right part of current shell word. The same I do
for zew-transpose-shell-words (former transpose-segments), block
cursor on first letter of word makes the word active, selected for
transposition.

I do other tricks in the code, e.g. trim left "$((   " out of spaces,
because in case of syntax error (Z+n+) will put spaces in returned
shell word. All this is quite hackish I think. But something could
clarify out of this, and most of all robust Alt-/ is on the horizon.

I have one problem. First compadd returns $found array
populated with matches:

A CURRENT: 3, words: >git,add,wid<
B CURRENT: 3, words: >git,add,wid<
C CURRENT: 3, CURSR: 11, lft: |wid|, rght: ||, words: >git,add,wid<
PREFIX: |wid|, SUFFIX: ||
Calling _history
widen_for_history widen_for_history widen_for_history
widen_for_history widen_for_history widen_for_history
widen_for_history widen_for_history

But nothing is displayed below the prompt, "en" replaces "wid" at
prompt and that's all, while $(<TAB> does:

A CURRENT: 1, words: ><
B CURRENT: 1, words: >$( <
C CURRENT: 1, CURSR: 2, lft: |$(|, rght: ||, words: >$(<
PREFIX: |$(|, SUFFIX: ||
Calling _history
$(( 0 + 1 )) $(( 0 + 1 )) $(( 試句相當長 ""  $(( 0 + 1 )) $( ( echo b ))
$(( echo b ) ) $(( echo a ) ) $(( 0 + 1 )) $(( 0 + 1 )) $(( i + 1 ))
$(( i + 1 )) $(( i + 1 )) $(( i+1 )) $(( i+1 )) $(( i+1 )) $(( i+1 ))
$(( i+1 ))

And below the prompt it's shown:
 ( echo b ))      ( 0 + 1 ))        ( echo a ) )      ( echo b ) )
 ( i + 1 ))        ( i+1 ))          ( 試句相當長 ""

The same problem is with "ls" about which I wrote to you earlier. So
it seems that the second compadd isn't fully working.

The code is here (and also attached):

https://raw.githubusercontent.com/psprint/zsh-editing-workbench/657d6a591bea5d26fbed88176ac7de80e8d73927/widen_for_history

Best regards,
Sebastian Gniazdowski

[-- Attachment #2: widen_for_history --]
[-- Type: application/octet-stream, Size: 2493 bytes --]

echo -e "\nA CURRENT: $CURRENT, words: >${(j:,:)words}<" >> /tmp/wfhistory

words="${(Z+n+)BUFFER}"
integer nwords="${#words}"
local buf="$BUFFER"

echo -e "B CURRENT: $CURRENT, words: >${(j:,:)words}<" >> /tmp/wfhistory

# Remove words one by one, counting characters,
# computing beginning of each word, to find
# place to break the word into 2 halves (for
# complete_in_word option)
# If would use $LBUFFER, then would have to be able
# to parse shell words, to extract word's halves

local i word
integer char_count=0
typeset -a word_beginnings
CURRENT="-1"

for (( i=1; i<=nwords; i++ )); do
    # Remove trailing spaces from $word - they may
    # appear in case of syntax error, like "$((   <TAB>"
    words[i]="${words[i]%% ##}"

    word="${words[i]}"

    # Remove white spaces
    buf="${buf##(#m)[^$word[1]]#}"
    # Count them
    char_count=char_count+"$#MATCH"
    # This is the beginning of current word
    word_beginnings[i]="$char_count"

    # Remove the word
    MATCH=""
    buf="${buf#(#m)$word}"

    # If shell word not found, return. This shoudln't happen
    [ -z "$MATCH" ] && { echo "Aborting" >> /tmp/wfhistory; return 0 }

    # Spaces point to previous shell word
    [[ "$CURRENT" -eq "-1" && "$char_count" -gt "$CURSOR" ]] && CURRENT=i-1

    # Actual characters point to current shell word
    char_count=char_count+"$#word"
    [[ "$CURRENT" -eq "-1" && "$char_count" -ge "$CURSOR" ]] && CURRENT=i
done 

# What's left in $buf can be only white spaces
char_count=char_count+"$#buf"
[[ "$CURRENT" -eq -1 && "$char_count" -ge "$CURSOR" ]] && CURRENT=i-1

# Divide words[CURRENT] into two halves
integer diff=$(( CURSOR - word_beginnings[CURRENT] ))
word="${words[CURRENT]}"
local left="${word[1,diff]}"
local right="${word[diff+1,-1]}"

echo "C CURRENT: $CURRENT, CURSR: $CURSOR, lft: |$left[*]|, rght: |$right|, words: >${(j:,:)words}<" >> /tmp/wfhistory

# History is searched for words that start with $PREFIX
PREFIX="${left}"
IPREFIX=''
# ... and end with $SUFFIX
SUFFIX="${right}"
ISUFFIX=''

echo "PREFIX: |$PREFIX|, SUFFIX: |$SUFFIX|" >> /tmp/wfhistory

{
    compadd () {
        local -a found
        builtin compadd -O found "$@"
        [[ "$#found" -eq 0 ]] && echo -n "_empty_ " >> /tmp/wfhistory || echo "${found[@]}" >> /tmp/wfhistory
        builtin compadd -Q -U "${(@)${(@)found#$PREFIX}%$SUFFIX}"
    }
    echo "Calling _history" >> /tmp/wfhistory
    _history
    echo >> /tmp/wfhistory
} always {
    unfunction compadd
}

# vim:ft=zsh

Re: _history-complete-older problems with $(

[Bart Schaefer]

On Jan 13,  5:40pm, Sebastian Gniazdowski wrote:
}
} Zsh doesn't make following word active, i.e. "ls |/Users/user_na"
} with the cursor where "|" is will not complete the "user_name".

That's specific to complete_in_word -- if you have not set that, then
the cursor moves to the end of the word before attempting completion.
But if you HAVE set that, you are saying that the (empty) part of the
word to the left of the cursor is important, so zsh is correctly
responding that there isn't anything that can be completed there.

} I have chosen different approach, in such case I treat
} what's after "|" as right part of current shell word.

Zsh DOES treat what's to the right of the cursor as part of the
current word.

You can see what's going on more clearly if you complete (in the zsh
source tree, "|" indicates the cursor when TAB is pressed):

torch% setopt completeinword
torch% ls |c
Doc/  Etc/  Src/

Here "c" is part of the word and there IS something that can appear
to the left of it, so you get the completions.

If you want other behavior you use a matcher (matcher-list style).

} The same I do for zew-transpose-shell-words (former
} transpose-segments), block cursor on first letter of word makes the
} word active, selected for transposition.

That should be the way both the builtin transpose-words and the newly
patched transpose-words-match work as well, so I'm not sure 

} I have one problem. First compadd returns $found array
} populated with matches:
} 
} But nothing is displayed below the prompt, "en" replaces "wid" at
} prompt and that's all

Ah.  Another wrinkle I overlooked.  You need to store the original
values of $PREFIX and $SUFFIX somewhere, e.g.

    local origPREFIX=$PREFIX origSUFFIX=$SUFFIX

Then, after "compadd -O found" but before trimming PREFIX and SUFFIX
off of $found, you need

    PREFIX=${PREFIX#$origPREFIX}
    SUFFIX=${SUFFIX%$origSUFFIX}

This is because compadd will still try to replace the original word
prefix ("wid") with whatever you pass to that second compadd, so if
there was an original prefix/suffix you must NOT trim those off.  I
didn't test my original version with anything other than an empty
$PREFIX so didn't think of this.

} The same problem is with "ls" about which I wrote to you earlier. So
} it seems that the second compadd isn't fully working.

When _history returns nonzero, your debugging line

    echo >> /tmp/wfhistory

is clobbering that with its own zero return, so completion stops even
if no matches were found.  Move that last "echo" into the block that
is after "always" (right before "unfunction compadd").

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

On 14 January 2016 at 05:48, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Jan 13,  5:40pm, Sebastian Gniazdowski wrote:
> Ah.  Another wrinkle I overlooked.  You need to store the original
> values of $PREFIX and $SUFFIX somewhere, e.g.
>
>     local origPREFIX=$PREFIX origSUFFIX=$SUFFIX
>
> Then, after "compadd -O found" but before trimming PREFIX and SUFFIX
> off of $found, you need
>
>     PREFIX=${PREFIX#$origPREFIX}
>     SUFFIX=${SUFFIX%$origSUFFIX}
>
> This is because compadd will still try to replace the original word
> prefix ("wid") with whatever you pass to that second compadd, so if
> there was an original prefix/suffix you must NOT trim those off.  I
> didn't test my original version with anything other than an empty
> $PREFIX so didn't think of this.
>
> } The same problem is with "ls" about which I wrote to you earlier. So
> } it seems that the second compadd isn't fully working.
>
> When _history returns nonzero, your debugging line
>
>     echo >> /tmp/wfhistory
>
> is clobbering that with its own zero return, so completion stops even
> if no matches were found.  Move that last "echo" into the block that
> is after "always" (right before "unfunction compadd").

Bart the menucomplete option has something to do with all those
things. I don't have the option set. When I set it it can be observed
how your code helps or harms $(<Tab> and ls<Tab>

https://asciinema.org/a/1qyjrg38s411m8rbsvmtnfwgl

Best regards,
Sebastian Gniazdowski

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

Better movie, I tried Tab multiple times with "ls" with no space after
it. I test "ls" and "$(" with and without spaces after them.

https://asciinema.org/a/bhslks3h04x328c72az3ugawz

Best regards,
Sebastian

On 14 January 2016 at 12:30, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> On 14 January 2016 at 05:48, Bart Schaefer <schaefer@brasslantern.com> wrote:
>> On Jan 13,  5:40pm, Sebastian Gniazdowski wrote:
>> Ah.  Another wrinkle I overlooked.  You need to store the original
>> values of $PREFIX and $SUFFIX somewhere, e.g.
>>
>>     local origPREFIX=$PREFIX origSUFFIX=$SUFFIX
>>
>> Then, after "compadd -O found" but before trimming PREFIX and SUFFIX
>> off of $found, you need
>>
>>     PREFIX=${PREFIX#$origPREFIX}
>>     SUFFIX=${SUFFIX%$origSUFFIX}
>>
>> This is because compadd will still try to replace the original word
>> prefix ("wid") with whatever you pass to that second compadd, so if
>> there was an original prefix/suffix you must NOT trim those off.  I
>> didn't test my original version with anything other than an empty
>> $PREFIX so didn't think of this.
>>
>> } The same problem is with "ls" about which I wrote to you earlier. So
>> } it seems that the second compadd isn't fully working.
>>
>> When _history returns nonzero, your debugging line
>>
>>     echo >> /tmp/wfhistory
>>
>> is clobbering that with its own zero return, so completion stops even
>> if no matches were found.  Move that last "echo" into the block that
>> is after "always" (right before "unfunction compadd").
>
> Bart the menucomplete option has something to do with all those
> things. I don't have the option set. When I set it it can be observed
> how your code helps or harms $(<Tab> and ls<Tab>
>
> https://asciinema.org/a/1qyjrg38s411m8rbsvmtnfwgl
>
> Best regards,
> Sebastian Gniazdowski

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

The same actions but without menucomplete. I'm sorry for quite of a
flood, I understand that not all people read the list by threads,
sorry, I will group my thoughts next time.

https://asciinema.org/a/82i53qbirassjyhxpjbb9u55k

Best regards,
Sebastian Gniazdowski


On 14 January 2016 at 12:37, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> Better movie, I tried Tab multiple times with "ls" with no space after
> it. I test "ls" and "$(" with and without spaces after them.
>
> https://asciinema.org/a/bhslks3h04x328c72az3ugawz
>
> Best regards,
> Sebastian
>
> On 14 January 2016 at 12:30, Sebastian Gniazdowski
> <sgniazdowski@gmail.com> wrote:
>> On 14 January 2016 at 05:48, Bart Schaefer <schaefer@brasslantern.com> wrote:
>>> On Jan 13,  5:40pm, Sebastian Gniazdowski wrote:
>>> Ah.  Another wrinkle I overlooked.  You need to store the original
>>> values of $PREFIX and $SUFFIX somewhere, e.g.
>>>
>>>     local origPREFIX=$PREFIX origSUFFIX=$SUFFIX
>>>
>>> Then, after "compadd -O found" but before trimming PREFIX and SUFFIX
>>> off of $found, you need
>>>
>>>     PREFIX=${PREFIX#$origPREFIX}
>>>     SUFFIX=${SUFFIX%$origSUFFIX}
>>>
>>> This is because compadd will still try to replace the original word
>>> prefix ("wid") with whatever you pass to that second compadd, so if
>>> there was an original prefix/suffix you must NOT trim those off.  I
>>> didn't test my original version with anything other than an empty
>>> $PREFIX so didn't think of this.
>>>
>>> } The same problem is with "ls" about which I wrote to you earlier. So
>>> } it seems that the second compadd isn't fully working.
>>>
>>> When _history returns nonzero, your debugging line
>>>
>>>     echo >> /tmp/wfhistory
>>>
>>> is clobbering that with its own zero return, so completion stops even
>>> if no matches were found.  Move that last "echo" into the block that
>>> is after "always" (right before "unfunction compadd").
>>
>> Bart the menucomplete option has something to do with all those
>> things. I don't have the option set. When I set it it can be observed
>> how your code helps or harms $(<Tab> and ls<Tab>
>>
>> https://asciinema.org/a/1qyjrg38s411m8rbsvmtnfwgl
>>
>> Best regards,
>> Sebastian Gniazdowski

Re: _history-complete-older problems with $(

[Bart Schaefer]

On Jan 14, 12:30pm, Sebastian Gniazdowski wrote:
}
} Bart the menucomplete option has something to do with all those
} things. I don't have the option set. When I set it it can be observed
} how your code helps or harms $(<Tab> and ls<Tab>
} 
} https://asciinema.org/a/1qyjrg38s411m8rbsvmtnfwgl

Sorry, Sebastian, but I lost interest in solving this problem about
three messages ago, and also have lost all tolerance for watching
screen capture movies and trying to match them to your descriptions
and log output.  It's too difficult to tell from the animations what
keystrokes you're executing when, among other things.

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

On 15 January 2016 at 04:18, Bart Schaefer <schaefer@brasslantern.com> wrote:
> Sorry, Sebastian, but I lost interest in solving this problem about
> three messages ago, and also have lost all tolerance for watching
> screen capture movies and trying to match them to your descriptions
> and log output.  It's too difficult to tell from the animations what
> keystrokes you're executing when, among other things.

Out of the chaos I've isolated one thing. "ls <TAB>" completes to "ls
ls" and this is the same as when "$((<TAB>" completes to "$(($((". All
this is quite exhausting but I'm off for the whole day and taking
zshcompwid.1 with me so I might learn required things. Sorry for my
verbosity, I worked for whole day on this yesterday, didn't want to
miss your insight which often help as when you came up with idea for
fpath-clean plugins or with idea that PREFIX should be restored to
value from before first compadd(), which I would rather miss as the
PREFIX value doesn't change.

Best regards,
Sebastian Gniazdowski

Re: _history-complete-older problems with $(

[Daniel Shahaf]

Bart Schaefer wrote on Thu, Jan 14, 2016 at 19:18:09 -0800:
> It's too difficult to tell from the animations what keystrokes you're
> executing when, among other things.

I agree:

    zle-line-pre-redraw() { zle -M "${(qq)KEYS} $WIDGET" }
    zle -N zle-line-pre-redraw

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

On 14 January 2016 at 05:48, Bart Schaefer <schaefer@brasslantern.com> wrote:
> Ah.  Another wrinkle I overlooked.  You need to store the original
> values of $PREFIX and $SUFFIX somewhere, e.g.
>
>     local origPREFIX=$PREFIX origSUFFIX=$SUFFIX
>
> Then, after "compadd -O found" but before trimming PREFIX and SUFFIX
> off of $found, you need
>
>     PREFIX=${PREFIX#$origPREFIX}
>     SUFFIX=${SUFFIX%$origSUFFIX}
>

Setting PREFIX and SUFFIX this way changes things (apparently for the
better, the code works very well with them, except for $(<TAB> as I
shown in other video, the "Better" one), but it's interesting that
PREFIX and SUFFIX do not change their values, I showed this in
following video:

https://asciinema.org/a/cf8yrmdzjqy7r2kz42091ujts

If I do PREFIX="$PREFIX" SUFFIX="$SUFFIX" then it doesn't change
things, interestingly, it's like no assignment would be done.

Best regards,
Sebastian

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

PS. I used the "find LBUFFER myself" approach to handle following case
better ("|" is cursor):

% $(( 0 |+ 1 ))

Doing ${(z)LBUFFER} and then (z) on right part of BUFFER would produce
SUFFIX="+" instead of "+ 1 ))".  If I find the word in ${(z)BUFFER}
and divide it into two halves myself, then I have PREFIX="$(( 0 "
SUFFIX "+ 1 ))". However, following case works better with LBUFFER
approach:

% ${|a some other $words

With LBUFFER approach, PREFIX="${" SUFFIX="a" and that's better than
PREFIX="${" SUFFIX="a some other $words". I can of course respect
no_complete_in_words and do PREFIX="${a" SUFFIX="".

I'm currently looking for a way to choose better strategy, maybe
someone has some ideas?

Best regards,
Sebastian Gniazdowski

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

On 14 January 2016 at 10:52, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> PS. I used the "find LBUFFER myself" approach to handle following case
> better ("|" is cursor):
>
> % $(( 0 |+ 1 ))
>
> Doing ${(z)LBUFFER} and then (z) on right part of BUFFER would produce
> SUFFIX="+" instead of "+ 1 ))".  If I find the word in ${(z)BUFFER}

That turned out to be not true, (z) isn't done on right part, instead
words[CURRENT] is utilized and chopped to obtain right part of shell
word.

    local -a left=( "${(z)LBUFFER}" )
    local right="${words[CURRENT]#${left[-1]}}"

So still clarifying things, maybe it's good idea to do (z) on right
part to obtain ${|a case behavior.

Best regards,
Sebastian Gniazdowski

Re: _history-complete-older problems with $(

[Sebastian Gniazdowski]

I have one idea. It's generally about detecting a syntax error that
makes shell word unbound. By appending " x" to shell word and running
(Z+n+) one can obtain either 1 or 2 elements and first signals a
syntax error. For such case, SUFFIX would consist only from a space
delimited part of the regular right part.

Best regards,
Sebastian Gniazdowski


On 14 January 2016 at 11:14, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> On 14 January 2016 at 10:52, Sebastian Gniazdowski
> <sgniazdowski@gmail.com> wrote:
>> PS. I used the "find LBUFFER myself" approach to handle following case
>> better ("|" is cursor):
>>
>> % $(( 0 |+ 1 ))
>>
>> Doing ${(z)LBUFFER} and then (z) on right part of BUFFER would produce
>> SUFFIX="+" instead of "+ 1 ))".  If I find the word in ${(z)BUFFER}
>
> That turned out to be not true, (z) isn't done on right part, instead
> words[CURRENT] is utilized and chopped to obtain right part of shell
> word.
>
>     local -a left=( "${(z)LBUFFER}" )
>     local right="${words[CURRENT]#${left[-1]}}"
>
> So still clarifying things, maybe it's good idea to do (z) on right
> part to obtain ${|a case behavior.
>
> Best regards,
> Sebastian Gniazdowski

Re: _history-complete-older problems with $(

[Bart Schaefer]

On Jan 14, 11:55am, Sebastian Gniazdowski wrote:
} Subject: Re: _history-complete-older problems with $(
}
} I have one idea. It's generally about detecting a syntax error that
} makes shell word unbound. By appending " x" to shell word and running
} (Z+n+) one can obtain either 1 or 2 elements and first signals a
} syntax error. For such case, SUFFIX would consist only from a space
} delimited part of the regular right part.

That should work, and in fact the completion internals do something
quite similar.

Re: _history-complete-older problems with $(

[Daniel Shahaf]

Sebastian Gniazdowski wrote on Thu, Jan 14, 2016 at 10:52:14 +0100:
> PS. I used the "find LBUFFER myself" approach to handle following case
> better ("|" is cursor):
> 
> % $(( 0 |+ 1 ))
> 
> Doing ${(z)LBUFFER} and then (z) on right part of BUFFER would produce
> SUFFIX="+" instead of "+ 1 ))".  If I find the word in ${(z)BUFFER}
> and divide it into two halves myself, then I have PREFIX="$(( 0 "
> SUFFIX "+ 1 ))". However, following case works better with LBUFFER
> approach:
> 
> % ${|a some other $words
> 
> With LBUFFER approach, PREFIX="${" SUFFIX="a" and that's better than
> PREFIX="${" SUFFIX="a some other $words". I can of course respect
> no_complete_in_words and do PREFIX="${a" SUFFIX="".
> 
> I'm currently looking for a way to choose better strategy, maybe
> someone has some ideas?

You could have the widget try both strategies.

Re: _history-complete-older problems with $(

[Daniel Shahaf]

Sebastian Gniazdowski wrote on Tue, Jan 12, 2016 at 14:09:47 +0100:
> On 12 January 2016 at 01:15, Bart Schaefer <schaefer@brasslantern.com> wrote:
> >   _widen_for_history () {
> >     local -a left=( "${(z)LBUFFER}" )

Strictly speaking, ${(z)} should always be applied to $PREBUFFER$LBUFFER
or $PREBUFFER$BUFFER, not merely to $BUFFER, otherwise things like
string literals with embedded newlines can confuse the result.

> > This needs some work around the issue of empty words, i.e., when
> > completing in empty parens such as $(<TAB>) then $words[CURRENT] is
> > empty and the trailing paren must be copied from $RBUFFER to SUFFIX.
> > I haven't dived into that detail.
> 
> I would gladly perform the work but the code is quite difficult. Could
> you state in one sentence what it is doing?

Bart's function runs the _history completer with $words modified to
reflect the entire command-line "list"¹ rather than only the "simple
command"¹ the cursor happens to be in.

Cheers,

Daniel

¹ Terms from zshmisc(1).

Re: _history-complete-older problems with $(

[Bart Schaefer]

On Jan 13,  1:01am, Daniel Shahaf wrote:
}
} Strictly speaking, ${(z)} should always be applied to $PREBUFFER$LBUFFER
} or $PREBUFFER$BUFFER, not merely to $BUFFER, otherwise things like
} string literals with embedded newlines can confuse the result.

Hmm, that's a difficult call.  That also means setting the $words array
to the full parse of $PREBUFFER$BUFFER, which I don't think matches what
the completion internals do.

OTOH the whole reason for this is to subvert the internals, so ...

Re: _history-complete-older problems with $(

[Bart Schaefer]

On Jan 10,  8:35pm, Sebastian Gniazdowski wrote:
}
} $(<Alt-/> doesn't complete $(( 0+1 )) correctly.
} 
} zstyle ':completion:*:history-words' remove-all-dups true
} 
} Also, the remove-all-dups zstyle doesn't work

So, just for the record, we've confirmed that:

The $( thing is because completion starts a new command context in a
subshell.

The remove-all-dups thing is because I gave the wrong style context, it
should be

    zstyle ':completion:history-words:*' remove-all-dups true