* Fwd: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases [not found] ` <20200504131830.3572e317@tarpaulin.shahaf.local2> @ 2020-05-04 17:13 ` Dan Arad 2020-05-05 16:49 ` Daniel Shahaf 0 siblings, 1 reply; 10+ messages in thread From: Dan Arad @ 2020-05-04 17:13 UTC (permalink / raw) To: zsh-users [-- Attachment #1: Type: text/plain, Size: 6354 bytes --] Hi there, I'm forwarding a conversation I started by mistake on the wrong mailing list (Sorry, I'm new to the concept). Any help on the topic would be greatly appreciated! Cheers, Dan ---------- Forwarded message --------- From: Daniel Shahaf <d.s@daniel.shahaf.name> Date: Mon, May 4, 2020 at 4:18 PM Subject: Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases To: Dan Arad <dan1994@gmail.com> Good morning Dan, Would you re-send that to the mailing list, please? You sent it only to me. Thanks for tracking down the rich text issue — it's much easier to talk this way. ☺ Cheers, Daniel Dan Arad wrote on Mon, 04 May 2020 09:46 +0300: > First of all, thank you for your swift response! > > For future reference, you should have addressed your question to > > zsh-users@, as the answer to it doesn't involve a change to zsh itself. > > > Noted! I searched online for the zsh bug tracker, and stumbled into this > mail address. > > Furthermore, it would be helpful to configure your email client to > > generate a text/plain part that doesn't surround every line by asterisks > > and add spurious blank lines in code blocks. > > > > I was trying to copy paste the stack overflow entry through Word, so it > won't keep the web formatting, and that's probably why this happened... > > Well, for starters, if you «shift words», you should decrement CURRENT > > as well. > > > > However, _python uses the «*::…» form of an _arguments optspec, > > which should take care of $words/$NUMERIC for you. I assume the reason > > it didn't is that you used «compdef -p». Try -P instead? If I'm not > > mistaken, that would also handle «python … script.py <TAB>» for you > > (where the ellipsis stands for python's --options). > > > I tested the 4 possible options, and found that switching -p with -P > doesn't change anything, but shifting words and decrementing CURRENT works! > But... I feel like it's a hack I'm not supposed to do (correct me if I'm > wrong). > > Your second paragraph made me look back at a previous solution (that > doesn't work). I would be glad to discuss it if you have the time. > > I was using `#compdef -p *.py` before, and what I saw was that _python was > being called before my script, and that my script didn't work well. What > was actually happening was that _python was shifting words before I was > being called, and that interfered with the logic of my script. > > In order to generate completions I rely on running what is currently > present in the command line, and so this may cause my script to generate no > completions (i.e. `python ps.py` becomes `ps.py` which can't be run, and > `python ./ps.py` becomes `./ps.py` which may or may not be run depending on > file permissions). > > I now tried using the BUFFER variable instead of words (since it remains > unchanged), but found that it also has caveats: The BUFFER variable > contains the raw value of the line, whereas the words array contains the > words after expansions (I think that's the right term). > To demonstrate this I tried running again completion for `python ps,py` > with `#compdef -P *`, and printing out BUFFER and words. The debug log (and > my terminal) showed the following: > First of all _python was called and dispatched my script which printed > `BUFFER=python ps.py` and `words=ps.py`. > After _python returned, my script was invoked again and printed > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3). > > I do think the right solution is to be dispatched by _python with `#compdef > -P *.py`, but to be able to run I need the words array as seen by _python > or a to understand how BUFFER can be expanded/converted to its correct form. > If you have any input on this, I would very much appreciate it. > > Best Regards, > Dan > > > On Sun, May 3, 2020 at 7:58 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > > > Dan Arad wrote on Sun, 03 May 2020 10:40 +0300: > > > Hi there, > > > > > > From what I understand, bug reports should be mailed here. If I'm wrong > > I'd > > > be happy to be redirected to the correct medium. > > > > For future reference, you should have addressed your question to > > zsh-users@, as the answer to it doesn't involve a change to zsh itself. > > Furthermore, it would be helpful to configure your email client to > > generate a text/plain part that doesn't surround every line by asterisks > > and add spurious blank lines in code blocks. > > > > > What I want to report is probably not a bug, and stems from my > > > misunderstanding of compsys. Any help will be greatly appreciated. > > > The following is a copy paste of the a this question > > > < > > https://stackoverflow.com/questions/61560687/comparguments-causes-arguments-to-fail-in-certain-cases > > > > > > I asked on stack overflow: > > > > > > > Thanks for the cross-reference. > > > > > *1. The script is invoked directly (e.g. ~/script.py)* > > > > > > *2. The script is invoked through python (e.g. python script.py)* > > > > > > *3. The script is invoked as a python module (e.g. python -m script)* > > > > > > > > > *I have so far managed to successfully handle the first case, but the > > > second case fails to retrieve any completions. Using zsh completion debug > > > log I was able to see where things went wrong:* > > ⋮ > > > *EDIT:* > > > > > > *In order to temporarily bypass the problem I tried adding a shift words > > > before calling _arguments. This caused comparguments to succeed (!), but > > > still causes _arguments to fail with a no arguments message later on.* > > > > > > *I added the log for this case to the gist linked above.* > > > > Well, for starters, if you «shift words», you should decrement CURRENT > > as well. > > > > However, _python uses the «*::…» form of an _arguments optspec, > > which should take care of $words/$NUMERIC for you. I assume the reason > > it didn't is that you used «compdef -p». Try -P instead? If I'm not > > mistaken, that would also handle «python … script.py <TAB>» for you > > (where the ellipsis stands for python's --options). > > > > Cheers, > > > > Daniel > > > > -- Dan Arad dan1994@gmail.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases 2020-05-04 17:13 ` Fwd: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases Dan Arad @ 2020-05-05 16:49 ` Daniel Shahaf 2020-05-07 19:24 ` Dan Arad 0 siblings, 1 reply; 10+ messages in thread From: Daniel Shahaf @ 2020-05-05 16:49 UTC (permalink / raw) To: Dan Arad; +Cc: zsh-users > > > Well, for starters, if you «shift words», you should decrement CURRENT > > > as well. > > > > > > However, _python uses the «*::…» form of an _arguments optspec, > > > which should take care of $words/$NUMERIC for you. I assume the reason > > > it didn't is that you used «compdef -p». Try -P instead? If I'm not > > > mistaken, that would also handle «python … script.py <TAB>» for you > > > (where the ellipsis stands for python's --options). > > > > > > I tested the 4 possible options, and found that switching -p with -P > > doesn't change anything, but shifting words and decrementing CURRENT > > works! But... I feel like it's a hack I'm not supposed to do > > (correct me if I'm wrong). Those parameters are documented, and manipulated by a number of functions (including _python, as it happens), so you _are_ using the API. See also the 'compset' builtin. However, as mentioned, I suspect that hardcoding a skip/decrement of one element won't DTRT if python is invoked with options (e.g., «python --foo script.py <TAB>») — not unless you reimplement _python's parsing of those options and any arguments to them. > > I now tried using the BUFFER variable instead of words (since it remains > > unchanged), but found that it also has caveats: The BUFFER variable > > contains the raw value of the line, whereas the words array contains the > > words after expansions (I think that's the right term). Partly. $words has undergone alias expansion, but not further expansions. In particular, the words are still quoted. > > To demonstrate this I tried running again completion for `python ps,py` > > with `#compdef -P *`, and printing out BUFFER and words. The debug log > > (and my terminal) showed the following: > > First of all _python was called and dispatched my script which printed > > `BUFFER=python ps.py` and `words=ps.py`. That's what I'd expect. > > After _python returned, my script was invoked again and printed > > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3). I'm not sure offhand where this comes from. $context, $funcstack, et al may have clues. > > I do think the right solution is to be dispatched by _python with > > `#compdef -P *.py`, but to be able to run I need the words array as > > seen by _python Once an element is shifted off $words, it's lost forever (free() is called on it), so there'll be no way to access a "previous" value of $words unless a copy had been made somewhere. Moreover, consider that python may be itself wrapped by some precommand: for example, «env X=Y python», «sudo -u foo python», or «ssh foo python». You might try looking up the place that originally populates $words and making it stash a copy of $words in another, read-only array; then you'll be able to use «compset -P» and access that array. (Compare vcs_info's use of *_orig keys in ${hook_com}.) > > or a to understand how BUFFER can be expanded/converted to its correct > > form. If you have any input on this, I would very much appreciate > > it. Something along these lines, I think: local -a words2=( ${(z):-"$PREBUFFER${LBUFFER}x"} ) words2[-1]=${words2[-1]%x} words2[1,${words2[(I)(;|&&|…)]}]=() To understand the second line, see addx() in Src/Zle/zle_tricky.c. With that out of the way, the first line's a textbook use of the ${(z)} tokenizer. The third line just throws out everything until the start of the current _simple command_ (see zshmisc(1)) — or, at least, it will once the ellipsis is filled out. Caveats: that snippet doesn't do alias expansion, doesn't try to deal with ${RBUFFER}, doesn't try to deal with SHORT_LOOPS, doesn't try to deal with precommands and precommand assignments, should be changed to use zZ+c+ if the INTERACTIVE_COMMENTS option is set… Although it's a plugin that I worked on myself, I should probably mention that zsh-syntax-highlighting has faced the same problem and you and implemented a solution to it, which handles a number of the above caveats. > There's also a third approach: instead of trying to run the binary, make it stash the output in a well-known location. For example, /usr/bin/foo could store the output in /usr/share/argcomplete/foo. (I've always wanted to standardize _some_ solution to this problem; I'm tired of writing completion functions by hand…) Cheers, Daniel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases 2020-05-05 16:49 ` Daniel Shahaf @ 2020-05-07 19:24 ` Dan Arad 2020-05-07 20:40 ` Daniel Shahaf 0 siblings, 1 reply; 10+ messages in thread From: Dan Arad @ 2020-05-07 19:24 UTC (permalink / raw) To: Daniel Shahaf, zsh-users [-- Attachment #1: Type: text/plain, Size: 6176 bytes --] > > > > After _python returned, my script was invoked again and printed > > > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3). > > I'm not sure offhand where this comes from. $context, $funcstack, et al > may have clues. I may have neglected to say that I aliased `python` to `python3`, so it makes sense since you said that words is the result of alias expansion. You might try looking up the place that originally populates $words and > making it stash a copy of $words in another, read-only array; then you'll > be able to use «compset -P» and access that array. (Compare vcs_info's > use of *_orig keys in ${hook_com}.) I tried looking at things related to `vcs_info`, and I saw they were outside of the Completion directory. I was able to find the place where the `*_orig` keys are set, but couldn't find where they are used. It would be nice to get some context (or reference) about how this part of zsh connects to compsys, and how variables like `hook_com` become available to completion functions. > Something along these lines, I think: > > local -a words2=( ${(z):-"$PREBUFFER${LBUFFER}x"} ) > words2[-1]=${words2[-1]%x} > words2[1,${words2[(I)(;|&&|…)]}]=() A lot for me to unpack here and in the paragraphs that follow. I'll try to understand as much as I can, but will probably ask about it later. Thanks! Dan On Tue, May 5, 2020 at 7:49 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > > > > Well, for starters, if you «shift words», you should decrement > CURRENT > > > > as well. > > > > > > > > However, _python uses the «*::…» form of an _arguments optspec, > > > > which should take care of $words/$NUMERIC for you. I assume the > reason > > > > it didn't is that you used «compdef -p». Try -P instead? If I'm not > > > > mistaken, that would also handle «python … script.py <TAB>» for you > > > > (where the ellipsis stands for python's --options). > > > > > > > > > I tested the 4 possible options, and found that switching -p with -P > > > doesn't change anything, but shifting words and decrementing CURRENT > > > works! But... I feel like it's a hack I'm not supposed to do > > > (correct me if I'm wrong). > > Those parameters are documented, and manipulated by a number of > functions (including _python, as it happens), so you _are_ using the API. > See also the 'compset' builtin. > > However, as mentioned, I suspect that hardcoding a skip/decrement of one > element won't DTRT if python is invoked with options (e.g., «python > --foo script.py <TAB>») — not unless you reimplement _python's parsing > of those options and any arguments to them. > > > > I now tried using the BUFFER variable instead of words (since it > remains > > > unchanged), but found that it also has caveats: The BUFFER variable > > > contains the raw value of the line, whereas the words array contains > the > > > words after expansions (I think that's the right term). > > Partly. $words has undergone alias expansion, but not further > expansions. In particular, the words are still quoted. > > > > To demonstrate this I tried running again completion for `python ps,py` > > > with `#compdef -P *`, and printing out BUFFER and words. The debug > log > > > (and my terminal) showed the following: > > > First of all _python was called and dispatched my script which printed > > > `BUFFER=python ps.py` and `words=ps.py`. > > That's what I'd expect. > > > > After _python returned, my script was invoked again and printed > > > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3). > > I'm not sure offhand where this comes from. $context, $funcstack, et al > may have clues. > > > > I do think the right solution is to be dispatched by _python with > > > `#compdef -P *.py`, but to be able to run I need the words array as > > > seen by _python > > Once an element is shifted off $words, it's lost forever (free() is > called on it), so there'll be no way to access a "previous" value of $words > unless a copy had been made somewhere. > > Moreover, consider that python may be itself wrapped by some precommand: > for example, «env X=Y python», «sudo -u foo python», or «ssh foo python». > > You might try looking up the place that originally populates $words and > making it stash a copy of $words in another, read-only array; then you'll > be able to use «compset -P» and access that array. (Compare vcs_info's > use of *_orig keys in ${hook_com}.) > > > > or a to understand how BUFFER can be expanded/converted to its > correct > > > form. If you have any input on this, I would very much appreciate > > > it. > > Something along these lines, I think: > > local -a words2=( ${(z):-"$PREBUFFER${LBUFFER}x"} ) > words2[-1]=${words2[-1]%x} > words2[1,${words2[(I)(;|&&|…)]}]=() > > To understand the second line, see addx() in Src/Zle/zle_tricky.c. With > that out of the way, the first line's a textbook use of the ${(z)} > tokenizer. The third line just throws out everything until the start of > the current _simple command_ (see zshmisc(1)) — or, at least, it will > once the ellipsis is filled out. > > Caveats: that snippet doesn't do alias expansion, doesn't try to deal > with ${RBUFFER}, doesn't try to deal with SHORT_LOOPS, doesn't try to > deal with precommands and precommand assignments, should be changed to > use zZ+c+ if the INTERACTIVE_COMMENTS option is set… > > Although it's a plugin that I worked on myself, I should probably > mention that zsh-syntax-highlighting has faced the same problem and you > and implemented a solution to it, which handles a number of the above > caveats. > > > > > There's also a third approach: instead of trying to run the binary, make > it stash the output in a well-known location. For example, /usr/bin/foo > could store the output in /usr/share/argcomplete/foo. (I've always > wanted to standardize _some_ solution to this problem; I'm tired of > writing completion functions by hand…) > > Cheers, > > Daniel > -- Dan Arad dan1994@gmail.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases 2020-05-07 19:24 ` Dan Arad @ 2020-05-07 20:40 ` Daniel Shahaf 2020-05-15 14:39 ` Dan Arad 0 siblings, 1 reply; 10+ messages in thread From: Daniel Shahaf @ 2020-05-07 20:40 UTC (permalink / raw) To: Dan Arad; +Cc: zsh-users > > > > After _python returned, my script was invoked again and printed > > > > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3). > > > > I'm not sure offhand where this comes from. $context, $funcstack, et al > > may have clues. > > > I may have neglected to say that I aliased `python` to `python3`, so it > makes sense since you said that words is the result of alias expansion. Yes, that'd be it. > > You might try looking up the place that originally populates $words and > > making it stash a copy of $words in another, read-only array; then you'll > > be able to use «compset -P» and access that array. (Compare vcs_info's > > use of *_orig keys in ${hook_com}.) > > > I tried looking at things related to `vcs_info`, and I saw they were > outside of the Completion directory. > I was able to find the place where the `*_orig` keys are set, but couldn't > find where they are used. > It would be nice to get some context (or reference) about how this part of > zsh connects to compsys, and how variables like `hook_com` become available > to completion functions. vcs_info has nothing to do with compsys. Sorry for the unclarity. *_orig variables would be used by hooks in people's dotfiles. There are examples of hooks in Misc/vcs_info-examples, though they don't access the *_orig keys. I was trying to explain by way of analogy: just like vcs_info sets ${hook_com[revision_orig]}, in order that if two hooks are installed the later one be able to access the original value even if the earlier one had modified ${hook_com[revision]}, so I imagined a ${words_orig} array, to let your completion function access the value of $words before the _arguments call in _python shifted some elements off it. > > Something along these lines, I think: > > > > local -a words2=( ${(z):-"$PREBUFFER${LBUFFER}x"} ) > > words2[-1]=${words2[-1]%x} > > words2[1,${words2[(I)(;|&&|…)]}]=() > > > A lot for me to unpack here and in the paragraphs that follow. > I'll try to understand as much as I can, but will probably ask about it > later. Sure, feel free. «${foo[(I)bar]}» expands to an index into the array $foo; which index is determined by the pattern «bar». (You can get quick help by tab completing after the opening round parenthesis; the full description of ${[(I)]} is in the manual.) Do consider the other two alternatives, though. Just because it's _possible_ to reimplement the $BUFFER-to-$words transformation doesn't imply that's the best way forward. Cheers, Daniel > Thanks! > Dan > > On Tue, May 5, 2020 at 7:49 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > > > > > > Well, for starters, if you «shift words», you should decrement > > CURRENT > > > > > as well. > > > > > > > > > > However, _python uses the «*::…» form of an _arguments optspec, > > > > > which should take care of $words/$NUMERIC for you. I assume the > > reason > > > > > it didn't is that you used «compdef -p». Try -P instead? If I'm not > > > > > mistaken, that would also handle «python … script.py <TAB>» for you > > > > > (where the ellipsis stands for python's --options). > > > > > > > > > > > > I tested the 4 possible options, and found that switching -p with -P > > > > doesn't change anything, but shifting words and decrementing CURRENT > > > > works! But... I feel like it's a hack I'm not supposed to do > > > > (correct me if I'm wrong). > > > > Those parameters are documented, and manipulated by a number of > > functions (including _python, as it happens), so you _are_ using the API. > > See also the 'compset' builtin. > > > > However, as mentioned, I suspect that hardcoding a skip/decrement of one > > element won't DTRT if python is invoked with options (e.g., «python > > --foo script.py <TAB>») — not unless you reimplement _python's parsing > > of those options and any arguments to them. > > > > > > I now tried using the BUFFER variable instead of words (since it > > remains > > > > unchanged), but found that it also has caveats: The BUFFER variable > > > > contains the raw value of the line, whereas the words array contains > > the > > > > words after expansions (I think that's the right term). > > > > Partly. $words has undergone alias expansion, but not further > > expansions. In particular, the words are still quoted. > > > > > > To demonstrate this I tried running again completion for `python ps,py` > > > > with `#compdef -P *`, and printing out BUFFER and words. The debug > > log > > > > (and my terminal) showed the following: > > > > First of all _python was called and dispatched my script which printed > > > > `BUFFER=python ps.py` and `words=ps.py`. > > > > That's what I'd expect. > > > > > > After _python returned, my script was invoked again and printed > > > > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3). > > > > I'm not sure offhand where this comes from. $context, $funcstack, et al > > may have clues. > > > > > > I do think the right solution is to be dispatched by _python with > > > > `#compdef -P *.py`, but to be able to run I need the words array as > > > > seen by _python > > > > Once an element is shifted off $words, it's lost forever (free() is > > called on it), so there'll be no way to access a "previous" value of $words > > unless a copy had been made somewhere. > > > > Moreover, consider that python may be itself wrapped by some precommand: > > for example, «env X=Y python», «sudo -u foo python», or «ssh foo python». > > > > You might try looking up the place that originally populates $words and > > making it stash a copy of $words in another, read-only array; then you'll > > be able to use «compset -P» and access that array. (Compare vcs_info's > > use of *_orig keys in ${hook_com}.) > > > > > > or a to understand how BUFFER can be expanded/converted to its > > correct > > > > form. If you have any input on this, I would very much appreciate > > > > it. > > > > Something along these lines, I think: > > > > local -a words2=( ${(z):-"$PREBUFFER${LBUFFER}x"} ) > > words2[-1]=${words2[-1]%x} > > words2[1,${words2[(I)(;|&&|…)]}]=() > > > > To understand the second line, see addx() in Src/Zle/zle_tricky.c. With > > that out of the way, the first line's a textbook use of the ${(z)} > > tokenizer. The third line just throws out everything until the start of > > the current _simple command_ (see zshmisc(1)) — or, at least, it will > > once the ellipsis is filled out. > > > > Caveats: that snippet doesn't do alias expansion, doesn't try to deal > > with ${RBUFFER}, doesn't try to deal with SHORT_LOOPS, doesn't try to > > deal with precommands and precommand assignments, should be changed to > > use zZ+c+ if the INTERACTIVE_COMMENTS option is set… > > > > Although it's a plugin that I worked on myself, I should probably > > mention that zsh-syntax-highlighting has faced the same problem and you > > and implemented a solution to it, which handles a number of the above > > caveats. > > > > > > > > > There's also a third approach: instead of trying to run the binary, make > > it stash the output in a well-known location. For example, /usr/bin/foo > > could store the output in /usr/share/argcomplete/foo. (I've always > > wanted to standardize _some_ solution to this problem; I'm tired of > > writing completion functions by hand…) > > > > Cheers, > > > > Daniel > > > > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases 2020-05-07 20:40 ` Daniel Shahaf @ 2020-05-15 14:39 ` Dan Arad 2020-05-16 20:48 ` Daniel Shahaf 0 siblings, 1 reply; 10+ messages in thread From: Dan Arad @ 2020-05-15 14:39 UTC (permalink / raw) To: Daniel Shahaf; +Cc: zsh-users [-- Attachment #1: Type: text/plain, Size: 8817 bytes --] I got it working using the hook method: I added another script with `#compdef -first-` that saves the words array. The thing is, that now I'm faced with a different problem: When the command line is `./ps.py`, completions are generated correctly. When the command line is `python ps.py`, the actual completions are joined by the list of files in the current directory. This appears to be a byproduct of going through the `_python` auto-completion script. I was wondering if there is a way to remove existing completion matches before adding my own to make sure I provide only correct matches. Also, might this be a bug in the `_python` completion script? Thanks for all the help, Dan On Thu, May 7, 2020 at 11:40 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > > > > > After _python returned, my script was invoked again and printed > > > > > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3). > > > > > > I'm not sure offhand where this comes from. $context, $funcstack, et > al > > > may have clues. > > > > > > I may have neglected to say that I aliased `python` to `python3`, so it > > makes sense since you said that words is the result of alias expansion. > > Yes, that'd be it. > > > > You might try looking up the place that originally populates $words and > > > making it stash a copy of $words in another, read-only array; then > you'll > > > be able to use «compset -P» and access that array. (Compare vcs_info's > > > use of *_orig keys in ${hook_com}.) > > > > > > I tried looking at things related to `vcs_info`, and I saw they were > > outside of the Completion directory. > > I was able to find the place where the `*_orig` keys are set, but > couldn't > > find where they are used. > > It would be nice to get some context (or reference) about how this part > of > > zsh connects to compsys, and how variables like `hook_com` become > available > > to completion functions. > > vcs_info has nothing to do with compsys. Sorry for the unclarity. > > *_orig variables would be used by hooks in people's dotfiles. There > are examples of hooks in Misc/vcs_info-examples, though they don't > access the *_orig keys. > > I was trying to explain by way of analogy: just like vcs_info sets > ${hook_com[revision_orig]}, in order that if two hooks are installed > the later one be able to access the original value even if the > earlier one had modified ${hook_com[revision]}, so I imagined > a ${words_orig} array, to let your completion function access the > value of $words before the _arguments call in _python shifted some > elements off it. > > > > Something along these lines, I think: > > > > > > local -a words2=( ${(z):-"$PREBUFFER${LBUFFER}x"} ) > > > words2[-1]=${words2[-1]%x} > > > words2[1,${words2[(I)(;|&&|…)]}]=() > > > > > > A lot for me to unpack here and in the paragraphs that follow. > > I'll try to understand as much as I can, but will probably ask about it > > later. > > Sure, feel free. > > «${foo[(I)bar]}» expands to an index into the array $foo; which index is > determined by the pattern «bar». (You can get quick help by tab > completing after the opening round parenthesis; the full description of > ${[(I)]} is in the manual.) > > Do consider the other two alternatives, though. Just because it's > _possible_ to reimplement the $BUFFER-to-$words transformation doesn't > imply that's the best way forward. > > Cheers, > > Daniel > > > Thanks! > > Dan > > > > On Tue, May 5, 2020 at 7:49 PM Daniel Shahaf <d.s@daniel.shahaf.name> > wrote: > > > > > > > > Well, for starters, if you «shift words», you should decrement > > > CURRENT > > > > > > as well. > > > > > > > > > > > > However, _python uses the «*::…» form of an _arguments optspec, > > > > > > which should take care of $words/$NUMERIC for you. I assume > the > > > reason > > > > > > it didn't is that you used «compdef -p». Try -P instead? If > I'm not > > > > > > mistaken, that would also handle «python … script.py <TAB>» for > you > > > > > > (where the ellipsis stands for python's --options). > > > > > > > > > > > > > > > I tested the 4 possible options, and found that switching -p with > -P > > > > > doesn't change anything, but shifting words and decrementing > CURRENT > > > > > works! But... I feel like it's a hack I'm not supposed to do > > > > > (correct me if I'm wrong). > > > > > > Those parameters are documented, and manipulated by a number of > > > functions (including _python, as it happens), so you _are_ using the > API. > > > See also the 'compset' builtin. > > > > > > However, as mentioned, I suspect that hardcoding a skip/decrement of > one > > > element won't DTRT if python is invoked with options (e.g., «python > > > --foo script.py <TAB>») — not unless you reimplement _python's parsing > > > of those options and any arguments to them. > > > > > > > > I now tried using the BUFFER variable instead of words (since it > > > remains > > > > > unchanged), but found that it also has caveats: The BUFFER variable > > > > > contains the raw value of the line, whereas the words array > contains > > > the > > > > > words after expansions (I think that's the right term). > > > > > > Partly. $words has undergone alias expansion, but not further > > > expansions. In particular, the words are still quoted. > > > > > > > > To demonstrate this I tried running again completion for `python > ps,py` > > > > > with `#compdef -P *`, and printing out BUFFER and words. The > debug > > > log > > > > > (and my terminal) showed the following: > > > > > First of all _python was called and dispatched my script which > printed > > > > > `BUFFER=python ps.py` and `words=ps.py`. > > > > > > That's what I'd expect. > > > > > > > > After _python returned, my script was invoked again and printed > > > > > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3). > > > > > > I'm not sure offhand where this comes from. $context, $funcstack, et > al > > > may have clues. > > > > > > > > I do think the right solution is to be dispatched by _python with > > > > > `#compdef -P *.py`, but to be able to run I need the words array as > > > > > seen by _python > > > > > > Once an element is shifted off $words, it's lost forever (free() is > > > called on it), so there'll be no way to access a "previous" value of > $words > > > unless a copy had been made somewhere. > > > > > > Moreover, consider that python may be itself wrapped by some > precommand: > > > for example, «env X=Y python», «sudo -u foo python», or «ssh foo > python». > > > > > > You might try looking up the place that originally populates $words and > > > making it stash a copy of $words in another, read-only array; then > you'll > > > be able to use «compset -P» and access that array. (Compare vcs_info's > > > use of *_orig keys in ${hook_com}.) > > > > > > > > or a to understand how BUFFER can be expanded/converted to its > > > correct > > > > > form. If you have any input on this, I would very much appreciate > > > > > it. > > > > > > Something along these lines, I think: > > > > > > local -a words2=( ${(z):-"$PREBUFFER${LBUFFER}x"} ) > > > words2[-1]=${words2[-1]%x} > > > words2[1,${words2[(I)(;|&&|…)]}]=() > > > > > > To understand the second line, see addx() in Src/Zle/zle_tricky.c. > With > > > that out of the way, the first line's a textbook use of the ${(z)} > > > tokenizer. The third line just throws out everything until the start > of > > > the current _simple command_ (see zshmisc(1)) — or, at least, it will > > > once the ellipsis is filled out. > > > > > > Caveats: that snippet doesn't do alias expansion, doesn't try to deal > > > with ${RBUFFER}, doesn't try to deal with SHORT_LOOPS, doesn't try to > > > deal with precommands and precommand assignments, should be changed to > > > use zZ+c+ if the INTERACTIVE_COMMENTS option is set… > > > > > > Although it's a plugin that I worked on myself, I should probably > > > mention that zsh-syntax-highlighting has faced the same problem and you > > > and implemented a solution to it, which handles a number of the above > > > caveats. > > > > > > > > > > > > > There's also a third approach: instead of trying to run the binary, > make > > > it stash the output in a well-known location. For example, > /usr/bin/foo > > > could store the output in /usr/share/argcomplete/foo. (I've always > > > wanted to standardize _some_ solution to this problem; I'm tired of > > > writing completion functions by hand…) > > > > > > Cheers, > > > > > > Daniel > > > > > > > > > -- Dan Arad dan1994@gmail.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases 2020-05-15 14:39 ` Dan Arad @ 2020-05-16 20:48 ` Daniel Shahaf [not found] ` <CAPPzoJDL_vpLHe5iNgiNP6desAt_SmwEHLurY0W-JN4YrwqvbA@mail.gmail.com> 0 siblings, 1 reply; 10+ messages in thread From: Daniel Shahaf @ 2020-05-16 20:48 UTC (permalink / raw) To: Dan Arad; +Cc: zsh-users Dan Arad wrote on Fri, 15 May 2020 17:39 +0300: > I got it working using the hook method: I added another script with > `#compdef -first-` that saves the words array. > Nice :) > The thing is, that now I'm faced with a different problem: > When the command line is `./ps.py`, completions are generated correctly. > When the command line is `python ps.py`, the actual completions are joined > by the list of files in the current directory. > > This appears to be a byproduct of going through the `_python` > auto-completion script. > > I was wondering if there is a way to remove existing completion matches > before adding my own to make sure I provide only correct matches. I don't think there's a way to remove already-added matches. How would that help if it existed? I assume -first- runs before _python, not after, so you should look for a way to prevent _python from running when your function has added matches. For starters, does your function return 0 when it has added matches? Or you could arrange for «python ps.py» to call your script without a -first- completion function. That should prevent _python from falling back to completing files. Additionally, overwriting -first- is not a composable approach: users who already overwrite -first- won't be able to install your completion function alongside their existing configuration. > Also, might this be a bug in the `_python` completion script? I don't see how. It's normal for completion to offer files as a fallback for unknown commands, as in «nosuchcommand <TAB>». I don't recall offhand how to disable or modify the fallback behaviour. > Thanks for all the help, > Dan You're welcome. Cheers, Daniel P.S. While I wrote the above I ran into this: diff --git a/Doc/Zsh/compsys.yo b/Doc/Zsh/compsys.yo index 98ab46d8a..8d2813c5a 100644 --- a/Doc/Zsh/compsys.yo +++ b/Doc/Zsh/compsys.yo @@ -1685,6 +1685,9 @@ example(zstyle ':completion:*' group-name '') All matches for which no group name is defined will be put in a group named tt(-default-). + +To display the group name in the output, see the tt(format) style (q.v.) +under the tt(descriptions) tag. ) kindex(group-order, completion style) item(tt(group-order))( ^ permalink raw reply [flat|nested] 10+ messages in thread
[parent not found: <CAPPzoJDL_vpLHe5iNgiNP6desAt_SmwEHLurY0W-JN4YrwqvbA@mail.gmail.com>]
* Fwd: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases [not found] ` <CAPPzoJDL_vpLHe5iNgiNP6desAt_SmwEHLurY0W-JN4YrwqvbA@mail.gmail.com> @ 2020-05-17 19:25 ` Dan Arad 2020-05-17 22:12 ` Bart Schaefer 0 siblings, 1 reply; 10+ messages in thread From: Dan Arad @ 2020-05-17 19:25 UTC (permalink / raw) To: Daniel Shahaf, zsh-users [-- Attachment #1: Type: text/plain, Size: 5602 bytes --] ---------- Forwarded message --------- From: Dan Arad <dan1994@gmail.com> Date: Sun, May 17, 2020, 11:05 Subject: Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases To: Daniel Shahaf <d.s@daniel.shahaf.name> I think you're mixing things up, so I'll try to clarify: The only thing I do in the `-first-` script is to backup `words` in another variable. Otherwise it doesn't affect the completion flow (and my other bug) at all. The `_python` script is called before my script (assuming your command line starts with python), and I actually use it to my advantage, since it updates `words` so that the script name is always the first element, making my job easier. To make things easier to understand, my main script is called `_python_script`, and the `-first-` script is called `_python_script_words_backup`. The two scenarios (starting from the first thing called by `_complete`): Given the command line `./ps.py`: `_python_script_words_backup`: Backs up `words` and returns to `_complete` `_complete`: Calls `_normal` which calls `_dispatch` which calls `_python_script` `_python_script`: Uses the first item in the `words` variable to determine the what is the script Given the command line `python ./ps.py`: `_python_script_words_backup`: Backs up `words` and returns to `_complete` `_complete`: Calls `_normal` which calls `_dispatch` which calls `_python` `_python`: Adds its own arguments, shifts `words` and calls `_normal` which calls `_dispatch` which calls `_python_script` `_python_script`: Uses the updated `words` the same as in the first scenario to determine which script is run, and adds the actual completion How would that help if it existed? I assume -first- runs before > _python, not after, so you should look for a way to prevent _python from > running when your function has added matches. For starters, does your > function return 0 when it has added matches? > So regarding what you wrote here, I add my matches after `_python` has run and not before. I also use `_compskip=all` and always return 0, but since it is after `_python` has already run, it doesn't solve this particular problem. In short, if I want to bypass `_python`, I would probably have to change my `compdef` definition from `-P *.py` to `-P *`, but then I'll have to parse everything myself again, losing the great advantage `_python` has given me. I also thought about using the `_ignore` directive, but I'm not exactly sure how to use it, or if it can even be used for this use case. And about this: Additionally, overwriting -first- is > not a composable approach: users who already overwrite -first- won't be > able to install your completion function alongside their existing > configuration. Would you propose using a different `compdef` specification for `_python_script_words_backup`? Using something like `-p *` is not right for this case, as it will be called multiple times (including after `_python` has run) and will overwrite the backup. Cheers, Dan On Sat, May 16, 2020 at 11:48 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > Dan Arad wrote on Fri, 15 May 2020 17:39 +0300: > > I got it working using the hook method: I added another script with > > `#compdef -first-` that saves the words array. > > > > Nice :) > > > The thing is, that now I'm faced with a different problem: > > When the command line is `./ps.py`, completions are generated correctly. > > When the command line is `python ps.py`, the actual completions are > joined > > by the list of files in the current directory. > > > > This appears to be a byproduct of going through the `_python` > > auto-completion script. > > > > I was wondering if there is a way to remove existing completion matches > > before adding my own to make sure I provide only correct matches. > > I don't think there's a way to remove already-added matches. > > How would that help if it existed? I assume -first- runs before > _python, not after, so you should look for a way to prevent _python from > running when your function has added matches. For starters, does your > function return 0 when it has added matches? > > Or you could arrange for «python ps.py» to call your script without > a -first- completion function. That should prevent _python from > falling back to completing files. Additionally, overwriting -first- is > not a composable approach: users who already overwrite -first- won't be > able to install your completion function alongside their existing > configuration. > > > Also, might this be a bug in the `_python` completion script? > > I don't see how. It's normal for completion to offer files as a > fallback for unknown commands, as in «nosuchcommand <TAB>». > > I don't recall offhand how to disable or modify the fallback behaviour. > > > Thanks for all the help, > > Dan > > You're welcome. > > Cheers, > > Daniel > > P.S. While I wrote the above I ran into this: > > diff --git a/Doc/Zsh/compsys.yo b/Doc/Zsh/compsys.yo > index 98ab46d8a..8d2813c5a 100644 > --- a/Doc/Zsh/compsys.yo > +++ b/Doc/Zsh/compsys.yo > @@ -1685,6 +1685,9 @@ example(zstyle ':completion:*' group-name '') > > All matches for which no group name is defined will be put in a group > named tt(-default-). > + > +To display the group name in the output, see the tt(format) style (q.v.) > +under the tt(descriptions) tag. > ) > kindex(group-order, completion style) > item(tt(group-order))( > -- Dan Arad dan1994@gmail.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases 2020-05-17 19:25 ` Fwd: " Dan Arad @ 2020-05-17 22:12 ` Bart Schaefer 2020-05-17 22:14 ` Bart Schaefer 0 siblings, 1 reply; 10+ messages in thread From: Bart Schaefer @ 2020-05-17 22:12 UTC (permalink / raw) To: Dan Arad; +Cc: Zsh Users Reordering discussion slightly ... On Sat, May 16, 2020 at 1:49 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > > Dan Arad wrote on Fri, 15 May 2020 17:39 +0300: > > The thing is, that now I'm faced with a different problem: > > When the command line is `python ps.py`, the actual completions are joined > > by the list of files in the current directory. > > > > This appears to be a byproduct of going through the `_python` > > auto-completion script. > > > > Also, might this be a bug in the `_python` completion script? > > I don't see how. It's normal for completion to offer files as a > fallback for unknown commands, as in «nosuchcommand <TAB>». This isn't happening because of an unknown command, it's because the historic behavior of _normal is to return all files when none match the specific completer. I've mentioned this before; the thought at the time was that it would be even more confusing for completion to beep failure at you when there was at least one file in the directory. > > I was wondering if there is a way to remove existing completion matches > > before adding my own to make sure I provide only correct matches. > > I don't think there's a way to remove already-added matches. Completers that want to do this sort of thing typically do a hack: Wrap the "compadd" builtin with a function that uses "builtin compadd -O array" (or "-A array") to capture the completions without actually adding them, and then later remove the wrapper and make the real compadd call. However, I don't think you need or want to do that here. On Sun, May 17, 2020 at 12:26 PM Dan Arad <dan1994@gmail.com> wrote: > > The `_python` script is called before my script (assuming your command line > starts with python), and I actually use it to my advantage, since it > updates `words` so that the script name is always the first element, making > my job easier. I think you're mis-stating this. The _python function (I'm going to be pedantic that a function is not the same as a script) is called AROUND your _python_script function. You have this call sequence correct: > Given the command line `python ./ps.py`: > `_python_script_words_backup`: Backs up `words` and returns to `_complete` > `_complete`: Calls `_normal` which calls `_dispatch` which calls `_python` > `_python`: Adds its own arguments, shifts `words` and calls `_normal` which > calls `_dispatch` which calls `_python_script` However, nothing will have been "compadd"ed by _python at the point at which it calls _normal. Anything that gets added is being added as consequence of calling _normal, not as a consequence of calling _python. > [Daniel Shahaf again in a later message]: > > you should look for a way to prevent _python from > > running when your function has added matches. For starters, does your > > function return 0 when it has added matches? > > So regarding what you wrote here, I add my matches after `_python` has run No, you add them while _python is still running, not after. > and not before. I also use `_compskip=all` That doesn't actually do very much once you are beyond -first-. It aborts the current level of _dispatch but doesn't prevent the outer _default that called _python from continuing on into -default-, which I think you'll find to be the place your extra matches are coming from. > and always return 0, but since > it is after `_python` has already run, it doesn't solve this particular > problem. What you would need here is for _python to set _compskip, because it's local to each level of _dispatch. > > Additionally, overwriting -first- is > > not a composable approach: users who already overwrite -first- won't be > > able to install your completion function alongside their existing > > configuration. > > Would you propose using a different `compdef` specification for > `_python_script_words_backup`? No, I'd suggest using a different compdef for "python" itself. E.g.: _python_or_script() { _python_script_words_backup _python "$@" && _compskip=all } compdef _python_or_script python (and remove your -first- compdef). ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases 2020-05-17 22:12 ` Bart Schaefer @ 2020-05-17 22:14 ` Bart Schaefer 2020-05-18 6:20 ` Dan Arad 0 siblings, 1 reply; 10+ messages in thread From: Bart Schaefer @ 2020-05-17 22:14 UTC (permalink / raw) To: Dan Arad; +Cc: Zsh Users On Sun, May 17, 2020 at 3:12 PM Bart Schaefer <schaefer@brasslantern.com> wrote: > > aborts the current level of _dispatch but doesn't prevent the outer > _default that called _python from continuing on into -default-, which Typo, that should say "... the outer _dispatch that ..." ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases 2020-05-17 22:14 ` Bart Schaefer @ 2020-05-18 6:20 ` Dan Arad 0 siblings, 0 replies; 10+ messages in thread From: Dan Arad @ 2020-05-18 6:20 UTC (permalink / raw) To: Bart Schaefer; +Cc: Zsh Users [-- Attachment #1: Type: text/plain, Size: 2594 bytes --] Thanks for the great clarifications! I'll try to stick to the correct technical terms. This isn't happening because of an unknown command, it's because the > historic behavior of _normal is to return all files when none match > the specific completer. ... However, nothing will have been "compadd"ed by _python at the point at > which it calls _normal. Anything that gets added is being added as > consequence of calling _normal, not as a consequence of calling _python. I suspected as much, but I'm new to zsh and completion, so I had some hard time following the debug logs and figuring this out. _python_or_script() { > _python_script_words_backup > _python "$@" && _compskip=all > } > compdef _python_or_script python This works like a charm! The only thing I did different is to use the `#compdef -p` form at the beginning of the file so I could give the same pattern as that of the `_python` function. > and not before. I also use `_compskip=all` > > That doesn't actually do very much once you are beyond -first-. It > aborts the current level of _dispatch but doesn't prevent the outer > _default that called _python from continuing on into -default-, which > I think you'll find to be the place your extra matches are coming > from. This `_compskip` prevents the inner `_dispatch` from calling the `_python_script` function more than once. An example I've seen, is with the command line `python .vscode/ps.py` (I use the .vscode directory as a place for temporary files as it is git ignored). In this case, without `_compskip=all`, the inner `_dispatch` calls `_python_script` once with `service=.vscode/ps.py` and once with `service=ps.py`. In my case this doubles the time to produce completions which may be critical depending on the way the user has written his script. To conclude, I would really like to thank everybody who took part in this discussion. This is my first open source project, and I was really hoping to provide a good user experience. It was a great experience for me to have such a productive conversation, and I really appreciate the time you've put into providing detailed answers in a welcoming manner. Have a great day! Dan On Mon, May 18, 2020 at 1:14 AM Bart Schaefer <schaefer@brasslantern.com> wrote: > On Sun, May 17, 2020 at 3:12 PM Bart Schaefer <schaefer@brasslantern.com> > wrote: > > > > aborts the current level of _dispatch but doesn't prevent the outer > > _default that called _python from continuing on into -default-, which > > Typo, that should say "... the outer _dispatch that ..." > -- Dan Arad dan1994@gmail.com ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2020-05-18 6:21 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <CAPPzoJDB3jB7pHUfE7UjY09OHk+r_pgLeYxuaoXhMFGHf-3UCQ@mail.gmail.com> [not found] ` <20200503165802.6540ad48@tarpaulin.shahaf.local2> [not found] ` <CAPPzoJDyK35bSMTyYKovEbRZNhDpKs1aeY2ZRjzBTxgdgCuGww@mail.gmail.com> [not found] ` <20200504131830.3572e317@tarpaulin.shahaf.local2> 2020-05-04 17:13 ` Fwd: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases Dan Arad 2020-05-05 16:49 ` Daniel Shahaf 2020-05-07 19:24 ` Dan Arad 2020-05-07 20:40 ` Daniel Shahaf 2020-05-15 14:39 ` Dan Arad 2020-05-16 20:48 ` Daniel Shahaf [not found] ` <CAPPzoJDL_vpLHe5iNgiNP6desAt_SmwEHLurY0W-JN4YrwqvbA@mail.gmail.com> 2020-05-17 19:25 ` Fwd: " Dan Arad 2020-05-17 22:12 ` Bart Schaefer 2020-05-17 22:14 ` Bart Schaefer 2020-05-18 6:20 ` Dan Arad
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).