menu-select interactive mode

menu-select interactive mode

[Mikael Magnusson]

We were talking on irc about this, and I couldn't get it to work.
Turns out if you just set
zstyle ':completion:*' menu interactive
and invoke the menu-select widget, nothing happens. What you have to
do is either
zstyle ':completion:*' menu select interactive
but then you always get menu selection which you might not want (I
don't), so I ended up with this instead
zle -C menu-select-interactive menu-complete _generic
zstyle ':completion:menu-select-interactive:*' menu select interactive
bindkey "^[m"  menu-select-interactive

(^[m was previously set to menu-select)

Is this intended or a bug? The _main_complete code leading up to
setting MENUMODE isn't very obvious to me.

-- 
Mikael Magnusson

Re: menu-select interactive mode

[Mikael Magnusson]

On 9 April 2011 16:56, Mikael Magnusson <mikachu@gmail.com> wrote:
> We were talking on irc about this, and I couldn't get it to work.
> Turns out if you just set
> zstyle ':completion:*' menu interactive
> and invoke the menu-select widget, nothing happens. What you have to
> do is either
> zstyle ':completion:*' menu select interactive
> but then you always get menu selection which you might not want (I
> don't), so I ended up with this instead
> zle -C menu-select-interactive menu-complete _generic
> zstyle ':completion:menu-select-interactive:*' menu select interactive
> bindkey "^[m"  menu-select-interactive
>
> (^[m was previously set to menu-select)
>
> Is this intended or a bug? The _main_complete code leading up to
> setting MENUMODE isn't very obvious to me.

Additionally, when you do the second one, ie set both select and
interactive, the menu-select widget does invoke interactiveness, so my
guess is the code simply checks the style instead of also checking
which widget invoked it.

-- 
Mikael Magnusson

Re: menu-select interactive mode

[Bart Schaefer]

On Apr 9,  4:56pm, Mikael Magnusson wrote:
} Subject: menu-select interactive mode
}
} We were talking on irc about this, and I couldn't get it to work.
} Turns out if you just set
} zstyle ':completion:*' menu interactive
} and invoke the menu-select widget, nothing happens.

First of all, make sure you're really invoking _main_complete via the
menu-select widget, rather than invoking the builtin menu-select.  
The builtin obviously won't pay any attention to the zstyles.  In
order to get this to work correctly, you have to explicitly load the
zsh/complist module BEFORE running compinit or compinit won't replace
the builtin widget.

I have the feeling this is documented somewhere but I can't find it
just now.

The builtin normally doesn't begin selection unless you either always
do menu completion or it's invoked in a context where menu completion
would start (such as after a listing has been printed).  Not being in
the correct context would correspond to your "nothing happens" case.

If I start zsh with a .zshrc that looks like this --

autoload -Uz compinit
compinit -D
zmodload zsh/complist
zstyle ':completion:*' menu interactive
bindey '\em' menu-select

-- then pressing ESC m does nothing unless I first hit TAB and get a
listing.  However, if I .zshrc looks like THIS --

zmodload zsh/complist
autoload -Uz compinit
compinit -D
zstyle ':completion:*' menu interactive
bindey '\em' menu-select

-- then ESC m goes directly into menu selection as I would expect.

However, what it does NOT do is go directly into interactive mode.
You do need both "select" and "interactive" to get into interactive
mode, even when the widget is named menu-select.  I'm not sure if
this is intentional (more below).

} What you have to do is either
} zstyle ':completion:*' menu select interactive
} but then you always get menu selection which you might not want

With the exception of what I described above, I don't think that has
accomplished what you think it has.  I suspect that what this has done
is to cause a *different* completion widget (one that does go through
_main_complete) to assign a value to the MENUSELECT variable, which
then later is observed by the menu-select builtin, which eventually
results in the builtin doing something.

} don't), so I ended up with this instead
} zle -C menu-select-interactive menu-complete _generic
} zstyle ':completion:menu-select-interactive:*' menu select interactive
} bindkey "^[m"  menu-select-interactive

What you've done here (I think) is mimic the setup that compinit would
have done for you if the complist module was loaded before compinit
was run.

} Additionally, when you do the second one, ie set both select and
} interactive, the menu-select widget does invoke interactiveness, so my
} guess is the code simply checks the style instead of also checking
} which widget invoked it.

That's sort of the right diagnosis.  There are several possible ways
to "fix" this if it's wrong, and I don't know which one is correct.

Here's the first, which has the smallest number of side-effects:

Index: Completion/Base/Core/_main_complete
===================================================================
--- _main_complete	21 Dec 2010 16:41:14 -0000	1.12
+++ _main_complete	9 Apr 2011 19:57:11 -0000
@@ -284,7 +284,7 @@
         unset MENUSELECT
       fi
     fi
-    if [[ -n "$MENUSELECT" ]]; then
+    if [[ -n "$MENUSELECT" || "$WIDGET" = menu-select ]]; then
       if [[ -n "$_menu_style[(r)interactive*]" ]]; then
         MENUMODE=interactive
       elif [[ -n "$_menu_style[(r)search*]" ]]; then

Here's the second, which has the effect of causing MENUSELECT to be
set and thereby make the "-" line of the above diff true (do *NOT*
apply both this and the above diff):

Index: Completion/Base/Core/_main_complete
===================================================================
--- _main_complete	21 Dec 2010 16:41:14 -0000	1.12
+++ _main_complete	9 Apr 2011 19:59:46 -0000
@@ -249,7 +249,7 @@
     fi
   fi
 
-  if [[ "$compstate[insert]" = *menu* ]]; then
+  if [[ "$compstate[insert]" = *menu*  || "$WIDGET" = menu-select ]]; then
     [[ "$MENUSELECT" = 00 ]] && MENUSELECT=0
     if [[ -n "$_menu_style[(r)no-select*]" ]]; then
       unset MENUSELECT

A third way might be to change the way _last_menu_style is determined
in _setup, but that makes me nervous.  PWS, any other ideas/opinions?

Re: menu-select interactive mode

[Mikael Magnusson]

On 9 April 2011 22:12, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Apr 9,  4:56pm, Mikael Magnusson wrote:
> } Subject: menu-select interactive mode
> }
> } We were talking on irc about this, and I couldn't get it to work.
> } Turns out if you just set
> } zstyle ':completion:*' menu interactive
> } and invoke the menu-select widget, nothing happens.
>
> First of all, make sure you're really invoking _main_complete via the
> menu-select widget, rather than invoking the builtin menu-select.
> The builtin obviously won't pay any attention to the zstyles.  In
> order to get this to work correctly, you have to explicitly load the
> zsh/complist module BEFORE running compinit or compinit won't replace
> the builtin widget.

I'm not sure why I wrote "nothing happens", what I actually meant was
"nothing changes" meaning that it goes into menu-selection as usual,
but not in interactive mode.

My menu-select is a widget, if this output is to be trusted,
% echo $widgets[menu-select]
completion:.menu-select:_main_complete
And I then have bindkey '^[m' menu-select
also this,
% zle -l|grep menu-select
menu-select -C .menu-select _main_complete
which I guess is pretty much the same piece of information.

> I have the feeling this is documented somewhere but I can't find it
> just now.

If you use the menu-select widget, which is part of  the zsh/complist
module,  you should make sure that that module is loaded before the
call to compinit so that that widget is also re-defined.

> The builtin normally doesn't begin selection unless you either always
> do menu completion or it's invoked in a context where menu completion
> would start (such as after a listing has been printed).  Not being in
> the correct context would correspond to your "nothing happens" case.
>
> If I start zsh with a .zshrc that looks like this --
>
> autoload -Uz compinit
> compinit -D
> zmodload zsh/complist
> zstyle ':completion:*' menu interactive
> bindey '\em' menu-select
>
> -- then pressing ESC m does nothing unless I first hit TAB and get a
> listing.  However, if I .zshrc looks like THIS --
>
> zmodload zsh/complist
> autoload -Uz compinit
> compinit -D
> zstyle ':completion:*' menu interactive
> bindey '\em' menu-select
>
> -- then ESC m goes directly into menu selection as I would expect.
>
> However, what it does NOT do is go directly into interactive mode.
> You do need both "select" and "interactive" to get into interactive
> mode, even when the widget is named menu-select.  I'm not sure if
> this is intentional (more below).

Right, this is the case I actually have, sorry about the poor communication.

> } What you have to do is either
> } zstyle ':completion:*' menu select interactive
> } but then you always get menu selection which you might not want
>
> With the exception of what I described above, I don't think that has
> accomplished what you think it has.  I suspect that what this has done
> is to cause a *different* completion widget (one that does go through
> _main_complete) to assign a value to the MENUSELECT variable, which
> then later is observed by the menu-select builtin, which eventually
> results in the builtin doing something.

I'm pretty sure when I tried this, I just pressed alt-m before
anything else and it went into interactive selection. But then again,
maybe not.

> } don't), so I ended up with this instead
> } zle -C menu-select-interactive menu-complete _generic
> } zstyle ':completion:menu-select-interactive:*' menu select interactive
> } bindkey "^[m"  menu-select-interactive
>
> What you've done here (I think) is mimic the setup that compinit would
> have done for you if the complist module was loaded before compinit
> was run.

What I've done here is copy the recipe pws gave me at some point for
setting a different set of completion styles when using another
keybind. Originally it was for making a keybind to complete files
sorted by access time or something like that.

> } Additionally, when you do the second one, ie set both select and
> } interactive, the menu-select widget does invoke interactiveness, so my
> } guess is the code simply checks the style instead of also checking
> } which widget invoked it.
>
> That's sort of the right diagnosis.  There are several possible ways
> to "fix" this if it's wrong, and I don't know which one is correct.
>
> Here's the first, which has the smallest number of side-effects:
>
> Index: Completion/Base/Core/_main_complete
> ===================================================================
> --- _main_complete      21 Dec 2010 16:41:14 -0000      1.12
> +++ _main_complete      9 Apr 2011 19:57:11 -0000
> @@ -284,7 +284,7 @@
>         unset MENUSELECT
>       fi
>     fi
> -    if [[ -n "$MENUSELECT" ]]; then
> +    if [[ -n "$MENUSELECT" || "$WIDGET" = menu-select ]]; then
>       if [[ -n "$_menu_style[(r)interactive*]" ]]; then
>         MENUMODE=interactive
>       elif [[ -n "$_menu_style[(r)search*]" ]]; then
>
> Here's the second, which has the effect of causing MENUSELECT to be
> set and thereby make the "-" line of the above diff true (do *NOT*
> apply both this and the above diff):
>
> Index: Completion/Base/Core/_main_complete
> ===================================================================
> --- _main_complete      21 Dec 2010 16:41:14 -0000      1.12
> +++ _main_complete      9 Apr 2011 19:59:46 -0000
> @@ -249,7 +249,7 @@
>     fi
>   fi
>
> -  if [[ "$compstate[insert]" = *menu* ]]; then
> +  if [[ "$compstate[insert]" = *menu*  || "$WIDGET" = menu-select ]]; then
>     [[ "$MENUSELECT" = 00 ]] && MENUSELECT=0
>     if [[ -n "$_menu_style[(r)no-select*]" ]]; then
>       unset MENUSELECT
>
> A third way might be to change the way _last_menu_style is determined
> in _setup, but that makes me nervous.  PWS, any other ideas/opinions?

I might be getting myself in a confused state by this point, but
neither of these seemed to do it for me with just setting 'menu
interactive' and using menu-complete, I did check that I zmodload
complist first too.

btw, both MENUSELECT and MENUMODE seem to "leak" out into the main
shell environment, is that necessary to keep the state consistent
during completion? I noticed a funny effect where echo $MENU<tab>
would offer some stuff and MENUSELECT, then when I typed s and tab
again, I got no hits, because that invocation of completion ended up
unsetting it again. :)

btw#2, this whole interactive feature is very under-advertised, I've
used zsh since 2003 and even have commit access, and I never heard
about it before today.

ps Sorry if something seems out of order, I went back and forth a few
times and added things, I hope it's not too confused.

-- 
Mikael Magnusson

Re: menu-select interactive mode

[Bart Schaefer]

On Apr 9, 10:58pm, Mikael Magnusson wrote:
}
} On 9 April 2011 22:12, Bart Schaefer <schaefer@brasslantern.com> wrote:
} 
} > } Additionally, when you do the second one, ie set both select and
} > } interactive, the menu-select widget does invoke interactiveness, so my
} > } guess is the code simply checks the style instead of also checking
} > } which widget invoked it.
} >
} > That's sort of the right diagnosis.  There are several possible ways
} > to "fix" this if it's wrong, and I don't know which one is correct.
} >
} I might be getting myself in a confused state by this point, but
} neither of these seemed to do it for me with just setting 'menu
} interactive' and using menu-complete, I did check that I zmodload
} complist first too.

Using menu-complete or using menu-select?  Note that the test looks
explicitly for the menu-select widget by name.  Should it instead be
looking for menu-* as a pattern?

I've just re-tested both of these and found that the second one does
not always work, and I can't figure out what I did to convince myself
it was working consistently before.

This first one, however (excerpted again below), does it for me when
starting from zsh -f and with no other styles defined:

torch% print $ZDOTDIR
/home/schaefer/.zsh-test
torch% head ~/.zsh-test/.zshrc
zmodload zsh/complist
autoload -Uz compinit
compinit -D
zstyle ':completion:*' menu interactive
bindkey '\em' menu-select
return
torch% zstyle -L
zstyle ':completion:*' menu interactive
torch% ls
interactive: []
BUILD              config.log         Etc/               Src/
Completion/        config.modules     Functions/         stamp-h
Config/            config.modules.sh  local              stamp-h.in
config.h           config.status*     Makefile           Test/
config.h-xx        Doc/               sleep              ul*

It does, of course, also depend on there being more than one match.

} > Here's the first, which has the smallest number of side-effects:
} >
} > Index: Completion/Base/Core/_main_complete
} > ===================================================================
} > --- _main_complete      21 Dec 2010 16:41:14 -0000      1.12
} > +++ _main_complete      9 Apr 2011 19:57:11 -0000
} > @@ -284,7 +284,7 @@
} >         unset MENUSELECT
} >       fi
} >     fi
} > -    if [[ -n "$MENUSELECT" ]]; then
} > +    if [[ -n "$MENUSELECT" || "$WIDGET" = menu-select ]]; then
} >       if [[ -n "$_menu_style[(r)interactive*]" ]]; then
} >         MENUMODE=interactive
} >       elif [[ -n "$_menu_style[(r)search*]" ]]; then
} 
} btw, both MENUSELECT and MENUMODE seem to "leak" out into the main
} shell environment, is that necessary to keep the state consistent
} during completion?

It's because the complist module needs both of them set (and MENUSCROLL
if you have that style) and on any given entry to _main_complete it's
not known whether they were previously set by either the user or by
_main_complete itself maintaining state for the next pass.  That is,
_main_complete gets menu selection started but then _main_complete
returns to the top level.  The menu-select internals are then in
control and need those variables to remain set.  To clean them up
properly we'd need a hook on menu-select exiting.

Hmm, nowadays it might be possible to do it with a zle-line-init hook.
That didn't exist when this code in _main_complete was written.

} btw#2, this whole interactive feature is very under-advertised, I've
} used zsh since 2003 and even have commit access, and I never heard
} about it before today.

Zsh doesn't really have a marketing department. :-)  Volunteers?

There are still some bugs in the interactive mode.  Starting from the
"ls" above, type "c":

torch% ls c
interactive: config.[]
config.h           config.log         config.modules.sh                   
config.h-xx        config.modules     config.status*           

Now press TAB:

torch% ls config.
interactive: config.[]
config.h           config.log         config.modules.sh                   
config.h-xx        config.modules     config.status*     

Then type "l":

torch% ls config.l
interactive: config.log[]
config.log

Then press ENTER to accept the one-and-only match:

torch% ls config.log l

Finally type ENTER again:

torch% ls config.logl 
ls: config.logl: No such file or directory

-- 

Re: menu-select interactive mode

[Mikael Magnusson]

On 10 April 2011 16:48, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Apr 9, 10:58pm, Mikael Magnusson wrote:
> }
> } On 9 April 2011 22:12, Bart Schaefer <schaefer@brasslantern.com> wrote:
> }
> } > } Additionally, when you do the second one, ie set both select and
> } > } interactive, the menu-select widget does invoke interactiveness, so my
> } > } guess is the code simply checks the style instead of also checking
> } > } which widget invoked it.
> } >
> } > That's sort of the right diagnosis.  There are several possible ways
> } > to "fix" this if it's wrong, and I don't know which one is correct.
> } >
> } I might be getting myself in a confused state by this point, but
> } neither of these seemed to do it for me with just setting 'menu
> } interactive' and using menu-complete, I did check that I zmodload
> } complist first too.

Okay, when I try it now it seems the first patch does work, so I must
have been confused yesterday. (A curious difference between this and
using my menu-select-interactive widget: pressing ctrl-c removes the
list with my widget but not with menu-select, subsequent presses of
menu-select then just cycle through the matches instead of going into
menu-selection again, but let's leave this for later :).)

> Using menu-complete or using menu-select?  Note that the test looks
> explicitly for the menu-select widget by name.  Should it instead be
> looking for menu-* as a pattern?

Intuitively I feel like the very place that checks if the 'menu' style
contains 'select' should also check if the widget is called
'menu-select'. menu-complete doesn't imply selection, does it? (I
admit all these slightly different terms are a bit confusing to me.)
This is not to say that that would be better than your patch, I
obviously don't know what I'm talking about.

> } btw, both MENUSELECT and MENUMODE seem to "leak" out into the main
> } shell environment, is that necessary to keep the state consistent
> } during completion?
>
> It's because the complist module needs both of them set (and MENUSCROLL
> if you have that style) and on any given entry to _main_complete it's
> not known whether they were previously set by either the user or by
> _main_complete itself maintaining state for the next pass.  That is,
> _main_complete gets menu selection started but then _main_complete
> returns to the top level.  The menu-select internals are then in
> control and need those variables to remain set.  To clean them up
> properly we'd need a hook on menu-select exiting.
>
> Hmm, nowadays it might be possible to do it with a zle-line-init hook.
> That didn't exist when this code in _main_complete was written.
>
> } btw#2, this whole interactive feature is very under-advertised, I've
> } used zsh since 2003 and even have commit access, and I never heard
> } about it before today.
>
> Zsh doesn't really have a marketing department. :-)  Volunteers?
>
> There are still some bugs in the interactive mode.  Starting from the
> "ls" above, type "c":
>
> torch% ls c
> interactive: config.[]
> config.h           config.log         config.modules.sh
> config.h-xx        config.modules     config.status*
>
> Now press TAB:
>
> torch% ls config.
> interactive: config.[]
> config.h           config.log         config.modules.sh
> config.h-xx        config.modules     config.status*
>
> Then type "l":
>
> torch% ls config.l
> interactive: config.log[]
> config.log
>
> Then press ENTER to accept the one-and-only match:
>
> torch% ls config.log l
>
> Finally type ENTER again:
>
> torch% ls config.logl
> ls: config.logl: No such file or directory

Yeah, I noticed a couple of things too, if you complete some levels of
directories and accept, then undo it all, you're left with the bits
you typed manually, only the completion-inserted parts are removed.
You can also not use more complex patterns than a simple * or ?, as
soon as you type a non-matching glob it drops you back into the main
shell. Even if you manage to type a ( or | it will just start
completing commands or worse instead of files :). I don't know if this
is a bug or not, the documentation doesn't really say what you can
type in it. Ie if i have files foo and bar and type (foo|bar) into the
interactive menu completion it will tell me no matches, then when I
press enter to accept my command it does match the files.

-- 
Mikael Magnusson

Re: menu-select interactive mode

[Bart Schaefer]

On Apr 10,  5:37pm, Mikael Magnusson wrote:
}
} Intuitively I feel like the very place that checks if the 'menu' style
} contains 'select' should also check if the widget is called
} 'menu-select'.

The only other possibility is a few lines farther up, like so:

Index: Completion/Base/Core/_main_complete
===================================================================
RCS file: /extra/cvsroot/zsh/zsh-4.0/Completion/Base/Core/_main_complete,v
retrieving revision 1.12
diff -c -r1.12 _main_complete
--- _main_complete	21 Dec 2010 16:41:14 -0000	1.12
+++ _main_complete	10 Apr 2011 18:22:22 -0000
@@ -260,7 +260,7 @@
       fi
     fi
     if [[ "$MENUSELECT" != 00 ]]; then
-      sel=( "${(@M)_menu_style:#select*}" )
+      sel=( "${(@M)_menu_style:#select*}" ${(M)WIDGET#menu-select} )
 
       if (( $#sel )); then
 	min=9999999

That has the effect of setting MENUSELECT=0 as if the style had been
set, but the side-effect is that MENUSELECT *remains* set to 0 after
_main_complete finishes, which my other patch doesn't do (both leave
MENUMODE set).

} menu-complete doesn't imply selection, does it?

No, but you wrote "using menu-complete" so I wanted to be sure what
behavior you were expecting.

} > There are still some bugs in the interactive mode.

I just managed to hit a really alarming one.  Starting zsh -f with
the ESC-m binding we've been discussing:

torch% MENUSELECT=0 MENUMODE=interactive

Next hit TAB after ls:

torch% ls
BUILD              config.log         Etc/               Src/             
Completion/        config.modules     Functions/         stamp-h          
Config/            config.modules.sh  local              stamp-h.in       
config.h           config.status*     Makefile           Test/            
config.h-xx        Doc/               sleep              ul*              

Now hit TAB again to enter menu completion:

torch% ls BUILD
BUILD              config.log         Etc/               Src/             
Completion/        config.modules     Functions/         stamp-h          
Config/            config.modules.sh  local              stamp-h.in       
config.h           config.status*     Makefile           Test/            
config.h-xx        Doc/               sleep              ul*              

And TAB again:

torch% ls Completion/
BUILD              config.log         Etc/               Src/
Completion/        config.modules     Functions/         stamp-h
Config/            config.modules.sh  local              stamp-h.in
config.h           config.status*     Makefile           Test/
config.h-xx        Doc/               sleep              ul*

Now hit ESC-m to enter menu-selection.  I'm not even going to attempt
to copy-paste the result, it's a screenful of garbage memory contents.

} Yeah, I noticed a couple of things too, if you complete some levels of
} directories and accept, then undo it all, you're left with the bits
} you typed manually, only the completion-inserted parts are removed.

Hmm.  I don't know whether the right thing would be to add those to the
undo system or something else.  Internally menu-select is handling it's
own undos; I'm not familiar enough with how the undo entries are kept
to know if extraneous entries would confuse things.

} You can also not use more complex patterns than a simple * or ?, as
} soon as you type a non-matching glob it drops you back into the main
} shell.

Yes, menu-select exits any time the number of matching completions is
zero after any single key press, and an incomplete glob pattern is
one such circumstance.

} Even if you manage to type a ( or | it will just start
} completing commands or worse instead of files :).

Actually for me it starts completing glob qualifiers.

} I don't know if this is a bug or not, the documentation doesn't really
} say what you can type in it.

It's pretty much exactly as if every alternate keystroke was a TAB,
and as soon as one of those returns no matches it bails out.

Re: menu-select interactive mode

[Mikael Magnusson]

On 10 April 2011 20:50, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Apr 10,  5:37pm, Mikael Magnusson wrote:
> }
> } Intuitively I feel like the very place that checks if the 'menu' style
> } contains 'select' should also check if the widget is called
> } 'menu-select'.
>
> The only other possibility is a few lines farther up, like so:
>
> Index: Completion/Base/Core/_main_complete
> ===================================================================
> RCS file: /extra/cvsroot/zsh/zsh-4.0/Completion/Base/Core/_main_complete,v
> retrieving revision 1.12
> diff -c -r1.12 _main_complete
> --- _main_complete      21 Dec 2010 16:41:14 -0000      1.12
> +++ _main_complete      10 Apr 2011 18:22:22 -0000
> @@ -260,7 +260,7 @@
>       fi
>     fi
>     if [[ "$MENUSELECT" != 00 ]]; then
> -      sel=( "${(@M)_menu_style:#select*}" )
> +      sel=( "${(@M)_menu_style:#select*}" ${(M)WIDGET#menu-select} )
>
>       if (( $#sel )); then
>        min=9999999
>
> That has the effect of setting MENUSELECT=0 as if the style had been
> set, but the side-effect is that MENUSELECT *remains* set to 0 after
> _main_complete finishes, which my other patch doesn't do (both leave
> MENUMODE set).
>
> } menu-complete doesn't imply selection, does it?
>
> No, but you wrote "using menu-complete" so I wanted to be sure what
> behavior you were expecting.

Oops, I must have meant menu-select, but if I didn't it clearly
explains why it didn't work yesterday.

> } > There are still some bugs in the interactive mode.
> }
> } Yeah, I noticed a couple of things too, if you complete some levels of
> } directories and accept, then undo it all, you're left with the bits
> } you typed manually, only the completion-inserted parts are removed.
>
> Hmm.  I don't know whether the right thing would be to add those to the
> undo system or something else.  Internally menu-select is handling it's
> own undos; I'm not familiar enough with how the undo entries are kept
> to know if extraneous entries would confuse things.

Just to clarify, I exited menu-select before doing the undo stuff.
What I sort of expected to happen was for the undo to remove the whole
thing in one step.

> } You can also not use more complex patterns than a simple * or ?, as
> } soon as you type a non-matching glob it drops you back into the main
> } shell.
>
> Yes, menu-select exits any time the number of matching completions is
> zero after any single key press, and an incomplete glob pattern is
> one such circumstance.

This strikes me as less than useful, I might have a look at it :).

> } Even if you manage to type a ( or | it will just start
> } completing commands or worse instead of files :).
>
> Actually for me it starts completing glob qualifiers.

That was "or worse".

Another thing I noticed was that the character to the left of the
cursor (or is that under the cursor?) gets suffix-highlighted when
interactive menu-completion starts, even a space that isn't part of
the resulting completion (this won't be visible with the default bold
style since a bold space is very similar to a regular space). This
then goes away when you start doing stuff. To see it, you have to make
sure no new text is inserted by starting the menu-selection.

-- 
Mikael Magnusson

Re: menu-select interactive mode

[Jun T.]

Sorry for a log post.

At 11:50 -0700 11/04/10, Bart Schaefer wrote:
>} > There are still some bugs in the interactive mode.
>
>I just managed to hit a really alarming one.
> .... (snip)
>Now hit ESC-m to enter menu-selection.  I'm not even going to attempt
>to copy-paste the result, it's a screenful of garbage memory contents.

I did some analysis on this problem.
I think I found what is causing the problem, but do not know how to fix it.
(The zsh is the latest CVS with ./configure --enable-zsh-debug).

[1] For easier reference, the problem is shown here again:

% ls                    # in a working directory
aabb aabc aacc foo      #   which contains these 4 files
% cat foo               # foo contains the setup for completion
zmodload zsh/complist
autoload -Uz compinit
compinit
zstyle ':completion:*' menu select=off interactive
bindkey '\em' menu-select
% zsh -f
iMac% source foo
iMac%                   # gdb is attatched to this zsh: see [2] below.

----- Type "ls <TAB>"

iMac% ls
aabb  aabc  aacc  foo   # aoutolist

----- Type <TAB>

iMac% ls aabb           # automenu
aabb  aabc  aacc  foo 

----- Type <TAB> again

iMac% ls aabc
aabb  aabc  aacc  foo 

----- Type ESC-m

iMac% ./aabc            # cursor is on the second "a"
interactive: []
aabb  aabc  aacc  foo

Note that the command line is broken.

[2] It seems the problem is that the memory allocated for orgline is
freed while origline is still in use.

Attatch gdb to the above "zsh -f" (after "source foo", bafore "ls TAB"):

% ps | grep 'zsh -f'
56223 ttys000    0:00.60 zsh -f
% gdb /usr/local/bin/zsh 56223
GNU gdb 6.3.50-20050815 (Apple version gdb-1515) (Sat Jan 15 08:33:48 UTC 2011)
....
Attaching to program: `/usr/local/bin/zsh', process 56223.
Reading symbols for shared libraries +++....... done
0x00007fff85e9b984 in read ()
(gdb) p origline
$1 = 0x0

## origline is not allocated yet.
## set watchpoint for origline, and continue:

(gdb) watch origline 
Hardware watchpoint 1: origline
(gdb) cont
Continuing.

----- Type "ls <TAB>" (in the zsh being debugged)

Hardware watchpoint 1: origline

Old value = 0x0
New value = 0x1008202a8 "ls "
docomplete (lst=4) at zle_tricky.c:633
633	    origcs = zlemetacs;
(gdb)

## origline is allocated at zle_tricky.c:632 (in docomplete())
##     origline = dupstring(zlemetaline);
##
## dupsrting() uses zhalloc() to allocate origline in the heap.
##
## set watchpoint for the address 0x1008202a8 and continue:

(gdb) watch *(char*)0x1008202a8
Hardware watchpoint 2: *(char *) 4303487656
(gdb) cont
Continuing.

----- Type <TAB>
----- Type <TAB> again

Reading symbols for shared libraries . done
Hardware watchpoint 2: *(char *) 4303487656

Old value = 108 'l'
New value = 46 '.'
0x00007fffffe007db in __memcpy ()
(gdb) bt
#0  0x00007fffffe007db in __memcpy ()
#1  0x00007fff85ea3225 in __sfvwrite ()
#2  0x00007fff85ea2cfc in __vfprintf ()
#3  0x00007fff85f55ea8 in vsprintf ()
#4  0x00007fff85ed2947 in __sprintf_chk ()
#5  0x00000001001bd9c2 in do_single (m=0x10022f6f0) at compresult.c:1094
#6  0x00000001001be128 in do_menucmp (lst=4) at compresult.c:1224
#7  0x00000001001a81ef in before_complete (dummy=0x1001869d0, lst=0x7fff5fbff1ac) at compcore.c:473
#8  0x0000000100050239 in runhookdef (h=0x1001869d0, d=0x7fff5fbff1ac) at module.c:996
#9  0x0000000100167058 in docomplete (lst=4) at zle_tricky.c:617
#10 0x0000000100166663 in expandorcomplete (args=0x100187360) at zle_tricky.c:315
#11 0x00000001001662a5 in completecall (args=0x100187360) at zle_tricky.c:208
#12 0x0000000100154e09 in execzlefunc (func=0x100183938, args=0x100187360, set_bindk=0) at zle_main.c:1311
#13 0x000000010015435e in zlecore () at zle_main.c:1058
#14 0x0000000100154a16 in zleread (lp=0x1000b8970, rp=0x0, flags=3, context=0) at zle_main.c:1219
#15 0x0000000100156b3d in zle_main_entry (cmd=1, ap=0x7fff5fbff560) at zle_main.c:1874
#16 0x000000010003a70d in zleentry (cmd=1) at init.c:1374
#17 0x000000010003b439 in inputline () at input.c:281
#18 0x000000010003b272 in ingetc () at input.c:217
#19 0x000000010002efc4 in ihgetc () at hist.c:279
#20 0x0000000100043ede in gettok () at lex.c:717
#21 0x000000010004352b in zshlex () at lex.c:395
#22 0x00000001000626a2 in parse_event () at parse.c:451
#23 0x00000001000371ec in loop (toplevel=1, justonce=0) at init.c:132
#24 0x000000010003ac8a in zsh_main (argc=2, argv=0x7fff5fbff900) at init.c:1528
#25 0x0000000100000ce7 in main (argc=2, argv=0x7fff5fbff900) at ./main.c:93
(gdb)

## Here is the problem.
## The address 0x1008202a8 has been already freed, and it is now allocated
## to another variable p at compresult.c:1092 (in do_single())
##             p = (char *) zhalloc(strlen(prpre) + strlen(str) +
##
## (it is not shown in this log, but I confirmed the value of p is 0x1008202a8)
##
## But the origline is still in use.
## Set the read-watchpoint for it, and continue:

(gdb) rwatch origline
Hardware read watchpoint 3: origline
(gdb) cont
Continuing.

----- Type ESC-m

Hardware read watchpoint 3: origline

Value = 0x1008202a8 "./aabc"
0x00000001001eeb3b in domenuselect (dummy=0x0, dat=0x0) at complist.c:2404
2404	            int l = strlen(origline);
(gdb)

## domenuselect() uses origline, but it has been corrupted
## (it should be "ls "), and the command line will be broken soon.

During "menu completion", the execution of zsh returns to
zlecore() after each TAB, and freeheap() is called at zle_main.c:1102.
I guess this frees the heap stack in which origline is allocated.

In the above case, it is freed between "Type TAB" and "Type TAB again".

Re: menu-select interactive mode

[Bart Schaefer]

On Apr 24,  7:40pm, Jun T. wrote:
}
} > .... (snip)
} >Now hit ESC-m to enter menu-selection.  I'm not even going to attempt
} >to copy-paste the result, it's a screenful of garbage memory contents.
} 
} I did some analysis on this problem.
} I think I found what is causing the problem, but do not know how to fix it.
} (The zsh is the latest CVS with ./configure --enable-zsh-debug).
} 
} ## domenuselect() uses origline, but it has been corrupted
} ## (it should be "ls "), and the command line will be broken soon.
} 
} During "menu completion", the execution of zsh returns to
} zlecore() after each TAB, and freeheap() is called at zle_main.c:1102.

The problem seems to be that the menuselect widget bypasses docomplete()
and jumps directly into domenuselect().  This misses out on some of the
setup performed by docomplete() -- normally domenuselect() is entered
from the after_complete() hook called at zle_tricky.c:869.

There's a code fragment at the top of domenuselect() that is partly
responsible for dealing with this situation:

    /*
     * Lots of the logic here doesn't really make sense if the
     * line isn't metafied, but the evidence was that it only used
     * to be metafied locally in a couple of places.
     * It's horrifically difficult to work out where the line
     * is metafied, so I've resorted to the following.
     * Unfortunately we need to unmetatfy in zrefresh() when
     * we want to display something.  Maybe this function can
     * be done better.
     */
    if (zlemetaline != NULL)
	wasmeta = 1;
    else {
	wasmeta = 0;
	metafy_line();
    }

In that else block I added code to populate origline/origcs/origll, as
happens in docomplete() right after metafy_line(), and that resolves
the memory issues -- but I'm not sure that menuselection is entered
in the correct state.  I think what *should* happen (from the user's
point of view) is that the line is restored back to just "ls " (the
starting point of menu completion) before selection begins.  That is
stored in the global struct "minfo" but I don't know the conditions
in which it's OK to access that and whether minfo.pos points into a
metafied or unmetafied line.

Re: menu-select interactive mode

[Bart Schaefer]

On Apr 26,  2:06am, Jun T. wrote:
}
} I have no idea why the else block is the appropriate place to
} set origline etc, but anyway the following patch seems to solve
} the *current* problem (but may break somewhere else...).

The else block is appropriate because that's where metafy_line() is
called ... if you look at docomplete() zle_tricky.c:629 you'll see
where the metafy and setup of origline occur when menu selection is
entered automatically rather than explicitly.

I can't explain why the code paths are so different, other than for
the usual historical reasons.

} By the way, is it possible to continue the interactive mode after
} accepting a match by hitting Return?

No, hitting accept-line takes you entirely out of menu completion,
so you're starting over with a new menu at that point.

} I tried hitting ESC-m again
} after the Return but it didn't work (either with or without the
} following patch, although the behavior was different).
} Hitting a key bound to accept-and-hold didn't work either. 

It should work to accept-and-hold if you do so *instead* of hitting
return, but not after.

} Index: Src/Zle/complist.c
} ===================================================================
} +	if(minfo.cur) {
} +	    origline = dupstrpfx(zlemetaline,minfo.pos);
} +	    origcs = minfo.pos;
} +	    origll = minfo.end;
} +	}

I think this is almost right -- origll should probably be set to
minfo.pos rather than minfo.end, and to be thorough there probably
ought to be an "else" for the case where minfo.cur is NULL (but I
suspect it's not possible for that to happen).


Index: Src/Zle/complist.c
===================================================================
--- complist.c	21 Dec 2010 16:41:16 -0000
+++ complist.c	26 Apr 2011 00:48:39 -0000
@@ -2390,8 +2390,19 @@
     else {
 	wasmeta = 0;
 	metafy_line();
+	if (minfo.cur) {
+	  /* Restore state when menu completion began */
+	  origline = dupstrpfx(zlemetaline, minfo.pos);
+	  origcs = minfo.pos;
+	  origll = minfo.pos;
+	} else {
+	  /* This branch may be unreachable */
+	  origline = dupstring(zlemetaline);
+	  origcs = zlemetacs;
+	  origll = zlemetall;
+	}
     }
-    
+
     if ((s = getsparam("MENUSCROLL"))) {
 	if (!(step = mathevali(s)))
 	    step = (lines - nlnct) >> 1;

Re: menu-select interactive mode

[Jun T.]

At 17:55 -0700 11/04/25, Bart Schaefer wrote:
>No, hitting accept-line takes you entirely out of menu completion,
>so you're starting over with a new menu at that point.

Starting over is OK, but if I want to start over another menu selection,
I need to type, after Return, first TAB and then ESC-m.
If I type ESC-m alone then the command line will be corrupted.

At the beginning of the command line (just after the prompt),
of course I can type "ls ESC-m" (or "ls foo ESC-m") to directly start
the menu selection without hitting TAB before ESC-m.

>It should work to accept-and-hold if you do so *instead* of hitting
>return, but not after.

I hit the key bound to accept-and-hold in the interactive mode without
hitting Return, but it didn't work.
I'm testing a binding like
bindkey -M menuselect '\ek' accept-and-hold

Re: menu-select interactive mode

[Jun T.]

Another problem reported by Bart Shaefer:

At 07:48 -0700 11/04/10, Bart Schaefer wrote:
>There are still some bugs in the interactive mode. 
> ..... (snip)
>Then press ENTER to accept the one-and-only match:
>
>torch% ls config.log l
>
>Finally type ENTER again:
>
>torch% ls config.logl 
>ls: config.logl: No such file or directory

The following patch *seenms* to solve this, 
but I'm not sure what I'm doing.
minfo.end should also be updated?
Or maybe I'm doing complete nonsense...

Index: Src/Zle/complist.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/complist.c,v
retrieving revision 1.123
diff -u -r1.123 complist.c
--- Src/Zle/complist.c	31 Aug 2010 18:09:16 -0000	1.123
+++ Src/Zle/complist.c	24 Apr 2011 11:28:54 -0000
@@ -2699,6 +2699,7 @@
                 else
                     selfinsertunmeta(zlenoargs);
 		metafy_line();
+		minfo.len++;
 
                 saveline = (char *) zhalloc(zlemetall);
                 memcpy(saveline, zlemetaline, zlemetall);

Re: menu-select interactive mode

[Bart Schaefer]

On Apr 24,  9:18pm, Jun T. wrote:
} Subject: Re: menu-select interactive mode
}
} Another problem reported by Bart Shaefer:
} 
} At 07:48 -0700 11/04/10, Bart Schaefer wrote:
} >torch% ls config.log l
} >torch% ls config.logl 
} >ls: config.logl: No such file or directory
} 
} The following patch *seenms* to solve this, 
} but I'm not sure what I'm doing.
} minfo.end should also be updated?
} Or maybe I'm doing complete nonsense...

That looks right to me.  And yes, it appears that minfo.end should
also be incremented.

I'll add the increment of end, and commit.

Re: menu-select interactive mode

[Jun T.]

At 11:38 -0700 11/04/24, Bart Schaefer wrote:
> I think what *should* happen (from the user's
>point of view) is that the line is restored back to just "ls " (the
>starting point of menu completion) before selection begins.

Yes, that is much more user friendly.

> That is
>stored in the global struct "minfo" but I don't know the conditions
>in which it's OK to access that

I *guess* it may be OK at least when minfo.cur != NULL.

> and whether minfo.pos points into a
>metafied or unmetafied line.

I believe minfo.pos points into zlemetaline. The only explicit assignment
to minfo.pos is at compresult.c:971 and :1329.

I have no idea why the else block is the appropriate place to
set origline etc, but anyway the following patch seems to solve
the *current* problem (but may break somewhere else...).


By the way, is it possible to continue the interactive mode after
accepting a match by hitting Return? I tried hitting ESC-m again
after the Return but it didn't work (either with or without the
following patch, although the behavior was different).
Hitting a key bound to accept-and-hold didn't work either. 
But I believe it is just too much hope.


Index: Src/Zle/complist.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/complist.c,v
retrieving revision 1.124
diff -u -r1.124 complist.c
--- Src/Zle/complist.c	24 Apr 2011 19:10:20 -0000	1.124
+++ Src/Zle/complist.c	25 Apr 2011 16:23:22 -0000
@@ -2390,6 +2390,11 @@
     else {
 	wasmeta = 0;
 	metafy_line();
+	if(minfo.cur) {
+	    origline = dupstrpfx(zlemetaline,minfo.pos);
+	    origcs = minfo.pos;
+	    origll = minfo.end;
+	}
     }
     
     if ((s = getsparam("MENUSCROLL"))) {