tab inserts literal tab instead of completing at beginning of line

tab inserts literal tab instead of completing at beginning of line

[Mikael Magnusson]

Right, so I know that's on purpose usually, but it can happen while
you're completing too:

% autoload compinit
% compinit
% mkdir foo' baz bar'
% mkdir 'bing baz bar'
% zstyle ':completion:*' matcher-list 'l:|=* r:|=*'
% setopt autocd
# this one is just to avoid completing _baz, too lazy to start over
% zstyle ':completion:*:functions' prefix-needed true
| means cursor
% baz|<tab>
results in
% |<tab>\ baz\ bar
results in
% \ baz\ bar    |

Yes, this actually happened to me. If I were to find where this
special tab-insert-at-start-of-line handling is, could I disable it,
or do I need to add an option? There's always ctrl-v tab, and why do
you want to write literal tabs at the start of the line anyway; to
paste random indented scriptlets? If that's why, maybe I can make the
thing check if the line is empty instead of $cursor==0?

-- 
Mikael Magnusson

Re: tab inserts literal tab instead of completing at beginning of line

[Bart Schaefer]

On Wed, May 18, 2011 at 3:37 PM, Mikael Magnusson <mikachu@gmail.com> wrote:
>
> Yes, this actually happened to me. If I were to find where this
> special tab-insert-at-start-of-line handling is, could I disable it,
> or do I need to add an option?

See the doc for the insert-tab zstyle.

> There's always ctrl-v tab, and why do
> you want to write literal tabs at the start of the line anyway; to
> paste random indented scriptlets?

Generally speaking, yes.

> If that's why, maybe I can make the
> thing check if the line is empty instead of $cursor==0?

Or try the "pending" setting of the style.  If testing for pending
input were more reliable, we might make that the default instead of
"true".

Re: tab inserts literal tab instead of completing at beginning of line

[Mikael Magnusson]

On 19 May 2011 07:47, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Wed, May 18, 2011 at 3:37 PM, Mikael Magnusson <mikachu@gmail.com> wrote:
>>
>> Yes, this actually happened to me. If I were to find where this
>> special tab-insert-at-start-of-line handling is, could I disable it,
>> or do I need to add an option?
>
> See the doc for the insert-tab zstyle.
>
>> There's always ctrl-v tab, and why do
>> you want to write literal tabs at the start of the line anyway; to
>> paste random indented scriptlets?
>
> Generally speaking, yes.
>
>> If that's why, maybe I can make the
>> thing check if the line is empty instead of $cursor==0?
>
> Or try the "pending" setting of the style.  If testing for pending
> input were more reliable, we might make that the default instead of
> "true".

Well, in my case there's no more input, just me pressing tab. How's
this? (it works)

--- a/Completion/Base/Core/_main_complete
+++ b/Completion/Base/Core/_main_complete
@@ -44,7 +44,8 @@

 if [[ ( "$tmp" = *pending(|[[:blank:]]*) && PENDING -gt 0 ) ||
       ( "$tmp" = *pending=(#b)([0-9]##)(|[[:blank:]]*) &&
-        PENDING -ge $match[1] ) ]]; then
+        PENDING -ge $match[1] ) ||
+      ( "$tmp" = empty && ${#${BUFFER##$'\t'#}} -eq 0 ) ]]; then
   compstate[insert]=tab

   return 0


-- 
Mikael Magnusson

Re: tab inserts literal tab instead of completing at beginning of line

[Bart Schaefer]

On Thu, May 19, 2011 at 2:36 AM, Mikael Magnusson <mikachu@gmail.com> wrote:
>
> Well, in my case there's no more input, just me pressing tab. How's
> this? (it works)

Sorry, I'm not sure what this does. Add an "empty" value to the
insert-tab style?  If so then that's OK, I think.

> --- a/Completion/Base/Core/_main_complete
> +++ b/Completion/Base/Core/_main_complete
> @@ -44,7 +44,8 @@
>
>  if [[ ( "$tmp" = *pending(|[[:blank:]]*) && PENDING -gt 0 ) ||
>       ( "$tmp" = *pending=(#b)([0-9]##)(|[[:blank:]]*) &&
> -        PENDING -ge $match[1] ) ]]; then
> +        PENDING -ge $match[1] ) ||
> +      ( "$tmp" = empty && ${#${BUFFER##$'\t'#}} -eq 0 ) ]]; then
>   compstate[insert]=tab
>
>   return 0
>
>
> --
> Mikael Magnusson
>

Re: tab inserts literal tab instead of completing at beginning of line

[Mikael Magnusson]

On 19 May 2011 17:19, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Thu, May 19, 2011 at 2:36 AM, Mikael Magnusson <mikachu@gmail.com> wrote:
>>
>> Well, in my case there's no more input, just me pressing tab. How's
>> this? (it works)
>
> Sorry, I'm not sure what this does. Add an "empty" value to the
> insert-tab style?  If so then that's OK, I think.

Ah, I was in my own little world there, I see more context is needed
:). What it does is what I described last in my first message: insert
a tab if the line is empty, or contains only tabs (presumably you want
to paste more than one sometimes), otherwise completion as usual.

-- 
Mikael Magnusson

Re: tab inserts literal tab instead of completing at beginning of line

[Mikael Magnusson]

On 19 May 2011 18:00, Mikael Magnusson <mikachu@gmail.com> wrote:
> On 19 May 2011 17:19, Bart Schaefer <schaefer@brasslantern.com> wrote:
>> On Thu, May 19, 2011 at 2:36 AM, Mikael Magnusson <mikachu@gmail.com> wrote:
>>>
>>> Well, in my case there's no more input, just me pressing tab. How's
>>> this? (it works)
>>
>> Sorry, I'm not sure what this does. Add an "empty" value to the
>> insert-tab style?  If so then that's OK, I think.
>
> Ah, I was in my own little world there, I see more context is needed
> :). What it does is what I described last in my first message: insert
> a tab if the line is empty, or contains only tabs (presumably you want
> to paste more than one sometimes), otherwise completion as usual.

tl;dr nm this whole thing

Hmmm, so I'm confused again. It turns out my patch didn't work as well
as I hoped. When I press ctrl-n or alt-m, it just inserts those
literally as well. I tried comparing _complete_debug output for
insert-tab=false and =true, but they were the same. I could do this,
but...
( "$KEYS" = $'\t' && "$tmp" = empty && ${#${BUFFER##$'\t'#}} -eq 0 )

So I just erased a big paragraph here explaining how I'm confused
since _complete_debug output is identical with insert-tab set to true
and false. It took a few minutes for me to realize this meant this is
handled in the C code. I got to this line, and decided to give up this
project for now.
uselist = (useline ?  ((isset(AUTOLIST) && !isset(BASHAUTOLIST)) ?
(isset(LISTAMBIGUOUS) ? 3 : 2) : 0) : 1);
useline is set by this earlier line
useline = (wouldinstab ? -1 : (lst != COMP_LIST_COMPLETE));
and wouldinstab is decided by this function, some time way before this
static int
usetab(void)
{
    ZLE_STRING_T s = zleline + zlecs - 1;

    if (keybuf[0] != '\t' || keybuf[1])
	return 0;
    for (; s >= zleline && *s != ZWC('\n'); s--)
	if (*s != ZWC('\t') && *s != ZWC(' '))
	    return 0;
    if (compfunc) {
	wouldinstab = 1;

	return 0;
    }
    return 1;
}

So I would have to somehow modify all the steps leading from
wouldinstab to useline/list/something to pass yet another value and
then compare this to my 'empty' value which I tracked down is called
'compinsert' in the C code. Since the code is already a mess of state
tracked by un-named magic integer values, I decided not to make
matters worse. Also to retain my sanity.

-- 
Mikael Magnusson

Re: tab inserts literal tab instead of completing at beginning of line

[Bart Schaefer]

On May 20, 11:55pm, Mikael Magnusson wrote:
} Subject: Re: tab inserts literal tab instead of completing at beginning of
}
} > Ah, I was in my own little world there, I see more context is needed
} > :). What it does is what I described last in my first message: insert
} > a tab if the line is empty, or contains only tabs (presumably you want
} > to paste more than one sometimes), otherwise completion as usual.
} 
} tl;dr nm this whole thing

OK, but the patch did intend to require that the insert-tab zstyle be
set to the value "empty" in order to create the described effect?

} Hmmm, so I'm confused again. It turns out my patch didn't work as well
} as I hoped. When I press ctrl-n or alt-m, it just inserts those
} literally as well.

That's because the style name assumes tab is the completion character.
Presumably you have ctl-n and alt-m bound to do completion as well?

You're running into a strange combination of effects.  The completion
internals initialize compstate[insert]=tab only when an actual tab is
pressed, but the effect of explicitly assigning compstate[insert]=tab
inside a completion function is to change the final keystroke into a
self-insert.

If you look at the code after line 53 in _main_complete, you'll note
that except for the case of "pending" on lines 45-47, insert-tab has
no effect unless compstate[insert]=tab is already true, i.e., unless
you actually did type a TAB character.

} I tried comparing _complete_debug output for
} insert-tab=false and =true, but they were the same.

This is one of those cases where you can't get useful output from
_complete_debug unless you actually bind it to the TAB key.  There
are a few such.  Here's the important part of the diff (blank lines
added for clarity):

 53> [[ 'tab automenu-unambiguous' == tab* ]]

-54> [[ true == (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*) ]]
-55> [[ ::: != :* || -z '' ]]
-56> return 0

+54> [[ false == (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*) ]]
+58> compstate[insert]=automenu-unambiguous

} ( "$KEYS" = $'\t' && "$tmp" = empty && ${#${BUFFER##$'\t'#}} -eq 0 )

You could do it that way, but the real problem is that you're testing
too early.  You want your test on line 54 for the "return 0" on line 56,
not on line 47 for the assignment to compstate[insert].  See below.

Index: Completion/Base/Core/_main_complete
===================================================================
diff -c -r1.12 _main_complete
--- Completion/Base/Core/_main_complete	21 Dec 2010 16:41:14 -0000	1.12
+++ Completion/Base/Core/_main_complete	23 May 2011 02:24:10 -0000
@@ -51,7 +51,9 @@
 fi
 
 if [[ "$compstate[insert]" = tab* ]]; then
-  { [[ "$tmp" = (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*) ]] &&
+  { [[ "$tmp" = (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*) ||
+	( "$tmp" = (|[[:blank:]]*)empty(|[[:blank:]]*) &&
+	  ${#${BUFFER##$'\t'#}} -eq 0 ) ]] &&
     { [[ "$curcontext" != :* || -z "$compstate[vared]" ]] ||
         zstyle -t ":completion:vared${curcontext}:" insert-tab } } && return 0
 

Re: tab inserts literal tab instead of completing at beginning of line

[Mikael Magnusson]

On 23 May 2011 04:27, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On May 20, 11:55pm, Mikael Magnusson wrote:
> } Subject: Re: tab inserts literal tab instead of completing at beginning of
> }
> } > Ah, I was in my own little world there, I see more context is needed
> } > :). What it does is what I described last in my first message: insert
> } > a tab if the line is empty, or contains only tabs (presumably you want
> } > to paste more than one sometimes), otherwise completion as usual.
> }
> } tl;dr nm this whole thing
>
> OK, but the patch did intend to require that the insert-tab zstyle be
> set to the value "empty" in order to create the described effect?

Yeah, I sort of got frustrated by looking at the C code for too long :).

> } Hmmm, so I'm confused again. It turns out my patch didn't work as well
> } as I hoped. When I press ctrl-n or alt-m, it just inserts those
> } literally as well.
>
> That's because the style name assumes tab is the completion character.
> Presumably you have ctl-n and alt-m bound to do completion as well?

Yeah, I should have spent more time composing this mail. ctrl-n completes
files by mtime and alt-m goes into menu selection. The mail also appears
extra confused because I wrote half of it before finding it is even
handled in the C code.

> You're running into a strange combination of effects.  The completion
> internals initialize compstate[insert]=tab only when an actual tab is
> pressed, but the effect of explicitly assigning compstate[insert]=tab
> inside a completion function is to change the final keystroke into a
> self-insert.
>
> If you look at the code after line 53 in _main_complete, you'll note
> that except for the case of "pending" on lines 45-47, insert-tab has
> no effect unless compstate[insert]=tab is already true, i.e., unless
> you actually did type a TAB character.

Yeah, all my confusion stems from not knowing where it was actually set.

> } I tried comparing _complete_debug output for
> } insert-tab=false and =true, but they were the same.
>
> This is one of those cases where you can't get useful output from
> _complete_debug unless you actually bind it to the TAB key.  There
> are a few such.  Here's the important part of the diff (blank lines
> added for clarity):
>
>  53> [[ 'tab automenu-unambiguous' == tab* ]]
>
> -54> [[ true == (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*) ]]
> -55> [[ ::: != :* || -z '' ]]
> -56> return 0
>
> +54> [[ false == (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*) ]]
> +58> compstate[insert]=automenu-unambiguous
>
> } ( "$KEYS" = $'\t' && "$tmp" = empty && ${#${BUFFER##$'\t'#}} -eq 0 )
>
> You could do it that way, but the real problem is that you're testing
> too early.  You want your test on line 54 for the "return 0" on line 56,
> not on line 47 for the assignment to compstate[insert].  See below.

A tangent, I was just staring at these tests, and the (|[[:blank:]]*)
thing that appears on both sides, shouldn't the asterisk be to the left
of the blank when the thing is to the left of the word? I don't see the
point of looking for a blank space, then random crap, then the word? Ie,
doesn't this
"$tmp" = (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*)
match the value " tralalayes bar" when the intention is to match "tralala
yes bar"? I tested now but I'm too lazy to rewrite, it does do that.

> Index: Completion/Base/Core/_main_complete
> ===================================================================
> diff -c -r1.12 _main_complete
> --- Completion/Base/Core/_main_complete 21 Dec 2010 16:41:14 -0000      1.12
> +++ Completion/Base/Core/_main_complete 23 May 2011 02:24:10 -0000
> @@ -51,7 +51,9 @@
>  fi
>
>  if [[ "$compstate[insert]" = tab* ]]; then
> -  { [[ "$tmp" = (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*) ]] &&
> +  { [[ "$tmp" = (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*) ||
> +       ( "$tmp" = (|[[:blank:]]*)empty(|[[:blank:]]*) &&
> +         ${#${BUFFER##$'\t'#}} -eq 0 ) ]] &&
>     { [[ "$curcontext" != :* || -z "$compstate[vared]" ]] ||
>         zstyle -t ":completion:vared${curcontext}:" insert-tab } } && return 0

Yeah, this works, and feels less gross than my version, thanks.

Here's diffs for the wrong * placement,

--- a/Completion/Base/Core/_main_complete
+++ b/Completion/Base/Core/_main_complete
@@ -51,8 +51,8 @@
 fi

 if [[ "$compstate[insert]" = tab* ]]; then
-  { [[ "$tmp" = (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*) ||
-       ( "$tmp" = (|[[:blank:]]*)empty(|[[:blank:]]*) &&
+  { [[ "$tmp" = (|*[[:blank:]])(yes|true|on|1)(|[[:blank:]]*) ||
+       ( "$tmp" = (|*[[:blank:]])empty(|[[:blank:]]*) &&
          ${#${BUFFER##$'\t'#}} -eq 0 ) ]] &&
     { [[ "$curcontext" != :* || -z "$compstate[vared]" ]] ||
         zstyle -t ":completion:vared${curcontext}:" insert-tab } } && return 0


and against current cvs

--- a/Completion/Base/Core/_main_complete
+++ b/Completion/Base/Core/_main_complete
@@ -51,7 +51,7 @@ if [[ ( "$tmp" = *pending(|[[:blank:]]*) && PENDING -gt 0 ) ||
 fi

 if [[ "$compstate[insert]" = tab* ]]; then
-  { [[ "$tmp" = (|[[:blank:]]*)(yes|true|on|1)(|[[:blank:]]*) ]] &&
+  { [[ "$tmp" = (|*[[:blank:]])(yes|true|on|1)(|[[:blank:]]*) ]] &&
     { [[ "$curcontext" != :* || -z "$compstate[vared]" ]] ||
         zstyle -t ":completion:vared${curcontext}:" insert-tab } } && return 0



-- 
Mikael Magnusson

Re: tab inserts literal tab instead of completing at beginning of line

[Bart Schaefer]

On May 23,  4:55am, Mikael Magnusson wrote:
}
} On 23 May 2011 04:27, Bart Schaefer <schaefer@brasslantern.com> wrote:
} > You're running into a strange combination of effects.  The completion
} > internals initialize compstate[insert]=tab only when an actual tab is
} > pressed, but the effect of explicitly assigning compstate[insert]=tab
} > inside a completion function is to change the final keystroke into a
} > self-insert.

I'm trying to decide if this is worth documenting.

} A tangent, I was just staring at these tests, and the (|[[:blank:]]*)
} thing that appears on both sides, shouldn't the asterisk be to the left
} of the blank when the thing is to the left of the word?

I think you're correct about that.  There may be a few places that need
that change.

The two patches at the end of your message look fine if you want to
commit one or the other (the "empty" one needs doc added, of course).

Re: tab inserts literal tab instead of completing at beginning of line

[Mikael Magnusson]

On 23 May 2011 18:57, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On May 23,  4:55am, Mikael Magnusson wrote:
> }
> } On 23 May 2011 04:27, Bart Schaefer <schaefer@brasslantern.com> wrote:
> } > You're running into a strange combination of effects.  The completion
> } > internals initialize compstate[insert]=tab only when an actual tab is
> } > pressed, but the effect of explicitly assigning compstate[insert]=tab
> } > inside a completion function is to change the final keystroke into a
> } > self-insert.
>
> I'm trying to decide if this is worth documenting.
>
> } A tangent, I was just staring at these tests, and the (|[[:blank:]]*)
> } thing that appears on both sides, shouldn't the asterisk be to the left
> } of the blank when the thing is to the left of the word?
>
> I think you're correct about that.  There may be a few places that need
> that change.

I did a quick grep for the exact pattern that occurs here, and only
found it in _pids, and that already has the * on the right side.

> The two patches at the end of your message look fine if you want to
> commit one or the other (the "empty" one needs doc added, of course).

Okay, I'll commit the one without (the one with doesn't add it, it was
just on top of your patch). Is the patch to add "empty" pretty
uncontroversial if I add docs to it? I was a bit worried the subst +
length check might be a bit slow, but I ran a loop doing it 10000
times on a 1000 or so char long string and it only took 2 seconds or
so.

Is there some problem with zsh.org? I can't access www.zsh.org for the
mla to look up the number of my patch (gmail discards mailing lists
mails from yourself (sigh)), and iirc it was out a few days ago too.

-- 
Mikael Magnusson

Re: tab inserts literal tab instead of completing at beginning of line

[Bart Schaefer]

On May 23,  7:11pm, Mikael Magnusson wrote:
}
} Is there some problem with zsh.org? I can't access www.zsh.org for the
} mla to look up the number of my patch (gmail discards mailing lists
} mails from yourself (sigh)), and iirc it was out a few days ago too.

I just noticed it down this morning, but I've been AFK for a few days.

Your message was 29338, by the way.  And the _make patch is 29365.