Slow completion

Slow completion

[Thorsten Kampe]

Zsh completion has always been kind of slow for me (in comparison to 
bash) but I always contributed this to the superior functionality (my 
.zshrc is two and a half times as long as my .bashrc).

Today I decided that it had become unbearable and started investigating. 

I tested a simple completion like

    /us<TAB>r/bi<TAB>n/cada<TAB>ver.exe

The directory completion (usr and bin) was already slow but the 
executable (cadaver.exe) took ages.

1. Testing revealed that "setopt path_dirs" was responsible for the slow 
directory completion.

The zshoptions manual page says that path_dirs should not apply in the 
above case[1] but a Process Monitor log showed that Zsh was actually 
searching through the whole executable path.

2. The cause for the even slower executable completion was the following 
line suggested by Bart Schaefer[2]

    zstyle ':completion:*:-command-:*' ignored-patterns '*.dll' '*.exe'

This works fine for a "relative" completion like

    cada<TAB>

where cadaver/and/ cadaver.exe are returned.

But for an "absolute" completion like

    /usr/bin/cada<TAB>

there is only cadaver.exe returned which the above completion style 
ignores. Since _ignored is at the end of my zstyle completer list, this 
results in cadaver.exe only being completed after everything else has 
been tried.

Bash in contrast actually completes

    cada<TAB>

and

    /usr/bin/cada<TAB>

to cadaver with completion_strip_exe set.

I think both issues (path_dirs and inconsistent completion) are actually 
bugs.

Thorsten
[1] "Commands explicitly beginning with `/' [...] are not subject to the 
path search."
[2] http://www.zsh.org/mla/users/2009/msg00391.html

Re: Slow completion

[Peter Stephenson]

On Wed, 06 Mar 2013 22:40:41 +0100
Thorsten Kampe <thorsten@thorstenkampe.de> wrote:

> Zsh completion has always been kind of slow for me (in comparison to 
> bash) but I always contributed this to the superior functionality (my 
> .zshrc is two and a half times as long as my .bashrc).
> 
> Today I decided that it had become unbearable and started investigating. 
> 
> I tested a simple completion like
> 
>     /us<TAB>r/bi<TAB>n/cada<TAB>ver.exe
> 
> The directory completion (usr and bin) was already slow but the 
> executable (cadaver.exe) took ages.
> 
> 1. Testing revealed that "setopt path_dirs" was responsible for the slow 
> directory completion.
> 
> The zshoptions manual page says that path_dirs should not apply in the 
> above case[1] but a Process Monitor log showed that Zsh was actually 
> searching through the whole executable path.

It's doing completion equivalent to:

/usr*/bin*/cada*

when you hit TAB in the last case.  This is a feature, but definitely
annoying in the case where path completion is slow.  You can turn it off
for the case where the directories exist by setting the style
accept-exact-dirs to true: I've been doing this under Cygwin.  In other
words, this should complete /usr/bin/cadaver.exe faster the way you're
doing it, since it's one component at a time, but still allows you to
complete /u/b/c in one go.

I don't think we have an option to turn completion of all path segments
off altogether at the moment.

pws

Re: Slow completion

[Mikael Magnusson]

On 07/03/2013, Peter Stephenson <p.stephenson@samsung.com> wrote:
> On Wed, 06 Mar 2013 22:40:41 +0100
> Thorsten Kampe <thorsten@thorstenkampe.de> wrote:
>
>> Zsh completion has always been kind of slow for me (in comparison to
>> bash) but I always contributed this to the superior functionality (my
>> .zshrc is two and a half times as long as my .bashrc).
>>
>> Today I decided that it had become unbearable and started investigating.
>>
>> I tested a simple completion like
>>
>>     /us<TAB>r/bi<TAB>n/cada<TAB>ver.exe
>>
>> The directory completion (usr and bin) was already slow but the
>> executable (cadaver.exe) took ages.
>>
>> 1. Testing revealed that "setopt path_dirs" was responsible for the slow
>> directory completion.
>>
>> The zshoptions manual page says that path_dirs should not apply in the
>> above case[1] but a Process Monitor log showed that Zsh was actually
>> searching through the whole executable path.
>
> It's doing completion equivalent to:
>
> /usr*/bin*/cada*
>
> when you hit TAB in the last case.  This is a feature, but definitely
> annoying in the case where path completion is slow.  You can turn it off
> for the case where the directories exist by setting the style
> accept-exact-dirs to true: I've been doing this under Cygwin.  In other
> words, this should complete /usr/bin/cadaver.exe faster the way you're
> doing it, since it's one component at a time, but still allows you to
> complete /u/b/c in one go.
>
> I don't think we have an option to turn completion of all path segments
> off altogether at the moment.

Isn't that what this does?

       accept-exact-dirs
...
              If you wish to inhibit this behaviour entirely, set the
path-comple‐
              tion style (see below) to false.

       path-completion
              Explicitly setting this style to false inhibits this behaviour for
              path components up to the / before the cursor; this
overrides the set‐
              ting of accept-exact-dirs.


-- 
Mikael Magnusson

Re: Slow completion

[Peter Stephenson]

On Thu, 07 Mar 2013 12:44:55 +0100
Mikael Magnusson <mikachu@gmail.com> wrote:
> > I don't think we have an option to turn completion of all path segments
> > off altogether at the moment.
> 
> Isn't that what this does?
> 
>        path-completion
>               Explicitly setting this style to false inhibits this behaviour for
>               path components up to the / before the cursor; this
> overrides the set‐
>               ting of accept-exact-dirs.

Oh, yes, I forgot we eventually added that, too.

pws

Re: Slow completion

[Bart Schaefer]

On Mar 6, 10:40pm, Thorsten Kampe wrote:
}
}     zstyle ':completion:*:-command-:*' ignored-patterns '*.dll' '*.exe'
} 
} This works fine for a "relative" completion like
} 
}     cada<TAB>
} 
} where cadaver/and/ cadaver.exe are returned.
} 
} But for an "absolute" completion like
} 
}     /usr/bin/cada<TAB>
} 
} there is only cadaver.exe returned which the above completion style 
} ignores. Since _ignored is at the end of my zstyle completer list, this 
} results in cadaver.exe only being completed after everything else has 
} been tried.

I'm not a regular cygwin user (in fact it's probably been 15 years since
I even touched a box with cygwin, now that I think about it) so you'll
have to (a) forgive any inaccuracies and (b) remind me how it is that
"cadaver" gets into the "relative" completion list in the first place,
since it is not a file on disk.

However, I think it's wrong to assert this is "inconsistent" completion.
You're not completing the same thing.  I'm pretty sure "cada" is being
completed for the "path-commands" group, while /usr/bin/cada is in the
"executables" (executable file or directory) group.  The latter uses
file completion rules, which are being applied quite consistently.
 
} Bash in contrast actually completes
} 
}     /usr/bin/cada<TAB>
} 
} to cadaver with completion_strip_exe set.

Well, yes.  Stripping the ".exe" off the matches when inserting into
the command line is a different operation from ignoring any matches
that contain the ".exe" suffix.

It might be possible to get the effect of completion_strip_exe with a
matcher style, something like (though I don't think this works)

    zstyle ':completion:*:-command-:*' matcher \
        'R:|.[eE][xX][eE]=.[eE][xX][eE]'

but I'm not really an expert with the matcher syntax and all the doc
examples are for left-anchored rather than right-anchored patterns.

Re: Slow completion

[Thorsten Kampe]

* Bart Schaefer (Thu, 07 Mar 2013 08:04:45 -0800)
> I'm not a regular cygwin user (in fact it's probably been 15 years 
since
> I even touched a box with cygwin, now that I think about it) so you'll
> have to (a) forgive any inaccuracies and (b) remind me how it is that
> "cadaver" gets into the "relative" completion list in the first place,
> since it is not a file on disk.

Not really sure about the "exe magic". I think Cygwin just adds it. 
Windows requires the .exe extension for the file itself but not for the 
invocation 
(PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC)

> However, I think it's wrong to assert this is "inconsistent" completion.
> You're not completing the same thing.  I'm pretty sure "cada" is being
> completed for the "path-commands" group, while /usr/bin/cada is in the
> "executables" (executable file or directory) group.

Okay, I got it.

> It might be possible to get the effect of completion_strip_exe with a
> matcher style, something like (though I don't think this works)
> 
>     zstyle ':completion:*:-command-:*' matcher \
>         'R:|.[eE][xX][eE]=.[eE][xX][eE]'

Doesn't work but thanks for trying.

Thorsten

Re: Slow completion

[Oliver Kiddle]

Bart wrote:
> It might be possible to get the effect of completion_strip_exe with a
> matcher style, something like (though I don't think this works)
> 
>     zstyle ':completion:*:-command-:*' matcher \
>         'R:|.[eE][xX][eE]=.[eE][xX][eE]'

To strip the .exe, you need to use a lowercase r, map the .exe to
nothing and use the .exe as the command-line pattern rather than as the
the anchor: 'r:.[eE][xX][eE]|='

Unfortunately, this somehow only seems to work when you position the
cursor before the .exe which makes it fairly useless. Note this will
only work at all when completeinword is set otherwise the .exe is not
seen by the matching at all. The same effect applies on the left but
that matters less because the cursor is usually going to be after a left
anchored pattern. I think this is a bug.

Oliver

Re: Slow completion

[joe M]

Hello,

I am wondering if there would be any benefit in compiling the
completion functions such as _main_complete, etc. Will that show a
performance boost?

Thanks
Joe]

Re: Slow completion

[Bart Schaefer]

On Mar 8, 10:45am, joe M wrote:
} 
} I am wondering if there would be any benefit in compiling the
} completion functions such as _main_complete, etc. Will that show a
} performance boost?

Nobody answered this, I guess.  The answer is complicated.

Although .zwc requires less parsing/interpretation than plain text, it's
often larger on disk because of storing both the bytecode and the raw
strings for xtrace.  On the other hand it doesn't include comments, so
heavily-commented functions might come out smaller.

It is probably not a win to compile all the functions into a single .zwc
file because most people don't use anywhere near the entire suite, so
you'll be loading a bunch of stuff you don't need.  The flip side of
this is that zsh is usually able to memory-map the whole .zwc file,
which is probably faster than reading individual small files, so it may
be a wash.

Whether there's a benefit to selectively compiling the parts that you do
use is system-dependent, and again is related to the file sizes.  If you
have fast disk I/O it may be a win (when zcompile was invented I/O was
consistently faster than parsing) but on say a laptop that does not have
SSD you may be better off parsing on the fly.

The best I can offer is, try it both ways and compare.