completion tricks

completion tricks

[Jeff Shipman]

I was wondering how to set up some completion tricks like
these:

1) Using 'man' will complete different man pages

2) Using ssh, rlogin, telnet, etc will complete
   different hosts on my network.

Any help would be greatly appreciated. Thanks!

Jeff "Shippy" Shipman     E-Mail: shippy@nmt.edu
Computer Science Major    ICQ: 1786493
New Mexico Institute of Mining and Technology
Homepage: http://www.nmt.edu/~shippy

Re: completion tricks

[Marius Strom]

Jeff,
I did the second portion as thus in my .zshrc:

hosts=( host1 host2 host3.fqdn.com )
compctl -k hosts ssh ftp telnet ping traceroute mtr rlogin rsh

On Mon, Mar 19, 2001 at 01:14:54PM -0700, Jeff Shipman wrote:
> I was wondering how to set up some completion tricks like
> these:
> 
> 1) Using 'man' will complete different man pages
> 
> 2) Using ssh, rlogin, telnet, etc will complete
>    different hosts on my network.
> 
> Any help would be greatly appreciated. Thanks!
> 
> Jeff "Shippy" Shipman     E-Mail: shippy@nmt.edu
> Computer Science Major    ICQ: 1786493
> New Mexico Institute of Mining and Technology
> Homepage: http://www.nmt.edu/~shippy

-- 
Marius Strom <marius@alpha1.net>
Professional Geek/Unix System Administrator
Alpha1 Internet <http://www.alpha1.net/>
http://www.marius.org/marius.pgp 0xF5D89089 *updated 2001-02-26*
 
Never underestimate the bandwidth of a mini-van full of DLT
tapes traveling down the highway at 65 miles per hour.
	-Andrew Tanenbaum, "Computer Networks"

Re: completion tricks

[Adam Spiers]

Marius Strom (marius@alpha1.net) wrote:
> On Mon, Mar 19, 2001 at 01:14:54PM -0700, Jeff Shipman wrote:
> > I was wondering how to set up some completion tricks like
> > these:
> > 
> > 1) Using 'man' will complete different man pages
> > 
> > 2) Using ssh, rlogin, telnet, etc will complete
> >    different hosts on my network.
>
> Jeff,
> I did the second portion as thus in my .zshrc:
> 
> hosts=( host1 host2 host3.fqdn.com )
> compctl -k hosts ssh ftp telnet ping traceroute mtr rlogin rsh

Or if you're using a development version (highly recommended), forget
compctl altogether, and do `autoload compinstall; compinstall' to
install the new completion system, which is much better.

Completion of man pages should then work automatically.  You can
also configure ssh etc. like so:

  # All my accounts:
  my_accounts=(
    {joe,root}@mymachine.com
    jbloggs@myothermachine.com
  )

  # Other people's accounts:
  other_accounts=(
    {fred,root}@hismachine.com
    vera@hermachine.com
  )

  zstyle ':completion:*:my-accounts' users-hosts $my_accounts
  zstyle ':completion:*:other-accounts' users-hosts $other_accounts

  telnet_users_hosts_ports=(
    user1@host1:
    user2@host2:
    @mail-server:{smtp,pop3}
    @news-server:nntp
    @proxy-server:8000
  )

  zstyle ':completion:*:*:telnet:*' users-hosts-ports $telnet_users_hosts_ports

Note to zsh-workers: it looks like completion within the -l option of
telnet is broken.

Re: completion tricks

[Matt Armstrong]

Adam Spiers <adam@spiers.net> writes:

> Marius Strom (marius@alpha1.net) wrote:
> > On Mon, Mar 19, 2001 at 01:14:54PM -0700, Jeff Shipman wrote:
> > > I was wondering how to set up some completion tricks like
> > > these:
> > > 
> > > 1) Using 'man' will complete different man pages
> > > 
> > > 2) Using ssh, rlogin, telnet, etc will complete
> > >    different hosts on my network.
> >
> > Jeff,
> > I did the second portion as thus in my .zshrc:
> > 
> > hosts=( host1 host2 host3.fqdn.com )
> > compctl -k hosts ssh ftp telnet ping traceroute mtr rlogin rsh
> 
> Or if you're using a development version (highly recommended), forget
> compctl altogether, and do `autoload compinstall; compinstall' to
> install the new completion system, which is much better.

I was using the new completion stuff for a while, but I switched back
for two reasons:

 - The learning curve to actually do stuff with the new completion
   stuff was above my personal threshold.

 - My shells started up many times slower with it in place.

compctl is cryptic but pretty simple.  I hope it stays around.

-- 
matt

Re: completion tricks

[Adam Spiers]

Matt Armstrong (matt.armstrong@openwave.com) wrote:
> I was using the new completion stuff for a while, but I switched back
> for two reasons:
> 
>  - The learning curve to actually do stuff with the new completion
>    stuff was above my personal threshold.
> 
>  - My shells started up many times slower with it in place.

This is a shame, but I can totally sympathise with your viewpoint.  I
would love to hear some tips from any zsh-workers as to how zsh can be
optimised with respect to startup time.  (Optimisation w.r.t. memory
usage is another topic I would like to hear people's thoughts on, but
it's off-topic here.)  Personally I tried messing with zcompile and
zrecompile once or twice to address this but ran out of time before I
got it working, and anyway, Peter's guide suggests that on most
computers these days it might not help much.  [Note to workers: would
it be worth having `make install' automatically zcompile the
completion system?]

As for the learning curve for the new completion system, this is a
known issue, but have you tried the `compinstall' command recently?
It goes a remarkably long way towards solving the problem.  If there's
something else which compinstall didn't help you with, if you could
find the time to describe more precisely what it was that you wanted
to do, we'll do our best to help you and/or improve the existing
documentation.

> compctl is cryptic but pretty simple.  I hope it stays around.

I don't think there are any plans to get rid of it.

Re: completion tricks

[Matt Armstrong]

Adam Spiers <adam@spiers.net> writes:

> Matt Armstrong (matt.armstrong@openwave.com) wrote:
> > I was using the new completion stuff for a while, but I switched back
> > for two reasons:
> > 
> >  - The learning curve to actually do stuff with the new completion
> >    stuff was above my personal threshold.
> > 
> >  - My shells started up many times slower with it in place.
> 
> As for the learning curve for the new completion system, this is a
> known issue, but have you tried the `compinstall' command recently?

I was way beyond compinstall -- making custom completers for my own
commands, etc.  Then I stepped back and analyzed the effort I was
expending for the actual benefit.  Using compctl exclusively removes
the temptation for spending time customizing every command I ever run.
;-)

> This is a shame, but I can totally sympathise with your viewpoint.
> I would love to hear some tips from any zsh-workers as to how zsh
> can be optimised with respect to startup time.

Thinking back, startup time was not the problem that caused me to
switch back.  It was the fact that simple tab file completion got
slower.  Occasionally I'm doing stuff on slower systems with NFS file
systems.  Completing files with TAB was noticeably slower with the new
system over the old.

-- 
matt

Re: completion tricks

[Adam Spiers]

Matt Armstrong (matt.armstrong@openwave.com) wrote:
> Adam Spiers <adam@spiers.net> writes:
> I was way beyond compinstall -- making custom completers for my own
> commands, etc.  Then I stepped back and analyzed the effort I was
> expending for the actual benefit.  Using compctl exclusively removes
> the temptation for spending time customizing every command I ever run.
> ;-)

Fair enough :-)  I've fallen into the same trap myself in the past,
although I think I've got the balance right now.  I'm finding that
writing really simple functions with maybe a _files -W type of
completion often come in incredibly handy, and hardly take any time to
write, for instance.

> > This is a shame, but I can totally sympathise with your viewpoint.
> > I would love to hear some tips from any zsh-workers as to how zsh
> > can be optimised with respect to startup time.
> 
> Thinking back, startup time was not the problem that caused me to
> switch back.  It was the fact that simple tab file completion got
> slower.  Occasionally I'm doing stuff on slower systems with NFS file
> systems.  Completing files with TAB was noticeably slower with the new
> system over the old.

This really bothered me too, so I got rid of the _approximate and
_correct completers (IIRC).

Re: completion tricks

[Sven Wischnowsky]

Adam Spiers wrote:

> Matt Armstrong (matt.armstrong@openwave.com) wrote:
> > I was using the new completion stuff for a while, but I switched back
> > for two reasons:
> > 
> >  - The learning curve to actually do stuff with the new completion
> >    stuff was above my personal threshold.
> > 
> >  - My shells started up many times slower with it in place.
> 
> This is a shame, but I can totally sympathise with your viewpoint.  I
> would love to hear some tips from any zsh-workers as to how zsh can be
> optimised with respect to startup time.

Normal users who have installed the functions in their home
directories or who can trust their system administrators can call
compinit with the -C option.  This avoids the security check
(compaudit) and it avoid parsing the tag lines of the completion files 
if there is already a dump file -- which means that people using -C
have to make sure the dump file is re-built when there are new or
changed completion functions (just removing the dump file is enough to 
trigger re-building it on the next invocation of compinit).

> (Optimisation w.r.t. memory
> usage is another topic I would like to hear people's thoughts on, but
> it's off-topic here.)  Personally I tried messing with zcompile and
> zrecompile once or twice to address this but ran out of time before I
> got it working, and anyway, Peter's guide suggests that on most
> computers these days it might not help much.  [Note to workers: would
> it be worth having `make install' automatically zcompile the
> completion system?]

On a stand-alone Linux box there probably really isn't that much
difference, but at least for me compiling the whole completion system
into a memory mapped digest file and using that makes things faster
(after the first function is loaded).  It also saves memory, of course 
because mapped files are shared by all instances of zsh running on the 
same host (if your OS supports that, but almost every Unix should).

> ...
> 
> > compctl is cryptic but pretty simple.  I hope it stays around.
> 
> I don't think there are any plans to get rid of it.

I once hoped we could get rid of it some years hence.  But since then
we've put it into a separate module, so I don't care about it anymore 
and it'll almost certainly survive.


Matt Armstrong wrote:

> ... [learning curve of the new completion system]
> 
> I was way beyond compinstall -- making custom completers for my own
> commands, etc.  Then I stepped back and analyzed the effort I was
> expending for the actual benefit.  Using compctl exclusively removes
> the temptation for spending time customizing every command I ever run.
> ;-)

I can easily understand reluctance to switch from compctl to the new
system once one has grokked the compctl man page ;-)

The difficulty is of course relative.  For very simple things compctls 
might be easier to write (if one doesn't have experience with the new
system), but as soon as we are talking about a completion for a
command with options and different arguments, the utility functions in 
the new system can make your live much easier (e.g. the _arguments
function).

> ...
> 
> Thinking back, startup time was not the problem that caused me to
> switch back.  It was the fact that simple tab file completion got
> slower.  Occasionally I'm doing stuff on slower systems with NFS file
> systems.  Completing files with TAB was noticeably slower with the new
> system over the old.

As Adam pointed out, this of course depends on the completers one uses 
(and the match specs, btw.).  It also heavily depends on the 3.1.x
version you tried.  Some time ago we added C-support for some of the
more heavily used completion functions, including, of course,
_path_files (which, not surprisingly, completes filenames and paths).
If you tried an earlier version I can understand your problems, but in 
a later version, and using a mapped function digest file completion is 
really pretty fast for me (we also had some other patches that made
the shell faster -- mainly due to the transition to the new internal
code representation (which also reduced the memory requirements)).


Anyway.  I can't and wouldn't want to force anyone to switch to the
new system -- but you miss a lot ;-)


Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: completion tricks

[Bart Schaefer]

On Mar 19,  1:46pm, Matt Armstrong wrote:
} Subject: Re: completion tricks
}
} I was using the new completion stuff for a while, but I switched back
} for two reasons:
} 
}  - The learning curve to actually do stuff with the new completion
}    stuff was above my personal threshold.

On the other hand, the new completion system does more in its shipping
configuration than was ever collected from all contributors of compctl
commands.  The amount of stuff that you _need_ to do yourself is less.

}  - My shells started up many times slower with it in place.

On Mar 19, 10:04pm, Adam Spiers wrote:
} Subject: Re: completion tricks
}
} This is a shame, but I can totally sympathise with your viewpoint.  I
} would love to hear some tips from any zsh-workers as to how zsh can be
} optimised with respect to startup time.  (Optimisation w.r.t. memory
} usage is another topic I would like to hear people's thoughts on, but
} it's off-topic here.)

Sven did a couple of optimizations posted in zsh-workers/13589 that
should both increase the speed and cut down the memory usage a lot.  If
you haven't tried again with the latest snapshot, you should.  Or at
least try again when 4.0.1 is released.

} [Note to workers: would it be worth having `make install'
} automatically zcompile the completion system?]

Maybe; the .zcompdump file actually does make a lot of difference.  In
tests I just tried (on a 233MHz Pentium II), "zsh -li" started in 1.05
seconds with a .zcompdump file, as opposed to 2.35s with "compinit -D"
(other garbage in my startup files being equal).  Incidentally, it took
3.64s to start up and write the .zcompdump in the first place.

On Tue Mar 20,  9:28am, Sven Wischnowsky wrote:
} Normal users who have installed the functions in their home
} directories or who can trust their system administrators can call
} compinit with the -C option.  This avoids the security check
} (compaudit) and it avoid parsing the tag lines of the completion files 

compinit -C cut the time from 1.05s to 1.01s.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: completion tricks

[Adam Spiers]

Sven Wischnowsky (wischnow@informatik.hu-berlin.de) wrote:
> On a stand-alone Linux box there probably really isn't that much
> difference, but at least for me compiling the whole completion system
> into a memory mapped digest file and using that makes things faster
> (after the first function is loaded).  It also saves memory, of course 
> because mapped files are shared by all instances of zsh running on the 
> same host (if your OS supports that, but almost every Unix should).

Would you mind showing how you do this quickly?  I tried:

  $ zcompile ~/.zsh/comp.zwc ~/local/share/zsh/4.0.1-pre-2/functions/Completion/*/_*

ripped from the example usage of zrecompile, but I got:

zsh: parse error near `|'
zcompile: can't read file: /nfs-home/adams/local/share/zsh/4.0.1-pre-2/functions/Completion/Builtins/_vars_eq

This is with the latest CVS.

Also, for this to work, wouldn't I have to include ~/.zsh/comp.zwc in
fpath?  Why did the example suggest only to compile Completion/*/_*,
not Completion/*/* ?

I get the impression there are several different strategies one could
take with compilation.  For example, how much is there to be gained by
replacing all the Completion/* directories from the fpath with a
single compiled foo.zwc digest?  Could you even do that, given that
presumably foo.zwc would be missing Completion/Core/comp* ?

The more I think about it, the more I like the idea of `make install'
(or maybe `make install.zwc') taking the most sensible compilation
route.  The distribution's completion functions probably aren't going
to change at all in between upgrades, after all.

> > Completing files with TAB was noticeably slower with the new
> > system over the old.
> 
> As Adam pointed out, this of course depends on the completers one uses 
> (and the match specs, btw.).

Perhaps these performance hits should be mentioned in the manual?

> Anyway.  I can't and wouldn't want to force anyone to switch to the
> new system -- but you miss a lot ;-)

You certainly do :-)

Re: completion tricks

[Stephane Bortzmeyer]

On Monday 19 March 2001, at 13 h 14, the keyboard of Jeff Shipman 
<shippy@nmt.edu> wrote:

> 2) Using ssh, rlogin, telnet, etc will complete
>    different hosts on my network.

Did you notice the 'ssh_completion' file in zsh's distribution?

Anyway, here is how I do it:

# Construct an array of common hosts
etchosts=( ${(s: :)${(ps:\t:)${${(f)"$(</etc/hosts)"}%%\#*}##[:blank:]#[^[:blan
k:]]#}} )
sshhosts=(${${${${(f)"$(<$HOME/.ssh/known_hosts)"}:#[^0-9]*}%%\ *}%%,*})
hosts=($etchosts $sshhosts)

# SSH (scp is much more difficult)
compctl -k hosts \
	-x "c[-1,-l]" -u \
	- "c[-1,-i]" -f \
	- "c[-1,-e]" -k "(~ none)" \
	- "c[-1,-c]" -k "(idea des 3des tss arcfour none)" \
	- "c[-1,-p]" -k ports \
	- "c[-1,-L] c[-1,-R] c[-1,-o]" -k "()" \
	-- ssh slogin

# various network programs
compctl -x 'N[-1,@]' -k hosts - 's[]' -u -qS '@' -- finger
compctl -k hosts traceroute mtr ping 

Re: completion tricks

[Stephane Bortzmeyer]

On Monday 19 March 2001, at 13 h 14, the keyboard of Jeff Shipman 
<shippy@nmt.edu> wrote:

> 1) Using 'man' will complete different man pages

You will find several at www.zsh.org (for instance http://sunsite.dk/zsh/Contrib/index.html) and at http://www.dotfiles.com/index.php3?app_id=4.

Re: completion tricks

[Felix Rosencrantz]

Zsh comes with a tool to convert tcsh completions to zsh compctl completions. 
It might be useful to have a tool to convert zsh compctl completions to the new
completion system.  Something like this would be useful for converting these
custom completions.

It could provide a good learning experience for how to use the new system for
folks familiar with the old (or are able to better understand the old system
that the new.)

For the timing issues, it might be useful if a completion could be stopped if a
certain time limit has been reached.  Maybe the check could occur between
completers.  So in addition to check if there were any matches, it could see if
the time limit was exceeded.

 Also, I thought there was a style to tell the completion system, if a given
filepath exists, accept it without trying to do completions on the whole
path...  Something like that might help prevent NFS from being excessively
tickled by completion.

-FR



__________________________________________________
Do You Yahoo!?
Get email at your own domain with Yahoo! Mail. 
http://personal.mail.yahoo.com/

Re: completion tricks

[Sven Wischnowsky]

Felix Rosencrantz wrote:

> Zsh comes with a tool to convert tcsh completions to zsh compctl completions. 
> It might be useful to have a tool to convert zsh compctl completions to the new
> completion system.  Something like this would be useful for converting these
> custom completions.

Yes, I once thought about trying to do that.  It can get pretty
complicated, though (with -T and -t and stuff).  I was just too fed up 
with compctl to mess with it again...

> It could provide a good learning experience for how to use the new system for
> folks familiar with the old (or are able to better understand the old system
> that the new.)

And I'm not sure about this.  Writing a working converter is already
non-trivial, writing one that produces nice code even more so.

> For the timing issues, it might be useful if a completion could be stopped if a
> certain time limit has been reached.  Maybe the check could occur between
> completers.  So in addition to check if there were any matches, it could see if
> the time limit was exceeded.

Hm, if seconds are accurate enough this would be extremely simple to
add using $SECONDS.

>  Also, I thought there was a style to tell the completion system, if a given
> filepath exists, accept it without trying to do completions on the whole
> path...  Something like that might help prevent NFS from being excessively
> tickled by completion.

It's the default behaviour: leading path segments are accepted as they 
are if there are files below them that can be matched by the rest of
the partially given path.


Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: completion tricks

[Sven Wischnowsky]

Adam Spiers wrote:

> Sven Wischnowsky (wischnow@informatik.hu-berlin.de) wrote:
> > On a stand-alone Linux box there probably really isn't that much
> > difference, but at least for me compiling the whole completion system
> > into a memory mapped digest file and using that makes things faster
> > (after the first function is loaded).  It also saves memory, of course 
> > because mapped files are shared by all instances of zsh running on the 
> > same host (if your OS supports that, but almost every Unix should).
> 
> Would you mind showing how you do this quickly?

I use this:

    zrec() {
      zrecompile "$@" -p -- \
        -R ~/.zshrc -- \
        -R ~/.zdump -- \
        -R ~/lib/zsh/comp/Core/compinit -- \
        -R ~/lib/zsh/comp/Core/compaudit -- \
        -R ~/lib/zsh/comp/Core/compdump -- \
        -M ~/lib/zsh/funcs.zwc \
           ~/funcs/*(.) \
           ~/lib/zsh/comp/*/_* \
           ~/lib/zsh/func/*/*
    }

Never mind the funny paths.  Since I just upgraded the zsh I use every 
day (above the development version) to 4.0.1-pre2 I can say that this
worked for me with the latest CVS, so I'm not sure why you get:

> ...
> 
> zsh: parse error near `|'
> zcompile: can't read file: /nfs-home/adams/local/share/zsh/4.0.1-pre-2/functions/Completion/Builtins/_vars_eq

There are only `||'s in _vars_eq, no `|'s.  Do you have any aliases
that might get in the way?  You'll have to use `zcompile -U ...', then.

> Also, for this to work, wouldn't I have to include ~/.zsh/comp.zwc in
> fpath? 

Right.

> Why did the example suggest only to compile Completion/*/_*,
> not Completion/*/* ?

So that one saves the (probably not too large) space for the comp*
functions which get called only seldom and unfunction themselves.  If
they are in a mapped zwc file they might still be kept in memory even
though they will never be called again through the lifetime of the shell.

> I get the impression there are several different strategies one could
> take with compilation.  For example, how much is there to be gained by
> replacing all the Completion/* directories from the fpath with a
> single compiled foo.zwc digest?  Could you even do that, given that
> presumably foo.zwc would be missing Completion/Core/comp* ?

Of course different strategies might be useful: everything into one
zwc (probably the fastest -- I even put my personal utility functions
in my master zwc file), one zwc digest per directory... Using one zwc
file per function would be a bad idea, though.  Most of them will not
be mapped unless forced with the -M option but then one would have so
many mapped files open that the kernel might get unhappy.

Using the zwc digest for only the completion functions (sans comp*)
works, of course (see above), but you'll have to put at least the Core 
directory in your $fpath.  Put it after the zwc file and the shell will 
starting searching that, find the function it's searching and not look 
at the directories, so it's quite fast (after the first function found 
in the digest file the `directory' from the zwc is mapped in memory
and searching it...).

> The more I think about it, the more I like the idea of `make install'
> (or maybe `make install.zwc') taking the most sensible compilation
> route.  The distribution's completion functions probably aren't going
> to change at all in between upgrades, after all.

I wouldn't be against that, I just wouldn't know what style to suggest:
one for all, one for each directory, and there are also the functions 
in the Functions directory... Although users still have to call
autoload with the function names they want so it isn't that big a
problem if the global zwc digest contains functions some users don't
want to use.  Hm.

> > > Completing files with TAB was noticeably slower with the new
> > > system over the old.
> > 
> > As Adam pointed out, this of course depends on the completers one uses 
> > (and the match specs, btw.).
> 
> Perhaps these performance hits should be mentioned in the manual?

Well, that putting many function into the completer style makes things 
slower shouldn't be that surprising.  And for the match specs I'm not
completely sure what to suggest (even thought I asked Peter to put
something about that into the Guide, which he did).  Specs with `*'
are slower than others, specs with `**' even more so.  But depending
on the number, length and complexity of the matches one probably
wouldn't notice that on modern computers.  Using many matchers with
the matcher-list style can make things slow, too, but I always think
that using to many different matchers isn't that good idea anyway
(because they will probably match too often) so that users will find
that out if they want or not anyway.

But of course I'm not against putting a little warning here and there
into the manual.  Maybe we could first collect some reports from
everyone here about things which they considered to be too slow (and
how they solved it).

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: completion tricks

[Adam Spiers]

Sven Wischnowsky (wischnow@informatik.hu-berlin.de) wrote:
> > Would you mind showing how you do this quickly?
> 
> I use this:
> 
>     zrec() {
>       zrecompile "$@" -p -- \
>         -R ~/.zshrc -- \
>         -R ~/.zdump -- \
>         -R ~/lib/zsh/comp/Core/compinit -- \
>         -R ~/lib/zsh/comp/Core/compaudit -- \
>         -R ~/lib/zsh/comp/Core/compdump -- \
>         -M ~/lib/zsh/funcs.zwc \
>            ~/funcs/*(.) \
>            ~/lib/zsh/comp/*/_* \
>            ~/lib/zsh/func/*/*
>     }

That's very instructive; thanks.  Why .zdump not .zcompdump?

> Never mind the funny paths.

But I'm curious now :-)

Here's my version:

  #autoload

  _Core_dir=$fpath[(r)*Core*]
  _func_dir=${_Core_dir%/Completion/Core}

  zrecompile "$@" -p -- \
    -R ~/.zshenv -- \
    -R ~/.zshrc  -- \
    -R ~/.zcompdump  -- \
    -R $_Core_dir/compinit -- \
    -R $_Core_dir/compaudit -- \
    -R $_Core_dir/compdump -- \
    -M -U ~/.zsh/funcs.zwc \
          ~/.zsh/functions/*(*) \
          $_func_dir/Completion/*/_* \
          $_func_dir/Prompts/*

Seems to work fine.

> > zsh: parse error near `|'
> > zcompile: can't read file: /nfs-home/adams/local/share/zsh/4.0.1-pre-2/functions/Completion/Builtins/_vars_eq
> 
> There are only `||'s in _vars_eq, no `|'s.  Do you have any aliases
> that might get in the way?  You'll have to use `zcompile -U ...', then.

Ahah, well spotted.  I have a few single capital letter global aliases
which were getting in the way.

> > Also, for this to work, wouldn't I have to include ~/.zsh/comp.zwc in
> > fpath? 
> 
> Right.

OK.  Presumably it has to come before the normal Completion/* paths?
I find the ELEMENT.zwc bit of the manual pretty hard to understand.

When I put it at the head of fpath, memory consumption appears to
grow from:

USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
adams     1421  0.2  0.7  3092 2052 pts/12   S    16:48   0:00 ..../zsh

to:

USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
adams     1617  8.0  0.8  3772 2172 pts/3    S    16:50   0:00 ..../zsh

Why's that?  Startup time is better though, as expected.

> > Why did the example suggest only to compile Completion/*/_*,
> > not Completion/*/* ?
> 
> So that one saves the (probably not too large) space for the comp*
> functions which get called only seldom and unfunction themselves.  If
> they are in a mapped zwc file they might still be kept in memory even
> though they will never be called again through the lifetime of the shell.

Makes a lot of sense.

> > The more I think about it, the more I like the idea of `make install'
> > (or maybe `make install.zwc') taking the most sensible compilation
> > route.  The distribution's completion functions probably aren't going
> > to change at all in between upgrades, after all.
> 
> I wouldn't be against that, I just wouldn't know what style to suggest:
> one for all, one for each directory, and there are also the functions 
> in the Functions directory... Although users still have to call
> autoload with the function names they want so it isn't that big a
> problem if the global zwc digest contains functions some users don't
> want to use.  Hm.

Exactly.

> > Perhaps these performance hits should be mentioned in the manual?
> 
> Well, that putting many function into the completer style makes things 
> slower shouldn't be that surprising.  And for the match specs I'm not
> completely sure what to suggest (even thought I asked Peter to put
> something about that into the Guide, which he did).  Specs with `*'
> are slower than others, specs with `**' even more so.  But depending
> on the number, length and complexity of the matches one probably
> wouldn't notice that on modern computers.  Using many matchers with
> the matcher-list style can make things slow, too, but I always think
> that using to many different matchers isn't that good idea anyway
> (because they will probably match too often) so that users will find
> that out if they want or not anyway.
> 
> But of course I'm not against putting a little warning here and there
> into the manual.  Maybe we could first collect some reports from
> everyone here about things which they considered to be too slow (and
> how they solved it).

Sounds good.  So far, we have already mentioned _approximate and
_correct (I still haven't got round to figuring out the difference
between the two), and match specs.  Some completions used to be really
expensive, e.g. _perl_modules, _man, and _rpm, but hopefully the
caching helps a lot with that.  (Incidentally there are some rough
edges with the caching mechanism which I *still* haven't got round to
smoothing.)  Anything else?

Re: completion tricks

[Sven Wischnowsky]

Adam Spiers wrote:

> ...
> 
> That's very instructive; thanks.  Why .zdump not .zcompdump?

As I said, I (now) use 4.0.1-pre2 as my normal shell -- that uses
~/.zdump.  But in at least one terminal I'm always running the latest
development version for testing -- that uses the normal ~/.zcompdump
which makes it easier with `zsh -f' and things like that.  And I can't 
use one dump file for both, because functions may change...

> ...
> 
> > > Also, for this to work, wouldn't I have to include ~/.zsh/comp.zwc in
> > > fpath? 
> > 
> > Right.
> 
> OK.  Presumably it has to come before the normal Completion/* paths?
> I find the ELEMENT.zwc bit of the manual pretty hard to understand.
> 
> When I put it at the head of fpath, memory consumption appears to
> grow from:
> 
> USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
> adams     1421  0.2  0.7  3092 2052 pts/12   S    16:48   0:00 ..../zsh
> 
> to:
> 
> USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
> adams     1617  8.0  0.8  3772 2172 pts/3    S    16:50   0:00 ..../zsh
> 
> Why's that?  Startup time is better though, as expected.

That's because the shell has to map the `directory' contained in the
digest file (i.e. the list of function names whose definitions are in
the digest).  Also, your OS may choose to map more pages than are
actually used.  If the digest file is at the end of $fpath, the shell
will first find the definition file, load the function from it and
never touch the digest -- which may result in slightly lower memory
consumption at the beginning.  However, as soon as there is a second
zsh running using the digest file will occupy less memory because the
second shell shares the mapped digest.  Given the different memory
mapping policies one probably can't predict which way to load
functions looks better at the beginning.

> ... [ giving tips about performance in the docs ]
> 
> Sounds good.  So far, we have already mentioned _approximate and
> _correct (I still haven't got round to figuring out the difference
> between the two), and match specs.  Some completions used to be really
> expensive, e.g. _perl_modules, _man, and _rpm, but hopefully the
> caching helps a lot with that.  (Incidentally there are some rough
> edges with the caching mechanism which I *still* haven't got round to
> smoothing.)  Anything else?

_cvs. I'm not sure if we should mention normal completion functions -- 
it could make some people not use the system even if their hardware is 
very well capable of supporting the completion system.


Bye
 Sven

P.S.: _approximate = _correct + _complete.  One could say that
      _approximate expects missing characters at the end of the word
      (and doesn't count them as errors), while _correct thinks that
      the word on the line is thought to be correct -- and checks if
      it is.

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de