zsh's answer to the bash completion fm project

zsh's answer to the bash completion fm project

[John Buttery]

[-- Attachment #1: Type: text/plain, Size: 1147 bytes --]

  I was just reading freshmeat and saw the "bash programmable
completion" project listed; does anybody have any interest in doing
something similar for/with zsh?
  For those not familiar with it, it's basically a project to create in
bash what in zsh would be a file containing compinit/zstyle completion
statements; just a big, sourceable .bashrc file (loaded from and in
addition to the user's normal .bashrc, presumably).
  Is there perhaps already a project like this that I don't know about?
If not, I think it would be neat to start one.  Question: if I/we were
to start one, what's the consensus on whether compinit or zstyle should
be used?  Admittedly I'm a relatively new zsh user, but it seems from
reading the docs that the "zstyle" builtin is pretty new, so although it
may be better, there may be vastly more voluminous content out there
using the older system (kinda like it was with ipchains/iptables when
iptables came out).  Perhaps it could use both, and if so would that be
a Bad Thing(tm)?  Assuming nobody felt like "porting" the old compinit
stuff to zstyle of course...

  Opinions? 

-- 
john@io.com

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: zsh's answer to the bash completion fm project

[Sven Wischnowsky]

John Buttery wrote:

>   I was just reading freshmeat and saw the "bash programmable
> completion" project listed; does anybody have any interest in doing
> something similar for/with zsh?

Actually, we already have that in recent distributions, see below...

> ...
>   Is there perhaps already a project like this that I don't know about?
> If not, I think it would be neat to start one.  Question: if I/we were
> to start one, what's the consensus on whether compinit or zstyle should
> be used?

Both compinit and zstyle ar new -- and they do different things.
Compinit is only a function that has to be called to get the new
(sic!) completion system going.  The real work, i.e. the code to
generate matches is distributed over many (many!) autoloaded function
files that will be loaded and called on demand by the things compinit
sets up.

When the completion system grew and grew we had more and more demand
for configurability and the final solution for that (after some other
attempts) was zstyle.  Its name doesn't start with `comp' because is
can (and has been) used elsewhere, because it is a pretty general
solution for context-dependent configurability.

So, compinit and zstyle play together.  The completion functions use
the zstyle builtin to look up users' settings (preferences) and the
user calls compinit to use the completion system at all and (s)he uses
zstyle to make it behave the way (s)he likes -- as far as the
completion functions allow themselves to be configured, of course, but
that's really very far.  I suggest you read chapter six of Peter's
user friendly user's guide available from www.zsh.org to get a taste
of all the possibilities.


Bye
  Sven

P.S.: The old system use[ds] the `compctl' builtin, which is very ugly
      and not nearly as powerful and flexible as the new system.  In
      some of the interims versions we had a builtin `compgen' which
      just used the internal C-code from `compctl' and added matches
      in a way usable for the new system.  That builtin has long
      vanished, and I mention it only because it was at that time that
      Chet Ramey posted a sneak preview to bash's completion features.
      And if you have a look, you'll see that it still uses a builtin
      called `compgen' and it still looks like a mixture of our old
      `compctl' and the new `compadd' (which is the builtin behind the
      scenes, used in the completion functions of the new system).
      I don't know exactly in which shape bash's `compgen' is
      nowadays, but the last time I looked there were still some
      features missing (i.e. promised but not yet implemented) and it
      wasn't nearly as powerful as our completion code (which isn't a
      surprise, because it is much younger).
      And, of course, they'll have a lot to do to reach the state the
      functions in our completion system -- the works of many people
      on the -workers list (thanks guys!) -- and even if they
      implement as many functions as we have, they probably won't be
      as powerful and configurable, at least at the beginning.
      Plus, we have a real autoloading mechanism (does bash have one
      nowadays?) and we have these digest files that can be used to
      make the whole completion system be mmap()ed, saving memory and
      loading time. I wouldn't want to have to load/define all of our
      334 completion functions at the startup of every shell.

-- 
Sven Wischnowsky                          wischnow@berkom.de

Re: zsh's answer to the bash completion fm project

[Thomas Köhler]

[-- Attachment #1: Type: text/plain, Size: 1911 bytes --]

John Buttery wrote [020403 13:33]:
>   I was just reading freshmeat and saw the "bash programmable
> completion" project listed; does anybody have any interest in doing
> something similar for/with zsh?
>   For those not familiar with it, it's basically a project to create in
> bash what in zsh would be a file containing compinit/zstyle completion
> statements; just a big, sourceable .bashrc file (loaded from and in
> addition to the user's normal .bashrc, presumably).
>   Is there perhaps already a project like this that I don't know about?
> If not, I think it would be neat to start one.  Question: if I/we were
> to start one, what's the consensus on whether compinit or zstyle should
> be used?  Admittedly I'm a relatively new zsh user, but it seems from
> reading the docs that the "zstyle" builtin is pretty new, so although it
> may be better, there may be vastly more voluminous content out there
> using the older system (kinda like it was with ipchains/iptables when
> iptables came out).  Perhaps it could use both, and if so would that be
> a Bad Thing(tm)?  Assuming nobody felt like "porting" the old compinit
> stuff to zstyle of course...
> 
>   Opinions? 

Seems like it's not needed. zsh comes with tons of functions
predefined for the new completion system ;)

Have you tried to put
   autoload -U compinit
   compinit
in your .zshrc and then type something like
   ssh tk<TAB>u<TAB>
and see how it is expanded to something like
   ssh tkoehler@unser.linux.laeuft.auf.s390.org
or something?

There's quite a lot of stuff there in zsh-4.0.x :-)

Ciao,
Thomas

-- 
 Thomas Köhler Email:   jean-luc@picard.franken.de     | LCARS - Linux
     <><        WWW:     http://jeanluc-picard.de      | for Computers
                IRC:             jeanluc               | on All Real
               PGP public key available from Homepage! | Starships

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: zsh's answer to the bash completion fm project

[John Buttery]

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]

* Sven Wischnowsky <wischnow@berkom.de> [2002-04- 3 14:09:09 +0200]:
> Actually, we already have that in recent distributions, see below...

  I didn't see a reference in the email, unless you mean section 6 of
the user guide...checking that out now.

> Both compinit and zstyle ar new -- and they do different things.

  Doh!  I meant compctl, not compinit.

  Basically, the old completion code vs. the new code.

-- 
john@io.com

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: zsh's answer to the bash completion fm project

[John Buttery]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="us-ascii:iso-8859-1:iso-8859-2:utf-8", Size: 892 bytes --]

* Thomas Köhler <jean-luc@picard.franken.de> [2002-04- 3 14:24:31 +0200]:
> Seems like it's not needed. zsh comes with tons of functions
> predefined for the new completion system ;)

  I know, but I've found that when you give people an avenue to
contribute their own, one Lego at a time, you can often wind up with
something pretty amazing in the end (sort of like Open Source in general
:).  Perhaps any output from this hypothetical project could be folded
into zsh's stock configuration periodically, at which point we would
zero our config files and start over.
  I've played around a lot with zsh's preloaded functions, and they are
certainly very extensive, but there's always more stuff to do.  The
example that pops to mind is a compctl "recipe" that allows dd to do
filename completion on arguments that start with (for example) "if=" and
"of=".

-- 
john@io.com

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: zsh's answer to the bash completion fm project

[Sven Wischnowsky]

John Buttery wrote:

>   I know, but I've found that when you give people an avenue to
> contribute their own, one Lego at a time, you can often wind up with
> something pretty amazing in the end (sort of like Open Source in general
> :).  Perhaps any output from this hypothetical project could be folded
> into zsh's stock configuration periodically, at which point we would
> zero our config files and start over.
>   I've played around a lot with zsh's preloaded functions, and they are
> certainly very extensive, but there's always more stuff to do.  The
> example that pops to mind is a compctl "recipe" that allows dd to do
> filename completion on arguments that start with (for example) "if=" and
> "of=".

Ah, I see, sorry I didn't understand you.

Well, anyone on this and all neighbouring planets are invited to share
their functions with us anyway, and several people not on the -workers
list have done that in the past.

If you have the time to start a project so that we get even more
completion functions or improvements for completion functions as
input, we'd all be glad and you'll almost certainly get any support
from the -workers list we can give. 

I'm not quite sure how support for zstyle would look like, though,
because that's so user-dependent, but a list of things people can just
copy and paste into their init files might already be of great value,
I think.


And as for compctl: I don't plan to give it any support anymore and
don't care if others do. There is still that file with several
compctls in the distribution as a starting point.


Bye
  Sven

-- 
Sven Wischnowsky                          wischnow@berkom.de

Re: zsh's answer to the bash completion fm project

[Thomas Köhler]

[-- Attachment #1: Type: text/plain, Size: 1557 bytes --]

John Buttery wrote [020403 16:57]:
> * Thomas Köhler <jean-luc@picard.franken.de> [2002-04- 3 14:24:31 +0200]:
> > Seems like it's not needed. zsh comes with tons of functions
> > predefined for the new completion system ;)
> 
>   I know, but I've found that when you give people an avenue to
> contribute their own, one Lego at a time, you can often wind up
> with something pretty amazing in the end (sort of like Open
> Source in general :).  Perhaps any output from this
> hypothetical project could be folded into zsh's stock
> configuration periodically, at which point we would zero our
> config files and start over.

Why "zero our config files"?
You put the completion functions in an extra file in a directory
in your fpath.
If you want the system functions take over precedence, put your
own dir at the end of fpath :)

>   I've played around a lot with zsh's preloaded functions, and they are
> certainly very extensive, but there's always more stuff to do.  The
> example that pops to mind is a compctl "recipe" that allows dd to do
> filename completion on arguments that start with (for example) "if=" and
> "of=".

Why would you want a compctl recipe if you get it for free in the
new completion system? :-)

Ciao,
Thomas

-- 
 Thomas Köhler Email:   jean-luc@picard.franken.de     | LCARS - Linux
     <><        WWW:     http://jeanluc-picard.de      | for Computers
                IRC:             jeanluc               | on All Real
               PGP public key available from Homepage! | Starships

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: zsh's answer to the bash completion fm project

[John Buttery]

[-- Attachment #1: Type: text/plain, Size: 3067 bytes --]

* Sven Wischnowsky <wischnow@berkom.de> [2002-04- 3 17:10:15 +0200]:
> If you have the time to start a project so that we get even more
> completion functions or improvements for completion functions as
> input, we'd all be glad and you'll almost certainly get any support
> from the -workers list we can give. 

  Sure, I don't mind being the central point for this.  Let me get a
starting file together; I'll start with some of the recipes I've picked
up from the web (both compctl and zstyle) and we can go from there.

> I'm not quite sure how support for zstyle would look like, though,
> because that's so user-dependent, but a list of things people can just
> copy and paste into their init files might already be of great value,
> I think.

  Well, this is how I envision it; and of course any improvement up to
and including a complete rewrite is welcome since, like I said, I'm a
newbie.  :)  Basically, I'm starting this project with motivation but
not necessarily a huge amount of knowledge on the subject.  But what a
great way to learn, huh?

> And as for compctl: I don't plan to give it any support anymore and
> don't care if others do. There is still that file with several
> compctls in the distribution as a starting point.

  That's the way I figured it: put the compctl recipes in there when
there isn't a zstyle equivalent (meaning, nobody's bothered to convert
it yet).  These are the questions I'd like to ask the group to give me
the basis for getting started:

  1) I'd like to encourage people to write any new submissions with
zstyle but take both, and initialize both completion systems so they can
all be used.  Is this possible and/or is there a compelling reason not
to do it this way?

 2) What are anybody's thoughts on the performance differences between
having everything in one file, or having one "master" file with no
actual content, but that just initializes the two completion systems and
sources other files for the actual recipes...does it make a difference?

 3) Would somebody mind writing a little code block that checks if each
completion system has already been initialized, and initializes it if
not?  Separate logic for each would be cool.

 4) What should the interpreter line for these files be?  In other
words, what's the most common path for zsh (mine is in /bin, but I've
seen it in /usr/bin and /usr/local/bin also), and what (if any)
command-line options should be used (such as -f, or the equivalent of
bash's -norc/-noprofile)?

  5) Should we zcompile the files?  I'm assuming yes, but just thought
I'd ask in case there are caveats.  Does it matter if the version of zsh
that zcompiles a file is different from the one that runs it?  If so, we
should just ship with instructions on how to zcompile that the users can
follow.

  6) Opinions on which of the startup files we should recommend people
call our setup from?  Before or after they initialize their own
completion recipes (in theory the user's should override ours)?

-- 
john@io.com

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: zsh's answer to the bash completion fm project

[Bruce Stephens]

John Buttery <john@io.com> writes:

[...]

>   That's the way I figured it: put the compctl recipes in there when
> there isn't a zstyle equivalent (meaning, nobody's bothered to convert
> it yet).  These are the questions I'd like to ask the group to give me
> the basis for getting started:

I think you misunderstand: compctl is obsoleted by the new completion
system, and the new completion system is one file per command
(roughly, anyway), so it's naturally easy for people to submit
completions for new commands---they just provide a new file.  

zstyle offers a way of individual users configuring the completion
system---for example, whether (on <TAB> when there's a choice) it
offers a list of completion options, or cycles through the
alternatives.

In zsh 4.0.4, there are already completions for if= and of= on dd,
BTW.  Look at the file
/usr/share/zsh/4.0.4/functions/Completion/Unix/_dd, suitably altered
if you installed somewhere else---this is using Debian unstable's
package.

[...]

Re: zsh's answer to the bash completion fm project

[John Buttery]

[-- Attachment #1: Type: text/plain, Size: 1345 bytes --]

* Bruce Stephens <bruce@cenderis.demon.co.uk> [2002-04- 3 23:05:47 +0100]:
> I think you misunderstand: compctl is obsoleted by the new completion
> system, and the new completion system is one file per command
> (roughly, anyway), so it's naturally easy for people to submit
> completions for new commands---they just provide a new file.  

  Woops, your right...ok, I said I was a newbie but I didn't think it
was that bad.  Yes, you're right.

  So that changes my original question to: Can/should we have a file for
compctl recipes, and then a directory containing the files for the stuff
made with the new system?

  Oh, and to the people who wrote me privately, pointing out that
compctl is strongly deprecated in favor of the new system, that it
sucks, etc etc...yes, I know this, I wasn't suggesting we advocate using
it.  All I meant was that if a recipe exists for compctl and hasn't been
ported to the new system yet, we should still include it unless using
both systems at once is impossible and/or a huge resource hit.

  So now that I'm up to speed on what's going on with the two completion
systems, my next project is to inspect the "onboard" completion
directories to see how many of my current compctl recipes are duplicated
there.  I should be able to have a starting point soon...

-- 
john@io.com

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: zsh's answer to the bash completion fm project

[Sven Wischnowsky]

John Buttery wrote:

> ...
> 
>   So that changes my original question to: Can/should we have a file for
> compctl recipes, and then a directory containing the files for the stuff
> made with the new system?

About the compctl part: I'd only consider such a list of
compctl-commands to be a todo-list of things that should be
implemented for the new system.  And that may actually be helpful.
I'm not sure that there will be many such cases, though, with the new
system being as comprehensive as it already is.

About the function-directory: the new system uses a hierarchy of
directories for classification and easier management.  New functions
will and should be put into it (and hence will end up in the
distribution).

For both parts: one of the things to look out for is opinions and
suggestions for different ways completion behaves.  We recently had
someone who user@host completions to be inserted in one go, not with
`us<TAB>ho<TAB>'.  Collecting such things would be interesting so that
we get a list of things we can try to make possible in the new system.

>   Oh, and to the people who wrote me privately, pointing out that
> compctl is strongly deprecated in favor of the new system, that it
> sucks, etc etc...yes, I know this, I wasn't suggesting we advocate using
> it.  All I meant was that if a recipe exists for compctl and hasn't been
> ported to the new system yet, we should still include it unless using
> both systems at once is impossible and/or a huge resource hit.

I think they can still be combined, at least they should.  The
overhead for compctl from a user's point of view is basically that
there is another module loaded (for systems without dynamic linking it
has to be linked into the zsh binary).  From my point of view there is
quite a bit of legacy code in the completion code that's only needed
to support compctl.  And I hate that, but don't see a way around it.


Bye
  Sven

-- 
Sven Wischnowsky                          wischnow@berkom.de

Re: zsh's answer to the bash completion fm project

[John Buttery]

[-- Attachment #1: Type: text/plain, Size: 1693 bytes --]

* Sven Wischnowsky <wischnow@berkom.de> [2002-04- 4 12:07:04 +0200]:
> About the compctl part: I'd only consider such a list of
> compctl-commands to be a todo-list of things that should be
> implemented for the new system.  And that may actually be helpful.
> I'm not sure that there will be many such cases, though, with the new
> system being as comprehensive as it already is.

  That's exactly the way I thought about it.

> About the function-directory: the new system uses a hierarchy of
> directories for classification and easier management.  New functions
> will and should be put into it (and hence will end up in the
> distribution).

  Ditto.

> For both parts: one of the things to look out for is opinions and
> suggestions for different ways completion behaves.  We recently had
> someone who user@host completions to be inserted in one go, not with
> `us<TAB>ho<TAB>'.  Collecting such things would be interesting so that
> we get a list of things we can try to make possible in the new system.

  Agree.

> I think they can still be combined, at least they should.  The
> overhead for compctl from a user's point of view is basically that
> there is another module loaded (for systems without dynamic linking it
> has to be linked into the zsh binary).  From my point of view there is
> quite a bit of legacy code in the completion code that's only needed
> to support compctl.  And I hate that, but don't see a way around it.

  Well, you just pretty much stated my exact vision for the project.  :) 

  I have a working prototype of the "master script"; I'll try and clean
it up a little more and post it later today.

-- 
john@io.com

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]