Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Sven Wischnowsky]

Bart Schaefer wrote:

> 
> Let's think about this a different way for a moment.
> 

Yes, I thought about similar things since your last mail...

> First, consider the manipulations that completion performs on the input
> line:
> 
> * Insert a word fragment at the cursor.
> * Erase the whole current word and replace it with another one.
> * Either of the above, but cycling through a list of replacements.
> * Append a suffix to the word or word fragment.
> * Set a flag indicating that the suffix is a candidate for removal.
> * Move the cursor to the end of the word.
> * Insert a space if the completion appears unique.
> * Etc. (I'm sure I've missed some.)
> 

I prefer thinking about collecting information, like:

- what should be shown in the case of ambiguous information
  (here and in menu-completion, which is the same as the behaviour for 
  a single match I nowadays only think of: `remove the current word,
  replace it with ...', makes things easier)
- which part of the inserted line should be automatically be removed
  (after which keys are typed)
- for each match: what should be inserted
- for each match: what should be listed
- for each match: is it a file? (if it is a file, we probably need a
  path for file type testing with LIST_TYPES)
- the things from the options (like recexact, automenu,
  listambiguous,...)

What should be inserted depends on things like:

- a -P prefix
- a -S suffix
- probably some ignored prefix that already was on the line
  (this also determines where the -P prefix should be inserted)
- some extra slash, space, etc.
- two strings that were temporarily removed if we are completing
  inside a brace expansion
- quoting or not

>
> ...
> 
> I suggest that we provide a builtin (or a collection of them) to apply
> the collection of tests to the current input line, plus a collection of
> what amount to zle widgets to permit the user to explicitly direct the
> changes to the input line that take place when the tests succeed.
> 

You mean that we should use a function for each case where we now use
a compctl (or probably you mean to use one fornction for all compctls
together), right?
I would prefer the solution with different functions which would
probably be called from some `master function (or widget)'. We could
add some support to make the selection of the function(s) easier.
The sub-functions would then call the builtin(s) to generate the
matches, they could access the command line (making compctl -x
superfluous, use `if ...' instead). They would probably also make
suggestions how to present things to the user (which could be
overridden in the master function).
The information collected could be stored (like the matches are stored 
now for later menu-completion) and made accessible via some builtin,
more precisely: all the `add matches for some kind of test' things
would only add to this information. We could add ways to create new
such collections, remove some things from them, add to them, change
them, and remove them. This would allow for things like: do
completion, do something other, continue completion at the place were
we just left.
For this `master function' we should add some builtin implementations
that use the well known options, compctl would be removed/changed.

Ok, I just tried to collect your ideas from various mails, did I
interpret them correctly?

I like this approach and it doesn't even look too hard to implement
(if we do it in multiple steps, the information gathering phase can
become quite tricky (internally)), my last patches may even help here
since they cleaned up some parts of zle_tricky.c.

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Sven Wischnowsky]

Bart Schaefer wrote:

> ...
> 
> } What we currently know as key functions that do completion would be
> } implemented as widgets. The completion functions we have now are
> } builtin widgets but others can be defined by the user.
> 
> Sort of, but not widgets in the sense of existing zle widgets (that is,
> I don't want to have to choose just one of them to bind to the TAB key).
> When you say:
> 

What I meant was: all the mechanisms needed to reimplement each of the 
currently existing completion key functions would be available to the
user, for compatibility we would preserve them as builtin key
functions.
Of course, since everything need to rimplement them should be broken
in several parts, users would be able to define completion functions
with behaviors different from all currently known completion
functions.

> } Such a widget for completion would select a shell function that
> } produces the matches. I.e., no more compctls but shell functions that
> } use the builtin to produce matches and that use `if...then' and other
> } things for things we now do with `-x' (which will become obsolete).
> 
> I wasn't thinking of quite so much differentiation between the "widget
> for completion" and the "function that produces the matches"; that is,
> there's no reason the widget couldn't produce the matches directly,
> and no reason the function that produces the matches couldn't also e.g.
> set the autoremovable suffix or specify that menu completion be used.
> 

Of course. The shell functions called to produce the matches would
just be shell functions in the same special `calling environment' as
the widget. So this widget itself would be able to produce matches and 
the functions called would be able to say something about how matches
should be inserted, presented to the user, or whatever.
If we use something like `completion contexts', these could be
modified even outside any of these functions.

(There would be a bit of a problem with the allocation behavior of the 
completion code, since we would need these context and the matches in
them be allocated in real memory where the completion code now uses
heap memory.)

> (It'd also be great if every match in the list could have some of its
> own attributes, e.g. prefix and suffix strings, -U-ness, etc.  But I
> think that can wait for the next overhaul.)
> 

My patch for inclusive-ors already does this for many of the different 
things for which it would be useful (-P, -S, -U, and for -Q it was
always done).

> 
> } After all selected completion-producing shell functions have been
> } executed, the widget uses the builtin to say what should be presented
> } to the user (e.g. show a list of matches, change the command line, or
> } both, or...).
> 
> Possibly.  I was hoping all that could be set up in advance, i.e. during
> generation of the matches, and then applied when the entire match list
> was finally available.  (Maybe that would require per-match attributes,
> which I just wrote off for now.  Hm.)
> 

As I wrote, here I was thinking only about things like `should we
show a list?', not about things like `how should this match be
inserted?', that can (and should) be defined when the matches are
generated.


Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[TGAPE!]

This thread was started by someone who wanted to add some functionality
to zsh.  What he was asking for sounded somewhat reasonable.  I went to
another window, and I tried a test.

	setopt dirs^[b^I
markdirs nohashdirs pathdirs

It worked, no problem.  I'm running 3.1.1.  What is the most depressing
about this is that it is very common.

The list then hashed about how to solve this problem, how to implement
it, how to make it simpler to use after implementation, and so forth.
My heart was gladdened when I finally bespied the word simplify, because
that is what zsh needs at this point; I've not upgraded since 3.1.1,
because it does everything I need, and I've not heard of improvements in
performance (i.e. speed) since.  I was horrified to see what their idea
of simplification was.  Zsh is getting seriously bloaty.

The only bright spot in this was, at long last,

Bart Schaefer wrote:
>
> Let's think about this a different way for a moment.
>
> First, consider the manipulations that completion performs on the input
> line:

<snip>

Someone finally seems to be actually studying the problem, rather than
doing a quick patch like this group is so wont on doing.  (Don't get me
wrong, it frequently is very useful, but this isn't a quick-fix problem,
this is something huge and complicated.)  Basically, the current problem
is probably best described as, "Well, we've added in virtually all the
completion functionality we can think of, but it's now a hairy mess.  We
need to bring it together under one metaphore, one unified structure."

Another tact, which has been being worked, is to make better
documentation for it.  This is also a good idea, but I haven't seen
anyone proposing any improvements on how to describe things on here.  I
think that is probably at least as important as the structure with which
you define the completions.

> It's sort of a RISC v. CISC argument, actually; it's easier to understand
> what's going on with RISC instructions, because each one is simpler, but
> you have to issue a lot more of them to get anything done.  The current
> compctl is confusing for two reasons:  (1) it uses single-character and
> thereby minimally mnemonic options to describe everything; and (2) the
> effect of any one of those options may be a very complex change to the
> input, and so combining them is either bewildering or impossible.
> 
> Addressing (1) with a flowery syntax doesn't help with (2), and I think
> that if (2) were cleaned up then the syntax for (1) wouldn't need to be
> as flowery.

The problems I've basically had with this have been one of flow, caused
by (2).  A helper program would do wonders to solving this.  This could
also aleiviate (1), since the person would no longer need to depend on
the key letters while they're programming.

I'm envisioning something which displays a menu, with the key letters in
brackets.  When an option is chosen, it changes to have a brief
description of the option, and guides for how to fill in any arguments
to the command.

As I'm thinking about it, I could even develop it utilizing completion
code, however, I think that this would send the wrong message; at some
point in time, functionality gets to be silly.  For example, ** may be
useful for simple cases, but when I do something that would take 240 or
more characters with find, it gets excessive.  Sure, I'm still under 80
characters with zsh globbing, but the point is that it is merely
confusing and inefficient; find runs much quicker, and I'll spend more
keystrokes moving through the zsh documentation, and flipping between
virtual consoles, than I'll save over the find line.  Not only that, but
find is easier to read later.  (Plus, find is not many keystrokes;
compctl has seen to that.)

Ed Grimm

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Bart Schaefer]

On Aug 26,  4:34pm, Sven Wischnowsky wrote:
} Subject: Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)
}
} We have a new builtin that allows us to use (parts of) the completion
} code. The builtin allows us to create, change, and destroy completion
} contexts. Changing them means to add matches, to remove matches, or to 
} change matches.

Ok ...

} What we currently know as key functions that do completion would be
} implemented as widgets. The completion functions we have now are
} builtin widgets but others can be defined by the user.

Sort of, but not widgets in the sense of existing zle widgets (that is,
I don't want to have to choose just one of them to bind to the TAB key).
When you say:

} Such a widget for completion would select a shell function that
} produces the matches. I.e., no more compctls but shell functions that
} use the builtin to produce matches and that use `if...then' and other
} things for things we now do with `-x' (which will become obsolete).

I wasn't thinking of quite so much differentiation between the "widget
for completion" and the "function that produces the matches"; that is,
there's no reason the widget couldn't produce the matches directly,
and no reason the function that produces the matches couldn't also e.g.
set the autoremovable suffix or specify that menu completion be used.

(It'd also be great if every match in the list could have some of its
own attributes, e.g. prefix and suffix strings, -U-ness, etc.  But I
think that can wait for the next overhaul.)

} The builtin may offer some support for selecting a shell function
} appropriate for the current command line (of course, this wouldn't
} depend on the command line itself, but would somehow allow us to
} quickly select a (few) function(s) for certain command names or
} whatever).

Yes.

} After all selected completion-producing shell functions have been
} executed, the widget uses the builtin to say what should be presented
} to the user (e.g. show a list of matches, change the command line, or
} both, or...).

Possibly.  I was hoping all that could be set up in advance, i.e. during
generation of the matches, and then applied when the entire match list
was finally available.  (Maybe that would require per-match attributes,
which I just wrote off for now.  Hm.)

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Sven Wischnowsky]

Bart Schaefer wrote:

> 
> ...
> 
> I suggest that we provide a builtin (or a collection of them) to apply
> the collection of tests to the current input line, plus a collection of
> what amount to zle widgets to permit the user to explicitly direct the
> changes to the input line that take place when the tests succeed.
> 

Ok, I'll try to collect the suggestions you made, combine with a few
additions from me, and come up with a description for that.

We have a new builtin that allows us to use (parts of) the completion
code. The builtin allows us to create, change, and destroy completion
contexts. Changing them means to add matches, to remove matches, or to 
change matches.

What we currently know as key functions that do completion would be
implemented as widgets. The completion functions we have now are
builtin widgets but others can be defined by the user.

Such a widget for completion would select a shell function that
produces the matches. I.e., no more compctls but shell functions that
use the builtin to produce matches and that use `if...then' and other
things for things we now do with `-x' (which will become obsolete).
The builtin may offer some support for selecting a shell function
appropriate for the current command line (of course, this wouldn't
depend on the command line itself, but would somehow allow us to
quickly select a (few) function(s) for certain command names or
whatever).

After all selected completion-producing shell functions have been
executed, the widget uses the builtin to say what should be presented
to the user (e.g. show a list of matches, change the command line, or
both, or...).

Using compltion contexts makes it possible to implement menu
completion easily (there might be an option to the builtin that says:
use the next match from context ...). It would also allow us to define 
widgets that do menu-completion while allowing other editing commands
between menu-completion commands. Of course, lots of other interesting 
things should be possible with that, too.

Hm, I think all this sounds good, and even not too hard to implement
(my last changes to zle_tricky.c might even help with this since they
make the completion code in question cleaner).

Some questions are:

- Are shell functions much bigger?
- Would this be too slow?
- Are there any problems I didn't see right now?

Anyway, did I understand you correctly (enough)? And what do the
others think about all this?

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Bart Schaefer]

Let's think about this a different way for a moment.

First, consider the manipulations that completion performs on the input
line:

* Insert a word fragment at the cursor.
* Erase the whole current word and replace it with another one.
* Either of the above, but cycling through a list of replacements.
* Append a suffix to the word or word fragment.
* Set a flag indicating that the suffix is a candidate for removal.
* Move the cursor to the end of the word.
* Insert a space if the completion appears unique.
* Etc. (I'm sure I've missed some.)

Now consider the set of tests that completion applies to decide upon an
action from the above list:

* Is the current word in "command position"?
* Is the current word a builtin command?
* Is the current word an external command?
* Is the current word a file name?
* Ditto, but matching a particular glob pattern?
* Is the current word a directory name?
* Etc., etc.

Presently, if a user wants to perform those tests, the only recourse is
to use a compctl -K function, use `read -nc nword;read -Ac args`, and
then write all the tests by hand.  And he has to specify in the compctl
(or in options for menucompletion), outside of that function, what the
resulting input line behavior is.

An example is multicomp -- it'd be nice to be able to say, "if this
completion is a directory name, then append a removable slash, else
append a space."

I suggest that we provide a builtin (or a collection of them) to apply
the collection of tests to the current input line, plus a collection of
what amount to zle widgets to permit the user to explicitly direct the
changes to the input line that take place when the tests succeed.

This is similar to what Peter did in his completion widgets patch, but
exposes even more of the mechanism.

It's sort of a RISC v. CISC argument, actually; it's easier to understand
what's going on with RISC instructions, because each one is simpler, but
you have to issue a lot more of them to get anything done.  The current
compctl is confusing for two reasons:  (1) it uses single-character and
thereby minimally mnemonic options to describe everything; and (2) the
effect of any one of those options may be a very complex change to the
input, and so combining them is either bewildering or impossible.

Addressing (1) with a flowery syntax doesn't help with (2), and I think
that if (2) were cleaned up then the syntax for (1) wouldn't need to be
as flowery.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Sven Wischnowsky]

Bart Schaefer wrote:

> 
> ...
> 
> I think implementing some new builtins is the way to go, but I don't
> think the right thing is to have them read a completely new syntax from
> some external file.  I actually have some experience with this:  When
> Z-Code implemented attachment processing extensions, similar to mailcap
> for MIME, in the Z-Mail product, we added a completely new config file
> with new syntax.  Prior to that, all configuration had been in "zscript,"
> the csh-like language accepted by zmail's command parser, and it became
> a hassle (for sysadmins especially) to have to keep track of the extra
> files and learn the extra syntax.  We soon had to add a zscript builtin
> with command-line arguments to replicate the new config's semantics, so
> customizations could all live in the same place.
> 
> In short, if you've already got a programming language, stick with it.
> 

I agree that we should use as much of the current shell syntax/lexer/parser/...
as possible, I just wouldn't want to change too much of that
code. I hadn't thought of using more than one new builtin, maybe there 
is a way we could go...

> } Also, we should have some kind of democratic decision about the
> } syntax. Let me start with a first suggestion:
> 
> I think what you've outlined is actually more verbose than necessary.
> For example:
> 

Yes, I know, it was done to improve readibility, which, if I
understood the complaints correctly, is one of the major problems with 
compctl.

> } pattern */X11/* do     # `pattern' is for defining pattern compctls
> } command xterm do       # `command' is for normal compctls
> 
> Why separate them?  Will it ever make sense to say
> 
> 	command *X* do
> 
> ??  Why not treat everything with wildcards as a pattern, and otherwise
> as a command?
> 

That's what I did with compctl...

> Another example:
> 
> }       strings display name  # this is -k '(display name)'
> 
> Since we're not talking compctl command lines any more, we don't need
> both "inline array syntax" and named arrays, so just use a standard
> array assignment and then refer to it by array name later.
> 

If we have a syntax for setting arrays in the context where the
completion control is defined (i.e. if we find a nice way to use the
things you hint at above and below): yes!

> I'd rather see something like "case" syntax:
> 
>    case $command[1] in	# command is set as if by `read -Ac command`
>       (*/X11/*)
>         if [[ $command[$current] == -* ]]	# `read -nc current`
> 	  reply=(display name)
> 	fi
> 	;&
>       (xterm)
>         if [[ $command[$current] == -* ]]
> 	  reply=($reply 132 title)
> 	fi
>         ;;
>     esac
> 

This is the reincarnation of your `hey we can do all compctls in one
compctl -T' (what you mentioned below), right?
For me (the way I use compctl), this is ok, but there may be problems:
  - you can't separate the thing above into different parts/files as
    easily as different commands
  - it doesn't make things simpler to read/understand (esp. if this
    becomes large)
  - a minor complaint: the `case $command[1] in ...(*/X11/*)' looks
    like matching the first word in the command, which is not what
    pattern compctls currently do: they first try to match the first
    word, if that doesn't match, the path name of the command is
    looked up and matching is done on that

> }   paths                # this is `-/'
> }   files *.dvi          # `files' with strings is `-g ...'
> 
> This kind of thing is a good idea.  So is PWS's suggestion of having
> longer names for the c[] and w[] and so forth extended patterns.  But
> both of those could be accomplished by introducing one new builtin
> that is otherwise equivalent to compctl.
> 
> On Aug 20, 11:48am, Bruce Stephens wrote:
> } Subject: Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)
> }
> } On the other hand, if you isolate the completion stuff into separate
> } files, then that probably makes things a lot saner (there're surely
> } going to be ambiguities otherwise), and it leaves open the possibility
> } of "byte compiling" the completion files for speed.
> 
> I think you could accomplish both of these goals by, for example, adding
> a new special function name (ala chpwd, precmd, and preexec) and getting
> the flow-control parts of completion by parsing that function.
> 
> (You can get this effect today, almost, by using
> 
> 	compctl -Tx 'W[0,*]' -K completer
> 
> but then you must reimplement all of compctl in the completer function.
> Essentially, I think this is the direction we should go, but provide a
> lot of builtin help towards "reimplement all of compctl in completer".)
> 
> If names like "command" and "current" in my example above had predefined
> meaning within this special function, $command[$current] == -* could be
> recognized and optimized to the equivalent of -x 's[-]'.  Conceivably.
> 

This is interesting but certainly doesn't simplify things (which was
almost all I was thinking about). It also makes me think again about
the hooks I would like to see in different places.

> On Aug 20,  1:17pm, Sven Wischnowsky wrote:
> } Subject: Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)
> }
> } Yes, I was aware of that problem. If the completion control command
> } would read stdin we could put it into the normal setup files with the
> } definitions in here-documents (which is a bit ugly).
> 
> That's an approach I'd also thought of, and it'd almost be OK, but I
> think we can do better.
> 

Agreed again.

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Bart Schaefer]

On Aug 19, 11:54am, Sven Wischnowsky wrote:
} Subject: Re:  EZ-compctl (was Re: ideas: free-search-complete, noexpand)
}
} Anyway, I would like to go in a direction where we have a simpler (and 
} more readable) form to define completions and than add some syntactic
} sugar for the most common cases. Bart suggested a syntax that is
} modeled after the normal shell syntax (if I remember and understood
} him correctly). I like that idea.
} 
} Since I wouldn't like to mess with the shell lexer/parser just to add
} a new syntax for completion control, I would (as a first step)
} implement it as a new builtin that gets some filenames arguments (or
} read stdin), reads these files and expects the new syntax there.

I think implementing some new builtins is the way to go, but I don't
think the right thing is to have them read a completely new syntax from
some external file.  I actually have some experience with this:  When
Z-Code implemented attachment processing extensions, similar to mailcap
for MIME, in the Z-Mail product, we added a completely new config file
with new syntax.  Prior to that, all configuration had been in "zscript,"
the csh-like language accepted by zmail's command parser, and it became
a hassle (for sysadmins especially) to have to keep track of the extra
files and learn the extra syntax.  We soon had to add a zscript builtin
with command-line arguments to replicate the new config's semantics, so
customizations could all live in the same place.

In short, if you've already got a programming language, stick with it.

} Also, we should have some kind of democratic decision about the
} syntax. Let me start with a first suggestion:

I think what you've outlined is actually more verbose than necessary.
For example:

} pattern */X11/* do     # `pattern' is for defining pattern compctls
} command xterm do       # `command' is for normal compctls

Why separate them?  Will it ever make sense to say

	command *X* do

??  Why not treat everything with wildcards as a pattern, and otherwise
as a command?

Another example:

}       strings display name  # this is -k '(display name)'

Since we're not talking compctl command lines any more, we don't need
both "inline array syntax" and named arrays, so just use a standard
array assignment and then refer to it by array name later.

I'd rather see something like "case" syntax:

   case $command[1] in	# command is set as if by `read -Ac command`
      (*/X11/*)
        if [[ $command[$current] == -* ]]	# `read -nc current`
	  reply=(display name)
	fi
	;&
      (xterm)
        if [[ $command[$current] == -* ]]
	  reply=($reply 132 title)
	fi
        ;;
    esac

}   paths                # this is `-/'
}   files *.dvi          # `files' with strings is `-g ...'

This kind of thing is a good idea.  So is PWS's suggestion of having
longer names for the c[] and w[] and so forth extended patterns.  But
both of those could be accomplished by introducing one new builtin
that is otherwise equivalent to compctl.

On Aug 20, 11:48am, Bruce Stephens wrote:
} Subject: Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)
}
} On the other hand, if you isolate the completion stuff into separate
} files, then that probably makes things a lot saner (there're surely
} going to be ambiguities otherwise), and it leaves open the possibility
} of "byte compiling" the completion files for speed.

I think you could accomplish both of these goals by, for example, adding
a new special function name (ala chpwd, precmd, and preexec) and getting
the flow-control parts of completion by parsing that function.

(You can get this effect today, almost, by using

	compctl -Tx 'W[0,*]' -K completer

but then you must reimplement all of compctl in the completer function.
Essentially, I think this is the direction we should go, but provide a
lot of builtin help towards "reimplement all of compctl in completer".)

If names like "command" and "current" in my example above had predefined
meaning within this special function, $command[$current] == -* could be
recognized and optimized to the equivalent of -x 's[-]'.  Conceivably.

On Aug 20,  1:17pm, Sven Wischnowsky wrote:
} Subject: Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)
}
} Yes, I was aware of that problem. If the completion control command
} would read stdin we could put it into the normal setup files with the
} definitions in here-documents (which is a bit ugly).

That's an approach I'd also thought of, and it'd almost be OK, but I
think we can do better.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Sven Wischnowsky]

Bruce Stephens wrote:

> 
> ...
> 
> > Since I wouldn't like to mess with the shell lexer/parser just to
> > add a new syntax for completion control, I would (as a first step)
> > implement it as a new builtin that gets some filenames arguments (or
> > read stdin), reads these files and expects the new syntax there.
> 
> I think there's potential for confusion.  If completions use
> functions, I'd want to put the function definitions in the same file
> as the completion.  Similarly, I might want to define an array in the
> same file, too.  There might be a case for having files which define
> functions together with completions for them.
> 

Yes, I was aware of that problem. If the completion control command
would read stdin we could put it into the normal setup files with the
definitions in here-documents (which is a bit ugly). Also we could
allow the definition of array/variables and functions in the
definition files (at least I think, we could do that).

> ...
>
> An extra feature I'd like to see (which may be in your suggestion, but
> I missed it if so) is sharing of components.  Most commands have
> basically the same structure: a set of options (with - or --) with
> arguments, and then an optional list of files.  Quite a few commands
> share a number of options, but not all of them.  
> 

There are two ways to do that: with some compctl stuff or with some
kind of `alias' in the command that interprets the definition
files. The second one doesn't look too hard and the first one is
possible with my last changes, more exactly the fixes to the `-l'
flag. With that, one can do:

  compctl ... __common_options
  compctl ... -l __common_options ... foo

to `include' the flags from `__common_options' into the definition for 
the command `foo' (in fact, this was the reason why I gave the `-l'
flag an optional argument, unfortunately this never quite worked until 
now).

>
> ...
> 
> Maybe it would be useful to have special support for cvs-like commands
> too, where cvs takes some options, and a subcommand, and each
> subcommand has its own set of options and arguments, many shared.  But
> maybe not: there probably aren't enough commands like that for it to
> be worthwhile.

This too should be easy enough with a combination of `-x' and the `-l' 
stuff mentioned above.
Of course, this could be one of the things for which some kind of
short-cut notation may be interesting.

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Bruce Stephens]

Sven Wischnowsky <wischnow@informatik.hu-berlin.de> writes:

> Also this is a question of personal taste, I think. At least I
> prefer a syntax that lets me get the whole power of a command even
> it is rather complicated.

Yes, I think I agree.  That's partly why I'd prefer something that
translated to compctl.  I'd definitely (as a user---presumably
developers would hate it even more) be against having two distinct
mechanisms, one for the easy stuff, and one providing the full power,
with no connection between the two.

> Anyway, I would like to go in a direction where we have a simpler
> (and more readable) form to define completions and than add some
> syntactic sugar for the most common cases. Bart suggested a syntax
> that is modeled after the normal shell syntax (if I remember and
> understood him correctly). I like that idea.

Yes, that sounds very attractive.

> Another question is if this should be implemented in an external
> program, in a new module, in the compctl module, or if we should
> make this new syntax the default and keep compctl in another module
> (which may become obsolete).

If the new syntax is better, then the last option is best: the
examples should all be provided in the new syntax, and users should be
encouraged to move to it.

> Since I wouldn't like to mess with the shell lexer/parser just to
> add a new syntax for completion control, I would (as a first step)
> implement it as a new builtin that gets some filenames arguments (or
> read stdin), reads these files and expects the new syntax there.

I think there's potential for confusion.  If completions use
functions, I'd want to put the function definitions in the same file
as the completion.  Similarly, I might want to define an array in the
same file, too.  There might be a case for having files which define
functions together with completions for them.

On the other hand, if you isolate the completion stuff into separate
files, then that probably makes things a lot saner (there're surely
going to be ambiguities otherwise), and it leaves open the possibility
of "byte compiling" the completion files for speed.  Anyway, as a
first step, using separate files looks fine.

I don't really have any comments about your proposed syntax: it looks
very readable.  With documentation (so that I know which words are
keywords and which are strings) it'll be great, but even so it looks
good.

An extra feature I'd like to see (which may be in your suggestion, but
I missed it if so) is sharing of components.  Most commands have
basically the same structure: a set of options (with - or --) with
arguments, and then an optional list of files.  Quite a few commands
share a number of options, but not all of them.  

I can't offhand think of a general example, but two of Isode's tools
have this property.  tcldish and ltcldish are basically the same:
they're shells which connect to X.500/LDAP directories.  Both take the
same kinds of arguments for authentication and things, but tcldish has
an -call option which has an X.500 presentation address argument,
whereas ltcldish has a -call with an LDAP URL.  

It would be nice to have the common arguments separated out, and be
able to define tcldish saying that it accepts these arguments,
together with these extra (unique) ones.  I'm sure there are lots of
other cases where this would be nice.  MH may be another one.  

Maybe it would be useful to have special support for cvs-like commands
too, where cvs takes some options, and a subcommand, and each
subcommand has its own set of options and arguments, many shared.  But
maybe not: there probably aren't enough commands like that for it to
be worthwhile.

Re: EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Sven Wischnowsky]

Bruce Stephens wrote:

> 
> Sven Wischnowsky <wischnow@informatik.hu-berlin.de> writes:
> 
> > If you meant that the description should be on a somewhat higher
> > level (something like: these options, or these options followed by
> > filenames, than an optional directory-name, then all files), then I
> > would like to agree, but I'm not sure, how easy we can make that
> > (thinking about commands like `find').
> 
> find is an extreme case.  Most people (me, for example) want to be
> able to construct completions for fairly simple things like the tar
> example, and some kind of easy way to do that would be useful, I
> think.  A Perl script or something that would produce human-readable
> compctl commands from something similar to the syntax you get in
> manpages (in as much as compctl commands are ever human-readable)
> would be ideal.
> 

I know that find is an extreme example, but I think some of the
problems would also appear for easier commands.
I would like to see a description of what people would want to easily
define and how we could turn that unambiguously into a compctl (a
compctl command string or the internal representation).

Also this is a question of personal taste, I think. At least I prefer
a syntax that lets me get the whole power of a command even it is
rather complicated. But compctl may be a bad example for me, since I
probably know to much about the code in zle_tricky.c to be wondering
what to do to get a certain effect.
Anyway, I would like to go in a direction where we have a simpler (and 
more readable) form to define completions and than add some syntactic
sugar for the most common cases. Bart suggested a syntax that is
modeled after the normal shell syntax (if I remember and understood
him correctly). I like that idea.
Another question is if this should be implemented in an external
program, in a new module, in the compctl module, or if we should make
this new syntax the default and keep compctl in another module (which
may become obsolete).

Since I wouldn't like to mess with the shell lexer/parser just to add
a new syntax for completion control, I would (as a first step)
implement it as a new builtin that gets some filenames arguments (or
read stdin), reads these files and expects the new syntax there.

Also, we should have some kind of democratic decision about the
syntax. Let me start with a first suggestion:


# Comments, as in the shell.

pattern */X11/* do     # `pattern' is for defining pattern compctls
                       # more than one pattern can be given before
                       # the `do'
  if cur_prefix = -         # this is `-x 's[-]'
  then
    group options do
      strings display name  # this is -k '(display name)'

      # -tc is implicit, we would need to have something for `do not
      # try other compctls'.
    end
  fi    
end

command xterm do       # `command' is for normal compctls
  if cur_prefix = -
  then
    group options do
      strings 132 title
    end 
  else                 # this is anything before a `-x'
    group files do
      strings -frob
      files            # `files' without following strings is `-f'
    end
  fi
end

command xdvi do
  paths                # this is `-/'
  files *.dvi          # `files' with strings is `-g ...'
end

command foo do
  if cur[-1] = -f ||        # this is `-x "c[-1,-f], C[-1,-*f*]"'
     cur[-1] ~ -*f*   
  then
     files; string foo bar  # `flags' can be separated by `;' or newlines

  elif word[1] = -s && pos >= 3  # this is `- "w[1,-s] p[3,-1]"'
  then
     xor                    # there is also `or' for `-t+ +'
       files *.s
       files *.S            # implicitly joining flags, either by
                            # combining their arguments or with `-t+ +'
     xor
       files
     rox                    # ending `xor'
  fi
end

... and so on.

We would have to agree on the names for the flags and the `-x'-patterns,
the syntax in the tests and which flags should be turned into
something like a construct (like the `group'-thing above) and which
flags should be turned into something like a command (e.g. `files' in
the example).

After that we could define shortcuts for some of the more interesting
things, e.g. `after -I do ... end' as a short form of
`if cur[-1] = -I then ...fi'.

So, any comments?


Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

EZ-compctl (was Re: ideas: free-search-complete, noexpand)

[Bruce Stephens]

Sven Wischnowsky <wischnow@informatik.hu-berlin.de> writes:

> If you meant that the description should be on a somewhat higher
> level (something like: these options, or these options followed by
> filenames, than an optional directory-name, then all files), then I
> would like to agree, but I'm not sure, how easy we can make that
> (thinking about commands like `find').

find is an extreme case.  Most people (me, for example) want to be
able to construct completions for fairly simple things like the tar
example, and some kind of easy way to do that would be useful, I
think.  A Perl script or something that would produce human-readable
compctl commands from something similar to the syntax you get in
manpages (in as much as compctl commands are ever human-readable)
would be ideal.

i.e., stuff the complex cases: just provide something that's handy.
If it produces compctl that one can read, then it'll probably help in
learning compctl too.  (Let's be honest, although the documentation to
compctl is *lots* better than it once was, this is still one of the
harder bits of zsh to learn.)