external completion command

external completion command

[Felix von Leitner]

Hello zsh-users,

I have not kept up to date with the zsh command line completion
functionality for more than ten years and decided to look at it again
recently, only to find it transformed beyond recognition.

I have written a few tools I use for my day job, that are based around
looking words up for example in a TAGS file, and I have also written a
search engine. I usually use these tools from inside my editor, but it
would be nice to have proper completion from the shell, too.

So, basically, what I want to do, is

  % s memc[TAB]
  memccpy
  memchr
  memcpy
  memcmp

However, since the database covers potentially gigabytes of source code,
I have written an optimized tool to list all the words that start with a
certain prefix. I want zsh to run that tool with the prefix I already
typed. I don't want to give zsh a list of all the words, because that
would be very inefficient. I would also like to tell zsh to abort after
reading 100 completions.

I also don't want zsh to cache anything, neither on disk nor in RAM. If
I type tab again, I want it to call the external command again.

To give you an idea about the numbers: for the zsh 5.9 source tree,
there are 76277 distinct words, but starting with memc there are 4.
I don't want zsh to waste time reading and discarding 76273 words.

Is this possible with the current completion system?

Thanks,

Felix

Re: external completion command

[Bart Schaefer]

On Mon, Feb 19, 2024 at 4:30 AM Felix von Leitner <felix-zsh@fefe.de> wrote:
>
> Hello zsh-users,

Hi, Felix.  I'm sure we've chatted before but I can't find anything
from you in the list archives going back 25 years.  Still maintaining
zsh.vim ?

>   % s memc[TAB]
>   memccpy
>   memchr
>   memcpy
>   memcmp
>
> However, since the database covers potentially gigabytes of source code,
> I have written an optimized tool to list all the words that start with a
> certain prefix. I want zsh to run that tool with the prefix I already
> typed.

There's a lot of scaffolding in the "new" completion system (which has
now been around for about 85% of the life of the "old" one), but it's
all pretty much in support of two commands:
 - compdef how what
   Tells completion "how" (e.g., what command to run) to complete for
"what" (command line context).
 - compadd details -- words...
   Tells completion to consider the listed words when invoked, with
various details like whether the words are file names and what
explanatory text to show.

The rest is all just to assist with breaking the command line and the
word adjacent to the cursor down into "what" context so that different
"how" can be selected.  The whole scaffold is loaded by running
  autoload -Uz compinit
  compinit
which also scans the directories in $fpath for files with "#compdef"
in the first line and automatically enables those.

The other minimum thing you need to know is that the $words array
contains the command and arguments for which completions are being
generated, and that the integer $CURRENT specifies the position in
$words that is adjacent to the cursor, so $words[CURRENT] is your
already typed prefix.

> I don't want to give zsh a list of all the words, because that
> would be very inefficient. I would also like to tell zsh to abort after
> reading 100 completions.

The simplest formulation of this, then, using here "taglist" as the
name of your custom tool and "s" from your example as the command name
for which completions are being offered, would be something like (in
.zshrc)

  compinit
  compdef 'compadd -- $(taglist $words[CURRENT] | head -n 100)' s

Or you could create a file named "_taglist" in a directory in your
$fpath and in that file have

#compdef s
compadd -- $(taglist $words[CURRENT] | head -n 100)

I'm glossing over the possible need to do other quoting on the $(...)
if taglist could produce results with embedded spaces, or what you
want to do if $words[CURRENT] is the empty string.

Everything beyond this is just frosting.  For example there's a
variable $curcontext that tells you what context the completion system
has figured out from the contents of the command line, and the
"zstyle" command to define and look up values based on that context,
so in the above _taglist you could change the limit of 100 to
different numbers based how many words are to the left of $CURRENT,
etc.

Hope this helps.

Re: external completion command

[Ray Andrews]

On 2024-02-19 11:14, Bart Schaefer wrote:
> On Mon, Feb 19, 2024 at 4:30 AM Felix von Leitner <felix-zsh@fefe.de> wrote:
>> Hello zsh-users,
> Hi, Felix.  I'm sure we've chatted before but I can't find anything
> from you in the list archives going back 25 years.
Bart, how does one search the archives?  It looks like one can pick a 
year, and after that it's just a monolith.  I've often wanted to look 
back, cuz I'm asking some question that I'm sure I've asked before.  It 
would save a few dumb questions.  Just brute force browser text searches?

Re: external completion command

[Bart Schaefer]

On Mon, Feb 19, 2024 at 12:44 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Bart, how does one search the archives?

Unfortunately search capability has not yet been restored on zsh.org
after the archives were forced to move from a university-hosted server
to a privately-hosted one.

In my case $DAYJOB has copies of all the -workers (and therefore
-users, because workers is copied) messages dating back to sometime in
1999, so I just run "grepmail ..." on a pile of gzip'd mbox files.  (
https://github.com/coppit/grepmail )

If anyone knows of a searchable mirror now that gmane.org is gone,
please speak up.

Re: external completion command

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 180 bytes --]



On 2024-02-19 14:14, Bart Schaefer wrote:
> If anyone knows of a searchable mirror now that gmane.org is gone,
> please speak up.
25 years of questions.  That's a resource.


>

[-- Attachment #2: Type: text/html, Size: 980 bytes --]

Re: external completion command

[Lawrence Velázquez]

On Mon, Feb 19, 2024, at 5:14 PM, Bart Schaefer wrote:
> On Mon, Feb 19, 2024 at 12:44 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>>
>> Bart, how does one search the archives?
>
> Unfortunately search capability has not yet been restored on zsh.org
> after the archives were forced to move from a university-hosted server
> to a privately-hosted one.
>
> In my case $DAYJOB has copies of all the -workers (and therefore
> -users, because workers is copied) messages dating back to sometime in
> 1999, so I just run "grepmail ..." on a pile of gzip'd mbox files.  (
> https://github.com/coppit/grepmail )
>
> If anyone knows of a searchable mirror now that gmane.org is gone,
> please speak up.

The Mail Archive seems relatively popular.

https://www.mail-archive.com/faq.html#newlist
https://www.mail-archive.com/faq.html#import

We're not on there currently, but I'd be happy to work on setting
it up, if we deem it suitable.

-- 
vq

Re: external completion command

[Jim]

[-- Attachment #1: Type: text/plain, Size: 1480 bytes --]

Hi all,

On Mon, Feb 19, 2024 at 5:25 PM Lawrence Velázquez <larryv@zsh.org> wrote:

> On Mon, Feb 19, 2024, at 5:14 PM, Bart Schaefer wrote:
> > On Mon, Feb 19, 2024 at 12:44 PM Ray Andrews <rayandrews@eastlink.ca>
> wrote:
> >>
> >> Bart, how does one search the archives?
> >
> > Unfortunately search capability has not yet been restored on zsh.org
> ...
> >
> > If anyone knows of a searchable mirror now that gmane.org is gone,
> > please speak up.
>
> The Mail Archive seems relatively popular.
>
> https://www.mail-archive.com/faq.html#newlist
> https://www.mail-archive.com/faq.html#import
>
> We're not on there currently, but I'd be happy to work on setting
> it up, if we deem it suitable.
>
>
Looked  at the links and noticed a few things. It's noted that attachments
like
images and other LARGE attachments are unlikely to be archived, which may
not be an issue for attached patches depending on what they consider LARGE.
But they also indicated that in the future maybe no attachments would be
archived.

I also looked at a couple of archives and noticed that email headers are
missing.
The site notes "raw" emails are not web accessible. Apparently this is done
to prevent address harvesting.  The one header item that I assume should be
displayed is X-Seq since this is used as a reference for things like
commits.
Couldn't find if this could be configured.

Just thought I would mention it.

Regards,

Jim Murphy

[-- Attachment #2: Type: text/html, Size: 2417 bytes --]