external completion command

[Felix von Leitner <felix-zsh@fefe.de>]

Hello zsh-users,

I have not kept up to date with the zsh command line completion
functionality for more than ten years and decided to look at it again
recently, only to find it transformed beyond recognition.

I have written a few tools I use for my day job, that are based around
looking words up for example in a TAGS file, and I have also written a
search engine. I usually use these tools from inside my editor, but it
would be nice to have proper completion from the shell, too.

So, basically, what I want to do, is

  % s memc[TAB]
  memccpy
  memchr
  memcpy
  memcmp

However, since the database covers potentially gigabytes of source code,
I have written an optimized tool to list all the words that start with a
certain prefix. I want zsh to run that tool with the prefix I already
typed. I don't want to give zsh a list of all the words, because that
would be very inefficient. I would also like to tell zsh to abort after
reading 100 completions.

I also don't want zsh to cache anything, neither on disk nor in RAM. If
I type tab again, I want it to call the external command again.

To give you an idea about the numbers: for the zsh 5.9 source tree,
there are 76277 distinct words, but starting with memc there are 4.
I don't want zsh to waste time reading and discarding 76273 words.

Is this possible with the current completion system?

Thanks,

Felix