Re: Completion test suite

[Sven Wischnowsky <wischnow@informatik.hu-berlin.de>]

Tanaka Akira wrote:

> In article <199911220934.KAA04967@beta.informatik.hu-berlin.de>,
>   Sven Wischnowsky <wischnow@informatik.hu-berlin.de> writes:
> 
> > Because it *is* quite complicated, so much so that I can't tests
> > everything.
> 
> So, definitely we need some automatic test suite for completion
> system.
> 
> When I tried it with expect, I found that it is very difficult because
> expect's support of screen based user interfaces like zsh (and
> other termcap/terminfo based program and curses based program) is not
> so good.
> 
> I think that zsh should have some noninteractive way to use completion
> system for this purpose.  If we can invoke completion system
> noninteractively with an arbitrary command line and zsh can dumps
> its completion result, it is enough to write test suite, maybe.

I don't think so. First, this would leave the problem unsolved that
we have to find out that the results or ok. I.e. either some pour
sould would have to go through the output and look if it's correct or
everyone who writes a completion function would have to extend the
test script so that it can check it automatically. Considering the
amount that would have to be tested (and all the dependencies on the
environment: files that have to existent, different settings for
styles, different _sort_tags functions, commands that have to be
existent in some version or not, compmatchers,...) noone will
volunteer for the first possibility. And since most people are even to 
lazy to write descriptions I'm sure that most people would be to lazy
to add test-code. And that would have to be correct under all
circumstances, too. Sounds completely impossible to me.

The other problem is that the completion code relies in some places on 
zle being active. In some cases it is even visible: the special zle
parameters that are used by some completion functions. In other cases
it is hidden, but, for example, the listing is really done by zle
(complete supplies the function but that is invoked by zle). Or think
about the auto_menu stuff, where we rely on zle to set some variables.

Etc, etc...

In short: I don't think I would even try to write that, but I would be 
willing to give the implementor every help I can, of course.

Bye
 Sven

P.S.: Yes, sure, we could at least try to test some of the things
      completion does: is the correct string inserted and are the
      correct matches being generated. But that would still leave us
      with the problems of the first form.

--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de