Re: Gnus automatic builds

[Ted Zlatanov <tzz@lifelogs.com>]

On Thu, 07 Apr 2011 21:21:46 +0200 David Engster <deng@randomsample.de> wrote: 

DE> I've now added a new rule 'check' which will run the whole test suite. I
DE> think it's a good thing to have a rule for every test in lisp/Makefile,
DE> since during development one often has to run single tests and not the
DE> whole suite. I've now started with "test-registry".

DE> Problem is: ERT does not run under Emacs 22, nor under XEmacs. So only
DE> Emacs 23 and 24 will currently run the tests. Maybe someone will port
DE> ERT to XEmacs, then we can add that any time.

I guess we could bundle ERT with the Gnus tests.  I was hoping to avoid
that.  Can it be installed in XEmacs as a package?

DE> I've also written a little test for NNTP connections; I've attached it
DE> to this mail. It also shows how one can test "real" Gnus functionality
DE> in batch mode on the buildbot server. It will connect news.gmane.org,
DE> get gmane.discuss, retrieve a single article from it (via message-id)
DE> and exit Gnus. The problem with these kinds of tests is that they depend
DE> on an external server. I added a simple check so that those test will
DE> only run if a ping to news.gmane.org succeeds, but still, the server
DE> might have other problems, which will lead to a failing test. So a
DE> failed NNTP test might not necessarily mean the last commit actually
DE> broke something, but a look in the logs should clear that up.

That's reasonable.  We could also run a fake NNTP server on a local port
that knows only a few commands and send back replies for them, so the
interaction is scripted.  This approach will definitely help us find
Gnus issues quickly.  I don't like test-driven development, but once the
code has been written, it's extremely useful to write tests too.

DE> The reason I did not commit gnustest-nntp.el yet is that I don't know
DE> where it should live. I'm hesitant to add it to gnus/lisp, because maybe
DE> we should have a dedicated directory for tests? We could also put other
DE> files in there we'll need for testing (some dummy mails for populating
DE> an IMAP folder, for instance). Suggestions?

For the *registry.el ERT tests I tried to provide tests that show how to
use the functionality (I used to just put comments with typical usage
above each defun).  Sort of a self-documenting library approach and one
of the reasons I like tests in the first place.

For standalone tests it makes sense to keep them separate.  So
gnus/tests would IMHO be OK and ERT can live there too.  Lars?

Ted