Re: Indexing Gnus (and other...) mails

[David Engster <deng@randomsample.de>]

The following message is a courtesy copy of an article
that has been posted to gmane.emacs.gnus.user as well.

Ted Zlatanov <tzz@lifelogs.com> writes:
> nnir is the backend that implements search at the highest level in Gnus.
> nnmairix is independent of it, but could probably be converted to a nnir
> backend. 

When I started with this, I thought about integrating mairix into nnir,
but the way nnir works internally doesn't really fit too well for
mairix.

Mairix does not care about mailboxes and article numbers; it works
strictly on the filesystem level, and search results are simply links to
the original message files. While this has some obvious advantages (it's
fast, and the resulting mailbox is "just there", but still occupies
almost no filespace), it makes other things pretty hard to do,
e.g. finding the original article in Gnus and propagating marks to it.

With IMAP SEARCH, it's pretty much the other way round - you know the
original articles, and the main work is to produce a mailbox which
integrates all the search results and transparently maps article numbers
in that mailbox to the original ones.

> TZ> I don't know how IMAP servers implement SEARCH.  Is the speed decent?

Tassilo already gave numbers on that. Usually, searching in the body is
slow. Since building indexes for full text search puts quite some load
on the server and can takes lots of filespace, it's usually only an
option for people managing their own IMAP servers (for example, Squat
takes about 30% of the mailbox size in the default configuration)

> If anyone has experience integrating mairix with Courier or Dovecot,
> please let me know.

You mean as a plugin? Otherwise, it's pretty straightforward. I call it
via ssh slave connections directly on the server.

-David