Re: Getting started with spam filtering

[Ted Zlatanov <tzz@lifelogs.com>]

On Tue, 07 Jan 2003, niklas.morberg@axis.com wrote:
> I've read the documentation on spam filtering in the latest
> CVS, and although it is very thorough, my head is spinning.
> I just don't understand what to do to get started...

Yes, there's no HOWTO yet.  I'm not done with the functionality yet :)
If anyone's willing to maintain a spam.el HOWTO meanwhile, feel free.
Below is what may be the first attempt at it.

>>From the documentation it looks like spam.el is all I need,
> but is this really the case? It seems like I need a spam
> processor (whatever that is) as well . I'm guessing spam.el
> is not a spam processor, right?. To further complicate
> things I'm using nnimap which seems to narrow down the
> options somewhat.

To start, set spam-split-group (the default of "spam" is sensible) to
where you want your spam to go.  Also (customize-variable
spam-junk-mailgroups) if you want to explicitly consider some groups
spam groups BY NAME.

A spam/ham processor is a "backend" that will take spam.  There are
internal ones like whitelists/blacklists/BBDB, and external ones like
ifile/bogofilter.

You set spam/ham processors either for a group/topic with `G c',
the spam-process parameter, or for a regex matching a group with
(customize-variable gnus-spam-process-newsgroups).  

To split incoming mail, most spam/ham processors have a corresponding
spam-use-PROCESSOR variable that you can set.  You add (: spam-split)
to your split rules, the rest is available with (customize-group
"spam").  The incoming split and the spam/ham processor work together.
For instance, if you use the bogofilter spam processor, you would
probably want to also set the spam-use-bogofilter variable to t so
your incoming mail gets classified into spam/non-spam according to
what bogofilter has learned from processing your spam.

The spam-use-blackholes is the only incoming spam-split variable that
does not have a corresponding spam/ham processor.

There are some rules as to how spam and ham articles are treated in
spam/ham/unclassified groups.  Basically, if it's spam, it will always
be processed by a spam processor you define for that group.
Otherwise, the group's content type matters.  You define the
spam/ham/unclassified content of a group either for a group/topic with
`G c', the spam-contents parameter, or for a regex matching a group
with (customize-variable gnus-spam-newsgroup-contents).

> The documentation would be more useful (to me) if it could
> explain the minimal stuff I need to get started. Is spam.el
> enough? Will spam.el and spam-stat.el suffice then?

spam-stat.el is not used by spam.el right now, and has a separate
manual section.  I'm considering adding it as a spam/ham processor to
spam.el, but I don't know if anyone needs that.

> A recommendation of what spam processor to use would also be
> helpful. 

It depends on what you want.  There's whitelists, blacklists, BBDB,
ifile, bogofilter...  spam.el is all about user choice.

I found the spam-use-blackholes almost essential.  I don't lose that
mail, it just goes to a "spam" folder.  I haven't had a positive yet.
It requires either a very recent Emacs, or the "dig" program in your
path.

> (Are these spam processors at all available for us Windows users,
> btw? Cygwin would work.) Do I have to use an external program?

BBDB, whitelists, and blacklists are internal.  Blackhole checks need
dig.el or dns.el to work.

Ifile and Bogofilter should work if executable-find can find the
program, and call-process-region works with that program.  I don't
have a Win32 machine to test on, so let me know if things break.

> I'm sorry if I'm not making any sense, as I said I'm very
> confused at the moment...

No problem, I'd like to help everyone interested in spam.el.  I just
can't write code, maintain the manual, and a HOWTO at the same time.

Ted