Re: Using Eric Raymond's bogofilter tool within Gnus

[jhbrown@ai.mit.edu (Jeremy H. Brown)]

pinard@iro.umontreal.ca (François Pinard) writes:
>Let me thank you for the two references above.  Here are other
> references I have on Bayes filtering.  

Let me second those thanks, and thank you as well!  With as many
projects as you've found, it'd be fun to have a running head-to-head
classifier competition.

> . @ http://www.ai.mit.edu/~jrennie/ifile/
> . @ http://www.ai.mit.edu/~jhbrown/ifile-gnus.html

I've been playing with ifile a great deal lately (thus leading to
ifile-gnus). A product review:

The good:

ifile is spectacularly accurate at classifying spam vs. non-spam.  I
love it.  

You can also use ifile as a more general classifier so that it will learn
to file your mail into arbitrary groups; it is reasonably accurate at
that (about 85% according to a paper the ifile author wrote, which
jives with my experience.)

The bad:

85% accurate classification isn't good enough to use across the board.
(Although it's a reasonable way of splitting mail that your
split-rules missed and would otherwise get dumped in your "misc"
group.)

Performance isn't stellar.  This is mostly because the database is
stored in a large flat text file.  With two classes (spam, non-spam)
it's usable with a DB around 200KB; with more classes, the database
rapidly heads towards a meg or so, and ifile becomes almost unusably
slow due to startup overhead.  (Caveat: I'm running it on a slow
computer, with the database stored in NFS, so I'm pretty much the
worst-case imaginable.  I have specific performance numbers; if anyone
wants to them, contact me personally.)


The ugly:

Given a bunch of messages to classify or learn from all at once, ifile
parses them all into memory before moving onto the next step; if your
mailboxes are as out of control as mine, this will cause ifile to run
out of memory and lose.  I don't think there're any fundamental
reasons that this couldn't be fixed.  I have dreams of doing this, and
making ifile run as a daemon to avoid the db-startup overhead.


I'd love to see more reviews of bayesian (or other) spamfilters; it'd
be trivial to mutate ifile-gnus to use just about any command-line
drivable filter.

Jeremy