Re: Email filing.

[clemens fischer <ino-eb2dced1@spotteswoode.de.eu.org>]

Scott A Crosby <scrosby@cs.rice.edu> writes:

> Been done. Look at ifile. I've heard its slow though. 
>
> And that although it works, it doesn't always work as nicely as you'd
> like, in that it classifies right only about 80-90% of the time, but
> that 10% will annoy you. IMHO, its only really useful when you have
> email thats uncatagorizable by any other means.

i have much better numbers with ifile.  where it fails it can be
attributed to not doing MIME, but /that would slow it!/.  mr. browne
has proposed a workaround, though, which just identifies the MIME
parts and/or encodings.  this would make ifile be more accurate in the
typical text-group, where it is intended to block spam.  then there's
always the possibility to mime-decode messages before classifying it.

my experiments show that ifile is good, both regarding accuracy and
speed, but i do use a tuned system with a procmail preprocessor.  the
"recipes" don't do any classification, they throw out chinese and
spamtool generated garbage.  incidentally, procmail uses most of the
time needed to categorize my email.

> It could try to identify mailing lists by noting list-headers but I
> wouldn't want to bet on perfect reliability.

it is easy to support this:  i have procmail tag messages to
mailinglists with a simple "X-Mailinglist: true" header early on, and
ifile adjusts nicely, including it in its statistics.

> For spam-checking, I'm, doing an implementation of something that does
> naive bayesean, but is flexible enough to be used for this. A *very
> fast* implementation.... my benchmark right now for the statistics
> building is 5 seconds on a 35mb, 7500 message corpus. V2 should be 30%
> faster.)

sounds very impressive.  is it for spam/non-spam checking only?

-- 
clemens