Re: spam-stat regeneration notes

[Ted Zlatanov <tzz@lifelogs.com>]

On Fri, 13 Jun 2003, billw@wolfram.com wrote:
> I've been using spam-stat for about 6 months now, and noticed lately
> that spam processing was getting mightly slow - thanks (I suspect)
> to the hashbusters spammers are putting in their messages.  I even
> do (spam-stat-reduce-size) when quitting gnus each day, so the thing
> was as small as possible.

Maybe we should add code that filters out terms seen only once from
the spam/ham database.  Doing that once a week should work fine.

Also, spam-stat.el creates hashtables without any optimizations: 

(make-hash-table :test 'equal)

maybe that could be improved.  There may be many such optimization
points.  I don't use spam-stat personally, but maybe you can
instrument it and see where the slowdowns are.

> So today I did my first rebuild of the spam-stat database.  That's
> not bad in my book - 6 months for one constantly-growing database.
> Here's the code, which I should probably put in a function
> "spam-reset" or something.

Please do!

> - Is there an easy way to run a function over an entire directory
>   tree, while specifying which dirs to include or avoid?

Maybe find-dired will help?  I don't know about anything like that
built-in...

Ted