Re: Splitting based on character sets

[David Engster <deng@randomsample.de>]

Ted Zlatanov writes:
> On Tue, 12 Apr 2011 18:44:49 +0200 David Engster <deng@randomsample.de> wrote: 
> DE> There is a crm114 plugin for spamassassin; it's in the "CoolThings"
> DE> section of the crm114 site. It may be that it's well suited for foreign
> DE> languages, but I tried it some time ago, and wasn't particularly
> DE> impressed, especially regarding the elaborated setup. The thing which
> DE> made me drop it was that I got false positives (yes, I read the docs and
> DE> trained it correctly). Middle-of-the-road Spamassassin in combination
> DE> with the Bayes-plugin, Razor and a few blacklists catches practically
> DE> all spam for me, without any false positives.
>
> I've been happy with CRM114 (since the first Spam Conference :) so I
> can't say why it didn't work for you.  I like that it only has one way
> to classify spam, as opposed to the SA multi-pronged approach.

See, that's exactly what I like about SA. :-)

I've long tried to find a single, pure black-box-machine-learning
spam detection which could rival Spamassassin, main reason being that SA
can be such a memory hog on smaller servers. But I always came back.

> It was definitely better against foreign languages than SA 5 years ago,
> when I tested it.

At least for German, I subscribe to a custom rule list which is
regularly updated. But most of the German spam is actually catched by
iXhash (similar to Razor/Pyzor) and iXRBL, which are also located in
Germany.

But I guess what Lars originally asked was not to classify mails in
Russian, but to just classify them as spam, since he doesn't get Russian
ham. For this, one can use the Textcat plugin from SA, which will try to
guess the language of the mail and include a X-Language header.

-David