bogofilter setup examples

bogofilter setup examples

[Sebastian Luque]

Hello, I'm trying to set bogofilter (0.94.4) up with gnus 5.10.7 (Debian
package) in GNU Emacs 21.4.1 (i386-pc-linux-gnu, X toolkit, Xaw3d scroll
bars) of 2005-03-17 on trouble, modified by Debian. I've read the relevant
sections of the manual, but I'm having trouble understanding how I should
proceed. Can somebody please point to some source where I can find
examples?

I have all my incoming mail in different nnmaildir groups, and would like
to have bogofilter tag all the incoming mail and put it in a
nnmaildir:Spam group. It also seems possible to manually mark messages as
spam or ham from within any group, and have gnus move the spam marked
messages to the spam group upon exit, or just leave it there if it's ham.
I understand it's also possible to use both actions to train bogofilter
and update its word database. Hopefully I can learn how to do these things
from an examples web page.

Any help is appreciated.

Best,
-- 
Sebastian P. Luque

Re: bogofilter setup examples

[Jonas Steverud]

Sebastian Luque <sluque@mun.ca> writes:

> Hello, I'm trying to set bogofilter (0.94.4) up with gnus 5.10.7 (Debian
> package) in GNU Emacs 21.4.1 (i386-pc-linux-gnu, X toolkit, Xaw3d scroll
> bars) of 2005-03-17 on trouble, modified by Debian. I've read the relevant
> sections of the manual, but I'm having trouble understanding how I should
> proceed. Can somebody please point to some source where I can find
> examples?

If you search the archives(¹) for mails I've written about spam
(spam.el) you will find my (still) current setup. There a several
lengthy threads there, read the Changelog for your Gnus version to see
which changes there are between the Gnus versions. I know spam.el
changed a bit around that time those threads where written.

HTH.

-------
(¹)  http://www.gnus.org/list-archives/ding/

-- 
(        http://hem.bredband.net/steverud/        !     Wei Wu Wei     )
(        Meaning of U2 Lyrics, Roleplaying        !  To Do Without Do  )

Re: bogofilter setup examples

[Ted Zlatanov]

On Tue, 19 Apr 2005, sluque@mun.ca wrote:

> Hello, I'm trying to set bogofilter (0.94.4) up with gnus 5.10.7 (Debian
> package) in GNU Emacs 21.4.1 (i386-pc-linux-gnu, X toolkit, Xaw3d scroll
> bars) of 2005-03-17 on trouble, modified by Debian. I've read the relevant
> sections of the manual, but I'm having trouble understanding how I should
> proceed. Can somebody please point to some source where I can find
> examples?

Besides Jonas' setup, you can look at the other spam.el-related posts,
quite a few have complete configurations.

You have to start with

(require 'spam)
(spam-initialize 'spam-use-bogofilter) ; plus whatever other backends
                                       ; you want to use, look in the manual

(setq spam-split-group "spam")         ; this is the default anyhow

> I have all my incoming mail in different nnmaildir groups, and would like
> to have bogofilter tag all the incoming mail and put it in a
> nnmaildir:Spam group.

You have to put (: spam-split) in your nnmail-split-fancy list.  It
will use whatever backends you provided to spam-initialize above.  You
can also specify a backend here, e.g.

(: spam-split 'spam-use-BBDB)

and even a spam group to override the default spam-split-group:

(: spam-split 'spam-use-bogofilter "SPAM")

so you can have different backends send spam to different places.

> It also seems possible to manually mark messages as spam or ham from
> within any group, and have gnus move the spam marked messages to the
> spam group upon exit, or just leave it there if it's ham.

Yes, the logic depends on whether the group is classified as a ham
group, spam group, or neither (but basically, spam is always
processed).  You can change the classification of a group with `G c'
as usual.  There, you can also set the spam/ham exit processors and
process destinations.  Spam groups have the added distinction that all
unseen articles are marked as spam when you enter the group.

The spam/ham processor processes spam or ham when you exit the group.

The process destination says what will happen to the spam or ham: it
will be moved to a group or groups; it will stay and (if spam) be
expired; or it will be respooled.  All this is visible under `G c' as
spam/ham process destination choices.

> I understand it's also possible to use both actions to train
> bogofilter and update its word database. Hopefully I can learn how
> to do these things from an examples web page.

Basically, when you quit a summary buffer, some spam.el-related hooks
will be run to do what I describe above.  So use `G c' to add
spam-use-bogofilter to your spam/ham processors, and the exit hooks
will do the right thing.

If you need to use `G c' on many groups, consider using topics or the
gnus-parameters variable, that will let you group settings together.

By the way, I found bogofilter slow and its database got corrupted a
lot when I tried it.  Maybe it was just me, but I like the
SpamAssassin database and speed much better for local spam
processing.  Plus, SA lets you talk to a remote server if needed.

Ted

Re: bogofilter setup examples

[Sebastian Luque]

Hi again,

Thanks Jonas and Ted for your advise, I did make some progress by putting
the following in my ~/.gnus.el:

(require 'spam)

(setq spam-split-group "Spam"
      spam-use-bogofilter t
      nnmail-split-methods 'nnmail-split-fancy
      nnmail-split-fancy
      '(| (: spam-split)
	  ("from" "someregexp" "somegroup")
	  (any ".*video4linux.*\\|.*Sympa\\|.*helixcommunity.*\\|.*aircanada.com.*\\|.*ostg.*" "Various")
	  ("to" ".*ecolog-l.*" "Ecolog-L")
	  ("to" ".*marmam.*" "Marmam")
	  ("to" ".*mammal-l.*" "Mammal-L")
	  ("from" ".*nature\\.com\\|.*source\\.cisti@.*\\|.*blackwell\\.synergy.*\\|.*CSIRO.*\\|.*@elsevier.*\\|.*oupjournals.*\\|.*cambridge\\.org\||.*cobjnls.*\\|.*springerlink.de" "Literature")
	  "mail.misc"))

(spam-initialize)


For some unknown (to me) reason, using Ted's suggestion:

(require 'spam)
(spam-initialize 'spam-use-bogofilter)
...

returned a "wrong number of arguments..." error for spam-initialize as
gnus was starting up. Anyway, the setup above seems to be working somewhat
fine for me. I don't understand, however, where (: spamp-split) needs to
go in the nnmail-split-fancy list. I guess putting it at the beginning as
I did splits the spam into my Spam group before doing the rest of my
splitting rules, but perhaps it's better to put it right before
"mail.misc". Another thing I had to do, although I would have preferred to
put my own expressions in my ~/.gnus.el, was to 'G c' on the "mail.misc"
group and set:

- Group contents spam/ham classification:
  gnus-group-spam-classification-ham

- Spam summary exit processor:
  Spam: Bogofilter
  Ham: Bogofilter

- Destination for spam-processed articles at summary exit:
  nnmaildir:Spam

- Spam-mark choices:
  gnus-spam-mark

My intention was to declare "mail.misc" as containing ham, but that whenever
I mark a message as spam ('S x'), it will be put in "Spam" and recorded by
bogofilter for the future.

Now, on the "Spam" group I did 'G c' and:

- Group contents spam/ham classification:
  gnus-group-spam-classification-spam

- Spam summary exit processor:
  Ham: Bogofilter

- Destination for spam-processed articles at summary exit:
  nnmaildir:mail.misc

- Spam-mark choices:
  gnus-spam-mark

and my intention here was to be able to mark as ham any ham that may have
been missclassified, send it to "mail.misc", and record it by bogofilter
for the future.

It seems to work fine, but some messages (particularly from ecolog-l) are
sent to "Spam". Marking them as ham does send them back "mail.misc"
(where I later redirect them by hand to "Ecolog-L"), but they don't seem
to be recorded by bogofilter, as (some) future messages from this address
still get put in "Spam".

How does this look? Thanks a lot so far,

-- 
Sebastian P. Luque

Re: bogofilter setup examples

[Ted Zlatanov]

On Wed, 27 Apr 2005, sluque@mun.ca wrote:

> Thanks Jonas and Ted for your advise, I did make some progress by putting
> the following in my ~/.gnus.el:
> 
> (require 'spam)
> 
> (setq spam-split-group "Spam" spam-use-bogofilter t nnmail-split-methods
> 	'nnmail-split-fancy nnmail-split-fancy '(| (: spam-split) ("from"
> 	"someregexp" "somegroup") (any
> 	".*video4linux.*\\|.*Sympa\\|.*helixcommunity.*\\|.*aircanada.com.*\\|.*ostg.*"
> 	"Various") ("to" ".*ecolog-l.*" "Ecolog-L") ("to" ".*marmam.*"
> 	"Marmam") ("to" ".*mammal-l.*" "Mammal-L") ("from"
> 	".*nature\\.com\\|.*source\\.cisti@.*\\|.*blackwell\\.synergy.*\\|.*CSIRO.*\\|.*@elsevier.*\\|.*oupjournals.*\\|.*cambridge\\.org\||.*cobjnls.*\\|.*springerlink.de"
> 	"Literature") "mail.misc"))
> 
> (spam-initialize)
> 
> 
> For some unknown (to me) reason, using Ted's suggestion:
> 
> (require 'spam)
> (spam-initialize 'spam-use-bogofilter)
> ...
> 
> returned a "wrong number of arguments..." error for spam-initialize as
> gnus was starting up. 

You are using an older version of Gnus - I would use CVS if possible,
5.10.8 otherwise.  spam.el has changed a bit since 5.10.7, including
some bug fixes.  If you don't want to upgrade, things will work but it
will be harder for me to figure out what went wrong if there's a
problem.

> Anyway, the setup above seems to be working somewhat fine for me. I
> don't understand, however, where (: spamp-split) needs to go in the
> nnmail-split-fancy list. I guess putting it at the beginning as I
> did splits the spam into my Spam group before doing the rest of my
> splitting rules, but perhaps it's better to put it right before
> "mail.misc". 

This is why I allow for a symbol parameter to spam-split.  You can
say, for example:

(: spam-split 'spam-use-blacklist)

at the beginning, which will catch only known spammers, and then

(: spam-split 'spam-use-bogofilter)

towards the end to catch more generic spam.  You can even give a group
name as a parameter, instead of the default ("spam" usually).

If you just use the default (: spam-split), I'd put it towards the end.

> Another thing I had to do, although I would have
> preferred to put my own expressions in my ~/.gnus.el, was to 'G c'
> on the "mail.misc" group and set:
> 
> - Group contents spam/ham classification:
>   gnus-group-spam-classification-ham
> 
> - Spam summary exit processor:
>   Spam: Bogofilter
>   Ham: Bogofilter
> 
> - Destination for spam-processed articles at summary exit:
>   nnmaildir:Spam
> 
> - Spam-mark choices:
>   gnus-spam-mark
> 
> My intention was to declare "mail.misc" as containing ham, but that whenever
> I mark a message as spam ('S x'), it will be put in "Spam" and recorded by
> bogofilter for the future.

Is "nnmaildir:Spam" correct?  I would use the full group name as it
appears in the Group buffer.

The rest is OK.

> Now, on the "Spam" group I did 'G c' and:
> 
> - Group contents spam/ham classification:
>   gnus-group-spam-classification-spam
> 
> - Spam summary exit processor:
>   Ham: Bogofilter
> 
> - Destination for spam-processed articles at summary exit:
>   nnmaildir:mail.misc
> 
> - Spam-mark choices:
>   gnus-spam-mark
> 
> and my intention here was to be able to mark as ham any ham that may have
> been missclassified, send it to "mail.misc", and record it by bogofilter
> for the future.

You want to set the ham-process-destination to "nnmaildir:mail.misc"
while leaving the spam-process-destination nil (so the spam will be
deleted in place).

You need to add the Spam: Bogofilter exit processor as well, or spam
messages will not be processed.  Right now you're just teaching what's
ham.

> It seems to work fine, but some messages (particularly from ecolog-l) are
> sent to "Spam". Marking them as ham does send them back "mail.misc"
> (where I later redirect them by hand to "Ecolog-L"), but they don't seem
> to be recorded by bogofilter, as (some) future messages from this address
> still get put in "Spam".

If you could update to CVS, that would make the job much easier.  Let
me know when you decide, and we'll go from there.

Ted

Re: bogofilter setup examples

[Reiner Steib]

On Fri, Apr 29 2005, Ted Zlatanov wrote:

> You are using an older version of Gnus - I would use CVS if possible,
> 5.10.8 otherwise.  spam.el has changed a bit since 5.10.7, including
> some bug fixes.

The last stable release was 5.10.6.  I guess you meant...

| You are using an older version of Gnus - I would use CVS if possible,
| 5.10.7 otherwise.  spam.el has changed a bit since 5.10.6, including
| some bug fixes.

Gnus 5.10.7 is not released yet (the release will be in May,
probably), but people can use the v5-10 branch in CVS which
corresponds to the upcoming 5.10.7 release.

BTW, have all bug fixes (especially in spam.el and spam-report.el)
been backported to v5-10?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: bogofilter setup examples

[Ted Zlatanov]

On Fri, 29 Apr 2005, reinersteib+gmane@imap.cc wrote:

> The last stable release was 5.10.6.  I guess you meant...
> 
> | You are using an older version of Gnus - I would use CVS if possible,
> | 5.10.7 otherwise.  spam.el has changed a bit since 5.10.6, including
> | some bug fixes.
> 
> Gnus 5.10.7 is not released yet (the release will be in May,
> probably), but people can use the v5-10 branch in CVS which
> corresponds to the upcoming 5.10.7 release.

That's what I meant - CVS if possible, 5.10.6 otherwise.  Thanks for
catching it.

> BTW, have all bug fixes (especially in spam.el and spam-report.el)
> been backported to v5-10?

As far as I know, bug fixes have been backported (please let me know
if you catch any discrepancies).  The recent gnus-registry.el changes
fix a broken feature but in a fundamental way, so it's not really a
bug fix but more of a overhaul IMO.  If anyone disagrees, I'll
backport it, it's not too hard.

Ted

Re: bogofilter setup examples

[Sebastian Luque]

Reiner Steib <reinersteib+gmane@imap.cc> wrote:

[...]

> Gnus 5.10.7 is not released yet (the release will be in May,
> probably), but people can use the v5-10 branch in CVS which
> corresponds to the upcoming 5.10.7 release.

Why do I get v5.10.7 when I say M-x gnus-version? I'm using the Debian
unstable package.

Best regards,
-- 
Sebastian P. Luque

Re: bogofilter setup examples

[Arne Jørgensen]

Sebastian Luque <sluque@mun.ca> writes:

> Reiner Steib <reinersteib+gmane@imap.cc> wrote:
>
> [...]
>
>> Gnus 5.10.7 is not released yet (the release will be in May,
>> probably), but people can use the v5-10 branch in CVS which
>> corresponds to the upcoming 5.10.7 release.
>
> Why do I get v5.10.7 when I say M-x gnus-version? I'm using the Debian
> unstable package.

Because the gnus package in Debian unstable is actually synchronised
against the v5_10 branch. According to the changelog in the package
the last sync was on March 18, 2005.

Kind regards,
-- 
Arne Jørgensen <http://arnested.dk/>