SPAM filtering.

SPAM filtering.

[Jorge Godoy]

Hi!


I've sent an abuse message to Hotmail and received an interesting
message telling about ways to identify if a message is originated from
them or not.

Such things included verifying if the address start with a number, if
theres a "X-Originated-From: ip" header, etc. 

Is there an easy way to implement such checkings on Gnus for messages
coming apparently from Hotmail? If they don't satisfy these checkings,
I'd like to automatically delete them (I use nnmail-split-fancy to
filter my messages).


Thanks in advance,
-- 
Godoy. <godoy@conectiva.com>

Departamento de Publicações       Conectiva S.A.
Publishing Department             Conectiva Inc.

Re: SPAM filtering.

[Kai Großjohann]

I suggest that you make a rule which goes like this:

(| ("x-originated-from" ""
    (| ...rest.of.split.rules.go.here...))
   junk)

`|' is like an if-then-else construct.  Since nnmail-split-fancy
doesn't offer a `not' construct, the trick is to find the right
condition and nesting to achieve what you want.

kai
-- 
The arms should be held in a natural and unaffected way and never
be conspicuous. -- Revised Technique of Latin American Dancing

Re: spam filtering

[Ted Zlatanov]

On Sun, 03 Oct 2004, sds@gnu.org wrote:

> exiting from Trash:
> 
> Retrieving newsgroup: nnimap+mail.podval.org:.imap/Trash...
> nnimap: Updating info for nnimap+mail.podval.org:.imap/Trash...done
> Fetching headers for nnimap+mail.podval.org:.imap/Trash...done
> Suppressing duplicates...done
> Scoring...done
> Sorting threads...done
> Generating summary...done
> Marking unseen articles as spam
> No more unread articles [3 times]
> No more articles
> No more unread articles
> Exiting summary buffer and applying spam rules
> Marking spam as expired without moving it
> Moving ham messages from spam group
> Expiring articles...done
> nnimap: Setting marks in .imap/Trash...done
> 
> 
> the article I ticked as read with "d" remains in Trash.
> 
> 
> gnus-parameters
> (("nnimap.*"
>   (spam-process-destination . "nnimap+mail.podval.org:.imap/Trash")
>   (spam-contents gnus-group-spam-classification-ham))
>  ("nnimap+mail.podval.org:.imap/Trash"
>   (ham-process-destination . "nnimap+mail.podval.org:INBOX")
>   (spam-contents gnus-group-spam-classification-spam)))

Sam,

would it be possible for you to try the latest Gnus from CVS?  The
"new" spam.el in CVS is much, much better than the old one you are
using.

Specifically regarding the problems you reported:

I think the 50,000 messages are due to either Gnus internals or the
agent - either way, spam.el doesn't have anything to do with them.

Can you explicitly set your parameters with `G c' instead of
gnus-parameters, and set your ham-marks to the ticked-mark (!).  Then
tick a ham article and see what happens.

Thanks
Ted

Re: spam filtering

[Sam Steingold]

> * Ted Zlatanov <gmm@yvsrybtf.pbz> [2004-09-30 10:35:16 -0400]:
>
> On Wed, 29 Sep 2004, sds@gnu.org wrote:
>
>>> * Reiner Steib <ervarefgrvo+tznar@vznc.pp> [2004-09-29 16:54:46 +0200]:
>>>
>>> On Wed, Sep 29 2004, Sam Steingold wrote:
>
>>>> so, do I use M-d in "Trash" to "ham" the message?
>>>
>>> No, just mark it as read (`d'):
>>>
>>> ,----[ (info "(gnus)Spam ELisp Package Configuration Examples") ]
>>> |      When I find a false positive, I mark the message with some
>>> |      other ham mark (`ham-marks', *Note Spam ELisp Package Global
>>> |      Variables::).
>>> ``----'
>> 
>> nope, I "d"'ed an article there and it stayed there after I exited the group.
>
> Are you sure the ham-process-destination is set for the group?  Set
> gnus-verbose to 10 and let us know what the *Messages* buffer says
> after you exit the group.

Gnus claims that there are 50,000+ articles in INBOX (actually, only
about 200 is there).
when I exit INBOX, I see 2(!) passes of messages like

gnus-agent-expire: nnimap+mail.podval.org:INBOX:49341: Kept unread article.
gnus-agent-expire: nnimap+mail.podval.org:INBOX:49342: Kept unread article.

with article numbers going from 20,000 (the oldest ticked message) to
~50,000 (the current one).

exiting from Trash:

Retrieving newsgroup: nnimap+mail.podval.org:.imap/Trash...
nnimap: Updating info for nnimap+mail.podval.org:.imap/Trash...done
Fetching headers for nnimap+mail.podval.org:.imap/Trash...done
Suppressing duplicates...done
Scoring...done
Sorting threads...done
Generating summary...done
Marking unseen articles as spam
No more unread articles [3 times]
No more articles
No more unread articles
Exiting summary buffer and applying spam rules
Marking spam as expired without moving it
Moving ham messages from spam group
Expiring articles...done
nnimap: Setting marks in .imap/Trash...done


the article I ticked as read with "d" remains in Trash.


gnus-parameters
(("nnimap.*"
  (spam-process-destination . "nnimap+mail.podval.org:.imap/Trash")
  (spam-contents gnus-group-spam-classification-ham))
 ("nnimap+mail.podval.org:.imap/Trash"
  (ham-process-destination . "nnimap+mail.podval.org:INBOX")
  (spam-contents gnus-group-spam-classification-spam)))


-- 
Sam Steingold (http://www.podval.org/~sds) running w2k
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.honestreporting.com>
There are two ways to write error-free programs; only the third one works.

Re: spam filtering

[Ted Zlatanov]

On Wed, 29 Sep 2004, sds@gnu.org wrote:

>> * Reiner Steib <ervarefgrvo+tznar@vznc.pp> [2004-09-29 16:54:46 +0200]:
>>
>> On Wed, Sep 29 2004, Sam Steingold wrote:

>>> so, do I use M-d in "Trash" to "ham" the message?
>>
>> No, just mark it as read (`d'):
>>
>> ,----[ (info "(gnus)Spam ELisp Package Configuration Examples") ]
>> |      When I find a false positive, I mark the message with some
>> |      other ham mark (`ham-marks', *Note Spam ELisp Package Global
>> |      Variables::).
>> ``----'
> 
> nope, I "d"'ed an article there and it stayed there after I exited the group.

Are you sure the ham-process-destination is set for the group?  Set
gnus-verbose to 10 and let us know what the *Messages* buffer says
after you exit the group.

Ted

Re: spam filtering

[Sam Steingold]

> * Reiner Steib <ervarefgrvo+tznar@vznc.pp> [2004-09-29 16:54:46 +0200]:
>
> On Wed, Sep 29 2004, Sam Steingold wrote:
>
>>> * Reiner Steib <ervarefgrvo+tznar@vznc.pp> [2004-09-29 09:12:55 +0200]:
> [...]
>>>         ("nnimap:\\(INBOX\\|other-folders\\)"
>>>          (spam-process-destination . "nnimap+...:Trash")
>>>          (spam-contents gnus-group-spam-classification-ham))
>>
>> this is for `gnus-parameters', not `gnus-spam-process-newsgroups',
>> right?
>
> Yes.

OK, thanks, it works, but exiting INBOX now takes forever (gnus thinks
there are 55000+ messages there)...

>>>> (it would also be nice if I could mark non-spam in Trash and have it
>>>> moved to INBOX on exit from Trash, but...)
>>>
>>> Same here:
>>>
>>>         ("nnimap+...:Trash"
>>>          (ham-process-destination . "nnimap:INBOX")
>>>          (spam-contents gnus-group-spam-classification-spam))
>>
>> so, do I use M-d in "Trash" to "ham" the message?
>
> No, just mark it as read (`d'):
>
> ,----[ (info "(gnus)Spam ELisp Package Configuration Examples") ]
> |      When I find a false positive, I mark the message with some
> |      other ham mark (`ham-marks', *Note Spam ELisp Package Global
> |      Variables::).
> ``----'

nope, I "d"'ed an article there and it stayed there after I exited the group.

-- 
Sam Steingold (http://www.podval.org/~sds) running w2k
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.honestreporting.com>
(let((a'(list'let(list(list'a(list'quote a)))a)))`(let((a(quote ,a))),a))

Re: spam filtering

[Ted Zlatanov]

On Wed, 29 Sep 2004, sds@gnu.org wrote:

> also, I have _many_ imap folders, listing them all is not feasible.

You can use topics, they support parameters just like groups do.

>>> (it would also be nice if I could mark non-spam in Trash and have it
>>> moved to INBOX on exit from Trash, but...)
>>
>> Same here:
>>
>>         ("nnimap+...:Trash"
>>          (ham-process-destination . "nnimap:INBOX")
>>          (spam-contents gnus-group-spam-classification-spam))
> 
> so, do I use M-d in "Trash" to "ham" the message?

You give the article a ham-mark, which can be many things (by default
the ham marks are (gnus-del-mark gnus-read-mark gnus-killed-mark
gnus-kill-file-mark gnus-low-score-mark) but I like to also use
gnus-ticked-mark.

You may also want spam-mark-ham-unread-before-move-from-spam-group,
which will undo the ham-mark (set the article to "Unread") before
moving the article to your favorite destination.  It's nil by default.

Ted

Re: spam filtering

[Reiner Steib]

On Wed, Sep 29 2004, Sam Steingold wrote:

>> * Reiner Steib <ervarefgrvo+tznar@vznc.pp> [2004-09-29 09:12:55 +0200]:
[...]
>>         ("nnimap:\\(INBOX\\|other-folders\\)"
>>          (spam-process-destination . "nnimap+...:Trash")
>>          (spam-contents gnus-group-spam-classification-ham))
>
> this is for `gnus-parameters', not `gnus-spam-process-newsgroups',
> right?

Yes.

> also, I have _many_ imap folders, listing them all is not feasible.

What wrong with "nnimap:.*"?  You probably want to exclude "Trash"?
When there is a later entry for "Thrash" in `gnus-parameters', the
last will "win", IIRC.

>>> (it would also be nice if I could mark non-spam in Trash and have it
>>> moved to INBOX on exit from Trash, but...)
>>
>> Same here:
>>
>>         ("nnimap+...:Trash"
>>          (ham-process-destination . "nnimap:INBOX")
>>          (spam-contents gnus-group-spam-classification-spam))
>
> so, do I use M-d in "Trash" to "ham" the message?

No, just mark it as read (`d'):

,----[ (info "(gnus)Spam ELisp Package Configuration Examples") ]
|      When I find a false positive, I mark the message with some
|      other ham mark (`ham-marks', *Note Spam ELisp Package Global
|      Variables::).
`----

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: spam filtering

[Sam Steingold]

> * Reiner Steib <ervarefgrvo+tznar@vznc.pp> [2004-09-29 09:12:55 +0200]:
>
> On Tue, Sep 28 2004, Sam Steingold wrote:
>
>> Now, how about my IMAP folders?
>>
>> I want spam to be moved to the spam folder on exit from the group, so,
>> I guess, I need to add to gnus-spam-process-newsgroups something like
>> ("nnimap.*" (spam spam-move-to-my-spam-imap-group))
>> or maybe (spam-process-destination . "nnimap+...:Trash")
>
> I think you can use my example from (info "(gnus)Spam ELisp Package
> Configuration Examples") with "nnimap:training.spam" replaced by
> "nnimap+...:Trash":
>
>         ("nnimap:\\(INBOX\\|other-folders\\)"
>          (spam-process-destination . "nnimap+...:Trash")
>          (spam-contents gnus-group-spam-classification-ham))

this is for `gnus-parameters', not `gnus-spam-process-newsgroups',
right?

also, I have _many_ imap folders, listing them all is not feasible.

>> (it would also be nice if I could mark non-spam in Trash and have it
>> moved to INBOX on exit from Trash, but...)
>
> Same here:
>
>         ("nnimap+...:Trash"
>          (ham-process-destination . "nnimap:INBOX")
>          (spam-contents gnus-group-spam-classification-spam))

so, do I use M-d in "Trash" to "ham" the message?


-- 
Sam Steingold (http://www.podval.org/~sds) running w2k
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.honestreporting.com>
Only a fool has no doubts.

Re: spam filtering

[Reiner Steib]

On Tue, Sep 28 2004, Sam Steingold wrote:

> Now, how about my IMAP folders?
>
> I want spam to be moved to the spam folder on exit from the group, so,
> I guess, I need to add to gnus-spam-process-newsgroups something like
> ("nnimap.*" (spam spam-move-to-my-spam-imap-group))
> or maybe (spam-process-destination . "nnimap+...:Trash")

I think you can use my example from (info "(gnus)Spam ELisp Package
Configuration Examples") with "nnimap:training.spam" replaced by
"nnimap+...:Trash":

        ("nnimap:\\(INBOX\\|other-folders\\)"
         (spam-process-destination . "nnimap+...:Trash")
         (spam-contents gnus-group-spam-classification-ham))

> (it would also be nice if I could mark non-spam in Trash and have it
> moved to INBOX on exit from Trash, but...)

Same here:

        ("nnimap+...:Trash"
         (ham-process-destination . "nnimap:INBOX")
         (spam-contents gnus-group-spam-classification-spam))

(When unsure about the correct syntax, I use `G c' first and put the
entries from `G E' into `gnus-parameters'.)

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

spam filtering

[Sam Steingold]

I set gnus-spam-process-newsgroups to
  (("gmane.*" (spam spam-use-gmane)))
and now I can mark gmane articles as spam with M-d and (apparently) they
are reported to gmane when I exit the group.

So far so good.

Now, how about my IMAP folders?

I want spam to be moved to the spam folder on exit from the group, so,
I guess, I need to add to gnus-spam-process-newsgroups something like
("nnimap.*" (spam spam-move-to-my-spam-imap-group))
or maybe (spam-process-destination . "nnimap+...:Trash")

(it would also be nice if I could mark non-spam in Trash and have it
moved to INBOX on exit from Trash, but...)

Thanks to Reiner & Ted for answering my previous questions.

-- 
Sam Steingold (http://www.podval.org/~sds) running w2k
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.honestreporting.com>
What was the best thing before sliced bread?