Wanted: Gnus plugin to submit spam to ordb.org

Wanted: Gnus plugin to submit spam to ordb.org

[Kirk Strauser]

I've just configured support for ORDB's service into my mailserver.  I'm
still getting a few errant SPAMs, though.  Being a programmer and therefore
inherently lazy, I'd like to automate this process.

The easiest way to report a spammer's mailserver to ORDB for relay testing
is to send an email to a particular address, the format of the body of which
is:

  Relay: xxx.xxx.xxx.xxx
  Relay: yyy.yyy.yyy.yyy
    ...
  Relay: zzz.zzz.zzz.zzz

where the variables are the IP addresses of candidate open relays.

The most promising options for me at this time are to 1) re-write Gnus in
Perl or C and then implement my little script in a language I'm actually
good at, or 2) beg the genii on this list to help a poor would-be spam
reporter by writing an elisp program to do the same thing.

Anybody feel like saving me from re-writing Gnus?  :)
-- 
Kirk Strauser

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Simon Josefsson]

On 28 Apr 2002, Kirk Strauser wrote:

> I've just configured support for ORDB's service into my mailserver.  I'm
> still getting a few errant SPAMs, though.  Being a programmer and therefore
> inherently lazy, I'd like to automate this process.
> 
> The easiest way to report a spammer's mailserver to ORDB for relay testing
> is to send an email to a particular address, the format of the body of which
> is:
> 
>   Relay: xxx.xxx.xxx.xxx
>   Relay: yyy.yyy.yyy.yyy
>     ...
>   Relay: zzz.zzz.zzz.zzz
> 
> where the variables are the IP addresses of candidate open relays.
> 
> The most promising options for me at this time are to 1) re-write Gnus in
> Perl or C and then implement my little script in a language I'm actually
> good at, or 2) beg the genii on this list to help a poor would-be spam
> reporter by writing an elisp program to do the same thing.
> 
> Anybody feel like saving me from re-writing Gnus?  :)

Isn't ORDB available via DNS?  Then you can use dns.el + spam.el or 
something.  Sending mail for each incoming mail seems slow?

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Oystein Viggen]

* [Simon Josefsson] 

> Isn't ORDB available via DNS?  Then you can use dns.el + spam.el or 
> something.  Sending mail for each incoming mail seems slow?

I think the point of the exercise is is to submit new IP addresses to
ordb for checking upon receiving a spam where none of the addresses in
the Received headers.  The actual matching of incoming messages would be
done using the normal DNS trick.

Oystein
-- 
When in doubt: Recompile.

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Oystein Viggen]

* [Oystein Viggen] 

> I think the point of the exercise is is to submit new IP addresses to
> ordb for checking upon receiving a spam where none of the addresses in
> the Received headers 

...are listed in ordb.

*goes to grab some coffee*

> The actual matching of incoming messages would be done using the
> normal DNS trick.

Oystein
-- 
When in doubt: Think again.

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Kirk Strauser]

At 2002-04-29T11:04:30Z, Oystein Viggen <oysteivi@tihlde.org> writes:

> I think the point of the exercise is is to submit new IP addresses to ordb
> for checking upon receiving a spam where none of the addresses in the
> Received headers.  The actual matching of incoming messages would be done
> using the normal DNS trick.

Yes, that's correct.  The goal is to make it easy for users to submit data
to ORDB (or other similar services) without getting out of Gnus, or even
leaving the current article:

  1) User sees a SPAM
  2) User presses M-x spam-ordb
  3) User goes on to read the rest of their mail
-- 
Kirk Strauser

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Ted Zlatanov]

On 29 Apr 2002, kirk@strauser.com wrote:
> Yes, that's correct.  The goal is to make it easy for users to
> submit data to ORDB (or other similar services) without getting out
> of Gnus, or even leaving the current article:
> 
>   1) User sees a SPAM
>   2) User presses M-x spam-ordb
>   3) User goes on to read the rest of their mail

I was working on spam.el, and I had similar ideas.  I'm waiting for
write access to the Gnus CVS repository, until then the spam work has
been on the back burner.

My plan was:

1) user defines spam reporting and catching methods
   (whitelist & blacklist/ordb/spamassassing/etc)

2) user defines a rule in the split rules that will classify spam as
   such using the spam catching methods.  For instance, the ordb
   method would do a DNS query, while the blacklist method would look
   in the user's blacklist.

3) user marks spam with the "H" mark, which triggers the reporting
   methods requested by the user.  For instance, the ordb method would
   send mail to ORDB, while the blacklist method would add the sender
   to the user's blacklist.

Does that make sense?  Any comments from the list?

Thanks
Ted

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Kirk Strauser]

At 2002-04-29T16:05:12Z, Ted Zlatanov <teodor.zlatanov@divine.com> writes:

> 2) user defines a rule in the split rules that will classify spam as
>    such using the spam catching methods.  For instance, the ordb
>    method would do a DNS query, while the blacklist method would look
>    in the user's blacklist.

I've already installed the dnsbl features on my mailserver, as have quite a
few other people.  I personally wouldn't be too interested in having Gnus
perform the checks again, although I'm certain that others would love this.

> 3) user marks spam with the "H" mark, which triggers the reporting
>    methods requested by the user.  For instance, the ordb method would
>    send mail to ORDB, while the blacklist method would add the sender
>    to the user's blacklist.

I particularly like this.  Anything that makes widespread use of anti-spam
tools trivially easy for end users can only be a good thing for all of us.

> Does that make sense?  Any comments from the list?

Sounds good to me!
-- 
Kirk Strauser

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Ted Zlatanov]

On 29 Apr 2002, kirk@strauser.com wrote:
> 
> At 2002-04-29T16:05:12Z, Ted Zlatanov <teodor.zlatanov@divine.com>
> writes:
> 
>> 2) user defines a rule in the split rules that will classify spam
>>    as such using the spam catching methods.  For instance, the ordb
>>    method would do a DNS query, while the blacklist method would
>>    look in the user's blacklist.
> 
> I've already installed the dnsbl features on my mailserver, as have
> quite a few other people.  I personally wouldn't be too interested
> in having Gnus perform the checks again, although I'm certain that
> others would love this.

Right, so your checking method list wouldn't contain ordb, but your
reporting method list would.
> 
>> 3) user marks spam with the "H" mark, which triggers the reporting
>>    methods requested by the user.  For instance, the ordb method
>>    would send mail to ORDB, while the blacklist method would add
>>    the sender to the user's blacklist.
> 
> I particularly like this.  Anything that makes widespread use of
> anti-spam tools trivially easy for end users can only be a good
> thing for all of us.

My major concern (and the reason why I haven't been doing too much
work on this) is that generally, in Gnus marks do not trigger actions
directly.  This would be a usability change that may not be welcome by
the majority, even though it makes perfect sense to me personally.

Thanks
Ted

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Kai Großjohann]

Ted Zlatanov <teodor.zlatanov@divine.com> writes:

> My major concern (and the reason why I haven't been doing too much
> work on this) is that generally, in Gnus marks do not trigger actions
> directly.  This would be a usability change that may not be welcome by
> the majority, even though it makes perfect sense to me personally.

There is the cache mark: applying it means that Gnus also enters the
article into the cache.

kai
-- 
Silence is foo!

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Paul Jarc]

Ted Zlatanov <teodor.zlatanov@divine.com> wrote:
> My major concern (and the reason why I haven't been doing too much
> work on this) is that generally, in Gnus marks do not trigger actions
> directly.

The 'expire mark does, sort of.  But if you'd like to avoid that, a
function could be added to gnus-summary-exit-hook to look for articles
with the spam mark and report them.  This isn't too great a one-time
burden for the users, is it?


paul

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Ted Zlatanov]

On Tue, 30 Apr 2002, prj@po.cwru.edu wrote:
> Ted Zlatanov <teodor.zlatanov@divine.com> wrote:
>> My major concern (and the reason why I haven't been doing too much
>> work on this) is that generally, in Gnus marks do not trigger
>> actions directly.
> 
> The 'expire mark does, sort of.  But if you'd like to avoid that, a
> function could be added to gnus-summary-exit-hook to look for
> articles with the spam mark and report them.  This isn't too great a
> one-time burden for the users, is it?

On the contrary, as I said, I think it makes perfect sense :) My only
concern was that other Gnus users would think it strange that marking
an article suddenly takes a few seconds and actually performs an
action.  From the lack of an outcry, and from yours and Kai's
comments, I guess everyone is OK with having the 'spam' mark trigger
the spam actions.

OK, for the plan of action.  Note that this is very general at this
point, I'm looking to define what's needed.  Variable names are
definitely subject to change - suggestions are welcome.  Ditto for
invocation methods.

Let's define spam checking and reporting backends.  Right now we
have blacklists, whitelists, and spam-blackhole-servers:

(defvar spam-blackhole-servers
  '("bl.spamcop.net" "relays.ordb.org" "dev.null.dk"
    "relays.visi.com" "rbl.maps.vix.com"))

I'd like to break the blackhole servers into individual symbols.  The
users would say something like:

(add-to-list spam-checking-backends 'spamcop)
(add-to-list spam-checking-backends 'ordb)
(add-to-list spam-checking-backends 'blacklist)
(add-to-list spam-checking-backends 'whitelist) ; scores everything but whitelisted senders down 1000, for instance

and then they would add the spam-check function to their nnml-split or
nnimap-split.  So we'll have spamcop, ordb, dev-null-dk, visi, vix.
Or should this be a generic 'blackhole symbol which uses the list of
blackhole servers?  How likely is the list of blackhole servers to
change or need modification by the user?

Should the split function just score things down or actually move them
to a "spam" forlder?

The spam-reporting-backends would be triggered (as a hook) by marking
an article as spam.  The user would do:
(add-to-list spam-reporting-backends 'blacklist)
(add-to-list spam-reporting-backends 'ordb)

I need lists of reporting and checking backends that don't follow the
blackhole-servers conventions (using DNS for checks).  See spam.el for
what's already in place for blackhole checks.

Thanks
Ted

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Paul Jarc]

Ted Zlatanov <teodor.zlatanov@divine.com> wrote:
> My only concern was that other Gnus users would think it strange
> that marking an article suddenly takes a few seconds and actually
> performs an action.  From the lack of an outcry, and from yours and
> Kai's comments, I guess everyone is OK with having the 'spam' mark
> trigger the spam actions.

I think it would be less surprising (and possibly more efficient) to
take action during exit from the group, like with 'expire marks.  But
I don't care whether this is built-in or done by adding something to
gnus-summary-exit-hook.


paul

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Kai Großjohann]

prj@po.cwru.edu (Paul Jarc) writes:

> Ted Zlatanov <teodor.zlatanov@divine.com> wrote:
>> My major concern (and the reason why I haven't been doing too much
>> work on this) is that generally, in Gnus marks do not trigger actions
>> directly.
>
> The 'expire mark does, sort of.

Hm?  Nothing happens when I hit `E'.  Only after quitting the group
does something happen.  Contrast with `B DEL' which takes effect
before exiting the group...

kai
-- 
Silence is foo!

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Oystein Viggen]

* [Paul Jarc] 

> I think it would be less surprising (and possibly more efficient) to
> take action during exit from the group, like with 'expire marks.  But
> I don't care whether this is built-in or done by adding something to
> gnus-summary-exit-hook.

Actually, I have this in my .gnus:

(remove-hook 'gnus-summary-prepare-exit-hook
             'gnus-summary-expire-articles)

I added this because with gnus-summary-expire-articles, exiting a large
groupw would just take too damn long.  Instead I run
gnus-group-expire-all-groups regurlarly using the gnus-demon.  This
makes Gnus feel a lot snappier, and the gnus-demon only does this when
I'm idling anyway.

It would be nice if the spam stuff can be hacked in a similar way.

I also think that pressing a key to call a function like
gnus-summary-submit-spam-and-mark-as-expirable might negate the need for
another mark altogether.  Of course, I only say that because that's what
I used for a similar hack a while back to forward mails to the now
defunct orbz.  (By the way, I found $ to be a very fitting key to bind
this to.)

Just my nkr 0.02.

Oystein
-- 
PHP: the wu-ftpd of www.

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Simon Josefsson]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> prj@po.cwru.edu (Paul Jarc) writes:
>
>> Ted Zlatanov <teodor.zlatanov@divine.com> wrote:
>>> My major concern (and the reason why I haven't been doing too much
>>> work on this) is that generally, in Gnus marks do not trigger actions
>>> directly.
>>
>> The 'expire mark does, sort of.
>
> Hm?  Nothing happens when I hit `E'.  Only after quitting the group
> does something happen.  Contrast with `B DEL' which takes effect
> before exiting the group...

Unless it's in IMAP group... the \Deleted flag is set when you press B
DEL but if you M-u the message, the \Deleted flag won't be around when
the expunging is ran.  But this is a undocumented feature, I guess.

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
> prj@po.cwru.edu (Paul Jarc) writes:
>> Ted Zlatanov <teodor.zlatanov@divine.com> wrote:
>>> generally, in Gnus marks do not trigger actions directly.
>>
>> The 'expire mark does, sort of.
>
> Hm?  Nothing happens when I hit `E'.  Only after quitting the group
> does something happen.

Hence "sort of".  But I guess some other marks act this way too.


paul

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Frank Tegtmeyer]

Ted Zlatanov <teodor.zlatanov@divine.com> writes:

> Should the split function just score things down or actually move them
> to a "spam" forlder?

I would prefer a folder.

Regards, Frank

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Malcolm Purvis]

>>>>> "Ted" == Ted Zlatanov <teodor.zlatanov@divine.com> writes:

Ted> From the lack of an outcry, and from yours and Kai's comments, I guess
Ted> everyone is OK with having the 'spam' mark trigger the spam actions.

If you mean by this that the act of applying the mark triggers the spam
actions, I must delurk to raise an objection.

I think that the proper behavior for the spam mark would be something like the
expire one.  It should do nothing until the group is quit and then something
like report-as-spam-and-expire is run on all marked articles before the expire
process is run.  In addition, any article that is split using the spam rule
should also get this mark automatically.

This combination should give the user a straightforward method for reporting
spam as well as maximum flexibility for dealing with false positives and
exceptional cases.

Malcolm
-- 
	       Malcolm Purvis <malcolmpurvis@optushome.com.au>

The hidden, terrible cost of nuclear warfare is Really Bad Public Art.
			        - Angus McIntyre, alt.peeves, 13/3/02.

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Jorge Godoy]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Simon Josefsson <jas@extundo.com> writes:

> Unless it's in IMAP group... the \Deleted flag is set when you press B
> DEL but if you M-u the message, the \Deleted flag won't be around when
> the expunging is ran.  But this is a undocumented feature, I guess.

It should become the default behaviour. Expunging message should only
occur when exiting the group. It would allow some 'undelete'
characteristic that might be interesting.

- -- 
Godoy. <godoy@conectiva.com>

Escritório de Projetos		-- 	  Conectiva S.A.
Projects Office			--	  Conectiva Inc.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE8z+NCEzC+baSjBiURArPXAJ9PCFcoQK02TJIHJeq9/tzkN98JHwCdE+9r
ooligLDpjp3YfWhjwa35lng=
=mGH3
-----END PGP SIGNATURE-----

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Kai Großjohann]

prj@po.cwru.edu (Paul Jarc) writes:

> Hence "sort of".  But I guess some other marks act this way too.

Cache takes effect immediately, I think.

kai
-- 
Silence is foo!

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
> prj@po.cwru.edu (Paul Jarc) writes:
>> Hence "sort of".  But I guess some other marks act this way too.
>
> Cache takes effect immediately, I think.

Yes, though I think of that more as an action which happens to
(unfortunately) use a mark to do its work.


paul

Re: Wanted: Gnus plugin to submit spam to ordb.org

[Steinar Bang]

>>>>> Jorge Godoy <godoy@conectiva.com>:

> Simon Josefsson <jas@extundo.com> writes:

>> Unless it's in IMAP group... the \Deleted flag is set when you press B
>> DEL but if you M-u the message, the \Deleted flag won't be around when
>> the expunging is ran.  But this is a undocumented feature, I guess.

> It should become the default behaviour. Expunging message should
> only occur when exiting the group. It would allow some 'undelete'
> characteristic that might be interesting.

I agree.  I like the current behaviour.