Re: A different spam model, which might be possible already

[Ted Zlatanov <tzz@lifelogs.com>]

On Fri, 02 May 2003, zack@codesourcery.com wrote:
> I use Gnus primarily to get at my mail, which is stored on an IMAP
> server.  Mail is sorted into folders by a maildrop script which runs
> on the server.  This script is also responsible for running
> bogofilter to weed out spam, which is sent to a separate folder;
> thus the bogofilter database is stored on that machine, too. I have
> shell access to the machine which stores the mail.
> 
> I have not yet been able to figure out how to use Gnus' spam
> features to feed error corrections back to bogofilter on the remote
> machine.  The way I would like it to work is:
> 
> * If I do 'B m' on message(s) in the spam group, they automagically
>   get their $ mark removed.  

Currently, the behavior is the reverse: if the articles have a
ham-mark in a spam group (see the ham-marks group parameter) they will
be moved out of the spam group to the ham-process-destination group
after being processed with the group's ham processors.  What you
describe could be coded as alternate behavior, but I don't see the
advantage (see later about article moving and tracking).

> Also, a hook fires which runs a program on the local machine.  This
> program gets the IMAP UIDs of all the moved messages as arguments.
> It does *not* get the text of the messages on its standard input or
> elsewhere.  It does not need to know the destination of the
> messages.  The move command is issued to the IMAP server only after
> this program finishes.

This hook can be a ham processor, invoked once per ham article.  The
processor will get each article number, and could then grab the IMAP
UIDs using the article number.  Then, an arbitrary shell command could
be executed after all the articles have been queued up (so this shell
command would be run after the ham processor has been invoked on each
hap article).  The shell command would login to the remote server and
get the article UIDs as standard input; on the other side you can run
something that can interpret the article UIDs and feed them to
bogofilter.

I'm not sure what's the benefit of doing the article move on the
server side.  Isn't it better to let Gnus do the move, since it will
involve about the same amount of (minimal) network traffic?  Or is the
nnimap remote article move somehow suboptimal?

> * If I mark messages in ham groups as spam ($), then on group exit
>   another hook fires which, again, runs a program on the local
>   machine, passing the IMAP UIDs of all the spam-marked messages as
>   arguments.  This program does need to know the name of the group
>   containing the messages - presumably that is an argument too.
>   After the program finishes, all the spam-marked messages are
>   deleted as if with 'B DEL', and then EXPUNGE is issued.

This works analogously to what's above for ham articles in spam
groups.  If the spam-destination parameter for a ham group is nil, the
spam articles will be only marked as expired.  If it's set to a group
name, the spam articles will be moved to that group and marked as
expired.  Regardless, the spam-processor for the ham group will be
invoked on each article, so it could queue up the IMAP UIDs and you
could pipe them to a single shell command afterwards.

> The programs run on the local machine are going to ssh over to the
> machine which stores the mail, dig the messages out of the
> mailboxes, and run bogofilter database update commands on them.
> There's no need to download the messages to the local machine and
> then send them back to the server, but there is a need for the
> messages not to get deleted or moved until the aforementioned
> digging happens.

Why?  As long as you don't expunge or expire the articles before
calling the shell pipe for ham articles in spam groups or spam
articles in ham groups, spam.el will know if they got moved to another
group or were left in the original group, marked as expired.  It can
tell that to the shell pipe to the remote server.

> All of this may be possible right now in Oort, but I don't know how
> to do it.  There is nothing in the manual about a setup like this,
> and I am not familiar enough with Gnus' internals to know where to
> look for appropriate hooks, internal functions, etc.  I do know
> enough Emacs Lisp that I can probably code this myself given enough
> pointers.

I think I understand what you want, and you definitely need a custom
spam/ham processor that will queue up message UIDs into a file or a
Lisp list.  You may also need a change in behavior, if you don't like
the mark-then-move procedure above and want the move-then-mark
procedure you described.  Finally, you will need a custom function
that will take the queued spam or ham message UIDs and feed them to a
remote shell script; this doesn't have to be a part of spam.el.

It sounds very useful, and I could certainly use something similar now
that my ISP has SpamAssassin on the mail server (but I was too lazy to
write it so far :)

Thanks
Ted