spam/ham processing

spam/ham processing

[Jake Colman]

I am getting a bit confused, although I think that I am almost there.

1) If a folder is declared a spam folder, and I set up a spam exit processor,
   the contents of that folder will be sent to spam processor so that it can
   be trained as spam.  Eventually, in an ideal world, I would expect all
   those messages to go directly to my general spambox as soon as they are
   seen, since my processor has been trained, and not even hit this folder.
   Is this correct?

2) In a spam folder, the default mark is the spam mark.  How do I mark
   sonmething as ham?  When I exit the folder, if I have specified both hm
   and spam exit processors, will both kinds of message be sent out to be
   trained?  

3) In a spam folder, the default action is to expire the spam contents.  What
   is the default action for the ham?

4) My spambox has the following configuration

((uidvalidity . "71437")
 (timestamp 16288 15245)
 (total-expire . t)
 (spam-contents gnus-group-spam-classification-spam)
 (spam-process
  (gnus-group-spam-exit-processor-bogofilter gnus-group-ham-exit-processor-bogofilter))
 (ham-process-destination . "nnimap+hamilton:INBOX/Misc")
 (ham-marks
  (gnus-read-mark))
 (expiry-wait . immediate))

   If I mark something in there as ham, by pressing 'd' and changing the mark
   to 'r', it should be moved to Misc box.  Instead, it disappared.  How do I
   figure out what happned to it?

5) I guess all groups should speicy both a ham and spam exit processor, no?
   This way all group's contents can be used for training.  The default
   classification for the folder (ham/spam) should reflect the status of the
   majortity of the mesages sent to that folder.  Does this make sense?

TIA!

...Jake

-- 
Jake Colman                     

Principia Partners LLC                    Phone: (201) 209-2467
Harborside Financial Center                 Fax: (201) 946-0320
902 Plaza Two                          E-mail: colman@ppllc.com
Jersey City, NJ 07311                 www.principiapartners.com

Re: spam/ham processing

[Ted Zlatanov]

On Wed, 29 Oct 2003, colman@ppllc.com wrote:

> 1) If a folder is declared a spam folder, and I set up a spam exit
>    processor, the contents of that folder will be sent to spam
>    processor so that it can be trained as spam.  Eventually, in an
>    ideal world, I would expect all those messages to go directly to
>    my general spambox as soon as they are seen, since my processor
>    has been trained, and not even hit this folder.  Is this correct?

Spam groups' classification matters only as far as first-seen or
unread mail.  Such mail be marked as spam when you enter that group.

Spam articles *anywhere* are processed by the exit spam processor.
Ham articles are treated differently based on the group
classification.  See the manual, please.

> 2) In a spam folder, the default mark is the spam mark.  How do I
>    mark sonmething as ham?  When I exit the folder, if I have
>    specified both hm and spam exit processors, will both kinds of
>    message be sent out to be trained?

Spam and ham marks are actually customizable (though spam marks should
probably be just the spam mark).  Do `G c' on a group or topic to
customize its spam/ham marks, or use M-x customize-variable on the
variables gnus-parameter-ham-marks-alist and
gnus-parameter-spam-marks-alist to achieve the same effect.  This is
in the manual.

> 3) In a spam folder, the default action is to expire the spam
>    contents.  What is the default action for the ham?

It depends on the folder, but by default nothing is done.

> 4) My spambox has the following configuration
> 
> ((uidvalidity . "71437")
>  (timestamp 16288 15245)
>  (total-expire . t)
>  (spam-contents gnus-group-spam-classification-spam)
>  (spam-process
>   (gnus-group-spam-exit-processor-bogofilter
>   gnus-group-ham-exit-processor-bogofilter))
>  (ham-process-destination . "nnimap+hamilton:INBOX/Misc")
>  (ham-marks
>   (gnus-read-mark))
>  (expiry-wait . immediate))
> 
>    If I mark something in there as ham, by pressing 'd' and changing
>    the mark to 'r', it should be moved to Misc box.  Instead, it
>    disappared.  How do I figure out what happned to it?

When it's marked as read, an article won't show up.  Try entering Misc
with C-u ENTER to see all the articles in it.  I set
spam-mark-ham-unread-before-move-from-spam-group [1] to t, personally,
and that way all ham that is moved out of a spam group is marked
unread.

> 5) I guess all groups should speicy both a ham and spam exit
>    processor, no?  

It's up to the user.  You can choose to do so.

> This way all group's contents can be used for training.  The default
> classification for the folder (ham/spam) should reflect the status
> of the majortity of the mesages sent to that folder.  Does this make
> sense?

Yes, but again, it's up to the user.  There's an incredible variety of
setups among Gnus users, and spam processing is done by them in
different ways, so spam.el tries to be as flexible as possible.

Ted

Re: spam/ham processing

[Jake Colman]

>>>>> "TZ" == Ted Zlatanov <tzz@lifelogs.com> writes:

   >> 4) My spambox has the following configuration
   >> 
   >> ((uidvalidity . "71437")
   >> (timestamp 16288 15245)
   >> (total-expire . t)
   >> (spam-contents gnus-group-spam-classification-spam)
   >> (spam-process
   >> (gnus-group-spam-exit-processor-bogofilter
   >> gnus-group-ham-exit-processor-bogofilter))
   >> (ham-process-destination . "nnimap+hamilton:INBOX/Misc")
   >> (ham-marks
   >> (gnus-read-mark))
   >> (expiry-wait . immediate))
   >> 
   >> If I mark something in there as ham, by pressing 'd' and changing
   >> the mark to 'r', it should be moved to Misc box.  Instead, it
   >> disappared.  How do I figure out what happned to it?

   TZ> When it's marked as read, an article won't show up.  Try entering Misc
   TZ> with C-u ENTER to see all the articles in it.  I set
   TZ> spam-mark-ham-unread-before-move-from-spam-group [1] to t, personally,
   TZ> and that way all ham that is moved out of a spam group is marked
   TZ> unread.

I've changed the config for the default spam mailbox (using G-c) and dumped
it out (using G-p) as follows:

((uidvalidity . "71437")
 (timestamp 16289 17610)
 (total-expire . t)
 (registry-ignore . t)
 (spam-contents gnus-group-spam-classification-spam)
 (spam-process
  (gnus-group-spam-exit-processor-bogofilter gnus-group-ham-exit-processor-bogofilter))
 (spam-process-destination)
 (ham-process-destination . "INBOX/Misc")
 (ham-marks nil)
 (expiry-wait . 2))

By default all messages are marked as spam.  I manually marked my ham using
the 'd' command.  When I exited the group, it presumably trained my spam and
ham, and should have moveed the ham.  It did not. I went back into this
mailbox and found my spam marked as 'E' (makes sense) and my ham marked as
'O'.

What am I doing wrong?

-- 
Jake Colman                     

Principia Partners LLC                    Phone: (201) 209-2467
Harborside Financial Center                 Fax: (201) 946-0320
902 Plaza Two                          E-mail: colman@ppllc.com
Jersey City, NJ 07311                 www.principiapartners.com

can gnus-unread-mark be a ham mark?

[Bill White]

On Thu Oct 30 2003 at 09:33, Ted Zlatanov <tzz@lifelogs.com> said:

>> 2) In a spam folder, the default mark is the spam mark.  How do I
>>    mark sonmething as ham?  When I exit the folder, if I have
>>    specified both hm and spam exit processors, will both kinds of
>>    message be sent out to be trained?
>
> Spam and ham marks are actually customizable (though spam marks
> should probably be just the spam mark).  Do `G c' on a group or
> topic to customize its spam/ham marks, or use M-x customize-variable
> on the variables gnus-parameter-ham-marks-alist and
> gnus-parameter-spam-marks-alist to achieve the same effect.  This is
> in the manual.

When I find ham in my spam group, I have a long-standing habit of
unmarking it with M-u (gnus-summary-clear-mark-forward), then moving
it somewhere useful to be acted on.  According to gnus-sum.el, it
looks as though M-u marks the message with gnus-unread-mark.

I suspect spam.el could move the message for me when I exit the group
if only it recognized gnus-unread-mark as one of the ham marks.
However, since spam.el doesn't currently notice gnus-unread-mark, my
newly-unmarked ham isn't handled by spam.el when I exit the group.

So I have a few questions:

- Does it make sense to consider these unmarked messages (messages
  marked with gnus-unread-mark) as ham when exiting a spam group?  It
  sure does to me, but maybe I'm weird.

- What's a good way to make gnus-unread-mark one of the recognized ham
  marks?  I hesitate to copy (gnus-define-group-parameter
  ham-marks... from gnus.el and redefine it in one of my startup
  files.

- Should gnus-unread-mark be added to the default value of
  gnus-parameter-ham-marks-alist?

Thanks for this magnificent spam-fighting machinery, Ted!

Cheers -

bw
-- 
Bill White . billw@wolfram.com . http://members.wri.com/billw
"No ma'am, we're musicians."

Re: can gnus-unread-mark be a ham mark?

[Ted Zlatanov]

On Fri, 31 Oct 2003, billw@wolfram.com wrote:

> When I find ham in my spam group, I have a long-standing habit of
> unmarking it with M-u (gnus-summary-clear-mark-forward), then moving
> it somewhere useful to be acted on.  According to gnus-sum.el, it
> looks as though M-u marks the message with gnus-unread-mark.

> I suspect spam.el could move the message for me when I exit the
> group if only it recognized gnus-unread-mark as one of the ham
> marks.  However, since spam.el doesn't currently notice
> gnus-unread-mark, my newly-unmarked ham isn't handled by spam.el
> when I exit the group.

> So I have a few questions:
> 
> - Does it make sense to consider these unmarked messages (messages
>   marked with gnus-unread-mark) as ham when exiting a spam group?
>   It sure does to me, but maybe I'm weird.

You can, by just adding the 'unread' mark to the list of ham marks
without using the customization interface.  That would be all that was
needed.  I have made a conscious effort to keep the 'unread' mark out
of the ham marks' available choices, because I want unread messages to
be considered unclassified.  Read below for my own solution to this.

> - What's a good way to make gnus-unread-mark one of the recognized
>   ham marks?  I hesitate to copy (gnus-define-group-parameter
>   ham-marks... from gnus.el and redefine it in one of my startup
>   files.

Here's what I do: I have the ! (ticked) mark as a ham mark.  I also
have the ham-process-destination parameter of my spam group set to
"mail" (my default mail group), and the
spam-mark-ham-unread-before-move-from-spam-group variable set to t.
Also I have my spam group's spam-process-destination set to "train"
(which is used for SpamAssassin training on my IMAP server), but you
don't need to do that.

That way, I go into my spam group, mark ham with `!', and just exit
the group.  

All new articles were marked as spam when I entered the spam group, so
now the ones still marked as spam are all moving to the "train"
(spam-process-destination) folder.  There is no spam-exit-processor
because the training is done on the IMAP server.

The ham-marked articles, namely the ticked ones, get moved to "mail"
and *unmarked* (this is the part I think you want), so they look fresh
and new.  I don't use a ham-exit-processor on them currently.

> - Should gnus-unread-mark be added to the default value of
>   gnus-parameter-ham-marks-alist?

No, I think we need an 'unclassified' mark, which 'unread' is right
now implicitly.  But it can be added to the customize choices.

> Thanks for this magnificent spam-fighting machinery, Ted!

It feels more like a swiss army knife with a saw blade attachment, but
thanks :)  I'm working on the promised integration with the registry
so articles won't get registered as spam/ham twice; after that I may
redo a lot of the spam.el code to be more generic and flexible.
Also, some "wizards" (heh, "angels") for configuring novice users'
setups would be nice.  All in all, I think spam.el is approaching a
stable 1.0 point, but it can be a lot better.

Ted

Re: can gnus-unread-mark be a ham mark?

[Bill White]

On Fri Oct 31 2003 at 10:47, Ted Zlatanov <tzz@lifelogs.com> said:

> That way, I go into my spam group, mark ham with `!', and just exit
> the group.  

[...]

> The ham-marked articles, namely the ticked ones, get moved to "mail"
> and *unmarked* (this is the part I think you want), so they look
> fresh and new.  I don't use a ham-exit-processor on them currently.

Excellent - this is working for me now, and ticked makes even more
sense to me than unmarked.  Thanks!

bw
-- 
Bill White                                               Office: 5N-X30
Documentation Programmer                      Phone: 217-398-0700 x 234
Wolfram Research                                      Fax: 217-398-0747
http://members.wri.com/billw         Office hours: MTuTh 8a-4p WF 4a-12

Re: can gnus-unread-mark be a ham mark?

[Jake Colman]

>>>>> "BW" == Bill White <billw@wolfram.com> writes:

   BW> On Fri Oct 31 2003 at 10:47, Ted Zlatanov <tzz@lifelogs.com> said:
   >> That way, I go into my spam group, mark ham with `!', and just exit
   >> the group.  

   BW> [...]

   >> The ham-marked articles, namely the ticked ones, get moved to "mail"
   >> and *unmarked* (this is the part I think you want), so they look
   >> fresh and new.  I don't use a ham-exit-processor on them currently.

   BW> Excellent - this is working for me now, and ticked makes even more
   BW> sense to me than unmarked.  Thanks!

You could also have used the gnus-del-mark ('r') for the same thing, right?

-- 
Jake Colman                     

Principia Partners LLC                    Phone: (201) 209-2467
Harborside Financial Center                 Fax: (201) 946-0320
902 Plaza Two                          E-mail: colman@ppllc.com
Jersey City, NJ 07311                 www.principiapartners.com

Re: can gnus-unread-mark be a ham mark?

[Ted Zlatanov]

On Fri, 31 Oct 2003, colman@ppllc.com wrote:

>>>>>> "BW" == Bill White <billw@wolfram.com> writes:
> 
>    BW> On Fri Oct 31 2003 at 10:47, Ted Zlatanov <tzz@lifelogs.com>
>    BW> said:
>    >> That way, I go into my spam group, mark ham with `!', and just
>    >> exit the group.
> 
>    BW> [...]
> 
>    >> The ham-marked articles, namely the ticked ones, get moved to
>    >> "mail" and *unmarked* (this is the part I think you want), so
>    >> they look fresh and new.  I don't use a ham-exit-processor on
>    >> them currently.
> 
>    BW> Excellent - this is working for me now, and ticked makes even
>    BW> more sense to me than unmarked.  Thanks!
> 
> You could also have used the gnus-del-mark ('r') for the same thing,
> right?

Any ham-mark, really, will work.

Ted

Re: can gnus-unread-mark be a ham mark?

[Bill White]

On Fri Oct 31 2003 at 11:35, Jake Colman <colman@ppllc.com> said:

>>>>>> "BW" == Bill White <billw@wolfram.com> writes:
>
>    BW> On Fri Oct 31 2003 at 10:47, Ted Zlatanov <tzz@lifelogs.com> said:
>    >> That way, I go into my spam group, mark ham with `!', and just exit
>    >> the group.  
>
>    BW> [...]
>
>    >> The ham-marked articles, namely the ticked ones, get moved to "mail"
>    >> and *unmarked* (this is the part I think you want), so they look
>    >> fresh and new.  I don't use a ham-exit-processor on them currently.
>
>    BW> Excellent - this is working for me now, and ticked makes even more
>    BW> sense to me than unmarked.  Thanks!
>
> You could also have used the gnus-del-mark ('r') for the same thing,
> right?

Yes, that would work too, since gnus-del-mark is one of the default
ham marks.  It doesn't fit into the way I think about ham in a spam
group, though - I like to do something to "rescue" the ham message,
then later start a different process that begins with reading the
message in the group where it belongs.  Starting the
reaading-thinking-replying process with the message still in a spam
group disturbs me somehow.  Ain't it amazing that gnus can be
customized to this extent?

Cheers -

bw
-- 
Bill White . billw@wolfram.com . http://members.wri.com/billw
"No ma'am, we're musicians."