Moving messages to unreadable nnimap group? (spam reporting)

Moving messages to unreadable nnimap group? (spam reporting)

[Steinar Bang]

Is it possible to somehow create an nnimap group for an IMAP folder
that cannot be read?

The way to report spam to the bayesian filter on the Exchange server
at work, is to move the messages to a folder that cannot be read by
ordinary users.  The folder cannot be read, even from Outlook.  But
messages can be moved there.

In Gnus, I'm not allowed to create this group with `G m', since the
IMAP folder for the group cannot be read.  Is there a way around this?

Thanx!


- Steinar

Re: Moving messages to unreadable nnimap group? (spam reporting)

[Simon Josefsson]

Steinar Bang <sb@dod.no> writes:

> Is it possible to somehow create an nnimap group for an IMAP folder
> that cannot be read?
>
> The way to report spam to the bayesian filter on the Exchange server
> at work, is to move the messages to a folder that cannot be read by
> ordinary users.  The folder cannot be read, even from Outlook.  But
> messages can be moved there.
>
> In Gnus, I'm not allowed to create this group with `G m', since the
> IMAP folder for the group cannot be read.  Is there a way around this?

`U'?

Re: Moving messages to unreadable nnimap group? (spam reporting)

[Steinar Bang]

>>>>> Simon Josefsson <jas@extundo.com>:

> Steinar Bang <sb@dod.no> writes:

>> The way to report spam to the bayesian filter on the Exchange server
>> at work, is to move the messages to a folder that cannot be read by
>> ordinary users.  The folder cannot be read, even from Outlook.  But
>> messages can be moved there.

>> In Gnus, I'm not allowed to create this group with `G m', since the
>> IMAP folder for the group cannot be read.  Is there a way around
>> this?

> `U'?

That let me create the group blindly.  But when I try to do a `B m'
into this group, I get the following message:

Please answer y or n.  No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam.  Create it? (y or n) 

What's blocking the move?  The IMAP interface to Exchange?  Gnus?  Is
there a way to find out?

Thanx!

Re: Moving messages to unreadable nnimap group? (spam reporting)

[Simon Josefsson]

Steinar Bang <sb@dod.no> writes:

>>>>>> Simon Josefsson <jas@extundo.com>:
>
>> Steinar Bang <sb@dod.no> writes:
>
>>> The way to report spam to the bayesian filter on the Exchange server
>>> at work, is to move the messages to a folder that cannot be read by
>>> ordinary users.  The folder cannot be read, even from Outlook.  But
>>> messages can be moved there.
>
>>> In Gnus, I'm not allowed to create this group with `G m', since the
>>> IMAP folder for the group cannot be read.  Is there a way around
>>> this?
>
>> `U'?
>
> That let me create the group blindly.  But when I try to do a `B m'
> into this group, I get the following message:
>
> Please answer y or n.  No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam.  Create it? (y or n) 
>
> What's blocking the move?  The IMAP interface to Exchange?  Gnus?  Is
> there a way to find out?

(setq imap-log t) and inspecting the *imap-log* buffer would help.  It
sounds as if the server says the group doesn't exist, so Gnus wants to
create it first.

Re: Moving messages to unreadable nnimap group? (spam reporting)

[Steinar Bang]

>>>>> Simon Josefsson <jas@extundo.com>:

>> That let me create the group blindly.  But when I try to do a `B m'
>> into this group, I get the following message:

>> Please answer y or n.  No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam.  Create it? (y or n) 

>> What's blocking the move?  The IMAP interface to Exchange?  Gnus?  Is
>> there a way to find out?

> (setq imap-log t) and inspecting the *imap-log* buffer would help.  It
> sounds as if the server says the group doesn't exist, so Gnus wants to
> create it first.

There doesn't seem to be any IMAP traffic related to the attempted
move.

I entered the folder in question, attempted a move of the spam article
with `B m'.  I then first tried responding with `y' to create the
foler, and then attempted a new move and pressed `n'.  I then pressed
`q' to leave the group.

What I see in the *imap-log* is a fetch of headers for all ticked
articles in the group, followed by:

572 OK FETCH completed.
573 STATUS "INBOX" (UNSEEN)
* STATUS INBOX (UNSEEN 2)
573 OK STATUS completed.
574 EXPUNGE
* 499 EXISTS
574 OK EXPUNGE completed.

Re: Moving messages to unreadable nnimap group? (spam reporting)

[Simon Josefsson]

Steinar Bang <sb@dod.no> writes:

>>>>>> Simon Josefsson <jas@extundo.com>:
>
>>> That let me create the group blindly.  But when I try to do a `B m'
>>> into this group, I get the following message:
>
>>> Please answer y or n.  No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam.  Create it? (y or n) 
>
>>> What's blocking the move?  The IMAP interface to Exchange?  Gnus?  Is
>>> there a way to find out?
>
>> (setq imap-log t) and inspecting the *imap-log* buffer would help.  It
>> sounds as if the server says the group doesn't exist, so Gnus wants to
>> create it first.
>
> There doesn't seem to be any IMAP traffic related to the attempted
> move.

Weird.  Hm.  Does it help to press M-g on the U'd group first?

> I entered the folder in question, attempted a move of the spam article
> with `B m'.  I then first tried responding with `y' to create the
> foler, and then attempted a new move and pressed `n'.  I then pressed
> `q' to leave the group.

What happened after you pressed 'y'?  Any errors?

>
> What I see in the *imap-log* is a fetch of headers for all ticked
> articles in the group, followed by:
>
> 572 OK FETCH completed.
> 573 STATUS "INBOX" (UNSEEN)
> * STATUS INBOX (UNSEEN 2)
> 573 OK STATUS completed.
> 574 EXPUNGE
> * 499 EXISTS
> 574 OK EXPUNGE completed.

Re: Moving messages to unreadable nnimap group? (spam reporting)

[Steinar Bang]

>>>>> Simon Josefsson <jas@extundo.com>:

>> There doesn't seem to be any IMAP traffic related to the attempted
>> move.

> Weird.  Hm.  Does it help to press M-g on the U'd group first?

From the *Help: lossage* buffer (this is XEmacs):

No more newsgroups
nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam error: 501 usage: group newsgroup 
501 usage: group newsgroup 

(What I see in the minibuffer is just "No more newsgroups")

>> I entered the folder in question, attempted a move of the spam article
>> with `B m'.  I then first tried responding with `y' to create the
>> foler, and then attempted a new move and pressed `n'.  I then pressed
>> `q' to leave the group.

> What happened after you pressed 'y'?  Any errors?

From the *Help: lossage* buffer:

No such function: nntp-request-create-group
No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam.  Create it? (y or n) Yes

Re: Moving messages to unreadable nnimap group? (spam reporting)

[Simon Josefsson]

Steinar Bang <sb@dod.no> writes:

>>>>>> Simon Josefsson <jas@extundo.com>:
>
>>> There doesn't seem to be any IMAP traffic related to the attempted
>>> move.
>
>> Weird.  Hm.  Does it help to press M-g on the U'd group first?
>
> From the *Help: lossage* buffer (this is XEmacs):
>
> No more newsgroups
> nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam error: 501 usage: group newsgroup 
> 501 usage: group newsgroup 
>
> (What I see in the minibuffer is just "No more newsgroups")
>
>>> I entered the folder in question, attempted a move of the spam article
>>> with `B m'.  I then first tried responding with `y' to create the
>>> foler, and then attempted a new move and pressed `n'.  I then pressed
>>> `q' to leave the group.
>
>> What happened after you pressed 'y'?  Any errors?
>
> From the *Help: lossage* buffer:
>
> No such function: nntp-request-create-group
> No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam.  Create it? (y or n) Yes

I think that, somehow, Gnus is trying to create the group on your news
server, which fails, and also explains why there isn't anything in
your *imap-log*.  Compare `G e' on the U'd group with `G e' on a
working nnimap group on that server.

Re: Moving messages to unreadable nnimap group? (spam reporting)

[Steinar Bang]

>>>>> Simon Josefsson <jas@extundo.com>:

> I think that, somehow, Gnus is trying to create the group on your
> news server, which fails, and also explains why there isn't anything
> in your *imap-log*.  Compare `G e' on the U'd group

"somehost.somecompany.com"

> with `G e' on a working nnimap group on that server.

(nnimap "somehost.somecompany.com")

Re: Moving messages to unreadable nnimap group? (spam reporting)

[Simon Josefsson]

Steinar Bang <sb@dod.no> writes:

>>>>>> Simon Josefsson <jas@extundo.com>:
>
>> I think that, somehow, Gnus is trying to create the group on your
>> news server, which fails, and also explains why there isn't anything
>> in your *imap-log*.  Compare `G e' on the U'd group
>
> "somehost.somecompany.com"
>
>> with `G e' on a working nnimap group on that server.
>
> (nnimap "somehost.somecompany.com")

Try changing the former into the latter?

Re: Moving messages to unreadable nnimap group? (spam reporting)

[Steinar Bang]

>>>>> Simon Josefsson <jas@extundo.com>:

> Steinar Bang <sb@dod.no> writes:

>> "somehost.somecompany.com"

>>> with `G e' on a working nnimap group on that server.

>> (nnimap "somehost.somecompany.com")

> Try changing the former into the latter?

That changed the colour of the nnimap group.  But the behaviour didn't
change. 

What follows are minibuffer messages and the imap log, from entering
the folder, attempting to move, responding with `y', and then
attempting to move again, responding with `n', and then quit.

First the minibuffer messages (in reverse order):
nnimap: Setting marks in INBOX...
Marking spam as expired without moving it
Exiting summary buffer and applying spam rules
No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam
No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam.  Create it? (y or n) No
No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam.  Create it? (y or n) 
No such function: nntp-request-create-group
No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam.  Create it? (y or n) Yes
No such group: nnimap+somehost.somecompany.com:Public Folders/All Public Folders/Somecompany/Processes/Sysadm/Spam.  Create it? (y or n) 
No more unread articles
Generating summary...done
Generating summary...
Fetching headers for nnimap+somehost.somecompany.com:INBOX...done
nnimap: Retrieving headers...done
nnimap: Retrieving headers... /
nnimap: Retrieving headers... |
nnimap: Retrieving headers... \
nnimap: Retrieving headers... -
nnimap: Retrieving headers... /
Fetching headers for nnimap+somehost.somecompany.com:INBOX...


Here's the imap log:
[snip! fetch of headers of all ticked articles]
295 OK FETCH completed.
296 STATUS "INBOX" (UNSEEN)
* STATUS INBOX (UNSEEN 1)
296 OK STATUS completed.
297 EXPUNGE
* 518 EXISTS
297 OK EXPUNGE completed.

Doesn't look like there's anything going on, on the imap side.

imap spam reporting (Was: Moving messages to unreadable nnimap group...)

[Steinar Bang]

Would it be possible to write a spam reporting function using the imap
library of Gnus, that would move an article to a spam reporting group
on the server?  Ie. without first transferring the article to Gnus as
is done in move operations today?

This function could then move to a group that Gnus would refuse to
subscribe to, because it couldn't be read.  As far as Gnus is
concerned, the spam reporting group name is just a string sent as-is
to the IMAP server.

Re: imap spam reporting

[Ted Zlatanov]

On Sat, 08 May 2004, sb@dod.no wrote:

> Would it be possible to write a spam reporting function using the
> imap library of Gnus, that would move an article to a spam reporting
> group on the server?  Ie. without first transferring the article to
> Gnus as is done in move operations today?

Yes, but I think it's a bad idea.  Maybe it's better to fix Gnus than
to spend time writing such hacks.

Ted

Re: imap spam reporting

[Steinar Bang]

>>>>> Ted Zlatanov <tzz@lifelogs.com>:

> On Sat, 08 May 2004, sb@dod.no wrote:

>> Would it be possible to write a spam reporting function using the
>> imap library of Gnus, that would move an article to a spam
>> reporting group on the server?  Ie. without first transferring the
>> article to Gnus as is done in move operations today?

> Yes, but I think it's a bad idea.  Maybe it's better to fix Gnus
> than to spend time writing such hacks.

I disagree that this is a hack.

A command like the one I'm suggesting means that spam can be reported
by moving it to a different folder on an IMAP server, without first
transferring it to Gnus, and then pushing it back to the IMAP server.

Sounds to me like a better approach than if I actually had been able
to move the spam articles between nnimap groups.

As to fixing Gnus in this respect... I'm not really sure what the
proper way to handle unreadable nnimap groups is...

Re: imap spam reporting

[Ted Zlatanov]

On Wed, 12 May 2004, sb@dod.no wrote:

>>>>>> Ted Zlatanov <tzz@lifelogs.com>:
> 
>> On Sat, 08 May 2004, sb@dod.no wrote:
> 
>>> Would it be possible to write a spam reporting function using the
>>> imap library of Gnus, that would move an article to a spam
>>> reporting group on the server?  Ie. without first transferring the
>>> article to Gnus as is done in move operations today?
> 
>> Yes, but I think it's a bad idea.  Maybe it's better to fix Gnus
>> than to spend time writing such hacks.
> 
> I disagree that this is a hack.

Well, it's a workaround that can be avoided :)

> A command like the one I'm suggesting means that spam can be
> reported by moving it to a different folder on an IMAP server,
> without first transferring it to Gnus, and then pushing it back to
> the IMAP server.

Shouldn't the nnimap backend do this?  If it doesn't do it already,
it seems like an awfully nice feature to move articles without
retrieving them.

Writing a spam reporting function to specifically move articles from
IMAP group A to IMAP group B because of unreadable IMAP groups is a
very, very narrow-focus task.  I doubt more than 5 users worldwide
would ever find it useful.  If I'm wrong and there's a crowd of such
users, I'll be more than happy to help them.

> As to fixing Gnus in this respect... I'm not really sure what the
> proper way to handle unreadable nnimap groups is...

I think we should fix nnimap so it does what you want!  I'll be glad
to put my efforts in that direction, since it will benefit many more
users and solve your problem too.

Ted

Re: imap spam reporting

[Zack Weinberg]

Ted Zlatanov <tzz@lifelogs.com> writes:

>> A command like the one I'm suggesting means that spam can be
>> reported by moving it to a different folder on an IMAP server,
>> without first transferring it to Gnus, and then pushing it back to
>> the IMAP server.
>
> Shouldn't the nnimap backend do this?  If it doesn't do it already,
> it seems like an awfully nice feature to move articles without
> retrieving them.

A general facility for moving articles between IMAP groups without
retrieving them would be very nice, yes.

Support for IMAP partial fetch (i.e. not downloading giant binary
attachments when the message is first opened) would also be very nice.

zw

Re: imap spam reporting

[Steinar Bang]

>>>>> Zack Weinberg <zack@codesourcery.com>:

> Support for IMAP partial fetch (i.e. not downloading giant binary
> attachments when the message is first opened) would also be very
> nice.

I agree with that, as well.  Here's a link to Simon's outlining of how
it could be done (from July 26 2002):
	http://article.gmane.org/gmane.emacs.gnus.general/45839
	nntp://news.gmane.org/gmane.emacs.gnus.general/45839

The theme was revisited one year ago (in a thread started by you,
actually):
	http://article.gmane.org/gmane.emacs.gnus.general/52159
	nntp://news.gmane.org/gmane.emacs.gnus.general/52159

It just keeps cropping up...:-)

Re: imap spam reporting

[Steinar Bang]

>>>>> Ted Zlatanov <tzz@lifelogs.com>:

> On Wed, 12 May 2004, sb@dod.no wrote:

>> A command like the one I'm suggesting means that spam can be
>> reported by moving it to a different folder on an IMAP server,
>> without first transferring it to Gnus, and then pushing it back to
>> the IMAP server.

> Shouldn't the nnimap backend do this? 

It should.  But it doesn't.

According to this message from Simon
	http://article.gmane.org/gmane.emacs.gnus.general/54020
	nntp://news.gmane.org/gmane.emacs.gnus.general/54020
it's because the Gnus backend interface doesn't have a move command.

> If it doesn't do it already, it seems like an awfully nice feature
> to move articles without retrieving them.

Oh, I completely agree.

Re: imap spam reporting

[Ted Zlatanov]

On Thu, 13 May 2004, sb@dod.no wrote:

> According to this message from Simon
> 	http://article.gmane.org/gmane.emacs.gnus.general/54020
> 	nntp://news.gmane.org/gmane.emacs.gnus.general/54020
> it's because the Gnus backend interface doesn't have a move command.

OK, so unless someone else wants to look into this, I'll get on it
after I finish up gnus-encrypt.el and the CRM114 spam.el additions.

Ted

Re: imap spam reporting

[Bjørn Mork]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]

Steinar Bang <sb@dod.no> writes:
>>>>>> Ted Zlatanov <tzz@lifelogs.com>:
>> On Wed, 12 May 2004, sb@dod.no wrote:
>
>>> A command like the one I'm suggesting means that spam can be
>>> reported by moving it to a different folder on an IMAP server,
>>> without first transferring it to Gnus, and then pushing it back to
>>> the IMAP server.
>
>> Shouldn't the nnimap backend do this? 
>
> It should.  But it doesn't.
>
> According to this message from Simon
> 	http://article.gmane.org/gmane.emacs.gnus.general/54020
> 	nntp://news.gmane.org/gmane.emacs.gnus.general/54020
> it's because the Gnus backend interface doesn't have a move command.
>
>> If it doesn't do it already, it seems like an awfully nice feature
>> to move articles without retrieving them.
>
> Oh, I completely agree.

I've been using this patch since that discussion (thought I posted it
too, but maybe I forgot?):


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: nnimap.patch --]
[-- Type: text/x-patch, Size: 1757 bytes --]

Index: lisp/nnimap.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/nnimap.el,v
retrieving revision 7.3
diff -u -r7.3 nnimap.el
--- lisp/nnimap.el	25 Apr 2004 18:01:33 -0000	7.3
+++ lisp/nnimap.el	13 May 2004 16:35:04 -0000
@@ -1418,14 +1418,23 @@
 
 (defun nnimap-expiry-target (arts group server)
   (unless (eq nnmail-expiry-target 'delete)
-    (with-temp-buffer
-      (dolist (art arts)
-	(nnimap-request-article art group server (current-buffer))
-	;; hints for optimization in `nnimap-request-accept-article'
-	(let ((nnimap-current-move-article art)
-	      (nnimap-current-move-group group)
-	      (nnimap-current-move-server server))
-	  (nnmail-expiry-target-group nnmail-expiry-target group))))
+    (if (and (not (functionp nnmail-expiry-target)) 
+	     (gnus-server-equal (gnus-group-method nnmail-expiry-target) 
+				(gnus-server-to-method 
+				 (format "nnimap:%s" server))))
+	;; moving article within same server, speed it up...
+	(and (nnimap-possibly-change-group group)
+	     (imap-message-copy (imap-range-to-message-set arts)
+				(gnus-group-short-name nnmail-expiry-target) 
+				'dontcreate nil nnimap-server-buffer))
+      (with-temp-buffer
+	(dolist (art arts)
+	  (nnimap-request-article art group server (current-buffer))
+	  ;; hints for optimization in `nnimap-request-accept-article'
+	  (let ((nnimap-current-move-article art)
+		(nnimap-current-move-group group)
+		(nnimap-current-move-server server))
+	    (nnmail-expiry-target-group nnmail-expiry-target group)))))
     ;; It is not clear if `nnmail-expiry-target' somehow cause the
     ;; current group to be changed or not, so we make sure here.
     (nnimap-possibly-change-group group server)))

[-- Attachment #3: Type: text/plain, Size: 162 bytes --]


It solves the problem for me.  May have bad side effects like static
electricity in cats or whatever.


Bjørn
-- 
You're probably feminist yourself.  

Re: imap spam reporting

[Steinar Bang]

>>>>> Bjørn Mork <bmork@dod.no>:

> I've been using this patch since that discussion (thought I posted it
> too, but maybe I forgot?):
[snip!]
> It solves the problem for me.

How does your patch work?  What does it do?

Re: imap spam reporting

[Bjørn Mork]

Steinar Bang <sb@dod.no> writes:
>>>>>> Bjørn Mork <bmork@dod.no>:
>
>> I've been using this patch since that discussion (thought I posted it
>> too, but maybe I forgot?):
> [snip!]
>> It solves the problem for me.
>
> How does your patch work?  What does it do?

It calls imap-message-copy on the range instead of retrieving every
article and calling nnmail-expiry-target-group on each of them,
provided the group being expired and the expiry-target is on the same
imap-server.  It falls back to the original behoviour if the servers
differ or expiry-target is a function.

Looking at it, I wonder why imap-message-copy is called with
'dontcreate?  It will probably fail silently unless the expiry-target
exists.  Well, bugs are guaranteed.  I don't know upside-down on a
lisp. 

But this is only for expiry and probably won't help you.  The problem
is still that there is no "move" interface. 


Bjørn
-- 
You're always totally wrong.