Automatically marking Gcc's as SEEN in nnimap

Automatically marking Gcc's as SEEN in nnimap

[Lloyd Zusman]

I'm using the latest CVS gnus under Emacs 23.0.60.1.

I use posting styles to set up the following kind of Gcc's for outgoing
messages in my nnimap groups:

  Gcc: nnimap+imap.server.tld:INBOX.archive.whatever

This properly writes copies my outgoing messages into the appropriate
INBOX.archive.whatever folder which I have previously created on my IMAP
server. The "whatever" part varies depending on which group the email is
being sent from.

What I'd like to do is for these archived messages to automatically get
marked as SEEN.

I was previously using Wanderlust, and archived messages going to the
correspondng IMAP-based Gcc destinations do indeed get marked as SEEN
under this software package. Therefore, I know that this is do-able.

If this is possible under gnus, how do I set this up?

Thanks in advance.


-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: Automatically marking Gcc's as SEEN in nnimap

[Paul R]

M-x apropos-variable RET gnus.*gcc.*read RET

gnus-gcc-mark-as-read
  Variable: If non-nil, automatically mark Gcc articles as read.


-- 
  Paul

Re: Automatically marking Gcc's as SEEN in nnimap

[Lloyd Zusman]

Paul R <paul.r.ml@gmail.com> writes:

> M-x apropos-variable RET gnus.*gcc.*read RET
>
> gnus-gcc-mark-as-read
>   Variable: If non-nil, automatically mark Gcc articles as read.

Thanks, but I forgot to mention in my original post that I am already
using that variable and it is set to `t' ... and the Gcc'ed messages are
never being marked as SEEN.

Have you actually tested this setting within an nnimap group and an
nnimap Gcc, or are you just going by the docs?

Here's pertinent the code fragment in gnus-msg.el ...

  (when (and group-art
             ;; FIXME: Should gcc-mark-as-read work when
             ;; Gnus is not running?
             (gnus-alive-p)
             (or gnus-gcc-mark-as-read
                 (and
                  (boundp 'gnus-inews-mark-gcc-as-read)
                  (symbol-value 'gnus-inews-mark-gcc-as-read))))
    (gnus-group-mark-article-read group (cdr group-art)))

This fragment appears within the `gnus-inews-do-gcc' function, and there
is no other gnus code which tests the value of the variable
`gnus-gcc-mark-as-read'. Could it be that this variable only applies
when an inews-type function is being performed under nntp, and not under
any other back ends such as nnimap?

However, if this code indeed gets executed in all backends, then I
should point out that gnus is indeed alive when the Gcc is being
performed; therefore, `(gnus-alive-p)' definitely returns `t'. Maybe
`group-art' isn't set? Is there some other setting that controls how
that variable is set?

Or perhaps the problem is related to nnimap. Maybe marking the messages
as "read" in gnus isn't the same as marking them as "SEEN" in IMAP. Or
maybe there is some other problem ... ???



-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: Automatically marking Gcc's as SEEN in nnimap

[Dave Goldberg]

>>>>> On Wed, 17 Dec 2008 19:57:22 -0500, Lloyd Zusman <ljz@asfast.com> said:

> Paul R <paul.r.ml@gmail.com> writes:
>> M-x apropos-variable RET gnus.*gcc.*read RET
>> 
>> gnus-gcc-mark-as-read
>> Variable: If non-nil, automatically mark Gcc articles as read.

> Have you actually tested this setting within an nnimap group and an
> nnimap Gcc, or are you just going by the docs?

I have.  I use nnimap exclusively at work (meaning no other backends,
not that I run Gnus only at work and not at home :-) and my gcc's are
always marked read when I next enter the gcc'd group.

> Or perhaps the problem is related to nnimap. Maybe marking the messages
> as "read" in gnus isn't the same as marking them as "SEEN" in IMAP. Or
> maybe there is some other problem ... ???

My gcc's are certainly marked "read" but they are not marked "seen".
If I do a `/ .' (gnus-summary-limit-to-unseen) in the group, my gcc
shows up with the other new messages.  So there is a difference
between "read" and "seen".

-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: Automatically marking Gcc's as SEEN in nnimap

[Lloyd Zusman]

Dave Goldberg <david.goldberg6@verizon.net> writes:

>>>>>> On Wed, 17 Dec 2008 19:57:22 -0500, Lloyd Zusman <ljz@asfast.com> said:
>>
>> [ ... ]
>>
>> Or perhaps the problem is related to nnimap. Maybe marking the messages
>> as "read" in gnus isn't the same as marking them as "SEEN" in IMAP. Or
>> maybe there is some other problem ... ???
>
> My gcc's are certainly marked "read" but they are not marked "seen".
> If I do a `/ .' (gnus-summary-limit-to-unseen) in the group, my gcc
> shows up with the other new messages.  So there is a difference
> between "read" and "seen".

Aha! So this seems to be the issue, then.

So, does anyone know how to cause articles marked as "read" in gnus to
be marked as "seen" under nnimap?

-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: Automatically marking Gcc's as SEEN in nnimap

[Lloyd Zusman]

Lloyd Zusman <ljz@asfast.com> writes:

> Dave Goldberg <david.goldberg6@verizon.net> writes:
>
>>>>>>> On Wed, 17 Dec 2008 19:57:22 -0500, Lloyd Zusman <ljz@asfast.com> said:
>>>
>>> [ ... ]
>>>
>>> Or perhaps the problem is related to nnimap. Maybe marking the messages
>>> as "read" in gnus isn't the same as marking them as "SEEN" in IMAP. Or
>>> maybe there is some other problem ... ???
>>
>> My gcc's are certainly marked "read" but they are not marked "seen".
>> If I do a `/ .' (gnus-summary-limit-to-unseen) in the group, my gcc
>> shows up with the other new messages.  So there is a difference
>> between "read" and "seen".
>
> [ ... ]

Does anyone have any thoughts about this? Is there a way to get messages
that are marked as "read" in nnimap to also get marked as "seen" on the
IMAP server?


-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: Automatically marking Gcc's as SEEN in nnimap

[David Engster]

Lloyd Zusman <ljz@asfast.com> writes:
> Lloyd Zusman <ljz@asfast.com> writes:
>
>> Dave Goldberg <david.goldberg6@verizon.net> writes:
>>
>>>>>>>> On Wed, 17 Dec 2008 19:57:22 -0500, Lloyd Zusman <ljz@asfast.com> said:
>>>>
>>>> [ ... ]
>>>>
>>>> Or perhaps the problem is related to nnimap. Maybe marking the messages
>>>> as "read" in gnus isn't the same as marking them as "SEEN" in IMAP. Or
>>>> maybe there is some other problem ... ???
>>>
>>> My gcc's are certainly marked "read" but they are not marked "seen".
>>> If I do a `/ .' (gnus-summary-limit-to-unseen) in the group, my gcc
>>> shows up with the other new messages.  So there is a difference
>>> between "read" and "seen".
>>
>> [ ... ]
>
> Does anyone have any thoughts about this? Is there a way to get messages
> that are marked as "read" in nnimap to also get marked as "seen" on the
> IMAP server?

Do you use nnimap as your primary select method? If so, manually edit
your GCC header

  Gcc: nnimap+imap.server.tld:INBOX.archive.whatever

so that it just reads

  Gcc: INBOX.archive.whatever

Does it work now?

-David

Re: Automatically marking Gcc's as SEEN in nnimap

[Lloyd Zusman]

David Engster <deng@randomsample.de> writes:

> Lloyd Zusman <ljz@asfast.com> writes:
>>
>> [ ... ]
>>
>> Does anyone have any thoughts about this? Is there a way to get messages
>> that are marked as "read" in nnimap to also get marked as "seen" on the
>> IMAP server?
>
> Do you use nnimap as your primary select method? If so, manually edit
> your GCC header
>
>   Gcc: nnimap+imap.server.tld:INBOX.archive.whatever
>
> so that it just reads
>
>   Gcc: INBOX.archive.whatever
>
> Does it work now?
>
> -David

Aha! Well, I have three nnimap select methods: one as my primary, and
two others which are secondary. When I refer to archives on the primary
server in the way you describe above, they are now indeed getting marked
as SEEN at the same time they're getting marked as "read".

However, what can I do about the secondaries? Anything?

Thanks!

-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: Automatically marking Gcc's as SEEN in nnimap

[David Engster]

[-- Attachment #1: Type: text/plain, Size: 1023 bytes --]

Lloyd Zusman <ljz@asfast.com> writes:
> David Engster <deng@randomsample.de> writes:
>
>> Lloyd Zusman <ljz@asfast.com> writes:
>>>
>>> [ ... ]
>>>
>>> Does anyone have any thoughts about this? Is there a way to get messages
>>> that are marked as "read" in nnimap to also get marked as "seen" on the
>>> IMAP server?
>>
>> Do you use nnimap as your primary select method? If so, manually edit
>> your GCC header
>>
>>   Gcc: nnimap+imap.server.tld:INBOX.archive.whatever
>>
>> so that it just reads
>>
>>   Gcc: INBOX.archive.whatever
>>
>> Does it work now?
>>
>> -David
>
> Aha! Well, I have three nnimap select methods: one as my primary, and
> two others which are secondary. When I refer to archives on the primary
> server in the way you describe above, they are now indeed getting marked
> as SEEN at the same time they're getting marked as "read".

It seems there is a bug in gnus-msg.el when the gcc is going to the
primary back end. Without changing the GCC manually, does the following
patch fix this for you?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gnus-msg-patch.diff --]
[-- Type: text/x-diff, Size: 487 bytes --]

diff --git a/lisp/gnus-msg.el b/lisp/gnus-msg.el
index 555b6a0..ce6fd3d 100644
--- a/lisp/gnus-msg.el
+++ b/lisp/gnus-msg.el
@@ -1624,6 +1624,8 @@ this is a reply."
 		  group (mm-encode-coding-string
 			 group
 			 (gnus-group-name-charset method group)))
+	    (when (gnus-native-method-p method)
+	      (setq group (gnus-group-short-name group)))
 	    (unless (gnus-check-server method)
 	      (error "Can't open server %s" (if (stringp method) method
 					      (car method))))

[-- Attachment #3: Type: text/plain, Size: 498 bytes --]


> However, what can I do about the secondaries? Anything?

Secondary or foreign servers should work if you give a fully qualified
group name; at least it works for me with a nnimap server in the
secondary select methods. Also, 'read' marks in Gnus should directly
correspond to the SEEN flags on the IMAP server. If you activate the
IMAP logging, you should see stuff like

595 UID STORE 2987 +FLAGS (\Seen)

when you mark message as read (and -FLAGS if you remove the marks on a
message)

-David

Re: Automatically marking Gcc's as SEEN in nnimap

[Lloyd Zusman]

David Engster <deng@randomsample.de> writes:

> Lloyd Zusman <ljz@asfast.com> writes:
>>
>> [ ... ]
>>
> It seems there is a bug in gnus-msg.el when the gcc is going to the
> primary back end. Without changing the GCC manually, does the following
> patch fix this for you?
>
> [ ... etc. ... ]

Yes, this does indeed work. Thank you!


>> However, what can I do about the secondaries? Anything?
>
> Secondary or foreign servers should work if you give a fully qualified
> group name; at least it works for me with a nnimap server in the
> secondary select methods. [ ... etc. ... ]

I'm sorry, but I misreported that Gcc's weren't being marked as SEEN for
the secondaries. They are indeed being marked correctly. The test that I
run on a "secondary" before writing that earlier message was actually
being incorrectly performed on a primary group.

So all is well now with the primaries and secondaries w/r/t Gcc.

There's another related issue, however.  When I do a
`gnus-group-catchup-current' on any nnimap-based group that is accessed
via my primary server, none of the un-SEEN articles get marked as SEEN.
This works fine for my nnimap-based groups accessed through one of my
secondary servers.

Any thoughts?

Thanks again for all your help.


-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: Automatically marking Gcc's as SEEN in nnimap

[David Engster]

Lloyd Zusman <ljz@asfast.com> writes:
> David Engster <deng@randomsample.de> writes:
>
>> Lloyd Zusman <ljz@asfast.com> writes:
>>>
>>> [ ... ]
>>>
>> It seems there is a bug in gnus-msg.el when the gcc is going to the
>> primary back end. Without changing the GCC manually, does the following
>> patch fix this for you?
>>
>> [ ... etc. ... ]
>
> Yes, this does indeed work. Thank you!

Thanks for testing. If no one objects, I'll commit this patch.

> There's another related issue, however.  When I do a
> `gnus-group-catchup-current' on any nnimap-based group that is accessed
> via my primary server, none of the un-SEEN articles get marked as SEEN.
> This works fine for my nnimap-based groups accessed through one of my
> secondary servers.
>
> Any thoughts?

Works for me. You can enable IMAP logging (by setting imap-log) and see
if catching up produces commands like

3365 UID STORE 5031 +FLAGS (\Seen)

If not, eval the following

(add-hook 'gnus-group-catchup-group-hook 
	  (lambda () (message "catch up: %s" gnus-newsgroup-name)))

and look at the group name in the message buffer after you have catched
up. For groups from the primary back end, you should just see the group
name without any prefix like "nnimap+SERVER:".

-David

Re: Automatically marking Gcc's as SEEN in nnimap

[Lloyd Zusman]

David Engster <deng@randomsample.de> writes:

> Lloyd Zusman <ljz@asfast.com> writes:
>>
>> [ ... ]
>>
> Thanks for testing. If no one objects, I'll commit this patch.

Well, I certainly don't object! :)

Thanks.


>> There's another related issue, however.  When I do a
>> `gnus-group-catchup-current' on any nnimap-based group that is accessed
>> via my primary server, none of the un-SEEN articles get marked as SEEN.
>> This works fine for my nnimap-based groups accessed through one of my
>> secondary servers.
>>
>> [ ... ]
>
> Works for me. You can enable IMAP logging (by setting imap-log) and see
> if catching up produces commands like
>
> 3365 UID STORE 5031 +FLAGS (\Seen)

Yes, I do see this in the *imap-log* buffer after issuing a
`gnus-group-catchup-current' command. Here are the entries that got
written to *imap-log* during the course of an invocation of that
command:

  1236 NOOP
  1236 OK NOOP completed.
  1237 UID STORE 3892 +FLAGS (\Seen)
  * 3296 FETCH (UID 3892 FLAGS (\Seen))
  1237 OK Store completed.

Furthermore, immediately after this command completes, the articles seem
to be marked as SEEN (or at least they are not flagged as unread/unseen
in the *Summary* buffer).  However, if I then run a
`gnus-group-get-new-news', the unread/unseen mark returns, and I can
only make it go away by actually reading the article.

This only occurs for my primary nnimap back end. For the secondaries,
the same sort of stuff gets written into *imap-log* (see below), but the
unread/unseen mark doesn't return after `gnus-group-get-new-news'.

Here's what got stored during `gnus-group-catchup-current' on a
secondary nnimap group:

  1780 NOOP
  1780 OK NOOP completed.
  1781 UID STORE 1874 +FLAGS (\Seen)
  * 37 FETCH (UID 1874 FLAGS (\Seen))
  1781 OK Store completed.

Could it be that some sort of local marks (local to gnus) don't get
properly changed from un-SEEN to SEEN when I do
`gnus-group-catchup-current' on a primary nnimap group, and could these
local marks somehow get re-propagated back to the IMAP server during
`gnus-group-get-new-news'?

That's just a shot in the dark, however.


> If not, eval the following
>
> (add-hook 'gnus-group-catchup-group-hook 
> 	  (lambda () (message "catch up: %s" gnus-newsgroup-name)))
>
> and look at the group name in the message buffer after you have catched
> up. For groups from the primary back end, you should just see the group
> name without any prefix like "nnimap+SERVER:".

I added this hook and re-ran `gnus-group-catchup-current', and I indeed
see the newsgroup name in the message buffer without any
`nnimap+SERVER:' prefix.

And I double-checked: this is definitely a group that is managed by my
nnimap primary back end.

For the secondary back end, this hook causes `nnimap+SERVER:groupname' to
be written in the message buffer.


-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: Automatically marking Gcc's as SEEN in nnimap

[David Engster]

Lloyd Zusman <ljz@asfast.com> writes:
>> Works for me. You can enable IMAP logging (by setting imap-log) and see
>> if catching up produces commands like
>>
>> 3365 UID STORE 5031 +FLAGS (\Seen)
>
> Yes, I do see this in the *imap-log* buffer after issuing a
> `gnus-group-catchup-current' command. Here are the entries that got
> written to *imap-log* during the course of an invocation of that
> command:
>
>   1236 NOOP
>   1236 OK NOOP completed.
>   1237 UID STORE 3892 +FLAGS (\Seen)
>   * 3296 FETCH (UID 3892 FLAGS (\Seen))
>   1237 OK Store completed.

Looks good.

> Furthermore, immediately after this command completes, the articles seem
> to be marked as SEEN (or at least they are not flagged as unread/unseen
> in the *Summary* buffer).  However, if I then run a
> `gnus-group-get-new-news', the unread/unseen mark returns, and I can
> only make it go away by actually reading the article.

I would guess this behavior comes from gaps in the article numbers. This
problem can't currently properly dealt with and also depends on the used
IMAP server. Does doing M-g on that group fix the number?

> Could it be that some sort of local marks (local to gnus) don't get
> properly changed from un-SEEN to SEEN when I do
> `gnus-group-catchup-current' on a primary nnimap group, and could these
> local marks somehow get re-propagated back to the IMAP server during
> `gnus-group-get-new-news'?

No, they don't get re-propagated to the IMAP server. Gnus simply doesn't
have all the necessary information for correctly calculating the number
of unread articles. When you do gnus-group-get-new-news, Gnus will try
to determine the number of unread articles purely based on the active
information (ie. lowest and highest article number in a group), and on
the group info, which contains the article numbers of already seen
articles. When you do M-g on a group, it will also re-scan the group on
the server, which can sometimes correct this problem.

> I added this hook and re-ran `gnus-group-catchup-current', and I indeed
> see the newsgroup name in the message buffer without any
> `nnimap+SERVER:' prefix.
>
> And I double-checked: this is definitely a group that is managed by my
> nnimap primary back end.
>
> For the secondary back end, this hook causes `nnimap+SERVER:groupname' to
> be written in the message buffer.

This is the correct behavior.

Regards,
David

Re: Automatically marking Gcc's as SEEN in nnimap

[Lloyd Zusman]

David Engster <deng@randomsample.de> writes:

> Lloyd Zusman <ljz@asfast.com> writes:
>>
>> [ ... ]
>>
>> Furthermore, immediately after this command completes, the articles seem
>> to be marked as SEEN (or at least they are not flagged as unread/unseen
>> in the *Summary* buffer).  However, if I then run a
>> `gnus-group-get-new-news', the unread/unseen mark returns, and I can
>> only make it go away by actually reading the article.
>
> I would guess this behavior comes from gaps in the article numbers. This
> problem can't currently properly dealt with and also depends on the used
> IMAP server. Does doing M-g on that group fix the number?

Yes, `M-g' fixes it.


>> [ ... ]
>
> No, they don't get re-propagated to the IMAP server. Gnus simply doesn't
> have all the necessary information for correctly calculating the number
> of unread articles. When you do gnus-group-get-new-news, Gnus will try
> to determine the number of unread articles purely based on the active
> information (ie. lowest and highest article number in a group), and on
> the group info, which contains the article numbers of already seen
> articles. When you do M-g on a group, it will also re-scan the group on
> the server, which can sometimes correct this problem.

OK. This explains and fixes the problem.

So do you see any disadvantage to my writing some function that runs
`gnus-group-get-new-news-this-group' (`M-g') on each of my groups, and
then rebinding this function to `g' (currently bound to
`gnus-group-get-new-news')?  It seems like such a function would be
more useful, especially for nnimap groups.

In any case, thanks for all your help!


-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.

Re: Automatically marking Gcc's as SEEN in nnimap

[David Engster]

Lloyd Zusman <ljz@asfast.com> writes:
> So do you see any disadvantage to my writing some function that runs
> `gnus-group-get-new-news-this-group' (`M-g') on each of my groups, and
> then rebinding this function to `g' (currently bound to
> `gnus-group-get-new-news')? 

No, it's just slower.

-David

Re: Automatically marking Gcc's as SEEN in nnimap

[Lloyd Zusman]

David Engster <deng@randomsample.de> writes:

> Lloyd Zusman <ljz@asfast.com> writes:
>> So do you see any disadvantage to my writing some function that runs
>> `gnus-group-get-new-news-this-group' (`M-g') on each of my groups, and
>> then rebinding this function to `g' (currently bound to
>> `gnus-group-get-new-news')? 
>
> No, it's just slower.

OK. Thanks.


-- 
 Lloyd Zusman
 ljz@asfast.com
 God bless you.