Messages still being marked as old in nnimap

Messages still being marked as old in nnimap

[John Sullivan]

I've been having this problem for years in gnus, and was hoping that the
new nnimap code would fix it (before that I hoped switching to dovecot
from courier would fix it) -- but it just happened again today.

My description of it is totally lame, because I can't figure out how to
reproduce it or even vaguely under what conditions it happens. But every
now and then, a bunch of unread messages get marked O, effectively
vanishing from my sight. This is very bad. 

Any ideas why this could happen or what I might do to get more useful
testing information? It happens so infrequently, and sometimes I don't
notice that it happened until a bit after...

I have a gut feeling that it is related to the fact that I move a lot of
messages out of my inbox, sometimes to a mailbox on a different server.
Maybe that confuses things. Or that it might be related to fetching new
mail to the local server (with offlineimap) while the gnus folder is
open.

Whatever it is, it's making me crazy.

-- 
John Sullivan
Emacs Planner Maintainer
http://wjsullivan.net/PlannerMode.html

Re: Messages still being marked as old in nnimap

[Dan Christensen]

John Sullivan <john@wjsullivan.net> writes:

> every now and then, a bunch of unread messages get marked O,
> effectively vanishing from my sight. This is very bad.

I'm not sure if it's the same problem, but I frequently lose all of my
expirable marks in nnimap groups.  Those are usually the only marks
there are, so it could be related to what you see.  But I think unread
articles stay unread for me.

I also can't reliably reproduce it, but it happens every day or two.
I also use offlineimap, to synchronize dovecot servers.

This happened a lot to me when the new nnimap was very new.  Then it
seemed to stop.  Now it's happening quite frequently again.

Dan

Re: Messages still being marked as old in nnimap

[John Sullivan]

Dan Christensen <jdc@uwo.ca> writes:

> John Sullivan <john@wjsullivan.net> writes:
>
>> every now and then, a bunch of unread messages get marked O,
>> effectively vanishing from my sight. This is very bad.
>
> I'm not sure if it's the same problem, but I frequently lose all of my
> expirable marks in nnimap groups.  Those are usually the only marks
> there are, so it could be related to what you see.  But I think unread
> articles stay unread for me.
>

What do you mean by "expirable marks"? Maybe I've noticed the same
thing -- articles that I'd expect to be marked with E, because they are
in a group where all read articles are supposed to be expired, are
marked with O instead. (While some others in the same group are still
marked with E.)

-- 
John Sullivan
Emacs Planner Maintainer
http://wjsullivan.net/PlannerMode.html

Re: Messages still being marked as old in nnimap

[Dan Christensen]

John Sullivan <john@wjsullivan.net> writes:

> Dan Christensen <jdc@uwo.ca> writes:
>
>> John Sullivan <john@wjsullivan.net> writes:
>>
>>> every now and then, a bunch of unread messages get marked O,
>>> effectively vanishing from my sight. This is very bad.
>>
>> I'm not sure if it's the same problem, but I frequently lose all of my
>> expirable marks in nnimap groups.  Those are usually the only marks
>> there are, so it could be related to what you see.  But I think unread
>> articles stay unread for me.
>
> What do you mean by "expirable marks"? Maybe I've noticed the same
> thing -- articles that I'd expect to be marked with E, because they are
> in a group where all read articles are supposed to be expired, are
> marked with O instead. (While some others in the same group are still
> marked with E.)

That's what I mean: articles marked with "E".  I often lose them all.
And I forgot to mention in my reply above that this doesn't only happen
for nnimap, it also happens for nnfolder.

Dan

Re: Messages still being marked as old in nnimap

[John Sullivan]

Dan Christensen <jdc@uwo.ca> writes:

> John Sullivan <john@wjsullivan.net> writes:
>
>> Dan Christensen <jdc@uwo.ca> writes:
>>
>>> John Sullivan <john@wjsullivan.net> writes:
>>>
>>>> every now and then, a bunch of unread messages get marked O,
>>>> effectively vanishing from my sight. This is very bad.
>>>
>>> I'm not sure if it's the same problem, but I frequently lose all of my
>>> expirable marks in nnimap groups.  Those are usually the only marks
>>> there are, so it could be related to what you see.  But I think unread
>>> articles stay unread for me.
>>
>> What do you mean by "expirable marks"? Maybe I've noticed the same
>> thing -- articles that I'd expect to be marked with E, because they are
>> in a group where all read articles are supposed to be expired, are
>> marked with O instead. (While some others in the same group are still
>> marked with E.)
>
> That's what I mean: articles marked with "E".  I often lose them all.
> And I forgot to mention in my reply above that this doesn't only happen
> for nnimap, it also happens for nnfolder.

Okay, then I can verify that this happens to me too in nnimap. I'm not
sure if they are related bugs, but as either or both of them getting
fixed would make me happier, I'm willing to help investigate either one.

-- 
John Sullivan
Emacs Planner Maintainer
http://wjsullivan.net/PlannerMode.html

Re: Messages still being marked as old in nnimap

[Lars Magne Ingebrigtsen]

John Sullivan <john@wjsullivan.net> writes:

> Okay, then I can verify that this happens to me too in nnimap. I'm not
> sure if they are related bugs, but as either or both of them getting
> fixed would make me happier, I'm willing to help investigate either one.

If you could find out what triggers this bug, then that would be very
helpful. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Messages still being marked as old in nnimap

[John Sullivan]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> John Sullivan <john@wjsullivan.net> writes:
>
>> Okay, then I can verify that this happens to me too in nnimap. I'm not
>> sure if they are related bugs, but as either or both of them getting
>> fixed would make me happier, I'm willing to help investigate either one.
>
> If you could find out what triggers this bug, then that would be very
> helpful. 

Sigh, it just happened again, but in a new way. This time, it lost a
bunch of R marks, so that a few hundred messages appear as unread that
were read.

I have an imap log of it, but sadly I have no idea where the important
part starts and stops, since I can't figure out anything I did
differently than usual. I'm going to send it to you off-list, because
it's semi-private and I don't want to have to scrub it at the moment..

-- 
John Sullivan
Emacs Planner Maintainer
http://wjsullivan.net/PlannerMode.html

Re: Messages still being marked as old in nnimap

[John Sullivan]

Dan Christensen <jdc@uwo.ca> writes:

> John Sullivan <john@wjsullivan.net> writes:
>
>> Dan Christensen <jdc@uwo.ca> writes:
>>
>>> John Sullivan <john@wjsullivan.net> writes:
>>>
>>>> every now and then, a bunch of unread messages get marked O,
>>>> effectively vanishing from my sight. This is very bad.
>>>
>>> I'm not sure if it's the same problem, but I frequently lose all of my
>>> expirable marks in nnimap groups.  Those are usually the only marks
>>> there are, so it could be related to what you see.  But I think unread
>>> articles stay unread for me.
>>
>> What do you mean by "expirable marks"? Maybe I've noticed the same
>> thing -- articles that I'd expect to be marked with E, because they are
>> in a group where all read articles are supposed to be expired, are
>> marked with O instead. (While some others in the same group are still
>> marked with E.)
>
> That's what I mean: articles marked with "E".  I often lose them all.
> And I forgot to mention in my reply above that this doesn't only happen
> for nnimap, it also happens for nnfolder.

You're not by chance also an offlineimap user, are you? 

It occurs to me that it's possible this isn't a problem in gnus, but a
problem in offlineimap. I doubt that because I don't think I've noticed
these issues in other mail clients talking to the same servers, but it's
also still a possibility, since I do use those other clients much less
frequently than gnus. I also wouldn't think that offlineimap would have
anything to do with the expirable marks, but I also don't know much.

(I use offlineimap to sync between a local dovecot and a remote dovecot
and courier.)

-- 
John Sullivan
Emacs Planner Maintainer
http://wjsullivan.net/PlannerMode.html

Re: Messages still being marked as old in nnimap

[James Cloos]

>>>>> "JS" == John Sullivan <john@wjsullivan.net> writes:

JS> Sigh, it just happened again, but in a new way. This time, it lost a
JS> bunch of R marks, so that a few hundred messages appear as unread that
JS> were read.

Do you use the cache?  Does you imap server support qresync?

I do; mine doesn't; and nnimap groups with cached articles are flakey.

But only recently.

The problems might be related.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: Messages still being marked as old in nnimap

[Dan Christensen]

John Sullivan <john@wjsullivan.net> writes:

>> That's what I mean: articles marked with "E".  I often lose them all.
>> And I forgot to mention in my reply above that this doesn't only happen
>> for nnimap, it also happens for nnfolder.
>
> You're not by chance also an offlineimap user, are you? 

Yes, I am.  But I keep logs of what offlineimap does, and have seen the
problem even though offlineimap didn't claim to be making changes.
Still, there is a possibility that it is offlineimap's fault.

Note that by default offlineimap doesn't synchronize any of the custom
flags that Gnus uses.  I use a slightly modified version of offlineimap
to handle this.

> (I use offlineimap to sync between a local dovecot and a remote dovecot
> and courier.)

I have dovecot at both ends.

Dan

Re: Messages still being marked as old in nnimap

[John Sullivan]

James Cloos <cloos@jhcloos.com> writes:

>>>>>> "JS" == John Sullivan <john@wjsullivan.net> writes:
>
> JS> Sigh, it just happened again, but in a new way. This time, it lost a
> JS> bunch of R marks, so that a few hundred messages appear as unread that
> JS> were read.
>
> Do you use the cache?  Does you imap server support qresync?
>

I don't use the cache. I believe my server supports qresync; I'm using
dovecot and I see 17:22:29 2 ENABLE QRESYNC in the log.

-- 
John Sullivan
Emacs Planner Maintainer
http://wjsullivan.net/PlannerMode.html