Is *anyone* else loosing expiry marks in nnimap buffers?

Is *anyone* else loosing expiry marks in nnimap buffers?

[Wes Hardaker]

It seems that 'every so often' I loose my expiry marks in my nnimap
groups.  I don't know why this would happen and I can't seem to
reproduce it easily either.

I expire by hand in about half my groups, and they're auto-expired
using the demon:

  (defun gnus-demon-get-new-news ()
    (save-excursion
      (gnus-group-get-new-news)
      (gnus-group-list-groups 5)))

  (defun gnus-demon-expire-all-groups ()
    (save-excursion
      (gnus-group-get-new-news)      ; find new stuff first so it keeps % marks
      (gnus-group-list-groups)
      (gnus-group-expire-all-groups)
      (gnus-group-get-new-news)
      (gnus-group-list-groups)
      ))

  (gnus-demon-add-handler 'gnus-demon-expire-all-groups 120 120)

But occasionally, I'll re-enter a group to see messages marked with
'O' that I'm *positive* I expired at some point.

Am I the only one seeing this?  sigh...
-- 
"In the bathtub of history the truth is harder to hold than the soap,
 and much more difficult to find."  -- Terry Pratchett

Re: Is *anyone* else loosing expiry marks in nnimap buffers?

[Niklas Morberg]

Wes Hardaker <wes@hardakers.net> writes:

> But occasionally, I'll re-enter a group to see messages marked with
> 'O' that I'm *positive* I expired at some point.
>
> Am I the only one seeing this?  sigh...

Are you using the agent? I had this problem when I was
using the agent, which made me run away screaming. I
haven't used the agent since and haven't seen the problem
either. But mailreading is oh so slow...

See e.g. the threads:

<URL: http://article.gmane.org/gmane.emacs.gnus.general/51217>
<URL: http://article.gmane.org/gmane.emacs.gnus.general/46635>

Searching for "lost marks" in the gmane.emacs.gnus.general
group on gmane.org seems to reveal that several people have
had problems with this or something similar. No apparent
solutions as far as I can tell.

Niklas

Re: Is *anyone* else loosing expiry marks in nnimap buffers?

[David S Goldberg]

>>>>> On Tue, 01 Jul 2003 10:14:12 -0700, Wes Hardaker
>>>>> <wes@hardakers.net> said:

> But occasionally, I'll re-enter a group to see messages marked with
> 'O' that I'm *positive* I expired at some point.

But are the messages actually there on the server?  I've found two
problems that might be related to what you're seeing.  The easiest to
deal with is when the agent overview gets out of sync and reports the
existence of messages that are gone.  I still see this happen on
occasion but gnus-agent-regenerate resolves it easily.  The more
difficult problem is when messages expired from the server don't get
expired from the agent.  This might have been fixed, though.  About a
month or two ago I got fed up, deleted my entire agent hierarchy and
started afresh and I haven't had that problem since.

-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: Is *anyone* else loosing expiry marks in nnimap buffers?

[Wes Hardaker]

>>>>> On Wed, 02 Jul 2003 11:18:47 -0400, David S Goldberg <david.goldberg6@verizon.net> said:

[and in answer to the other message: yes I'm using the agent and I was
pretty sure it was the culprit as well].

David> But are the messages actually there on the server?  I've found two
David> problems that might be related to what you're seeing.  The easiest to
David> deal with is when the agent overview gets out of sync and reports the
David> existence of messages that are gone.

I suspect that actually might be the problem.  I'll try regenerating
the .overview file next time.  I actually thought this problem had
been fixed as well (in fact, a long time ago I reported that the
.overview files got to be huge because duplicate information was being
entered into them, but that may have been a separate bug because that
was fixed by Lars)

David> The more difficult problem is when messages expired from the
David> server don't get expired from the agent.  This might have been
David> fixed, though.  About a month or two ago I got fed up, deleted
David> my entire agent hierarchy and started afresh and I haven't had
David> that problem since.

Thats the thing.  I thought it was supposed to have been fixed too.

-- 
"In the bathtub of history the truth is harder to hold than the soap,
 and much more difficult to find."  -- Terry Pratchett

Re: Is *anyone* else loosing expiry marks in nnimap buffers?

[Nevin Kapur]

Niklas Morberg <niklas.morberg@axis.com> writes:

> Wes Hardaker <wes@hardakers.net> writes:
>
>> But occasionally, I'll re-enter a group to see messages marked with
>> 'O' that I'm *positive* I expired at some point.
>>
> Are you using the agent? I had this problem when I was
> using the agent, which made me run away screaming. 

Let me add my voice to this thread: I too see articles in my agentized
nnimap groups that don't exist in the group.  Running
gnus-agent-regenerate-group doesn't help at all.

-Nevin

Re: Is *anyone* else loosing expiry marks in nnimap buffers?

[David S Goldberg]

>>>>> On Wed, 02 Jul 2003 12:54:24 -0700, Wes Hardaker <wes@hardakers.net> said:

>>>>> On Wed, 02 Jul 2003 11:18:47 -0400, David S Goldberg
>>>>> <david.goldberg6@verizon.net> said:

David> But are the messages actually there on the server?  I've found two
David> problems that might be related to what you're seeing.  The easiest to
David> deal with is when the agent overview gets out of sync and reports the
David> existence of messages that are gone.

> I suspect that actually might be the problem.  I'll try regenerating
> the .overview file next time.  I actually thought this problem had
> been fixed as well (in fact, a long time ago I reported that the
> .overview files got to be huge because duplicate information was being
> entered into them, but that may have been a separate bug because that
> was fixed by Lars)

I still get warnings during expiry that my overview either has
information about a non-existing message or is missing info about an
existing one and the recommendation to run gnus-agent-regenerate.  So
the bug may not be fixed, but there's code added to detect it.

David> The more difficult problem is when messages expired from the
David> server don't get expired from the agent.  This might have been
David> fixed, though.  About a month or two ago I got fed up, deleted
David> my entire agent hierarchy and started afresh and I haven't had
David> that problem since.

> Thats the thing.  I thought it was supposed to have been fixed too.

I think this may be fixed since I've been free of it for so long (used
to happen pretty regularly).  However, the fix may only be good for
dealing with new messages.  If you have messages in your agent that
were downloaded before the fix was committed they may not be dealt
with.  I've had that issue with several bugs I reported in the agent.
I went nuts trying to figure out why the fix worked for Kevin but not
for me.  As soon as I would delete my agent directory and let the
agent rebuild it, the positive effects of the fix would be realized.

-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: Is *anyone* else loosing expiry marks in nnimap buffers?

[Niklas Morberg]

David S Goldberg <david.goldberg6@verizon.net> writes:

> I still get warnings during expiry that my overview either has
> information about a non-existing message or is missing info about an
> existing one and the recommendation to run gnus-agent-regenerate.  So
> the bug may not be fixed, but there's code added to detect it.

Where is this overview (or .overview that Wes mentioned)?
I'm willing to give the agent another go, starting fresh,
but I need to know what to delete first. I've removed the
News/agent dir.

Should I get rid of News/overview as well? The index of the
gnus info has no mentioning of overview files.

Niklas

Re: Is *anyone* else loosing expiry marks in nnimap buffers?

[David S Goldberg]

>>>>> On Thu, 03 Jul 2003 09:32:05 +0200, Niklas Morberg
>>>>> <niklas.morberg@axis.com> said:

> Where is this overview (or .overview that Wes mentioned)?
> I'm willing to give the agent another go, starting fresh,
> but I need to know what to delete first. I've removed the
> News/agent dir.

That should be all you need.  Under News/agent you'll get a directory
for each method (nnimap, nntp) you have agentized and under each of
those a directory for each server in the method followed by
directories for each group.  In each of those will be a .overview
file.

> Should I get rid of News/overview as well? The index of the
> gnus info has no mentioning of overview files.

I think News/overview is a remnant from an earlier version of the
agent, or maybe it was even nnimap specific.  I recall deleting mine
ages ago and it hasn't come back.

-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: Is *anyone* else loosing expiry marks in nnimap buffers?

[Wes Hardaker]

>>>>> On Wed, 02 Jul 2003 11:18:47 -0400, David S Goldberg <david.goldberg6@verizon.net> said:

David> I still see this happen on occasion but gnus-agent-regenerate
David> resolves it easily.

Well, I found a group with the problem and regenerating within emacs
definitely didn't help.  I even answered the mini-buffer question
(which I now forget.  Too early in the morning) with all 3 possible
values and they still showed the buffer.  Interestingly enough,
answering whatever the equivalent to 'all' to it ended up marking them
as unread as well (though they still didn't exist).

Removing the .overview file, however, did fix the problem.
-- 
"In the bathtub of history the truth is harder to hold than the soap,
 and much more difficult to find."  -- Terry Pratchett

Re: Is *anyone* else loosing expiry marks in nnimap buffers?

[Wes Hardaker]

>>>>> On Thu, 03 Jul 2003 09:32:05 +0200, Niklas Morberg <niklas.morberg@axis.com> said:

Niklas> Where is this overview (or .overview that Wes mentioned)?
Niklas> I'm willing to give the agent another go, starting fresh,
Niklas> but I need to know what to delete first. I've removed the
Niklas> News/agent dir.

There is 1 per group in ~/News/agent/nnimap/SERVER/GROUP/.overview

Niklas> Should I get rid of News/overview as well?

shouldn't matter.

Niklas> The index of the gnus info has no mentioning of overview
Niklas> files.

That's because its a behind-the-scenes thing and you shouldn't have to
worry about it.

-- 
"In the bathtub of history the truth is harder to hold than the soap,
 and much more difficult to find."  -- Terry Pratchett

Re: Is *anyone* else loosing expiry marks in nnimap buffers?

[Simon Josefsson]

Niklas Morberg <niklas.morberg@axis.com> writes:

> Should I get rid of News/overview as well? The index of the
> gnus info has no mentioning of overview files.

It is only used by nnimap, when nnimap-nov-is-evil is non-nil, which I
guess it will be if you have disabled the agent.  If you try the
agent, better make sure the nnimap overview cache is disabled, to
minimize problem areas.

(defvoo nnimap-nov-is-evil gnus-agent
  ...