Deleted messages don't go away

Deleted messages don't go away

[David Abrahams]

I keep seeing some messages in my NNIMAP groups even long after I've
removed them using a different mail client.  Is there some way to get
Gnus to forget what it thinks it knows so the record can be accurate
again?

Thannks in advance,

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

Re: Deleted messages don't go away

[gdt]

David Abrahams <dave@boost-consulting.com> writes:

> I keep seeing some messages in my NNIMAP groups even long after I've
> removed them using a different mail client.  Is there some way to get
> Gnus to forget what it thinks it knows so the record can be accurate
> again?

I had this problem too, and turned off the agent.  Then gnus seems to
no longer remember the stale messages.

I suspect this is a cache-consistency issue arising from the
newsreader focus of Gnus, where articles don't (properly) go away.
With nnimap, articles deleted from the server should in my view be
deleted from the agent.


-- 
	Greg Troxel <gdt@work.lexort.com>

Re: Deleted messages don't go away

[Dave Goldberg]

I periodically run gnus-agent-expire to solve this.  I think it's only
an issue with messages that I delete or move by hand (though I haven't
looked very closely).  Those that are deleted by the expiry process
seem to be handled correctly.
-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: Deleted messages don't go away

[Reiner Steib]

On Tue, Mar 28 2006, Dave Goldberg wrote:

> I periodically run gnus-agent-expire to solve this.  I think it's only
> an issue with messages that I delete or move by hand (though I haven't
> looked very closely).  Those that are deleted by the expiry process
> seem to be handled correctly.

(I don't use the agent, so I might be completely wrong.)  Does setting 
`gnus-agent-synchronize-flags' to t help?

,----[ (info "(gnus)Agent Variables") ]
| `gnus-agent-synchronize-flags'
|      If `gnus-agent-synchronize-flags' is `nil', the Agent will never
|      automatically synchronize flags.  If it is `ask', which is the
|      default, the Agent will check if you made any changes and if so
|      ask if you wish to synchronize these when you re-connect.  If it
|      has any other value, all flags will be synchronized automatically.
`----

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Deleted messages don't go away

[Wolfram Fenske]

gdt@work.lexort.com writes:

> David Abrahams <dave@boost-consulting.com> writes:
>
>> I keep seeing some messages in my NNIMAP groups even long after I've
>> removed them using a different mail client.  Is there some way to get
>> Gnus to forget what it thinks it knows so the record can be accurate
>> again?
>
> I had this problem too, and turned off the agent.  Then gnus seems to
> no longer remember the stale messages.
>
> I suspect this is a cache-consistency issue arising from the
> newsreader focus of Gnus, where articles don't (properly) go away.

I agree that it's a cache issue.  A while back, when this problem got
really bad for me (hundreds of ghost emails), I deleted everything in
~/News/agent/nnimap and the dead articles were gone.  I think Dave
Goldberg's workaround might be better, though.

This problem might not arise if I used just one mail client on one
computer but that's not possible for me.  In fact, that's the whole
reason I'm using IMAP.  Ironic, isn't it?

> [...]


Regards
Wolfram

Re: Deleted messages don't go away

[Steven E. Harris]

Dave Goldberg <david.goldberg6@verizon.net> writes:

> I periodically run gnus-agent-expire to solve this.

I still don't understand why this function (or the related
gnus-agent-expire-group) needs to run manually. I've been doing it for
a few years now and have never been able to figure out why it's
necessary.

The problem appears for me when I've been reading mail at work all
week and not at all at home, then start reading mail at home over the
weekend. My mail folders viewed from home will be full of old messages
marked as expirable long ago. Despite having both the variables
gnus-agent-expire-days and nnmail-expiry-wait set to 4, I find old
messages piling up in my IMAP folders for several weeks, Gnus having
forgotten or ignored the expire (expirable?) mark.

-- 
Steven E. Harris

[nnimap] Deleted messages don't go away

[David Abrahams]

gdt@work.lexort.com writes:

> David Abrahams <dave@boost-consulting.com> writes:
>
>> I keep seeing some messages in my NNIMAP groups even long after I've
>> removed them using a different mail client.  Is there some way to get
>> Gnus to forget what it thinks it knows so the record can be accurate
>> again?
>
> I had this problem too, and turned off the agent.  Then gnus seems to
> no longer remember the stale messages.

Yeah, but once you turn it back on, they come back, right?  I can't
even get them to go away by removing the group from the agent.

> I suspect this is a cache-consistency issue arising from the
> newsreader focus of Gnus, where articles don't (properly) go away.
> With nnimap, articles deleted from the server should in my view be
> deleted from the agent.

Agreed.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

Re: [nnimap] Deleted messages don't go away

[gdt]

David Abrahams <dave@boost-consulting.com> writes:

> gdt@work.lexort.com writes:
>
>> David Abrahams <dave@boost-consulting.com> writes:
>>
>>> I keep seeing some messages in my NNIMAP groups even long after I've
>>> removed them using a different mail client.  Is there some way to get
>>> Gnus to forget what it thinks it knows so the record can be accurate
>>> again?
>>
>> I had this problem too, and turned off the agent.  Then gnus seems to
>> no longer remember the stale messages.
>
> Yeah, but once you turn it back on, they come back, right?  I can't
> even get them to go away by removing the group from the agent.

I didn't try this, but I'd expect them to come back.  Since the agent
seems broken with nnimap (if messages are deleted by other imap
clients), I just disabled the agent for the entire nnimap server.

It would be nice to have the agent work, and be able to use cached
copies of messages and at mail offline.  It seems that some other uses
of the agent want articles only in the agent (but not on the server)
to appear.  So perhaps this should be a per-group or per-server
property, defaulting to "sync with server" for nnimap.

-- 
	Greg Troxel <gdt@work.lexort.com>

Re: [nnimap] Deleted messages don't go away

[David Abrahams]

gdt@work.lexort.com writes:

> David Abrahams <dave@boost-consulting.com> writes:
>
>> gdt@work.lexort.com writes:
>>
>>> David Abrahams <dave@boost-consulting.com> writes:
>>>
>>>> I keep seeing some messages in my NNIMAP groups even long after I've
>>>> removed them using a different mail client.  Is there some way to get
>>>> Gnus to forget what it thinks it knows so the record can be accurate
>>>> again?
>>>
>>> I had this problem too, and turned off the agent.  Then gnus seems to
>>> no longer remember the stale messages.
>>
>> Yeah, but once you turn it back on, they come back, right?  I can't
>> even get them to go away by removing the group from the agent.
>
> I didn't try this, but I'd expect them to come back.  Since the agent
> seems broken with nnimap (if messages are deleted by other imap
> clients), I just disabled the agent for the entire nnimap server.

But of course I want offline access to my IMAP messages.  And (of
course) my IMAP server is slow so having a local cache is valuable.

> It would be nice to have the agent work, and be able to use cached
> copies of messages and at mail offline.  It seems that some other uses
> of the agent want articles only in the agent (but not on the server)
> to appear.  So perhaps this should be a per-group or per-server
> property, defaulting to "sync with server" for nnimap.

That would sure be nice.  I wish I could do it myself, but I've given
up trying to modify the Gnus system (or even really understand it --
I'm sometimes amazed I can use it at all, but it's still the most
productive client I can find).

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

Re: Deleted messages don't go away

[Dave Goldberg]

Well, I'm pretty sure the default is to expire a group upon exit.  I
find I don't like that at all.  So I run them manually because I
removed them from any hooks on which I found them.  In general, I want
to choose when to clean house.  Using total expire with a couple days
wait time and running expiry only periodically gives me the and
convenience I want.  The main thing with gnus-agent-expire is that it
cleans out the overview files in the agent directory which is key to
getting rid of deleted and moved messages.

-- 
Dave Goldberg
david.goldberg6@verizon.net

moved messages don't go away neither (was: Deleted messages don't go away)

[Uwe Brauer]

>>>>> "Dave" == Dave Goldberg <david.goldberg6@verizon.net> writes:

   Dave> I periodically run gnus-agent-expire to solve this.  I think it's only
   Dave> an issue with messages that I delete or move by hand (though I haven't
   Dave> looked very closely).  Those that are deleted by the expiry process
   Dave> seem to be handled correctly.

If in the same setting, nnimap and agent I move messages from imap
folder 1 to imap folder 2, with gnus-summary-move-article, they remain
in folder 1. Even gnus-agent-expire does not help.

Can somebody please confirm that.

Thanks

Uwe Brauer

Re: moved messages don't go away neither

[Simon Josefsson]

Uwe Brauer <oub@mat.ucm.es> writes:

>>>>>> "Dave" == Dave Goldberg <david.goldberg6@verizon.net> writes:
>
>    Dave> I periodically run gnus-agent-expire to solve this.  I think it's only
>    Dave> an issue with messages that I delete or move by hand (though I haven't
>    Dave> looked very closely).  Those that are deleted by the expiry process
>    Dave> seem to be handled correctly.
>
> If in the same setting, nnimap and agent I move messages from imap
> folder 1 to imap folder 2, with gnus-summary-move-article, they remain
> in folder 1. Even gnus-agent-expire does not help.
>
> Can somebody please confirm that.

I've experienced the same thing several times, but I cannot reproduce
it.  Perhaps it happens if you C-k the summary buffer, rather than
exiting it with 'q'?

When it happens, I usually remove the entire agent directory for that
particular nnimap group.  E.g. rm -rf ~/News/agent/nnimap/foo/bar.  It
is safe to do so (although, of course, you have then lost the local
cache of that group).

Re: moved messages don't go away neither

[Uwe Brauer]

>>>>> "Simon" == Simon Josefsson <jas@extundo.com> writes:

   Simon> I've experienced the same thing several times, but I cannot
   Simon> reproduce it.  Perhaps it happens if you C-k the summary
   Simon> buffer, rather than exiting it with 'q'?

No I always leave with `q.'

   Simon> When it happens, I usually remove the entire agent directory
   Simon> for that particular nnimap group.  E.g. rm -rf
   Simon> ~/News/agent/nnimap/foo/bar.  It is safe to do so (although,
   Simon> of course, you have then lost the local cache of that
   Simon> group).

Right, sometimes I do the same or similar, I only wanted to know, if
gnus itself can solve that problem.

Re: moved messages don't go away neither

[Dave Goldberg]

I used to see that.  A couple months ago someone provided a patch that
has been incorporated in No Gnus that appears to have solved it for me.
-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: moved messages don't go away neither

[Wolfram Fenske]

Dave Goldberg <david.goldberg6@verizon.net> writes:

> I used to see that.  A couple months ago someone provided a patch
> that has been incorporated in No Gnus that appears to have solved it
> for me.

That was me ;-) and I used to have the problem of "ghost mail," too.
But when I saw this thread I checked again, and it seems I don't have
it anymore.  Thinking about it, it makes a little sense (but not a
whole lot): without the patch, when nnimap moved an email, it created
a copy in the destination folder (like it was supposed to), but didn't
switch back to the source folder before setting the `Deleted' flag,
which had the effect that the mail in the source folder never got
expired.  My guess is that this threw off the agent; at least it's
easy to imagine how this would cause cache inconsistencies of some
sort.

Anyway, it seems I don't have the problem anymore and I'd suggest to
try and see if my patch [1] keeps it from re-occurring.  Any
non-existent mail you have now, you have to get rid off by hand,
though.


Regards
Wolfram

Footnotes: 
[1]  <http://thread.gmane.org/gmane.emacs.gnus.general/61669/focus=61669>

-- 
A: Yes.
>Q: Are you sure?
>>A: Because it reverses the logical flow of conversation.
>>>Q: Why is top posting frowned upon?

Re: moved messages don't go away neither

[Uwe Brauer]

>>>>> "Wolfram" == Wolfram Fenske
>>>>> <Wolfram.Fenske@Student.Uni-Magdeburg.DE> writes: 

   Wolfram> Dave Goldberg <david.goldberg6@verizon.net> writes:
   >> I used to see that.  A couple months ago someone provided a patch
   >> that has been incorporated in No Gnus that appears to have solved it
   >> for me.


   Wolfram> Anyway, it seems I don't have the problem anymore and I'd
   Wolfram> suggest to try and see if my patch [1] keeps it from
   Wolfram> re-occurring.  Any non-existent mail you have now, you
   Wolfram> have to get rid off by hand, though.

Thanks, but it does not work for me, when moving an article, the
articles gets copied to the other folder and *marked* for deletion,
however when exiting the group and reentering it is still there
(contrary to deleting it my hand).

I ran gnus-agent-regenerate and gnus-agent-expire but without
success. Odd.

Uwe Brauer 

Re: moved messages don't go away neither

[Wolfram Fenske]

Uwe Brauer <oub@mat.ucm.es> writes:

>>>>>> "Wolfram" == Wolfram Fenske
> [...]
>
>    Wolfram> Anyway, it seems I don't have the problem anymore and I'd
>    Wolfram> suggest to try and see if my patch [1] keeps it from
>    Wolfram> re-occurring.  Any non-existent mail you have now, you
>    Wolfram> have to get rid off by hand, though.
>
> Thanks, but it does not work for me, when moving an article, the
> articles gets copied to the other folder and *marked* for deletion,
> however when exiting the group and reentering it is still there
> (contrary to deleting it my hand).

Did you recompile "nnimap.el" or move the old "nnimap.elc" out of the
way?  IIRC, Emacs tries to use the compiled version of a file, even if
it's older.

Also, how exactly are you're articles still there?  I ask, because
there are two problems:

  1. Articles that appears in an nnimap group but aren't really there.
  When you try to read such an article, Gnus beeps at you and tells
  you the article is no longer present (something like that).  IIRC,
  you can't delete (B DEL) it, either.  This is what I referred to as
  "ghost mail" in my previous post, and this is what I believe the
  original poster was talking about.  I may be wrong, though.

  2. Moving articles in the nnimap backend didn't work because the
  article in the source folder was not expired.  But in contrast to
  problem no. 1, the article in the source folder could still be read.
  My patch should have fixed this bug.

The reason I brought up the second issue is that I suspect that
somehow the broken move function confused the agent and thus caused
the "ghost mail" problem as well.  Dave says he doesn't have this
problem anymore since he's been using the patch, and neither do I.

> I ran gnus-agent-regenerate and gnus-agent-expire but without
> success. Odd.

Can't really help you with that.  I deleted everything in
~/News/agent/nnimap/ but previous posters have suggested other
methods which sounded less brutal.


Greetings
Wolfram

-- 
A: Yes.
>Q: Are you sure?
>>A: Because it reverses the logical flow of conversation.
>>>Q: Why is top posting frowned upon?

Re: moved messages don't go away neither

[Uwe Brauer]

>>>>> "Wolfram" == Wolfram Fenske
>>>>> <Wolfram.Fenske@Student.Uni-Magdeburg.DE> writes:


   Wolfram> Uwe Brauer <oub@mat.ucm.es> writes:

   Wolfram> Did you recompile "nnimap.el" or move the old "nnimap.elc"
   Wolfram> out of the way?  IIRC, Emacs tries to use the compiled
   Wolfram> version of a file, even if it's older.

Of course  I did.

   Wolfram> Also, how exactly are you're articles still there?  I ask, because
   Wolfram> there are two problems:

   Wolfram>   1. Articles that appears in an nnimap group but aren't
   Wolfram>   really there. When you try to read such an article, Gnus
   Wolfram>   beeps at you and tells you the article is no longer
   Wolfram>   present (something like that).  IIRC, you can't delete
   Wolfram>   (B DEL) it, either.  This is what I referred to as
   Wolfram>   "ghost mail" in my previous post, and this is what I
   Wolfram>   believe the original poster was talking about.  I may be
   Wolfram>   wrong, though.
No this is not my case. I can read the article.

   Wolfram>   2. Moving articles in the nnimap backend didn't work
   Wolfram>   because the article in the source folder was not
   Wolfram>   expired.  But in contrast to problem no. 1, the article
   Wolfram>   in the source folder could still be read. My patch
   Wolfram>   should have fixed this bug.

But the article has been copied? If so then it is my case. I use the
gnus-summary-move-article function, say in a imap folder INBOX, then
the articles *gets* copied to the new imap folder, say IMPORTANT, and
the article is then *marked* as deleted. However when I reenter the
group INBOX, the article is still there.

Uwe 

Re: moved messages don't go away neither

[Wolfram Fenske]

Uwe Brauer <oub@mat.ucm.es> writes:

> [...]
>
>    Wolfram> Did you recompile "nnimap.el" or move the old "nnimap.elc"
>    Wolfram> out of the way?  IIRC, Emacs tries to use the compiled
>    Wolfram> version of a file, even if it's older.
>
> Of course  I did.

Sorry, I didn't mean to offend you.

>    Wolfram> Also, how exactly are you're articles still there?  I
>    Wolfram> ask, because there are two problems:
>
>    [...]
>
>    Wolfram>   2. Moving articles in the nnimap backend didn't work
>    Wolfram>   because the article in the source folder was not
>    Wolfram>   expired.  But in contrast to problem no. 1, the article
>    Wolfram>   in the source folder could still be read. My patch
>    Wolfram>   should have fixed this bug.
>
> But the article has been copied? If so then it is my case.

Yes, that's the one I meant.

> I use the gnus-summary-move-article function, say in a imap folder
> INBOX, then the articles *gets* copied to the new imap folder, say
> IMPORTANT, and the article is then *marked* as deleted. However when
> I reenter the group INBOX, the article is still there.

Weird.  This should be working now.  It is for me and at least one
more person [1] (not much, I know ;-)).  I just checked again,
explicitly saying "M-x gnus-summary-move-article" -- everything OK.  I
have no clue why it doesn't work in your Gnus :-/


Regards
Wolfram

Footnotes: 
[1]  <http://article.gmane.org/gmane.emacs.gnus.general/62047>

-- 
A: Yes.
>Q: Are you sure?
>>A: Because it reverses the logical flow of conversation.
>>>Q: Why is top posting frowned upon?

Re: moved messages don't go away neither

[Uwe Brauer]

>>>>> "Wolfram" == Wolfram Fenske
>>>>> <Wolfram.Fenske@Student.Uni-Magdeburg.DE> writes: 

   >> 
   >> Of course  I did.

   Wolfram> Sorry, I didn't mean to offend you.

Sorry from my side, didn't mean that neither, you provide the code :)
and I complain :(
   >> 
   >> But the article has been copied? If so then it is my case.

   Wolfram> Yes, that's the one I meant.

   Wolfram> Weird.  This should be working now.  It is for me and at
   Wolfram> least one more person [1] (not much, I know ;-)).  I just
   Wolfram> checked again, explicitly saying "M-x
   Wolfram> gnus-summary-move-article" -- everything OK.  I have no
   Wolfram> clue why it doesn't work in your Gnus :-/

I am on no gnus 0.3, another possibility would be some strange
variable setting but I looked for it, I don't find anything strange.

Thanks again

Uwe 

Ps I like you signature.

Re: moved messages don't go away neither

[Uwe Brauer]

>>>>> "Wolfram" == Wolfram Fenske
>>>>> <Wolfram.Fenske@Student.Uni-Magdeburg.DE> writes: 

   >> 
   >> [...]
   >> 
   Wolfram> 2. Moving articles in the nnimap backend didn't work
   Wolfram> because the article in the source folder was not
   Wolfram> expired.  But in contrast to problem no. 1, the article
   Wolfram> in the source folder could still be read. My patch
   Wolfram> should have fixed this bug.
   >> 
   >> But the article has been copied? If so then it is my case.

   Wolfram> Yes, that's the one I meant.

Hello 

I have to modify my statement:

    -  I restarted my xemacs and the articles which I have moved,
       after having applied your patch, seems now to be ghost
       articles, in the sense of 1. Of your last mail. 

    -  I then run (gnus-agent-expire) and they indeed have
       disappeared.

So it seems it was not enough to restart gnus, I had to restart
xemacs.

Thanks again

Uwe 
       
Ps so I   have doubled the number of success reports.

Re: moved messages don't go away neither

[Wolfram Fenske]

Uwe Brauer <oub@mat.ucm.es> writes:

> [...] 
>
> I have to modify my statement:
>
>     -  I restarted my xemacs and the articles which I have moved,
>        after having applied your patch, seems now to be ghost
>        articles, in the sense of 1. Of your last mail. 
>
>     -  I then run (gnus-agent-expire) and they indeed have
>        disappeared.
>
> [...]
>        
> Ps so I   have doubled the number of success reports.

I gather everything's working now?  Glad to hear it :-)


Greetings
Wolfram

-- 
A: Yes.
>Q: Are you sure?
>>A: Because it reverses the logical flow of conversation.
>>>Q: Why is top posting frowned upon?