Bug in nnimap?

Bug in nnimap?

[Sven Bretfeld]

Hi to all

I'm using Gnus as an imap client since two years now. What annoys me
ever since, is that Gnus keeps record of old mail headers of long
deleted emails (not expired, deleted!).

When entering a Group with C-u, Gnus asks if I want to see the several
thousands of messages in that group. In reality there are only about one
or two hundred mails within that imap folder. But Gnus actually displays the
headers of every email that had ever lived in that group -- no matter if
it's long deleted or shifted to another imap-folder. The Summary buffer
displays the headers with the "O" mark, turning to "G" when one selects
the message.

This is annoying, especially if you have to worry about space
limitations on the server. Gnus is not capable to tell how many mails
actually exist in the respective folder.

Is that a bug in nnimap? Is it a feature? Can somebody tell me how to
prevent Gnus from keeping track of old headers or how to refresh its
memory? 

Can anybody even reproduce that behavior?

Thanks

Sven

Re: Bug in nnimap?

[Frank Schmitt]

"Sven Bretfeld" <sven.bretfeld@gmx.ch> writes:

> Hi to all
>
> I'm using Gnus as an imap client since two years now. What annoys me
> ever since, is that Gnus keeps record of old mail headers of long
> deleted emails (not expired, deleted!).
>
> When entering a Group with C-u, Gnus asks if I want to see the several
> thousands of messages in that group. In reality there are only about one
> or two hundred mails within that imap folder. But Gnus actually displays the
> headers of every email that had ever lived in that group -- no matter if
> it's long deleted or shifted to another imap-folder. The Summary buffer
> displays the headers with the "O" mark, turning to "G" when one selects
> the message.
>
> This is annoying, especially if you have to worry about space
> limitations on the server. Gnus is not capable to tell how many mails
> actually exist in the respective folder.
>
> Is that a bug in nnimap? Is it a feature? Can somebody tell me how to
> prevent Gnus from keeping track of old headers or how to refresh its
> memory? 
>
> Can anybody even reproduce that behavior?

I regularly watch #gnus on freenode and this comes up very
frequently. The solution is to turn off Gnus agent.

-- 
Have you ever considered how much text can fit in eighty columns?  Given that a
signature typically contains up to four lines of text, this space allows you to
attach a tremendous amount of valuable information to your messages.  Seize the
opportunity and don't waste your signature on bullshit that nobody cares about.

Re: Bug in nnimap?

[Sven Bretfeld]

Hi Frank

Frank Schmitt <ich@frank-schmitt.net> writes:

> "Sven Bretfeld" <sven.bretfeld@gmx.ch> writes:
>
>> Hi to all
>>
>> I'm using Gnus as an imap client since two years now. What annoys me
>> ever since, is that Gnus keeps record of old mail headers of long
>> deleted emails (not expired, deleted!).

...

> I regularly watch #gnus on freenode and this comes up very
> frequently. The solution is to turn off Gnus agent.

Yes, that works. Perfect. Thank you. 

Sven

Re: Bug in nnimap?

[Tibor Simko]

On Wed, 01 Sep 2010, Frank Schmitt wrote:
> The solution is to turn off Gnus agent.

BTW, due to similar issues, I have been running with agent off for IMAP
groups since ages.  But incidentally I tried to re-test the agent
behaviour for IMAP groups afresh a few month ago, and it seems to have
worked much better than it used to N years ago.

So, is anyone using agent with IMAP groups regularly these days?  Or has
everyone switched to offlineimap or the like?

Best regards
-- 
Tibor Simko

Re: Bug in nnimap?

[Frank Schmitt]

Tibor Simko <tibor.simko@cern.ch> writes:

> On Wed, 01 Sep 2010, Frank Schmitt wrote:
>> The solution is to turn off Gnus agent.
>
> BTW, due to similar issues, I have been running with agent off for IMAP
> groups since ages.  But incidentally I tried to re-test the agent
> behaviour for IMAP groups afresh a few month ago, and it seems to have
> worked much better than it used to N years ago.
>
> So, is anyone using agent with IMAP groups regularly these days?  Or has
> everyone switched to offlineimap or the like?

Like I said before: I'd strongly vote for disabling the agent for nnimap
by default. The whole point of IMAP is that several clients access the
same mails on the central server but the agent just isn't up to this in
my experience.

-- 
Have you ever considered how much text can fit in eighty columns?  Given that a
signature typically contains up to four lines of text, this space allows you to
attach a tremendous amount of valuable information to your messages.  Seize the
opportunity and don't waste your signature on bullshit that nobody cares about.

Re: Bug in nnimap?

[Lars Magne Ingebrigtsen]

Frank Schmitt <ich@frank-schmitt.net> writes:

> Like I said before: I'd strongly vote for disabling the agent for nnimap
> by default.

I've now done this.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Bug in nnimap?

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> Frank Schmitt <ich@frank-schmitt.net> writes:
>> Like I said before: I'd strongly vote for disabling the agent for nnimap
>> by default.

> I've now done this.

Wasn't it before?  Ie. if you created a new nnimap server, wasn't it not
initially not agentized?  Seemed to be the last time I created one.

FWIW I've used nnimap with agent since the agent's inception, and apart
from the odd "wrong unread article number" sometime in the 2001-2004
time frame, I haven't had many issues with it.

I've been running CVS Gnus since 1997 or thereabouts.

Re: Bug in nnimap?

[Lars Magne Ingebrigtsen]

Steinar Bang <sb@dod.no> writes:

> Wasn't it before?  Ie. if you created a new nnimap server, wasn't it not
> initially not agentized?  Seemed to be the last time I created one.

There was a variable saying that nnimap was agentized by default, so...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Bug in nnimap?

[Tibor Simko]

On Fri, 03 Sep 2010, Steinar Bang wrote:
> FWIW I've used nnimap with agent since the agent's inception, and
> apart from the odd "wrong unread article number" sometime in the
> 2001-2004 time frame, I haven't had many issues with it.

This time frame correlates well with the N years period I mentioned.  
I had some troubles with nnimap agent in around 2002, then I mostly
stopped using agent for nnimap, until revival in 2010 due to some needs.
It seems to work better now.

Best regards
-- 
Tibor Simko

Re: Bug in nnimap?

[Sven Bretfeld]

Tibor Simko <tibor.simko@cern.ch> writes:

> On Wed, 01 Sep 2010, Frank Schmitt wrote:
>> The solution is to turn off Gnus agent.
>
> BTW, due to similar issues, I have been running with agent off for IMAP
> groups since ages.  But incidentally I tried to re-test the agent
> behaviour for IMAP groups afresh a few month ago, and it seems to have
> worked much better than it used to N years ago.
>
> So, is anyone using agent with IMAP groups regularly these days?  Or has
> everyone switched to offlineimap or the like?

To be true, I have not really understood what Gnus Agent exactly does.
So I don't really know what your question is aiming at.

I just used the Agent without knowing, because it's turned on by
default. To store and access my emails offline, I expire them from the
server after two weeks with a kind of "spool folder" on the imap server
as the expiry-target. fetchmail clears this folder via POP and stores
the mails on my disk where I access them via nnml. In this way, I have
the messages available from every computer for a while, and after two
weeks I have them saved for eternity on a local machine without running
out of server space.

Sven

Re: Bug in nnimap?

[Steinar Bang]

>>>>> "Sven Bretfeld" <sven.bretfeld@gmx.ch>:

> To be true, I have not really understood what Gnus Agent exactly does.
> So I don't really know what your question is aiming at.

The agent caches the NOV information of the nnimap folders, as well as
articles.  How it caches the articles depends on the setup.  For me, it
caches articles upon reading them.

The cached information is stored under
  ̃/News/agent/backend/server/
Eg. 
  ̃/News/agent/nnimap/myimapserver/

I won't go in detail on the file structure under there.  Take a look, or
read the manual.

Re: Bug in nnimap?

[Steinar Bang]

>>>>> Tibor Simko <tibor.simko@cern.ch>:

> So, is anyone using agent with IMAP groups regularly these days? 

Yes.  Have been, since the agent was introduced, whenever that was.

> Or has everyone switched to offlineimap or the like?

Why would they, or I, want to?

Re: Bug in nnimap?

[Frank Schmitt]

Steinar Bang <sb@dod.no> writes:

>>>>>> Tibor Simko <tibor.simko@cern.ch>:
>
>> So, is anyone using agent with IMAP groups regularly these days? 
>
> Yes.  Have been, since the agent was introduced, whenever that was.

And you access your mail from more than one Gnus instance, right?

-- 
Have you ever considered how much text can fit in eighty columns?  Given that a
signature typically contains up to four lines of text, this space allows you to
attach a tremendous amount of valuable information to your messages.  Seize the
opportunity and don't waste your signature on bullshit that nobody cares about.

Re: Bug in nnimap?

[Steinar Bang]

>>>>> Frank Schmitt <ich@frank-schmitt.net>:

> Steinar Bang <sb@dod.no> writes:
>>>>>>> Tibor Simko <tibor.simko@cern.ch>:

>>> So, is anyone using agent with IMAP groups regularly these days? 

>> Yes.  Have been, since the agent was introduced, whenever that was.

> And you access your mail from more than one Gnus instance, right?

Yep.

Re: Bug in nnimap?

[Tibor Simko]

On Fri, 03 Sep 2010, Steinar Bang wrote:
>> Or has everyone switched to offlineimap or the like?
>
> Why would they, or I, want to?

For example, to be able to copy/move messages between IMAP folders while
being unplugged.  Useful for occasional offline triage and other such
housekeeping.

(BTW, Wanderlust offline IMAP mode supports this natively; wouldn't it
be cool if Gnus nnimap agent unplugged supported that natively too.)

Best regards
-- 
Tibor Simko

Re: Bug in nnimap?

[Steinar Bang]

>>>>> Tibor Simko <tibor.simko@cern.ch>:

> For example, to be able to copy/move messages between IMAP folders
> while being unplugged.  Useful for occasional offline triage and other
> such housekeeping.

Ah, yes.  I've wanted that for, like, ages.

> (BTW, Wanderlust offline IMAP mode supports this natively; wouldn't it
> be cool if Gnus nnimap agent unplugged supported that natively too.)

Indeed!  AOL!

Re: Bug in nnimap?

[Steinar Bang]

>>>>> Frank Schmitt <ich@frank-schmitt.net>:

> I regularly watch #gnus on freenode and this comes up very
> frequently. The solution is to turn off Gnus agent.

You don't have to do it permanently.  Sometimes it's enough to just
toggle off agent and revisit the group, and then toggle the agent on
again. 

Sometimes you need to toggle off agent, trash the .agentview and
.overview files for the folder, toggle the agent back on, and revisit.

Re: Bug in nnimap?

[Lars Magne Ingebrigtsen]

Steinar Bang <sb@dod.no> writes:

> Sometimes you need to toggle off agent, trash the .agentview and
> .overview files for the folder, toggle the agent back on, and revisit.

That doesn't sound very user-friendly.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Bug in nnimap?

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> Steinar Bang <sb@dod.no> writes:
>> Sometimes you need to toggle off agent, trash the .agentview and
>> .overview files for the folder, toggle the agent back on, and revisit.

> That doesn't sound very user-friendly.  :-)

Like I said: I haven't had to do it for 5-6 years.

Re: Bug in nnimap?

[Gijs Hillenius]

On  1 Sep 2010, Sven Bretfeld wrote:


[...]

>
> When entering a Group with C-u, Gnus asks if I want to see the several
> thousands of messages in that group. In reality there are only about one
> or two hundred mails within that imap folder. But Gnus actually displays the
> headers of every email that had ever lived in that group -- no matter if
> it's long deleted or shifted to another imap-folder. The Summary buffer
> displays the headers with the "O" mark, turning to "G" when one selects
> the message.

[...]


> Can anybody even reproduce that behavior?

Here the message count gets out of sync sometimes, too. But it seems
different from what you describe. Gnus will ask "how many of 39584
messages ..." or something, when there is actually just 30 ...

What I do, when it starts to bug me: 

for instance in the nnmimap INBOX: I select all messages, and move them
to the nnimimap INBOX (so the very same box they're in). And then q that
buffer, and M-g the INBOX. When I enter the INBOX again, it is fine.

Re: Bug in nnimap?

[Sven Bretfeld]

Gijs Hillenius <gijs@hillenius.net> writes:

> On  1 Sep 2010, Sven Bretfeld wrote:
>
>
> [...]
>
>>
>> When entering a Group with C-u, Gnus asks if I want to see the several
>> thousands of messages in that group. In reality there are only about one
>> or two hundred mails within that imap folder. But Gnus actually displays the
>> headers of every email that had ever lived in that group -- no matter if
>> it's long deleted or shifted to another imap-folder. The Summary buffer
>> displays the headers with the "O" mark, turning to "G" when one selects
>> the message.
>
> [...]
>
>
>> Can anybody even reproduce that behavior?
>
> Here the message count gets out of sync sometimes, too. But it seems
> different from what you describe. Gnus will ask "how many of 39584
> messages ..." or something, when there is actually just 30 ...

That's exactly what I mean. Turning off Gnus Agent, as Frank suggests,
also works.

Greetings,

Sven