4266 is the 666 of the IMAP world (retrieval issues and odd behaviours)

4266 is the 666 of the IMAP world (retrieval issues and odd behaviours)

[Matt Ford]

Hi,

Returned to Gnus after hearing that all IMAP code is spiffy fast
again...which it is :-) But I do have some issues...

I've checked out the latest (as of a couple of days ago) git development
branch and configured Gnus to run against Gmail and my works Exchange
2010 server.

Gmail is great! :-D But I don't have so much mail in there.

Exchange not so :`(

Without the Gnus agent on when fetching my largish Inbox on Exchange I
have found that I can happily enter the Inbox file from the group buffer
if I request 4265 articles (C-u SPACE 4265 or gnus-group-read-group).

If I request 4266 articles then Gnus reports "No unread news" and moves
the cursor to the bottom of the buffer.

In both cases the total number of available articles is reported
correctly.

I can get the data though if I jump into the buffer and then run "/ o"
or gnus-summary-insert-old-articles.  Interestingly this function
returns not the number of messages but what looks like the max article
UID in the prompt (bug or feature?).

Switching on debugging (via nnimap-record-commands t) shows a
spectacularly big fetch line for gnus-group-read-group when asking for
everything.  I'm wondering if the request is too large based on this
rather flimsy evidence...to say the UIDs are not contiguous is an
understatement.  How does this code work?  Are requests split into
sensible chunks?

Interestingly the gnus-summary-insert-old-articles command doesn't log
any activity in *imap log* so that's probably a bug in itself but also
hints something is going on quite different.

I replicated this on two different machines...

If you grow your mailbox slowly and use Gnus agent then you don't hit
any issues as it kicks in when you request everything...

Anyone seen anything similar or have thoughts how I might get around
this?

Phew!

-- 
Matt

Re: 4266 is the 666 of the IMAP world (retrieval issues and odd behaviours)

[Steinar Bang]

When you initially connected to the exchange server, did you `M-g' on
all of the nnimap groups?  It's probably not necessary when you start
initial subscription (my subscriptions were old when I transitioned to
the new nnimap backend) but it is something to try.

AFAIK `M-g' does a fully sync of what's available in that particular
nnimap group, and put that information in the .newsrc.eld, so that it
can do quick QRESYNC requests later.

Re: 4266 is the 666 of the IMAP world (retrieval issues and odd behaviours)

[Matt Ford]

Hi,

Yep, I did.  The unread count information where used is correct.  It's
the actual fetching that's the problem.
-- 
Matt

Re: 4266 is the 666 of the IMAP world (retrieval issues and odd behaviours)

[Matt Ford]

To be clear(er). The issue can be created by:

1. Disabling Gnus agent for the Exchange 2010 nnimap backend.
2. Clearing down all Gnus agent files `rm -r ~/News/agent/nnimap/XXX'
3. Attempting to download all mail via `C-u SPC`

What happens: fails when number of messages is over a certain size.

To work-around:

1. Clear down all Gnus agent files `rm -r ~/News/agent/nnimap/XXX`
2. Ensure the Gnus agent is enabled.
3. Download mail under the fail size limit via `C-u 4265 SPC'
4. Fetch remaining articles with `/ o'

What happens: avoids failing fetch and builds agent indexes so that
subsequent complete fetches don't query the server.

-- 
Matt

Re: 4266 is the 666 of the IMAP world (retrieval issues and odd behaviours)

[Lars Magne Ingebrigtsen]

Matt Ford <matt@dancingfrog.co.uk> writes:

> To be clear(er). The issue can be created by:
>
> 1. Disabling Gnus agent for the Exchange 2010 nnimap backend.
> 2. Clearing down all Gnus agent files `rm -r ~/News/agent/nnimap/XXX'
> 3. Attempting to download all mail via `C-u SPC`
>
> What happens: fails when number of messages is over a certain size.

That's interesting.  What's in the " *nnimap ... server...*" (note
leading space) buffer when this happens?  And also set
`nnimap-record-commands' and see what's in the "*imap log*" buffer.

I suspect that there may be some kind of timing or time-out issue.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: 4266 is the 666 of the IMAP world (retrieval issues and odd behaviours)

[Matt Ford]

Hi,

To re-create:

1. Delete the gnus agent cache (rm -rf ~/New/agent/nnimap/Folder)
2. Turn off gnus agent for nnimap on Exchnage
3. Try to fetch all messages

Fails, instantly when the message count is equal to or above 4266 (this
number)

To work-around:

1. Enable Gnus agent
2. Fetch only 4265 messages
3. Add the remaining messages via `/ o'

With the agent cache populated there is no issue.

Anyone any thoughts on why this might be happening?

-- 
Matt