Re: Faster nnimap: nnimap-retrieve-groups-asynchronous

[Simon Josefsson <jas@extundo.com>]

Bjørn Mork <bmork@dod.no> writes:

> Simon Josefsson <jas@extundo.com> writes:
>
>> I tried to speed up new mail checking (`g') on IMAP groups.  Startup
>> uses old technique, but subsequent `g' can be faster.  The speedup
>> depends on how many groups usually receives new mail each time you
>> press `g' (the p below).  It might depend on the server as well.  For
>> me, it is faster against Cyrus IMAPD -- results from other servers
>> would be interesting.
>
> This speeded up mail checking noticably on all my imap servers, but
> unfortunately it's causing a problem with one of them. I could of
> course turn it off, but then I lose the performance gain... So I'm
> hoping the problem can be fixed.
>
> I currently use three imap servers:
> a) IMAP4 server (InterMail vM.4.01.03.23 201-229-121-123-20010418)
> b) Courier-IMAP ready. Copyright 1998-2001 Double Precision, Inc.
> c) Courier-IMAP ready. Copyright 1998-2002 Double Precision, Inc.

I suspect the same problem as Ilpo reported, also Courier IMAPD from
what I recall.  I haven't debug that yet though.

> The problem is that Gnus doesn't detect new mail split into a folder
> on b) when pressing 'g'. Doing a 'M-g' on the folder will however show
> the new mail. The funny thing is that everything works fine on server
> a) and c). So it looks like a combination of a Courier server and
> client side splitting is causing the problem, unless there is a
> specific problem with the Courier version running on b).

Can you find out the server versions used on server a and b?

Ilpo was not using client side splitting from what I recall.

> 95 EXAMINE "INBOX.Trash"
> * FLAGS (\Answered \Flagged \Deleted \Seen \Recent)
> * OK [PERMANENTFLAGS ()] No permanent flags permitted
> * 0 EXISTS
> * 0 RECENT
> * OK [UIDVALIDITY 979914089] Ok
> 95 OK [READ-ONLY] Ok
> [..]
> 128 STATUS "INBOX.Trash" (uidnext)
> * STATUS "INBOX.Trash" (UIDNEXT 1)
> 128 OK STATUS Completed.
> [..]
> 170 UID FETCH 370 BODY.PEEK[HEADER]
> * 131 FETCH (UID 370 BODY[HEADER] {1628}
> [snip mail]
> 170 OK FETCH completed.
> 171 UID COPY 370 "INBOX.Trash"
> 171 OK COPY completed.
> 172 UID STORE 370 +FLAGS (\Seen \Deleted)
> * 131 FETCH (FLAGS (\Seen \Deleted))
> 172 OK STORE completed.
> 173 EXPUNGE
> * 131 EXPUNGE
> * 130 EXISTS
> * 0 RECENT
> 173 OK EXPUNGE completed
> 174 CLOSE
> 174 OK mailbox closed.
> 175 STATUS "INBOX.Trash" (uidnext)
> * STATUS "INBOX.Trash" (UIDNEXT 1)
> 175 OK STATUS Completed.
>
> Hmm, definitely looks like a server problem when I'm looking at it
> now. Shouldn't the UIDNEXT value change between command 128 and 175? 

Hm.  At first glance, it looks like it, but it could be valid
behaviour if UIDVALIDITY changed. Could you try running with this
patch?  Does the UIDVALIDITY change between 128- and 175-like commands
then?  Nnimap is buggy, it should look at UIDVALIDITY as well here.
I'll fix it.

--- nnimap.el.~6.36.~	Sat May 25 16:16:03 2002
+++ nnimap.el	Thu Jun  6 15:05:15 2002
@@ -975,7 +975,8 @@
 			     'asyncgroups
 			   'slowgroups)
 			 (list group (imap-mailbox-status-asynch
-				      group 'uidnext nnimap-server-buffer))))
+				      group '(uidnext uidvalidity)
+				      nnimap-server-buffer))))
 	  (dolist (asyncgroup asyncgroups)
 	    (let ((group (nth 0 asyncgroup))
 		  (tag   (nth 1 asyncgroup))
@@ -984,7 +985,8 @@
 		(if (nnimap-string-lessp-numerical
 		     (car (gnus-gethash
 			   (concat server group) nnimap-mailbox-info))
-		     (imap-mailbox-get 'uidnext group nnimap-server-buffer))
+		     (imap-mailbox-get '(uidnext uidvalidity)
+				       group nnimap-server-buffer))
 		    (push (list group) slowgroups)
 		  (insert (cdr (gnus-gethash (concat server group)
 					     nnimap-mailbox-info))))))))


> Is this a known problem with the Courier server?

No, but there are other known problems with it.

The workaround is to disable nnimap-retrieve-groups-asynchronous for
the server.