Re: [PATCH] nnimap performance improvement for large or old groups

[Simon Josefsson <jas@extundo.com>]

Daniel Pittman <daniel@rimspace.net> writes:

> One of the nagging problems with using nnimap as my primary mail store
> is that it takes absolutely forever to enter my INBOX, as well as
> causing XEmacs to allocate around 150MB of memory.
>
> This is because I have been using the same INBOX for a long while now,
> and the 'read' info range starts with '(1 . 695705)'
>
> The code in `nnimap-request-update-info-internal' called
> `gnus-uncompress-range' on this, resulting in a list containing around
> seven million numbers -- an awful lot of memory, and time spent working
> through it.
>
> To address this I rewrote the code in that routine to work with
> compressed ranges, rather than uncompressed, which gives me a huge
> performance improvement (almost instant entry, vs ten to fifteen
> seconds) and reduces the memory use significantly.
>
>
> I don't think this is too performance-inefficient to include as is, and
> it doesn't seem to adversely effect entry into small groups or anything
> like that.

Your patch seem correct.  Installed, thanks!

It should probably be tested for a while before being included in
v5-10 though, the logic seem somewhat complicated and while I
convinced myself that it does the same thing, I might be mistaken.

> 	Daniel
>
> ? build.info
> ? build.sh
> ? nnimap-performance.patch
> ? contrib/auto-autoloads.el
> ? contrib/semantic.cache
> Index: lisp/nnimap.el
> ===================================================================
> RCS file: /usr/local/cvsroot/gnus/lisp/nnimap.el,v
> retrieving revision 7.30
> diff -u -u -r7.30 nnimap.el
> --- lisp/nnimap.el	21 Feb 2006 07:14:23 -0000	7.30
> +++ lisp/nnimap.el	3 Mar 2006 01:37:36 -0000
> @@ -1183,18 +1183,12 @@
>  	  (let (seen unseen)
>  	    ;; read info could contain articles marked unread by other
>  	    ;; imap clients!  we correct this
> -	    (setq seen (gnus-uncompress-range (gnus-info-read info))
> -		  unseen (imap-search "UNSEEN UNDELETED")
> -		  seen (gnus-set-difference seen unseen)
> -		  ;; seen might lack articles marked as read by other
> -		  ;; imap clients! we correct this
> -		  seen (append seen (imap-search "SEEN"))
> -		  ;; remove dupes
> -		  seen (sort seen '<)
> -		  seen (gnus-compress-sequence seen t)
> -		  ;; we can't return '(1) since this isn't a "list of ranges",
> -		  ;; and we can't return '((1)) since g-list-of-unread-articles
> -		  ;; is buggy so we return '((1 . 1)).
> +	    (setq unseen (gnus-compress-sequence 
> +			  (imap-search "UNSEEN UNDELETED"))
> +		  seen (gnus-range-difference (gnus-info-read info) unseen)
> +		  seen (gnus-range-add seen 
> +				       (gnus-compress-sequence 
> +					(imap-search "SEEN")))
>  		  seen (if (and (integerp (car seen))
>  				(null (cdr seen)))
>  			   (list (cons (car seen) (car seen)))