PATCH to nnfolder-retrieve-headers to speed up a request for all articles

PATCH to nnfolder-retrieve-headers to speed up a request for all articles

[Jason Merrill]

[-- Attachment #1: Type: text/plain, Size: 908 bytes --]

I haven't actually been using nnfolder for months now (I've switched to
nnimap/agent), but when I was it annoyed me that if I asked gnus for all
the articles in a folder, gnus would search through the file for each
article number between the lowest and highest active articles.  Most of
these had been deleted, but gnus would search through the whole file for
each of them, which for a large folder became incredibly slow.

This patch improves things so that if we fail to find one article, and see
that our pointer into the folder is between articles with lower and higher
numbers than the one we are looking for, we also skip any other requested
articles in that range.  This dramatically sped up requests for all
articles in a folder.

2001-11-04  Jason Merrill  <jason@redhat.com>

	* nnfolder.el (nnfolder-retrieve-headers): Avoid searching the entire
	file for each of a sequence of missing articles.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 3326 bytes --]

*** nnfolder.el.~1~	Fri Jul 19 13:07:23 2002
--- nnfolder.el	Sun May 19 04:18:41 2002
*************** the group.  Then the marks file will be 
*** 158,164 ****
    (save-excursion
      (set-buffer nntp-server-buffer)
      (erase-buffer)
!     (let (article start stop)
        (nnfolder-possibly-change-group group server)
        (when nnfolder-current-buffer
  	(set-buffer nnfolder-current-buffer)
--- 158,164 ----
    (save-excursion
      (set-buffer nntp-server-buffer)
      (erase-buffer)
!     (let (article start stop num)
        (nnfolder-possibly-change-group group server)
        (when nnfolder-current-buffer
  	(set-buffer nnfolder-current-buffer)
*************** the group.  Then the marks file will be 
*** 173,188 ****
  			    (nnfolder-existing-articles)))
  	    (while (setq article (pop articles))
  	      (set-buffer nnfolder-current-buffer)
! 	      (when (nnfolder-goto-article article)
! 		(setq start (point))
! 		(setq stop (if (search-forward "\n\n" nil t)
! 			       (1- (point))
! 			     (point-max)))
! 		(set-buffer nntp-server-buffer)
! 		(insert (format "221 %d Article retrieved.\n" article))
! 		(insert-buffer-substring nnfolder-current-buffer start stop)
! 		(goto-char (point-max))
! 		(insert ".\n")))
  	    (set-buffer nntp-server-buffer)
  	    (nnheader-fold-continuation-lines)
  	    'headers))))))
--- 173,225 ----
  			    (nnfolder-existing-articles)))
  	    (while (setq article (pop articles))
  	      (set-buffer nnfolder-current-buffer)
! 	      (cond ((nnfolder-goto-article article)
! 		     (setq start (point))
! 		     (setq stop (if (search-forward "\n\n" nil t)
! 				    (1- (point))
! 				  (point-max)))
! 		     (set-buffer nntp-server-buffer)
! 		     (insert (format "221 %d Article retrieved.\n" article))
! 		     (insert-buffer-substring nnfolder-current-buffer
! 					      start stop)
! 		     (goto-char (point-max))
! 		     (insert ".\n"))
! 
! 		    ;; If we couldn't find this article, skip over ranges
! 		    ;; of missing articles so we don't search the whole file
! 		    ;; for each of them.
! 		    ((numberp article)
! 		     (setq start (point))
! 		     (and
! 		      ;; Check that we are either at BOF or after an
! 		      ;; article with a lower number.  We do this so we
! 		      ;; won't be confused by out-of-order article numbers,
! 		      ;; as caused by active file bogosity.
! 		      (cond
! 		       ((bobp))
! 		       ((search-backward (concat "\n" nnfolder-article-marker)
! 					 nil t)
! 			(goto-char (match-end 0))
! 			(setq num (string-to-int
! 				   (buffer-substring
! 				    (point) (progn (end-of-line) (point)))))
! 			(goto-char start)
! 			(< num article)))
! 		      ;; Check that we are before an article with a
! 		      ;; higher number.
! 		      (search-forward (concat "\n" nnfolder-article-marker)
! 				      nil t)
! 		      (progn
! 			(setq num (string-to-int
! 				   (buffer-substring
! 				    (point) (progn (end-of-line) (point)))))
! 			(> num article))
! 		      ;; Discard any article numbers before the one we're
! 		      ;; now looking at.
! 		      (while (and articles
! 				  (< (car articles) num))
! 			(setq articles (cdr articles))))
! 		     (goto-char start))))
  	    (set-buffer nntp-server-buffer)
  	    (nnheader-fold-continuation-lines)
  	    'headers))))))

Re: PATCH to nnfolder-retrieve-headers to speed up a request for all articles

[Kai Großjohann]

Committed.
kai
-- 
A large number of young women don't trust men with beards.  (BFBS Radio)