From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/45734 Path: main.gmane.org!not-for-mail From: Jason Merrill Newsgroups: gmane.emacs.gnus.general Subject: PATCH to nnfolder-retrieve-headers to speed up a request for all articles Date: Fri, 19 Jul 2002 13:58:59 +0100 Sender: owner-ding@hpc.uh.edu Message-ID: NNTP-Posting-Host: localhost.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: main.gmane.org 1027083803 364 127.0.0.1 (19 Jul 2002 13:03:23 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Fri, 19 Jul 2002 13:03:23 +0000 (UTC) Cc: Jason Merrill Return-path: Original-Received: from malifon.math.uh.edu ([129.7.128.13]) by main.gmane.org with esmtp (Exim 3.33 #1 (Debian)) id 17VXPl-00005j-00 for ; Fri, 19 Jul 2002 15:03:21 +0200 Original-Received: from sina.hpc.uh.edu ([129.7.128.10] ident=lists) by malifon.math.uh.edu with esmtp (Exim 3.20 #1) id 17VXO9-0003zI-00; Fri, 19 Jul 2002 08:01:41 -0500 Original-Received: by sina.hpc.uh.edu (TLB v0.09a (1.20 tibbs 1996/10/09 22:03:07)); Fri, 19 Jul 2002 08:02:05 -0500 (CDT) Original-Received: from sclp3.sclp.com (qmailr@sclp3.sclp.com [209.196.61.66]) by sina.hpc.uh.edu (8.9.3/8.9.3) with SMTP id IAA13429 for ; Fri, 19 Jul 2002 08:01:53 -0500 (CDT) Original-Received: (qmail 26574 invoked by alias); 19 Jul 2002 13:01:19 -0000 Original-Received: (qmail 26569 invoked from network); 19 Jul 2002 13:01:19 -0000 Original-Received: from dell-paw-3.cambridge.redhat.com (HELO executor.cambridge.redhat.com) (195.224.55.237) by gnus.org with SMTP; 19 Jul 2002 13:01:19 -0000 Original-Received: from prospero.cambridge.redhat.com (dell-paw-2.cambridge.redhat.com [195.224.55.226]) by executor.cambridge.redhat.com (Postfix) with ESMTP id 19504ABB13; Fri, 19 Jul 2002 14:00:50 +0100 (BST) Original-Received: by prospero.cambridge.redhat.com (Postfix, from userid 4046) id EEC9FF83C1; Fri, 19 Jul 2002 13:58:59 +0100 (BST) Original-To: ding@gnus.org Original-Lines: 110 User-Agent: Gnus/5.090007 (Oort Gnus v0.07) Emacs/21.1 (i686-pc-linux-gnu) Precedence: list X-Majordomo: 1.94.jlt7 Xref: main.gmane.org gmane.emacs.gnus.general:45734 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:45734 --=-=-= I haven't actually been using nnfolder for months now (I've switched to nnimap/agent), but when I was it annoyed me that if I asked gnus for all the articles in a folder, gnus would search through the file for each article number between the lowest and highest active articles. Most of these had been deleted, but gnus would search through the whole file for each of them, which for a large folder became incredibly slow. This patch improves things so that if we fail to find one article, and see that our pointer into the folder is between articles with lower and higher numbers than the one we are looking for, we also skip any other requested articles in that range. This dramatically sped up requests for all articles in a folder. 2001-11-04 Jason Merrill * nnfolder.el (nnfolder-retrieve-headers): Avoid searching the entire file for each of a sequence of missing articles. --=-=-= Content-Type: text/x-patch Content-Disposition: inline *** nnfolder.el.~1~ Fri Jul 19 13:07:23 2002 --- nnfolder.el Sun May 19 04:18:41 2002 *************** the group. Then the marks file will be *** 158,164 **** (save-excursion (set-buffer nntp-server-buffer) (erase-buffer) ! (let (article start stop) (nnfolder-possibly-change-group group server) (when nnfolder-current-buffer (set-buffer nnfolder-current-buffer) --- 158,164 ---- (save-excursion (set-buffer nntp-server-buffer) (erase-buffer) ! (let (article start stop num) (nnfolder-possibly-change-group group server) (when nnfolder-current-buffer (set-buffer nnfolder-current-buffer) *************** the group. Then the marks file will be *** 173,188 **** (nnfolder-existing-articles))) (while (setq article (pop articles)) (set-buffer nnfolder-current-buffer) ! (when (nnfolder-goto-article article) ! (setq start (point)) ! (setq stop (if (search-forward "\n\n" nil t) ! (1- (point)) ! (point-max))) ! (set-buffer nntp-server-buffer) ! (insert (format "221 %d Article retrieved.\n" article)) ! (insert-buffer-substring nnfolder-current-buffer start stop) ! (goto-char (point-max)) ! (insert ".\n"))) (set-buffer nntp-server-buffer) (nnheader-fold-continuation-lines) 'headers)))))) --- 173,225 ---- (nnfolder-existing-articles))) (while (setq article (pop articles)) (set-buffer nnfolder-current-buffer) ! (cond ((nnfolder-goto-article article) ! (setq start (point)) ! (setq stop (if (search-forward "\n\n" nil t) ! (1- (point)) ! (point-max))) ! (set-buffer nntp-server-buffer) ! (insert (format "221 %d Article retrieved.\n" article)) ! (insert-buffer-substring nnfolder-current-buffer ! start stop) ! (goto-char (point-max)) ! (insert ".\n")) ! ! ;; If we couldn't find this article, skip over ranges ! ;; of missing articles so we don't search the whole file ! ;; for each of them. ! ((numberp article) ! (setq start (point)) ! (and ! ;; Check that we are either at BOF or after an ! ;; article with a lower number. We do this so we ! ;; won't be confused by out-of-order article numbers, ! ;; as caused by active file bogosity. ! (cond ! ((bobp)) ! ((search-backward (concat "\n" nnfolder-article-marker) ! nil t) ! (goto-char (match-end 0)) ! (setq num (string-to-int ! (buffer-substring ! (point) (progn (end-of-line) (point))))) ! (goto-char start) ! (< num article))) ! ;; Check that we are before an article with a ! ;; higher number. ! (search-forward (concat "\n" nnfolder-article-marker) ! nil t) ! (progn ! (setq num (string-to-int ! (buffer-substring ! (point) (progn (end-of-line) (point))))) ! (> num article)) ! ;; Discard any article numbers before the one we're ! ;; now looking at. ! (while (and articles ! (< (car articles) num)) ! (setq articles (cdr articles)))) ! (goto-char start)))) (set-buffer nntp-server-buffer) (nnheader-fold-continuation-lines) 'headers)))))) --=-=-=--