Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders

[Romain Francoise <romain@orebokech.com>]

Elias Oltmanns <oltmanns@uni-bonn.de> writes:

> So far I managed to rule out search-* and re-search-*.

How?

> Still, I have the feeling that it might be one of those seeking
> commands, however, I didn't manage to profile goto-char which was my
> next guess.

Quite unlikely... `goto-char' can take its sweet time to move around in
very large buffers, but *three seconds in one call*?

I think the problem with this function is that we call `search-forward'
and `re-search-forward' alternatively for each message, which duplicates
all the work of setting up Boyer-Moore tables, and possibly just
thrashes search caches between each call.

I get a very significant performance boost with this simple patch which
just merges the two searches into one: on a 65M file, calling the
function now takes 10 seconds instead of 23 on my (slow) machine.  Could
you try it out?

--- nnfolder.el	09 Feb 2006 08:33:47 +0100	7.16
+++ nnfolder.el	20 Mar 2006 22:39:42 +0100	
@@ -405,12 +405,10 @@ the group.  Then the marks file will be 
     (when nnfolder-current-buffer
       (set-buffer nnfolder-current-buffer)
       (goto-char (point-min))
-      (let ((marker (concat "\n" nnfolder-article-marker))
-	    (number "[0-9]+")
-	    numbers)
-	(while (and (search-forward marker nil t)
-		    (re-search-forward number nil t))
-	  (let ((newnum (string-to-number (match-string 0))))
+      (let ((head (concat "\n" nnfolder-article-marker "\\([0-9]+\\)"))
+	     numbers)
+	(while (re-search-forward head nil t)
+	  (let ((newnum (string-to-number (match-string 1))))
 	    (if (nnmail-within-headers-p)
 		(push newnum numbers))))
       ;; The article numbers are increasing, so this result is sorted.

-- 
Romain Francoise <romain@orebokech.com> | The sea! the sea! the open
it's a miracle -- http://orebokech.com/ | sea! The blue, the fresh, the
                                        | ever free! --Bryan W. Procter