From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/62327 Path: news.gmane.org!not-for-mail From: Romain Francoise Newsgroups: gmane.emacs.gnus.general Subject: Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders Date: Mon, 20 Mar 2006 22:44:38 +0100 Organization: orebokech dot com Message-ID: <87acbku6yh.fsf@pacem.orebokech.com> References: <878xr6klq1.fsf@denkblock.local> <87ek0xzm9c.fsf@rimspace.net> <87u09tjvdk.fsf@denkblock.local> <871wwxz73o.fsf@rimspace.net> <87mzfl87eu.fsf@denkblock.local> NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1142891105 2562 80.91.229.2 (20 Mar 2006 21:45:05 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Mon, 20 Mar 2006 21:45:05 +0000 (UTC) Original-X-From: ding-owner+m10854@lists.math.uh.edu Mon Mar 20 22:45:03 2006 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from malifon.math.uh.edu ([129.7.128.13]) by ciao.gmane.org with esmtp (Exim 4.43) id 1FLSBO-0003Ay-9M for ding-account@gmane.org; Mon, 20 Mar 2006 22:44:58 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu ident=lists) by malifon.math.uh.edu with smtp (Exim 3.20 #1) id 1FLSBH-0004T7-00; Mon, 20 Mar 2006 15:44:51 -0600 Original-Received: from nas02.math.uh.edu ([129.7.128.40]) by malifon.math.uh.edu with esmtp (Exim 3.20 #1) id 1FLSBA-0004SU-00 for ding@lists.math.uh.edu; Mon, 20 Mar 2006 15:44:44 -0600 Original-Received: from quimby.gnus.org ([80.91.224.244]) by nas02.math.uh.edu with esmtp (Exim 4.52) id 1FLSB6-00005k-Qw for ding@lists.math.uh.edu; Mon, 20 Mar 2006 15:44:44 -0600 Original-Received: from yeast.orebokech.com ([82.67.41.165]) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1FLSB4-0006Nx-00 for ; Mon, 20 Mar 2006 22:44:38 +0100 Original-Received: from pacem.orebokech.com (pacem.orebokech.com [192.168.1.3]) by yeast.orebokech.com (Postfix) with ESMTP id D5B6412CF7 for ; Mon, 20 Mar 2006 22:44:37 +0100 (CET) Original-Received: by pacem.orebokech.com (Postfix, from userid 1000) id 20BB9523CE; Mon, 20 Mar 2006 22:44:38 +0100 (CET) Original-To: ding@gnus.org X-Face: }9mYu,e_@+e!`Z-P5kVXa3\_b:hdJ"B)ww[&=b<2=awG:GOIM (Elias Oltmanns's message of "Mon, 20 Mar 2006 16:25:45 +0100") X-Spam-Score: -2.5 (--) Precedence: bulk Original-Sender: ding-owner@lists.math.uh.edu Xref: news.gmane.org gmane.emacs.gnus.general:62327 Archived-At: Elias Oltmanns writes: > So far I managed to rule out search-* and re-search-*. How? > Still, I have the feeling that it might be one of those seeking > commands, however, I didn't manage to profile goto-char which was my > next guess. Quite unlikely... `goto-char' can take its sweet time to move around in very large buffers, but *three seconds in one call*? I think the problem with this function is that we call `search-forward' and `re-search-forward' alternatively for each message, which duplicates all the work of setting up Boyer-Moore tables, and possibly just thrashes search caches between each call. I get a very significant performance boost with this simple patch which just merges the two searches into one: on a 65M file, calling the function now takes 10 seconds instead of 23 on my (slow) machine. Could you try it out? --- nnfolder.el 09 Feb 2006 08:33:47 +0100 7.16 +++ nnfolder.el 20 Mar 2006 22:39:42 +0100 @@ -405,12 +405,10 @@ the group. Then the marks file will be (when nnfolder-current-buffer (set-buffer nnfolder-current-buffer) (goto-char (point-min)) - (let ((marker (concat "\n" nnfolder-article-marker)) - (number "[0-9]+") - numbers) - (while (and (search-forward marker nil t) - (re-search-forward number nil t)) - (let ((newnum (string-to-number (match-string 0)))) + (let ((head (concat "\n" nnfolder-article-marker "\\([0-9]+\\)")) + numbers) + (while (re-search-forward head nil t) + (let ((newnum (string-to-number (match-string 1)))) (if (nnmail-within-headers-p) (push newnum numbers)))) ;; The article numbers are increasing, so this result is sorted. -- Romain Francoise | The sea! the sea! the open it's a miracle -- http://orebokech.com/ | sea! The blue, the fresh, the | ever free! --Bryan W. Procter