Performance regression expiring nnml groups

Performance regression expiring nnml groups

[Detlev Zundel]

Hi,

recently I upgraded my emacs from git://repo.or.cz/emacs.git effectively
jumping from commit b6bd53c (EMACS_23_2-3198-gb6bd53c) to commit d1be0ce
(EMACS_PRETEST_23_2_90-3710-gd1be0ce).  For gnus this was a lot of
changes:

dzu@ohwell:emacs[0]$ git diff --stat b6bd53c..d1be0ce lisp/gnus
[...]
 69 files changed, 6387 insertions(+), 3632 deletions(-)

Since this upgrade, expiring an article from an nnml group to another
nnmail group now roughly takes one second per article compared to below
a second for the whole group as before.  I should mention that the PC
running this Emacs is somewhat aged (Pentium III) so maybe nobody else
noticed this regression before as it most likely will not be that
noticeable for faster hardware.

Doing a profile, it seems like the problem is somewhere below
the call to gnus-request-accept-article as shown by these numbers from elp:

gnus-summary-exit                                                             1           91.959608     91.959608
gnus-run-hooks                                                                5           91.917669000  18.383533800
gnus-summary-expire-articles                                                  1           91.917626     91.917626
gnus-request-expire-articles                                                  1           91.892586     91.892586
gnus-request-accept-article                                                   78          88.905859999  1.1398187179
gnus-group-mark-article-read                                                  78          0.692419      0.0088771666
gnus-group-make-articles-read                                                 78          0.5176209999  0.0066361666
gnus-request-set-mark                                                         79          0.4452899999  0.0056365822
gnus-activate-group                                                           78          0.0944839999  0.0012113333
gnus-group-update-group                                                       79          0.0871199999  0.0011027848
gnus-topic-update-topic-line                                                  158         0.0624190000  0.0003950569
gnus-request-group                                                            156         0.0553440000  0.0003547692
gnus-topic-update-topics-containing-group                                     79          0.0480610000  0.0006083670
gnus-summary-update-info                                                      1           0.038921      0.038921

Is this a known behaviour?

The configuration for the group in question is something like this:

(setq gnus-auto-expirable-newsgroups "^\\(nnml:\\)*\\(mail\\|list\\)\\.*")
(setq gnus-parameters
      `(((,(regexp-quote "nnml:mail.foo")
	 (expiry-target . "nnml:mail.foo-old")
	 (expiry-wait 100))

The nnml group where expiration takes place contains ~2900 articles and
the target group ~35000.

Thanks for any help.

Cheers
  Detlev

-- 
X-Windows has to be the most expensive way ever of popping up an Emacs
window.
                                          -- The UNIX Haters Handbook

Re: Performance regression expiring nnml groups

[Lars Magne Ingebrigtsen]

Detlev Zundel <dzu@denx.de> writes:

> Since this upgrade, expiring an article from an nnml group to another
> nnmail group now roughly takes one second per article compared to below
> a second for the whole group as before. 

That has to be the result of this change:

diff --git a/lisp/ChangeLog b/lisp/ChangeLog
index 121b9f0..9ac822f 100644
--- a/lisp/ChangeLog
+++ b/lisp/ChangeLog
@@ -1,5 +1,9 @@
 2010-10-09  Lars Magne Ingebrigtsen  <larsi@gnus.org>
 
+	* nnmail.el (nnmail-expiry-target-group): Say that every expiry target
+	group is the "last", so that the backends like nnfolder actually save
+	their folders.

Hm...  so it's now slow, but actually works as it's supposed to.  I
think the right way to fix this is to add another backend function, like
-request-flush, to make them flush whatever they have stored.  I'll add
that.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen