From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/55458 Path: main.gmane.org!not-for-mail From: Reiner Steib <4.uce.03.r.s@nurfuerspam.de> Newsgroups: gmane.emacs.gnus.general Subject: Spam processing slows down group exit (was: Profiling) Date: Thu, 01 Jan 2004 14:51:27 +0100 Sender: ding-owner@lists.math.uh.edu Message-ID: References: Reply-To: reiner.steib@gmx.de NNTP-Posting-Host: deer.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1072965908 27849 80.91.224.253 (1 Jan 2004 14:05:08 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Thu, 1 Jan 2004 14:05:08 +0000 (UTC) Original-X-From: ding-owner+M3998@lists.math.uh.edu Thu Jan 01 15:05:02 2004 Return-path: Original-Received: from malifon.math.uh.edu ([129.7.128.13]) by deer.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 1Ac3Re-0005WB-00 for ; Thu, 01 Jan 2004 15:05:02 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by malifon.math.uh.edu with smtp (Exim 3.20 #1) id 1Ac3RA-0000ez-00; Thu, 01 Jan 2004 08:04:32 -0600 Original-Received: from justine.libertine.org ([66.139.78.221] ident=postfix) by malifon.math.uh.edu with esmtp (Exim 3.20 #1) id 1Ac3R2-0000er-00 for ding@lists.math.uh.edu; Thu, 01 Jan 2004 08:04:24 -0600 Original-Received: from main.gmane.org (main.gmane.org [80.91.224.249]) by justine.libertine.org (Postfix) with ESMTP id ECFE73A0048 for ; Thu, 1 Jan 2004 08:04:22 -0600 (CST) Original-Received: from list by main.gmane.org with local (Exim 3.35 #1 (Debian)) id 1Ac3Qz-00055M-00 for ; Thu, 01 Jan 2004 15:04:21 +0100 Mail-Followup-To: ding@gnus.org X-Injected-Via-Gmane: http://gmane.org/ Original-To: ding@gnus.org Original-Received: from sea.gmane.org ([80.91.224.252]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 1Ac3Qy-000557-00 for ; Thu, 01 Jan 2004 15:04:20 +0100 Original-Received: from news by sea.gmane.org with local (Exim 3.35 #1 (Debian)) id 1Ac3Qy-0007E6-00 for ; Thu, 01 Jan 2004 15:04:20 +0100 Original-Lines: 63 Original-X-Complaints-To: usenet@sea.gmane.org X-Face: /U7=m^"/-Dn61mAl{g9e3>\G5Tp,oEX|V)g2I1hBk\ML;)7A?6cmB-y7y?'NA^J<=oz7syB =(McAwIHgLX!.B?R3X}98d@?>CrT094KLWh]WU4gDpnL/")MS(XoQTv`Oq225uL>+;CpPXo$N5e>N> $tPd-gbB^F{gQS#1ase]XO~D4p4M"3+F-7~u]dy3I?Pb8RO*H-EFeWDUf?Rf,d]pv\Jvh2Cht!A=im yKAS2Z%Ao^;}W/qzMvMm Mail-Copies-To: nobody User-Agent: Gnus/5.1004 (Gnus v5.10.4) Emacs/21.3 (gnu/linux) Cancel-Lock: sha1:MEGEESu/Qnng7zGsxQ0BV+JBcqo= Precedence: bulk Xref: main.gmane.org gmane.emacs.gnus.general:55458 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:55458 On Tue, Dec 30 2003, Lars Magne Ingebrigtsen wrote: > Gnus was feeling a bit more sluggish than usual, so I've spent a > couple of hours profiling and fixing obvious things. I noticed that leaving large nntp or nnml groups became kinda slow (I didn't profile it, though). I found that the group and topic parameters (spam/ham) are calculated[1] for *every article* rather than once for each group exit. I have set up spam processing for some IMAP groups and all Gmane groups[2], but not for nnml (and other nntp groups). I added the following debug statements in `spam-group-ham-mark-p'... (gnus-message 9 "DEBUG: spam-group-ham-mark-p %s %s %s" group mark spam) ... and got the following output after leaving a group with 10 articles (entered with `10 RET'): ,---- | Exiting summary buffer and applying spam rules | DEBUG: spam-group-ham-mark-p nnml+foo:bar 69 t [9 times] | DEBUG: spam-group-ham-mark-p nnml+foo:bar 33 t | DEBUG: spam-group-ham-mark-p nnml+foo:bar 69 nil [9 times] | DEBUG: spam-group-ham-mark-p nnml+foo:bar 33 nil | Marking spam as expired without moving it `---- The function seems to be called from `spam-ham-copy-or-move-routine': (dolist (article articles) (when (spam-group-ham-mark-p gnus-newsgroup-name (gnus-summary-article-mark article)) (push article todo))) Couldn't each possible mark be checked _once_ per group exit instead once for every article? (==> O(1) instead of O(n), AFAIKS.) Why is `spam-ham-copy-or-move-routine' called in the first place? I have not requested any spam processing in this group. Bye, Reiner. [1] Using C-g after toggle-debug-on-quit, I saw that Gnus was doing this most of the time. [2] See also : ("nnimap:spam\\.detected" (gnus-article-sort-functions '(gnus-article-sort-by-chars)) (ham-process-destination "nnimap:INBOX" "nnimap:training.ham") (spam-contents gnus-group-spam-classification-spam)) ("nnimap:\\(INBOX\\|other-folders\\)" (spam-process-destination . "nnimap:training.spam") (spam-contents gnus-group-spam-classification-ham)) ("^gmane\\." (spam-process (gnus-group-spam-exit-processor-report-gmane))) -- ,,, (o o) ---ooO-(_)-Ooo--- PGP key available via WWW http://rsteib.home.pages.de/