From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/41970 Path: main.gmane.org!not-for-mail From: David Z Maze Newsgroups: gmane.emacs.gnus.general Subject: Gnus slow incorporating vast quantities of mail Date: Mon, 14 Jan 2002 11:22:40 -0500 Sender: owner-ding@hpc.uh.edu Message-ID: NNTP-Posting-Host: coloc-standby.netfonds.no Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1035177282 8899 80.91.224.250 (21 Oct 2002 05:14:42 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Mon, 21 Oct 2002 05:14:42 +0000 (UTC) Return-Path: Original-Received: (qmail 27535 invoked from network); 14 Jan 2002 16:24:46 -0000 Original-Received: from malifon.math.uh.edu (mail@129.7.128.13) by mastaler.com with SMTP; 14 Jan 2002 16:24:46 -0000 Original-Received: from sina.hpc.uh.edu ([129.7.128.10] ident=lists) by malifon.math.uh.edu with esmtp (Exim 3.20 #1) id 16Q9t8-0003fA-00; Mon, 14 Jan 2002 10:23:10 -0600 Original-Received: by sina.hpc.uh.edu (TLB v0.09a (1.20 tibbs 1996/10/09 22:03:07)); Mon, 14 Jan 2002 10:23:02 -0600 (CST) Original-Received: from sclp3.sclp.com (qmailr@sclp3.sclp.com [209.196.61.66]) by sina.hpc.uh.edu (8.9.3/8.9.3) with SMTP id KAA15912 for ; Mon, 14 Jan 2002 10:22:51 -0600 (CST) Original-Received: (qmail 27512 invoked by alias); 14 Jan 2002 16:22:52 -0000 Original-Received: (qmail 27507 invoked from network); 14 Jan 2002 16:22:52 -0000 Original-Received: from pacific-carrier-annex.mit.edu (18.7.21.83) by gnus.org with SMTP; 14 Jan 2002 16:22:52 -0000 Original-Received: from grand-central-station.mit.edu (GRAND-CENTRAL-STATION.MIT.EDU [18.7.21.82]) by pacific-carrier-annex.mit.edu (8.9.2/8.9.2) with ESMTP id LAA16341 for ; Mon, 14 Jan 2002 11:22:52 -0500 (EST) Original-Received: from melbourne-city-street.mit.edu (MELBOURNE-CITY-STREET.MIT.EDU [18.7.21.86]) by grand-central-station.mit.edu (8.9.2/8.9.2) with ESMTP id LAA18949; Mon, 14 Jan 2002 11:22:41 -0500 (EST) Original-Received: from nerd-xing.mit.edu (NERD-XING.MIT.EDU [18.7.16.74]) by melbourne-city-street.mit.edu (8.9.2/8.9.2) with ESMTP id LAA20779; Mon, 14 Jan 2002 11:22:41 -0500 (EST) Original-To: ding@gnus.org Original-Lines: 34 User-Agent: Gnus/5.090005 (Oort Gnus v0.05) XEmacs/21.4 (Artificial Intelligence, sparc-sun-solaris2.8) X-Attribution: DZM Precedence: list X-Majordomo: 1.94.jlt7 Xref: main.gmane.org gmane.emacs.gnus.general:41970 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:41970 I was travelling on business for a few days last week, and so my incoming mail spool got sort of big. (Being on debian-* lists will do that, somehow... :-) Looking at the Incoming... file: {15} Mail% grep -c ^L Incoming-4FNdp 2475 {16} Mail% ls -l Incoming-4FNdp -rw------- 1 dmaze mit 8785983 Jan 12 14:07 Incoming-4FNdp Gnus seems to have pulled the mail in just fine, but AFAICT sorting it into folders was really slow; I didn't get to the group buffer for about half an hour. (Hardware is a Sun Fire 280R, two 750 MHz UltraSPARC-IIIs, 2.0 GB of RAM, I think I was the only serious user of the machine at the time; CPU load wasn't an issue, but network latency of getting to the mail store over AFS probably was.) My suspicion is that nnmail-split-fancy can be O(n^2) in the size of the incoming mail spool, meaning that you'll get acceptable performance unless you're trying to slurp in a truly vast quantity of mail (as I did here). More configuration details: using nnml (into an AFS directory) fed by a (Kerberized) POP server, sorting using a moderately complex nnmail-split-fancy rule set. Most of the time it works reasonably; 'ls -lSr ~/Mail' says that the new three largest Incoming... files I have are 2.5, 1.4, and 1.2 MB, respectively, and I haven't noticed sorting those being too painful. Any hints as to what might cause this? Is this an issue in Gnus, or my splitting rules, or something else? Is there any easy way to debug it without letting my mail back up for a week? :-) TIA... -- David Maze dmaze@mit.edu http://www.mit.edu/~dmaze/ "Theoretical politics is interesting. Politicking should be illegal." -- Abra Mitchell