Gnus slow incorporating vast quantities of mail

[David Z Maze <dmaze@MIT.EDU>]

I was travelling on business for a few days last week, and so my
incoming mail spool got sort of big.  (Being on debian-* lists will do
that, somehow... :-)  Looking at the Incoming... file:

{15} Mail% grep -c ^L Incoming-4FNdp 
2475
{16} Mail% ls -l Incoming-4FNdp      
-rw-------   1 dmaze    mit       8785983 Jan 12 14:07 Incoming-4FNdp

Gnus seems to have pulled the mail in just fine, but AFAICT sorting it
into folders was really slow; I didn't get to the group buffer for
about half an hour.  (Hardware is a Sun Fire 280R, two 750 MHz
UltraSPARC-IIIs, 2.0 GB of RAM, I think I was the only serious user of
the machine at the time; CPU load wasn't an issue, but network latency
of getting to the mail store over AFS probably was.)  My suspicion is
that nnmail-split-fancy can be O(n^2) in the size of the incoming mail
spool, meaning that you'll get acceptable performance unless you're
trying to slurp in a truly vast quantity of mail (as I did here).

More configuration details: using nnml (into an AFS directory) fed by
a (Kerberized) POP server, sorting using a moderately complex
nnmail-split-fancy rule set.  Most of the time it works reasonably;
'ls -lSr ~/Mail' says that the new three largest Incoming... files I
have are 2.5, 1.4, and 1.2 MB, respectively, and I haven't noticed
sorting those being too painful.

Any hints as to what might cause this?  Is this an issue in Gnus, or
my splitting rules, or something else?  Is there any easy way to debug
it without letting my mail back up for a week?  :-)  TIA...

-- 
David Maze             dmaze@mit.edu          http://www.mit.edu/~dmaze/
"Theoretical politics is interesting.  Politicking should be illegal."
	-- Abra Mitchell