From mboxrd@z Thu Jan  1 00:00:00 1970
X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/11970
Path: main.gmane.org!not-for-mail
From: Mark Eichin <eichin@cygnus.com>
Newsgroups: gmane.emacs.gnus.general
Subject: Re: opening speed for nnml groups
Date: 03 Sep 1997 15:45:53 -0400
Message-ID: <xe1k9gyb2q6.fsf@maneki-neko.cygnus.com>
References: <whbu2bq604.fsf@norne.troll.no> <m0x6F7g-0005kWC@wandle.dra.hmg.gb> <0f4t82zbjt.fsf@odin.mjolner.dk>
NNTP-Posting-Host: coloc-standby.netfonds.no
X-Trace: main.gmane.org 1035151591 1148 80.91.224.250 (20 Oct 2002 22:06:31 GMT)
X-Complaints-To: usenet@main.gmane.org
NNTP-Posting-Date: Sun, 20 Oct 2002 22:06:31 +0000 (UTC)
Cc: ding@gnus.org
Return-Path: <ding-request@ifi.uio.no>
Original-Received: from xemacs.org (xemacs.cs.uiuc.edu [128.174.252.16])
	by altair.xemacs.org (8.8.7/8.8.7) with ESMTP id RAA17205
	for <steve@altair.xemacs.org>; Wed, 3 Sep 1997 17:01:48 -0700
Original-Received: from ifi.uio.no (0@ifi.uio.no [129.240.64.2])
	by xemacs.org (8.8.5/8.8.5) with SMTP id SAA29420
	for <steve@xemacs.org>; Wed, 3 Sep 1997 18:57:31 -0500 (CDT)
Original-Received: from claymore.vcinet.com (claymore.vcinet.com [208.205.12.23]) by ifi.uio.no with SMTP (8.6.11/ifi2.4) 
	id <AAA29119@ifi.uio.no> for <ding@ifi.uio.no> ; Thu, 4 Sep 1997 00:32:43 +0200
Original-Received: (qmail 5323 invoked by uid 504); 3 Sep 1997 19:46:01 -0000
Original-Received: (qmail 5318 invoked from network); 3 Sep 1997 19:45:59 -0000
Original-Received: from tweedledumb.cygnus.com (192.80.44.1)
  by claymore.vcinet.com with SMTP; 3 Sep 1997 19:45:59 -0000
Original-Received: from maneki-neko.cygnus.com (eichin@maneki-neko.cygnus.com [192.80.44.51])
	by tweedledumb.cygnus.com (8.8.5/8.8.5) with ESMTP id PAA19928;
	Wed, 3 Sep 1997 15:45:55 -0400 (EDT)
Original-Received: (from eichin@localhost) by maneki-neko.cygnus.com (8.8.5/8.7.3) id PAA13165; Wed, 3 Sep 1997 15:45:54 -0400
Original-To: Lars Balker Rasmussen <lbr@mjolner.dk>
In-Reply-To: Lars Balker Rasmussen's message of "03 Sep 1997 17:01:26 +0200"
Original-Lines: 26
X-Mailer: Gnus v5.4.56/Emacs 19.34
Xref: main.gmane.org gmane.emacs.gnus.general:11970
X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:11970

Indeed, 
+% mkdir test
+% time perl -e 'foreach $a (1..10000) { open FOO, ">test/$a" }'
1.490u 72.550s 1:31.65 80.7%    0+0k 0+0io 287pf+0w
+% time sync
0.000u 0.040s 0:00.21 19.0%     0+0k 0+0io 21pf+0w
+% time rm -rf test
0.200u 8.120s 0:09.26 89.8%     0+0k 0+0io 86pf+0w

But that's a p133 running linux, run of the mill 2G SCSI disks.

> Some of the NNML groups that I use for mailing lists have
> become quite large (over 1,000 messages) and have started to

Heh -- my mail.misc has 16000 files, mail.personal has 6500... and I
usually enter them with 33 or 99 (whatever is easiest to type) to read
new stuff, or maybe 300 or so if I want to search back a bit.  *That*
isn't slow... but when I hit $ to refile some spam, from mail.misc to
mail.spam (which only has 600 messages at the moment) it takes as long
as 5 to 10 seconds to recover.  I'd guess most of that has to do with
rewriting the 3.5M .overview file in mail.misc -- having 64M of RAM
does help, especially with linux aggressive caching...

Needless to say I'm pushing gnus quite a bit here.  All I really need
to do is write some scripts to refile all of mail.misc during 1996
into archive.1996.mail.misc, or something like that...