Re: opening speed for nnml groups

[Sean Doran <smd@ebone.net>]

Lars Balker Rasmussen <lbr@mjolner.dk> writes:

> % mkdir test
> % perl -e 'foreach $a (1..10000) { open FOO, ">test/$a" }'
> % time rm -rf test
> 0.60u 11.18s 1:55.40 10.2%
> % uname -a
> SunOS heimdal 5.5.1 Generic_103640-08 sun4u sparc

Ich, perl is a pig.  This makes me feel good about my
choice for a scripting shell:

 : sean.ebone.net ; uname -a        
NetBSD sean.ebone.net 1.2G NetBSD 1.2G (EBONE) #0: Sun Aug 17 20:19:52 CEST 1997     smd@sean.ebone.net:/usr/src/sys/arch/i386/compile/EBONE i386
: sean.ebone.net ; scsh
Scsh 0.5
> (create-directory "test")
> (run (time scsh -c 
	"(let loop ((i 1))
		(close-output-port 
		 (open-output-file (string-append \"test/\"
				    (number->string i))))
		(if (< i 10001) 
		    (loop (+ i 1))))"))
      175.68 real         6.71 user        68.59 sys
> (run (time rm -rf test))
       94.45 real         0.12 user         3.38 sys
> (create-directory "test")
> (run (time perl -e "foreach $a (1..10000) { open FOO, \">test/$a\" }"))
      198.90 real	  5.81 user	   68.99 sys
> (run (time scsh -c "(begin (for-each (lambda (f)
                                        (delete-file f))
                                       (glob \"./test/*\"))
			     (delete-directory \"test\"))"))
       89.79 real         7.71 user         7.83 sys

Well I cheat in that I use a statically-linked scshvm,
which makes an impressive difference. :)

> ls is primarily slow because it has to sort the files.

Yes, quite likely.  The globbing is certainly the most
CPU-intensive part of the quick rm equivalent.

> But of course, the Unix filesystem gets slower as more files are in a
> directory, but it's not _that_ bad.

Ayup.  I can't see any reason to blame the filesystem for 
nnml's slow behaviour on large groups.   The obvious CPU
eaters are mostly in preparing the *Summary* buffer, 
in the work done upon exiting the group, and in anything
else that does significant marching through and manipulation of
the *Summary* buffer.

Copying, respooling or moving articles also has ugly time
consumption properties having to do with maintaining
.overview files.

	Sean.