From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/39759 Path: main.gmane.org!not-for-mail From: "Matt Armstrong" Newsgroups: gmane.emacs.gnus.general Subject: Re: Archiving large NNML groups quickly. Date: Tue, 30 Oct 2001 08:58:18 -0700 (MST) Sender: owner-ding@hpc.uh.edu Message-ID: <20011030155818.100BFBD4F@squeaker.lickey.com> References: NNTP-Posting-Host: coloc-standby.netfonds.no Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Trace: main.gmane.org 1035175418 29651 80.91.224.250 (21 Oct 2002 04:43:38 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Mon, 21 Oct 2002 04:43:38 +0000 (UTC) Cc: Kai =?iso-8859-1?q?Gro=DFjohann?= , Daniel Pittman , Return-Path: Original-Received: (qmail 21860 invoked from network); 30 Oct 2001 16:14:07 -0000 Original-Received: from malifon.math.uh.edu (mail@129.7.128.13) by mastaler.com with SMTP; 30 Oct 2001 16:14:07 -0000 Original-Received: from sina.hpc.uh.edu ([129.7.128.10] ident=lists) by malifon.math.uh.edu with esmtp (Exim 3.20 #1) id 15ybW5-0008Tf-00; Tue, 30 Oct 2001 10:13:29 -0600 Original-Received: by sina.hpc.uh.edu (TLB v0.09a (1.20 tibbs 1996/10/09 22:03:07)); Tue, 30 Oct 2001 10:13:08 -0600 (CST) Original-Received: from epithumia.math.uh.edu (epithumia.math.uh.edu [129.7.128.2]) by sina.hpc.uh.edu (8.9.3/8.9.3) with ESMTP id KAA08578 for ; Tue, 30 Oct 2001 10:13:01 -0600 (CST) Original-Received: (from tibbs@localhost) by epithumia.math.uh.edu (8.11.2/8.11.1) id f9UGDHt04883 for ding@hpc.uh.edu; Tue, 30 Oct 2001 10:13:17 -0600 Original-Received: from sclp3.sclp.com (qmailr@sclp3.sclp.com [209.196.61.66]) by sina.hpc.uh.edu (8.9.3/8.9.3) with SMTP id JAA08399 for ; Tue, 30 Oct 2001 09:58:22 -0600 (CST) Original-Received: (qmail 21575 invoked by alias); 30 Oct 2001 15:58:37 -0000 Original-Received: (qmail 21570 invoked from network); 30 Oct 2001 15:58:37 -0000 Original-Received: from hank.lickey.com (64.81.100.235) by gnus.org with SMTP; 30 Oct 2001 15:58:37 -0000 Original-Received: from squeaker.lickey.com (squeaker.lickey.com [192.168.100.10]) by hank.lickey.com (Postfix) with ESMTP id 9C9D4EE33 for ; Tue, 30 Oct 2001 08:58:21 -0700 (MST) Original-Received: from localhost (localhost [127.0.0.1]) by squeaker.lickey.com (Postfix) with ESMTP id C8D34BD69 for ; Tue, 30 Oct 2001 08:58:20 -0700 (MST) Original-Received: by squeaker.lickey.com (Postfix, from userid 1000) id 100BFBD4F; Tue, 30 Oct 2001 08:58:18 -0700 (MST) Original-To: Simon Josefsson In-Reply-To: (Simon Josefsson's message of "Tue, 30 Oct 2001 12:54:05 +0100 (CET)") User-Agent: Gnus/5.090004 (Oort Gnus v0.04) Emacs/21.1 (i386-debian-linux-gnu) Mail-Followup-To: jas@extundo.com, Kai.Grossjohann@CS.Uni-Dortmund.DE, daniel@rimspace.net, ding@gnus.org Original-Lines: 37 X-Delivery-Agent: TMDA v0.39/Python 1.5.2 (linux2) X-Virus-Scanned: by AMaViS snapshot-20010714 Precedence: list X-Majordomo: 1.94.jlt7 Xref: main.gmane.org gmane.emacs.gnus.general:39759 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:39759 Simon Josefsson writes: > On Tue, 30 Oct 2001, Kai Gro=DFjohann wrote: > >> Simon Josefsson writes: >>=20 >> > IMHO the active file should be eliminated, and a ".active" file in >> > each nnml group should be used instead. >>=20 >> This means that Gnus has to recursively search ~/Mail, at least if >> long file names are not in effect. This could be quite slow. > > No recursive searching is needed, I think, just use the subscribed > group name in Gnus to find the filename. Of course, instead of > reading only 1 file (active) you need to read N files, but it is > only necessary to do it once and after that keep things in memory -- > right now the active file is often read from disk. So in the end, > it would probably be about the same speed. When listing all groups, > a recursive find is needed though. > > Or maybe the active file should simply be removed completely (not > replaced by .active), the active info could be kept in .newsrc.eld > (.marks) instead. If we change the format of .marks it would even > be simple to manipulate it from shell/perl scripts. Hm, what do you > think? Keeping the active file out of the backend's directory tree might ease problems generated by having nnfolder and nnml backends using the same active file. But my personal preference would be to see .newsrc.eld contain less, and push more of the meta data into mostly-human-readable files like the active and marks files. This would require some serious effort though. --=20 matt