From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/13668 Path: main.gmane.org!not-for-mail From: Jason L Tibbitts III Newsgroups: gmane.emacs.gnus.general Subject: Re: Huge groups Date: 20 Jan 1998 03:29:19 -0600 Sender: owner-ding@hpc.uh.edu Message-ID: References: NNTP-Posting-Host: coloc-standby.netfonds.no Mime-Version: 1.0 (generated by tm-edit 7.100) Content-Type: text/plain; charset=US-ASCII X-Trace: main.gmane.org 1035152990 10413 80.91.224.250 (20 Oct 2002 22:29:50 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Sun, 20 Oct 2002 22:29:50 +0000 (UTC) Return-Path: Original-Received: from xemacs.org (xemacs.cs.uiuc.edu [128.174.252.16]) by altair.xemacs.org (8.8.8/8.8.8) with ESMTP id BAA10166 for ; Tue, 20 Jan 1998 01:32:17 -0800 Original-Received: from sina.hpc.uh.edu (root@Sina.HPC.UH.EDU [129.7.3.5]) by xemacs.org (8.8.5/8.8.5) with ESMTP id DAA29736 for ; Tue, 20 Jan 1998 03:30:19 -0600 (CST) Original-Received: from sina.hpc.uh.edu (lists@Sina.HPC.UH.EDU [129.7.3.5]) by sina.hpc.uh.edu (8.7.3/8.7.3) with ESMTP id DAH09696; Tue, 20 Jan 1998 03:30:13 -0600 (CST) Original-Received: by sina.hpc.uh.edu (TLB v0.09a (1.20 tibbs 1996/10/09 22:03:07)); Tue, 20 Jan 1998 03:29:31 -0600 (CST) Original-Received: from claymore.vcinet.com (claymore.vcinet.com [208.205.12.23]) by sina.hpc.uh.edu (8.7.3/8.7.3) with SMTP id DAA09683 for ; Tue, 20 Jan 1998 03:29:23 -0600 (CST) Original-Received: (qmail 25152 invoked by uid 504); 20 Jan 1998 09:29:22 -0000 Original-Received: (qmail 25149 invoked from network); 20 Jan 1998 09:29:22 -0000 Original-Received: from sina.hpc.uh.edu (tibbs@129.7.3.5) by claymore.vcinet.com with SMTP; 20 Jan 1998 09:29:22 -0000 Original-Received: (from tibbs@localhost) by sina.hpc.uh.edu (8.7.3/8.7.3) id DAA09678; Tue, 20 Jan 1998 03:29:20 -0600 (CST) Original-To: ding@gnus.org In-Reply-To: Steinar Bang's message of 20 Jan 1998 08:39:33 +0100 Original-Lines: 24 X-Mailer: Gnus v5.4.46/Emacs 19.34 Precedence: list X-Majordomo: 1.94.jlt7 Xref: main.gmane.org gmane.emacs.gnus.general:13668 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:13668 >>>>> "SB" == Steinar Bang writes: SB> A faster backend? Are there some formats that could be used? Isn't the SB> qmail native format supposed to be fast? (at least according to Dan SB> Bernstein...) Well, I think it would be hard to get much faster than nnml. You could have something like nnfolder but keep an overview file that stores byte offsets within the folder for each message, so it's one open and one seek to grab a message. (I use this format in an archiver I'm working with.) But the question here is how to handle tens of thousands of messages, when even the overhead of loading the overview file is too much. The solution is, of course, a database, but a simpler solution would be to just split the messages up into smaller collections and optimize the common operations to only deal with one piece of the collection. I suppose this is easy enough to do manually in the cases where it is really needed. The only place where it can't be done is with the list archive (which is why I brought the idea up ages ago). The solution to that was the ding-list-recent archive, accomplished through procmail trickery. - J<