From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/47500 Path: main.gmane.org!not-for-mail From: kai.grossjohann@uni-duisburg.de (Kai =?iso-8859-1?q?Gro=DFjohann?=) Newsgroups: gmane.emacs.gnus.general Subject: Re: overview -> .db files? Date: Thu, 31 Oct 2002 19:21:44 +0100 Organization: University of Dortmund, Germany Sender: owner-ding@hpc.uh.edu Message-ID: <844rb2o4ef.fsf@crybaby.uni-duisburg.de> References: NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1036089426 24173 80.91.224.249 (31 Oct 2002 18:37:06 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Thu, 31 Oct 2002 18:37:06 +0000 (UTC) Return-path: Original-Received: from malifon.math.uh.edu ([129.7.128.13]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 187KBl-0006HY-00 for ; Thu, 31 Oct 2002 19:37:05 +0100 Original-Received: from sina.hpc.uh.edu ([129.7.128.10] ident=lists) by malifon.math.uh.edu with esmtp (Exim 3.20 #1) id 187KAC-0008Dk-00; Thu, 31 Oct 2002 12:35:28 -0600 Original-Received: by sina.hpc.uh.edu (TLB v0.09a (1.20 tibbs 1996/10/09 22:03:07)); Thu, 31 Oct 2002 12:36:09 -0600 (CST) Original-Received: from sclp3.sclp.com (qmailr@sclp3.sclp.com [209.196.61.66]) by sina.hpc.uh.edu (8.9.3/8.9.3) with SMTP id MAA19180 for ; Thu, 31 Oct 2002 12:35:41 -0600 (CST) Original-Received: (qmail 1550 invoked by alias); 31 Oct 2002 18:34:47 -0000 Original-Received: (qmail 1545 invoked from network); 31 Oct 2002 18:34:47 -0000 Original-Received: from quimby.gnus.org (80.91.224.244) by gnus.org with SMTP; 31 Oct 2002 18:34:47 -0000 Original-Received: from news by quimby.gnus.org with local (Exim 3.12 #1 (Debian)) id 187KDx-0008SX-00 for ; Thu, 31 Oct 2002 19:39:21 +0100 Original-To: ding@gnus.org Original-Path: not-for-mail Original-Newsgroups: gnus.ding Original-Lines: 41 Original-NNTP-Posting-Host: pd951fe3b.dip.t-dialin.net Original-X-Trace: quimby.gnus.org 1036089561 32398 217.81.254.59 (31 Oct 2002 18:39:21 GMT) Original-X-Complaints-To: usenet@quimby.gnus.org Original-NNTP-Posting-Date: 31 Oct 2002 18:39:21 GMT User-Agent: Gnus/5.090008 (Oort Gnus v0.08) Emacs/21.3.50 (i686-pc-linux-gnu) Cancel-Lock: sha1:R6170TJqtmtjiQfMYY93r55kj08= Precedence: list X-Majordomo: 1.94.jlt7 Xref: main.gmane.org gmane.emacs.gnus.general:47500 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:47500 Wes Hardaker writes: > Here's a thought: XEmacs at least (I don't know about Emacs) offers > the ability to make use of true database files for things. How > difficult would it be to (optionally of course) change the .overview > files into db files for speed purposes? I think it might not be so easy. I think the backend interface works like this: Gnus tells the backend to retrieve some headers, the backend puts the headers into a buffer in NOV format, then Gnus reads that buffer. But I think there are already provisions for using headers instead of NOV format. So maybe that's a spot where you can hook in and return the headers to Gnus in a Lisp data structure without going via a buffer. It would be a major undertaking. Please work on it :-) Another possibility which might be easier is to keep the backend interface and to just use the db files as a faster way to extract the right headers. Currently, I think nnml just inserts the whole .overview file into a buffer and then uses search-forward to find things. (Or maybe it has a binary search algorithm? Then it would be difficult to make it a lot faster.) But if you set gnus-verbose and gnus-verbose-backends to 10 I think you will see that fetching headers is not the most time-consuming part when using Gnus. There are some cases where header fetching is slow, but now we have the agent cache which speeds things up a lot. At least for me, the slow part about using Gnus is the generation of the summary buffer. I wonder if that can be made faster somehow. Is there a way to create a buffer with a lot of text properties and stuff really quickly? What happens if we princ (buffer-string) to a file and then read that representation into memory and use a single insert call to create a buffer from it? kai -- ~/.signature is: umop ap!sdn (Frank Nobis)