From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/52994 Path: main.gmane.org!not-for-mail From: Ted Zlatanov Newsgroups: gmane.emacs.gnus.general Subject: Re: fast local-file storage for Gnus Date: Tue, 03 Jun 2003 11:37:07 -0400 Organization: =?koi8-r?q?=F4=C5=CF=C4=CF=D2=20=FA=CC=C1=D4=C1=CE=CF=D7?= @ Cienfuegos Sender: ding-owner@lists.math.uh.edu Message-ID: <4nadcz895o.fsf@lockgroove.bwh.harvard.edu> References: <4nvfw19zgy.fsf@lockgroove.bwh.harvard.edu> <87smqwedvf.fsf@mail.paradoxical.net> <4n1xygihkm.fsf@lockgroove.bwh.harvard.edu> NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1054654617 30556 80.91.224.249 (3 Jun 2003 15:36:57 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Tue, 3 Jun 2003 15:36:57 +0000 (UTC) Cc: ding@gnus.org Original-X-From: ding-owner+M1538@lists.math.uh.edu Tue Jun 03 17:36:53 2003 Return-path: Original-Received: from malifon.math.uh.edu ([129.7.128.13]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 19NDoy-0007oZ-00 for ; Tue, 03 Jun 2003 17:35:32 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by malifon.math.uh.edu with smtp (Exim 3.20 #1) id 19NDqj-0007sD-00; Tue, 03 Jun 2003 10:37:21 -0500 Original-Received: from sclp3.sclp.com ([64.157.176.121]) by malifon.math.uh.edu with smtp (Exim 3.20 #1) id 19NDqc-0007s7-00 for ding@lists.math.uh.edu; Tue, 03 Jun 2003 10:37:14 -0500 Original-Received: (qmail 60316 invoked by alias); 3 Jun 2003 15:37:14 -0000 Original-Received: (qmail 60311 invoked from network); 3 Jun 2003 15:37:14 -0000 Original-Received: from clifford.bwh.harvard.edu (134.174.9.41) by sclp3.sclp.com with SMTP; 3 Jun 2003 15:37:14 -0000 Original-Received: from lockgroove.bwh.harvard.edu (lockgroove [134.174.9.133]) by clifford.bwh.harvard.edu (8.10.2+Sun/8.11.0) with ESMTP id h53Fb7I19393; Tue, 3 Jun 2003 11:37:07 -0400 (EDT) Original-Received: (from tzz@localhost) by lockgroove.bwh.harvard.edu (8.11.6+Sun/8.11.0) id h53Fb7T15504; Tue, 3 Jun 2003 11:37:07 -0400 (EDT) Original-To: Dave Love X-Face: bd.DQ~'29fIs`T_%O%C\g%6jW)yi[zuz6;d4V0`@y-~$#3P_Ng{@m+e4o<4P'#(_GJQ%TT= D}[Ep*b!\e,fBZ'j_+#"Ps?s2!4H2-Y"sx" Mail-Followup-To: Dave Love , ding@gnus.org In-Reply-To: (Dave Love's message of "Tue, 03 Jun 2003 12:02:31 +0100") User-Agent: Gnus/5.1003 (Gnus v5.10.3) Emacs/21.3 (usg-unix-v) Precedence: bulk Xref: main.gmane.org gmane.emacs.gnus.general:52994 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:52994 On Tue, 03 Jun 2003, d.love@dl.ac.uk wrote: > Ted Zlatanov writes: > >> it seems like a very simple feature to add BerkeleyDB/DBM >> interfaces that can be configured at compile-time like a lot of >> other GNU packages do (at least cfengine, that I know of), > > [Ah, the cfengine libdb configuration nightmare... You'd certainly > have to do better than that!] I'm sure the implementation will not be hard for people who know autoconf well. Emacs detects other optional libraries just fine. > I don't know for sure, but I doubt rms would want to insist on libdb > being available to build Emacs. Otherwise, instances of Emacs > wouldn't be able to run the Gnus distributed with it if you insist > on such support. Well, there are two options then: distribute Emacs with its own file storage mechanism (not a bad thing, if it's optimized for Lisp data structures), or bundle something with Emacs. But I only want this functionality optionally available; if Emacs doesn't find libdb for instance maybe it can switch to something slower but universally available such as plain text storage. I really just want an API like the one in XEmacs, and what happens behind the curtain is not my concern. That way, even a relational database could be accessed transparently through the same mechanism; I'm definitely not suggesting that as a goal though. You mentioned dynamic loading, that would be nice too but could be a problem if libdb is modified or removed. >> Gnus could probably use fast file databases in aspects other than >> the registry. > > You haven't explained why this is necessary. If just speed is the > issue, how are Emacs hash tables too slow? Hash tables are not persistent. The Gnus newsrc.eld file, for instance, converts a hashtable into an alist and saves that. The gnus-registry does the same thing with the registry hashtable. It's not a bad approach, but it can be slow with large alists. I'm sure there are other applications (BBDB, for instance) that could use this functionality. Here's what I want, in an ideal world: a new data structure called a persistent-hashtable. It should be configurable to save on every modification or only when requested. It can use the alist-to-file conversion right now, there's no external linking needed. Then, optionally, it can load another backend, for instance BerkeleyDB or (some day) a RDBMS gateway to a table with key-value columns. Is this possible? Ted