From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/42671 Path: main.gmane.org!not-for-mail From: Rob Browning Newsgroups: gmane.emacs.gnus.general Subject: Re: db-backed mail back end Date: Sun, 27 Jan 2002 16:32:52 -0600 Sender: owner-ding@hpc.uh.edu Message-ID: <87bsffjugb.fsf@raven.i.defaultvalue.org> References: NNTP-Posting-Host: coloc-standby.netfonds.no Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1035177881 13065 80.91.224.250 (21 Oct 2002 05:24:41 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Mon, 21 Oct 2002 05:24:41 +0000 (UTC) Return-Path: Original-Received: (qmail 24009 invoked from network); 27 Jan 2002 22:33:39 -0000 Original-Received: from malifon.math.uh.edu (mail@129.7.128.13) by mastaler.com with SMTP; 27 Jan 2002 22:33:39 -0000 Original-Received: from sina.hpc.uh.edu ([129.7.128.10] ident=lists) by malifon.math.uh.edu with esmtp (Exim 3.20 #1) id 16UxrK-0002Ns-00; Sun, 27 Jan 2002 16:33:11 -0600 Original-Received: by sina.hpc.uh.edu (TLB v0.09a (1.20 tibbs 1996/10/09 22:03:07)); Sun, 27 Jan 2002 16:33:07 -0600 (CST) Original-Received: from sclp3.sclp.com (qmailr@sclp3.sclp.com [209.196.61.66]) by sina.hpc.uh.edu (8.9.3/8.9.3) with SMTP id QAA04824 for ; Sun, 27 Jan 2002 16:32:56 -0600 (CST) Original-Received: (qmail 23989 invoked by alias); 27 Jan 2002 22:32:55 -0000 Original-Received: (qmail 23984 invoked from network); 27 Jan 2002 22:32:54 -0000 Original-Received: from dsl-209-87-109-2.constant.com (HELO defaultvalue.org) (postfix@209.87.109.2) by gnus.org with SMTP; 27 Jan 2002 22:32:54 -0000 Original-Received: from raven.i.defaultvalue.org (raven.i.defaultvalue.org [192.168.1.7]) by defaultvalue.org (Postfix) with ESMTP id 58293F93 for ; Sun, 27 Jan 2002 16:32:53 -0600 (CST) Original-Received: by raven.i.defaultvalue.org (Postfix, from userid 1000) id ECE581B0C; Sun, 27 Jan 2002 16:32:52 -0600 (CST) Original-To: ding@gnus.org In-Reply-To: (Lars Magne Ingebrigtsen's message of "Wed, 23 Jan 2002 00:33:58 +0100") Original-Lines: 74 User-Agent: Gnus/5.090006 (Oort Gnus v0.06) Emacs/21.1 (i386-debian-linux-gnu) Precedence: list X-Majordomo: 1.94.jlt7 Xref: main.gmane.org gmane.emacs.gnus.general:42671 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:42671 Lars Magne Ingebrigtsen writes: > Now that there are nice, free SQL-based database servers out there > (PostgreSQL, and, er, PostgreSQL), would it be an idea to have a > db-backed mail back end? What would you gain by such a beast, and > what would you lose? FWIW, I've been thinking about what might be involved for a while now (mostly wondering about an nnsql backend) -- things I'd been pondering: * if the db layout were well specified, and if certain precautions were taken, would it be possible to use other tools to manipulate your mail (perl, python, whatever) without stepping on gnus' toes? At the very least, it would be nice if there were a simple elisp library for manipulating your mail. i.e.: (gnus-for-each-msg-in-group (lambda (msg) (gnus-msg-mark msg 'expirable)) "spam") * upon arrival each msg could be given a guaranteed unique id, and everything else in the DB could key off of that. This makes "msg urls" possible so that you can easily refer to your mail from other documents, like a to-do file, etc, even if you move/re-file the message, etc. Only deletion would render the msg unreachable via that url. i.e. gnus://my-msgs/20333111 (or whatever) * as you suggest, it seems like a given message could easily belong to multiple mail groups via a table containing (msg-id group-name) rows. Deleting a message from a group would just mean removing the corresponding row, and the message might or might not be deleted when its last "group membership" link were removed. Though there might be performance issues, depending on how you handled this. * it would also be easy to add a table (or tables) for user defined per-msg information (user tags, or whatever) * transactions and locking might be helpful on the safety front. * you might be able to (eventually) provide fairly fancy remote mail access, though I'm not sure how hard it is to reasonably secure a postgres connection. * here, we've figured how to easily launch a copy of postgres in a sandbox, and so it's easy to use it during "make check" etc. without worrying about mucking up a system shared db. It would also make it easy to "automate" setup and installation for people who don't need all their mail in the system shared DB. You can just ask "Where should we put your mail repository?", and then create that tree and launch a postmaster there. The user wouldn't have to bother the system DB admin or know any DB commands just to set up their sql mail system. * originally, I had figured that for my purposes, I might just use the DB for the meta-data and store the messages on disk. I'd have one table that mapped the msg unique IDs to filesystem paths and I wouldn't store all the message bodies in the database. It seems like whether or not this is a good idea really depends on the system's other goals (do you want the db to search for content, or other tools, i.e. glimpse, etc.). If I had tried this, I was thinking of storing messages with paths chosen according to arrival time like ./msgs/2002/01/02/1 or similar. * deletion/expiry might be able to be more completely divorced from readness as a state. Personally, I don't really want to lose the information that I've read/ticked/not-read something just because I mark it as expired/deleted/etc. -- Rob Browning rlb @defaultvalue.org, @linuxdevel.com, and @debian.org Previously @cs.utexas.edu GPG=1C58 8B2C FB5E 3F64 EA5C 64AE 78FE E5FE F0CB A0AD