Re: fast local-file storage for Gnus

[Ted Zlatanov <tzz@lifelogs.com>]

On Thu, 12 Jun 2003, d.love@dl.ac.uk wrote:
> Ted Zlatanov <tzz@lifelogs.com> writes:
> 
>> I'm sure the implementation will not be hard for people who know
>> autoconf well.
> 
> [That might be me, which is why I made the comment!  The problem is
> in guessing where on earth the library is, and checking versions.]

Unfortunately there's no libdb-config like for instance imlib2-config,
that sort of wrapper is very useful.  I would imagine a --with-libdb
switch, optional and left empty (no libdb) by default, would do the
job.  Anyhow, what I want is a generic interface, libdb is just a
container library and I don't want to write specifically for it.

>> It's not a bad approach, but it can be slow with large alists.
> 
> You don't actually have to cons an alist to transfer data from a
> file to a hash table.  (You can read data directly into a table,
> which is probably faster.  If necessary, you could make a primitive
> to dump and restore hash tables fast.)

Right, but I don't know how to write that primitive in C, and in Lisp
it would probably be too slow.  If we're going to write that
primitive, why not make it a generic interface with
get/set/delete/commit functions with a plain text backend?  

> If you want `fast' access to the data, you presumably have to read
> them into memory anyway, so I think it needs measurements, unless
> you've already made them.  I've had surprising results from
> measuring this sort of thing in the past.

I really don't know what to measure, since there's only the
alist-to-plain-file mechanism right now :)

>> Here's what I want, in an ideal world: a new data structure called
>> a persistent-hashtable.
> 
> [...]
> 
>> Is this possible?
> 
> I don't see why not, if it's well designed, but you'd want to talk
> to rms about it and be prepared to do the implementation.  I suspect
> it's more difficult to get right than you might guess, but that's
> not a reason for not doing it.

So emacs-devel would be the right place?

Thanks
Ted