Re: overview -> .db files?

[kai.grossjohann@uni-duisburg.de (Kai Großjohann)]

Wes Hardaker <wes@hardakers.net> writes:

> Here's a thought:  XEmacs at least (I don't know about Emacs) offers
> the ability to make use of true database files for things.  How
> difficult would it be to (optionally of course) change the .overview
> files into db files for speed purposes?

I think it might not be so easy.  I think the backend interface works
like this: Gnus tells the backend to retrieve some headers, the
backend puts the headers into a buffer in NOV format, then Gnus reads
that buffer.

But I think there are already provisions for using headers instead of
NOV format.  So maybe that's a spot where you can hook in and return
the headers to Gnus in a Lisp data structure without going via a
buffer.

It would be a major undertaking.  Please work on it :-)

Another possibility which might be easier is to keep the backend
interface and to just use the db files as a faster way to extract the
right headers.  Currently, I think nnml just inserts the whole
.overview file into a buffer and then uses search-forward to find
things.  (Or maybe it has a binary search algorithm?  Then it would
be difficult to make it a lot faster.)

But if you set gnus-verbose and gnus-verbose-backends to 10 I think
you will see that fetching headers is not the most time-consuming
part when using Gnus.  There are some cases where header fetching is
slow, but now we have the agent cache which speeds things up a lot.

At least for me, the slow part about using Gnus is the generation of
the summary buffer.  I wonder if that can be made faster somehow.
Is there a way to create a buffer with a lot of text properties and
stuff really quickly?  What happens if we princ (buffer-string) to a
file and then read that representation into memory and use a single
insert call to create a buffer from it?

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)