Re: nndb mailing list?

[bmiller@cs.umn.edu (Brad Miller)]

>>>>> "Dave" == Dave Blacka <dblacka@fuentez.com> writes:

>>>>> "Joe" == Joe Hildebrand <hildjj@fuentez.com> writes:
Joe> Also BTW, we did some more testing on the differences between gdbm and
Joe> berkeley db.  Gdbm wins *big* on disk space (like one less digit in
Joe> the bytes column), and is not noticeably slower.  It will probably be
Joe> the default in 0.8.  We'll also include a script to convert one to the
Joe> other.

Dave> Ok.  Joe lied here a bit.  GDBM does actually win big on disk space:
Dave> Berkeley DB isn't really very space efficient when the data you are
Dave> storing has widely different sizes (like mail bodies), so the Berkely
Dave> DB file grows to enormous size.  For instance, I probably have at most
Dave> 1 Meg of mail stored in a 7 MB mail.db file.  GDBM gets you much
Dave> closer to the actual size of the data.  However, GDBM and Berkeley DB
Dave> only have comparable speeds when you are hitting a local disk.  In my
Dave> testing so far, GDBM suffers from incredible performance problems over
Dave> NFS, while Berkeley DB behaves about the same.  Thus, if you are
Dave> storing your mail.db file on a local disk, switching over to GDBM is
Dave> probably a real good idea, otherwise it would probably be best to wait
Dave> until Joe and I rework the whole backend of nndb.

The GroupLens Better Bit Bureau uses gdbm as its database, and I can
vouch for the fact that performance over NFS is really really bad.
Also, if you happen to get into a situation where you are frequently
updating the data associated with one key, the database will start to
grow all out of proportion.  The only solution to this seems to be to
call gdbm_reorganize, but that is terribly slow.  I've had some database
files go from 46Meg back down to 8Meg with one call to reorg!

\Brad
----------------------------------------------------------------------------
Brad Miller                    | e-mail: bmiller@cs.umn.edu
University of Minnesota        | phone:  (612) 626-8396     
Department of Computer Science | www:    http://www.cs.umn.edu/~bmiller
EE/CS 5-244                    | 
----------------------------------------------------------------------------
**  Save USENET Try GroupLens: http://www.cs.umn.edu/Research/GroupLens   **
----------------------------------------------------------------------------