Re: [9fans] fossil/venti/manber

[William Josephson <jkw@eecs.harvard.edu>]

On Mon, Apr 28, 2003 at 05:55:34PM -0400, Russ Cox wrote:
> I think the notes happen to answer your questions, mainly
> because I think they clear up your misunderstandings about
> how applying Manber's idea would actually work.
> If you have questions after reading the notes, I'd be happy
> to address them directly, but I think we're not on the same
> page at the moment.

<snip good discussion of chunking>

For what it is worth, on common datasets (e.g. /usr and /home on Linux
and FreeBSD boxes, the root disk on Windows boxes, and sets containing
a majority of CAD and graphics files), using chunking turns out to be
a significant win compared to fixed size blocks, at least over time.
For the first run (i.e. first write of a set of files) and for small
files, the difference is usually fairly trivial.  The chunking
approach is not as big a win as using content-addressed blocks in the
first place, but still a win.  Whether or not it makes sense to go to
the effort of implementing it for Plan 9 is another question, of
course.  I'm afraid I can't name numbers as I'd like to do, but
suffice it to say that this idea is being pursued commercially....