Re: [9fans] Streaming on venti

[Enrico Weigelt <weigelt@metux.de>]

* Russ Cox <rsc@swtch.com> wrote:
> > VAC eg. is good for archiving, but it's tree-based structure
> > is probably not optimal for streaming (on large files, a lot
> > of blocks IMHO have to be loaded before getting the first
> > payload block can be reached).
>
> A typical venti tree has a branching factor of 409 (8192/20).

I gues, 8k is vac's index block size ?
So, maybe it could even be improved (for my case) by increasing
it to the 56k venti limit ?

> For a 1GB file, that means you have to load two extra blocks to find
> the first one, and 322 interior blocks to find all 131,072 data blocks.
> Is improving that 0.2% really your justification for a less capable
> data structure?

Well, I'll have to think about this. My primary goal is to
make the sequential read as fast as possible. There won't be
any non-sequential access.

> Venti's performance is dominated much more by fragmentation
> in where the blocks are laid out in the arena logs (that causes seeks)
> than anything in higher level data structures.  There is a paper about
> this in the upcoming Usenix.  See http://swtch.com/~rsc/papers/
> for a link to PDF and HTML.  (Because the paper is targeted at a
> non-Plan 9 audience, "Venti" in that paper refers to venti as described
> in the original paper.  The current venti sources implement all the
> improvements described as "Foundation" in the paper.)

thx, I'll have a look at this (as soon as time allows ;-o).


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 cellphone: +49 174 7066481   email: info@metux.de   skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------