Re: [9fans] Hardware for Plan9

[Venkatesh Srinivas <me@acm.jhu.edu>]

On Mon, Jan 11, 2010 at 10:22:49AM -0500, erik quanstrom wrote:
>>
>> An alternate configuration, which takes more memory, but might offer
>> a bit more in the way of survivability, would be to not use fs for venti. Instead,
>> run one venti daemon per disk, with independent arenas/indexes. Insert a little
>> Venti proxy between fossil and your daemons; it should try reads on each
>> venti until one returns a block; it should issue writes to all of them.
>
>why would that be more survivable?

When an fs mirror is out of sync, which mirror holds the right data? Fs has no
way of knowing. Venti at least has the block hashes.

Imagine cutting power after a first disk in a mirror has data written but
subsequent ones don't? With fs, disaster. (well, sorta. devfs always reads from
the first device in a mirror first, and writes to the devices in order as well.
you might get lucky, but you wouldn't know about errors until you have to hit
the second device. at which point its too late.)

Venti deals with incompletely written blocks; the arenas and index structures
are still workable. The situation is even recoverable - a proxy could notice
that one of the backends failed to return a read, so it rewrite the data from
an other copy (which it can verify) to the failed one.

Also, the isolation granted by writing data to two venti daemons is nicer than
scribbling blocks to both disks; you can bring down either back-end venti while
the system is running. You can even move one of the pairs to a remote system.
If disks are removable in your configuration, you can even grow the available
space live.

-- vs