[9fans] Clearing venti arena during install

[9fans] Clearing venti arena during install

[bakul+plan9]

If fossil+venti is chosen during installation, the entire
venti arena partition is cleared.  Is there a way to avoid
clearing all of the arena space?  Since in a sense it is an
append only log, and the purpose for clearing it is,
presumably, to allow recreating the index, may be a shortcut
exists?  Doing so will save a lot of time and space
particularly on a qemu VM.  In qemu one can use a sparse file
or a qcow format file which allocates space only for data
actually written.  Writing even zeroes can eat up space.

Perhaps it is easier to add a do-not-zero option for install
as a fresh qemu disk is already "cleared".

Of course, it doesn't make much sense to use venti under qemu
but p9p does not run on all the platforms (notably windows).

Re: [9fans] Clearing venti arena during install

[Russ Cox]

> If fossil+venti is chosen during installation, the entire
> venti arena partition is cleared.  Is there a way to avoid
> clearing all of the arena space?  Since in a sense it is an
> append only log, and the purpose for clearing it is,
> presumably, to allow recreating the index, may be a shortcut
> exists?  Doing so will save a lot of time and space
> particularly on a qemu VM.  In qemu one can use a sparse file
> or a qcow format file which allocates space only for data
> actually written.  Writing even zeroes can eat up space.

Any sparse file worth its salt would catch the case
where you write zeros to a hole and not do anything.

That said, I'd rather leave the zeroing in for now.
The new venti, which I am testing on Plan 9 right now,
does not require the zeroing.

Russ

Re: [9fans] Clearing venti arena during install

[bakul+plan9]

> > If fossil+venti is chosen during installation, the entire
> > venti arena partition is cleared.  Is there a way to avoid
> > clearing all of the arena space?  Since in a sense it is an
> > append only log, and the purpose for clearing it is,
> > presumably, to allow recreating the index, may be a shortcut
> > exists?  Doing so will save a lot of time and space
> > particularly on a qemu VM.  In qemu one can use a sparse file
> > or a qcow format file which allocates space only for data
> > actually written.  Writing even zeroes can eat up space.
>
> Any sparse file worth its salt would catch the case
> where you write zeros to a hole and not do anything.

Creating a hole is as easy as seeking beyond the end of a
file and scribbling something but maintaining its holeyness
by checking every write for a block of zeroes is quite
expensive.

> That said, I'd rather leave the zeroing in for now.
> The new venti, which I am testing on Plan 9 right now,
> does not require the zeroing.

This is a much better solution -- I didn't realize this was
imminent.  Thanks!

Re: [9fans] Clearing venti arena during install

[Russ Cox]

> This is a much better solution -- I didn't realize this was
> imminent.  Thanks!

I don't know if I'd say imminent.  But on its way.

Re: [9fans] Clearing venti arena during install

[Russ Cox]

> Creating a hole is as easy as seeking beyond the end of a
> file and scribbling something but maintaining its holeyness
> by checking every write for a block of zeroes is quite
> expensive.

It only needs to be done when writing to a hole,
and for the small cost of a scan over the block,
you get to avoid a few disk writes (more expensive).
It's worth doing.

Aux/disksim does this, specifically so that I can run
tests on enormous files without needing the backing
store.  For example, that's how I checked tar's
recently-added support for files > 4GB.

Russ

Re: [9fans] Clearing venti arena during install

[geoff]

Furthermore, the check for a zero block can be done pretty cheaply (I
thought about doing this years ago for v7 Unix).  First see if the
first word of the buffer is zero; if so, see if the last word is zero;
if so, run through the buffer a word at a time, stopping at a non-zero
word or end of buffer.

In the common (real-life) case, the test of the first word suffices.
The word-at-a-time scan can be made to run fairly fast, especially if
you have the option of first putting a non-zero word just after the
buffer, so that no explicit bounds test is necessary in the loop.

Re: [9fans] Clearing venti arena during install

[Russ Cox]

> In the common (real-life) case, the test of the first word suffices.
> The word-at-a-time scan can be made to run fairly fast, especially if
> you have the option of first putting a non-zero word just after the
> buffer, so that no explicit bounds test is necessary in the loop.

Or temporarily unzeroing the last word in the buffer.

Russ

Re: [9fans] Clearing venti arena during install

[Bakul Shah]

> > In the common (real-life) case, the test of the first word suffices.
> > The word-at-a-time scan can be made to run fairly fast, especially if
> > you have the option of first putting a non-zero word just after the
> > buffer, so that no explicit bounds test is necessary in the loop.
>
> Or temporarily unzeroing the last word in the buffer.

So if you are asked to allocate a disk block only to write a
block of zeroes, don't!  I can dig that!  That fills a hole
in my education.  Thanks guys!

Re: [9fans] Clearing venti arena during install

[Lluís Batlle i Rossell]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

bakul+plan9@bitblocks.com wrote:
> If fossil+venti is chosen during installation, the entire
> venti arena partition is cleared.  Is there a way to avoid
> clearing all of the arena space?  Since in a sense it is an
> append only log, and the purpose for clearing it is,
> presumably, to allow recreating the index, may be a shortcut
> exists?  Doing so will save a lot of time and space
> particularly on a qemu VM.  In qemu one can use a sparse file
> or a qcow format file which allocates space only for data
> actually written.  Writing even zeroes can eat up space.
As I've tested, if you avoid using "qcow" and otherwise use a "raw" qemu
image, the venti formatting is normally quick in my Linux system. Using
'qcow' terribly slows down that format process.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3311 bytes --]