[9fans] venti arenas

[9fans] venti arenas

[nigel]

If they don't fit exactly into the arenas file, according to the
storage stats, the last arena is bigger than the requested
arena size.

Please tell me that only the first 'arenasize' bytes are used,
otherwise I have to set arena size to just less than half
the size of my backup media...

Re: [9fans] venti arenas

[nigel]

[-- Attachment #1: Type: text/plain, Size: 969 bytes --]

I'll answer my own question.

It is true; the last arena takes the remaining space. This is not good if
the intention is to dump to fixed size removable media, which is the
whole reason for the arenas in the first place. I know that if the last
arena is filling, then it's time to get a new disk drive, but it would be
nice to be able to dump the last arena before getting a new disk.

This is the code in fmtarenas..

		if(limit >= ap->size || ap->size - limit < MinArenaSize){
			limit = ap->size;
			if(limit - addr < MinArenaSize)
				fatal("bad arena set math: runt arena at %lld,%lld %lld\n", addr, limit, ap->size);
		}

Now I think that should be

		if(limit > ap->size || ap->size - limit < MinArenaSize){
			limit = ap->size;
			if(limit - addr < MinArenaSize)
				fatal("bad arena set math: runt arena at %lld,%lld %lld\n", addr, limit, ap->size);
		}

anyway.

I'd suggest replacing this with

		if(limit > ap->size)
			break;

[-- Attachment #2: Type: message/rfc822, Size: 1454 bytes --]

From: nigel@9fs.org
To: 9fans@cse.psu.edu
Subject: [9fans] venti arenas
Date: Wed, 30 Apr 2003 08:22:07 +0100
Message-ID: <6562c935043ced09a324b5835ed1de06@9fs.org>

If they don't fit exactly into the arenas file, according to the
storage stats, the last arena is bigger than the requested
arena size.

Please tell me that only the first 'arenasize' bytes are used,
otherwise I have to set arena size to just less than half
the size of my backup media...

Re: [9fans] venti arenas

[Richard Miller]

> If they don't fit exactly into the arenas file, according to the
> storage stats, the last arena is bigger than the requested
> arena size.
>
> Please tell me that only the first 'arenasize' bytes are used,
> otherwise I have to set arena size to just less than half
> the size of my backup media...

I think the last arena can only exceed arenasize by up to MinArenaSize
bytes, which is 1 megabyte.  So you only have to set arenasize to
1MB less than the size of your backup media.

Re: [9fans] venti arenas

[Lucio De Re]

On Wed, Apr 30, 2003 at 10:03:21AM +0100, Richard Miller wrote:
>
> I think the last arena can only exceed arenasize by up to MinArenaSize
> bytes, which is 1 megabyte.  So you only have to set arenasize to
> 1MB less than the size of your backup media.

That doesn't make a great deal of sense.  If it is going to be limited
to an extra megabyte, why not keep it the exact size and throw the
remainder away?  What does the extra megabyte buy you?

++L

Re: [9fans] venti arenas

[Boyd Roberts]

From: "Lucio De Re" <lucio@proxima.alt.za>
> That doesn't make a great deal of sense.  If it is going to be limited
> to an extra megabyte, why not keep it the exact size and throw the
> remainder away?  What does the extra megabyte buy you?

yup:

    Always go for overkill -- Lou Reed

--
The Beretta 92FS -- don't leave home without it ...

Re: [9fans] venti arenas

[nigel]

[-- Attachment #1: Type: text/plain, Size: 270 bytes --]

OK. I can see that the max-oversize is 1Mb, and not 2 * 'arenasize',
but that 1Mb is a constant embedded in the code.

Externally, I can't tell how to choose my arena size.

Surely that's untidy?

Unless, of course, there's a reason to have 1Mb free at the end.

[-- Attachment #2: Type: message/rfc822, Size: 1757 bytes --]

From: Richard Miller <miller@hamnavoe.demon.co.uk>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] venti arenas
Date: Wed, 30 Apr 2003 10:03:21 +0100
Message-ID: <6641ba56fe369dcba9498b55574627c9@hamnavoe.demon.co.uk>

> If they don't fit exactly into the arenas file, according to the
> storage stats, the last arena is bigger than the requested
> arena size.
>
> Please tell me that only the first 'arenasize' bytes are used,
> otherwise I have to set arena size to just less than half
> the size of my backup media...

I think the last arena can only exceed arenasize by up to MinArenaSize
bytes, which is 1 megabyte.  So you only have to set arenasize to
1MB less than the size of your backup media.

Re: [9fans] venti arenas

[rsc]

Jmk and I think the solution is this:

	apsize = ap->size - ap->arenaBase;
	n = apsize / asize;

	fprint(2, "configuring %s with arenas=%d for a total storage of bytes=%lld and directory bytes=%d\n",
		file, n, apsize, ap->tabSize);

	ap->narenas = n;
	ap->map = MKNZ(AMap, n);
	ap->arenas = MKNZ(Arena*, n);

	addr = ap->arenaBase;
	for(i = 0; i < n; i++){
		limit = addr + asize;
		snprint(aname, ANameSize, "%s%d", name, i);
		fprint(2, "adding arena %s at [%lld,%lld)\n", aname, addr, limit);

Even on a small system there are 200+ arenas.
Another half-sized one isn't going to help much.

Russ

[9fans] venti arenas

[nigel]

Forgive me for being a venti newcomer...

I've just read out an arena, which is about 20% full.
I was expecting the last 80% of the image to be null,
but it isn't. The last ~16k of the image contains some
data. If I pop it through strings, the name of the
arena appears amongst other less meaningful stuff.

Is this data important, or an artifact?

Re: [9fans] venti arenas

[Martin C.Atkins]

On Tue, 1 Apr 2003 07:34:18 +0100 nigel@9fs.org wrote:
> Forgive me for being a venti newcomer...
>
> I've just read out an arena, which is about 20% full.
> I was expecting the last 80% of the image to be null,
> but it isn't. The last ~16k of the image contains some
> data. If I pop it through strings, the name of the
> arena appears amongst other less meaningful stuff.
>
> Is this data important, or an artifact?
>
>
Hi Nigel,

I thought (from the venti paper) that copies of the block headers
were written to the end of each arena (starting from the end and
working back). This gives more robustness, and speeds up rebuilding
the index, though without trying it, I wonder how great the second
benefit really is. (But maybe it's an example of where thought
experiments don't match reality :-)

Martin
--
Martin C. Atkins				martin@mca-ltd.com
Mission Critical Applications Ltd, U.K.		http://www.mca-ltd.com