[9fans] growing/shrinking venti arena files -OR- arena file format

[9fans] growing/shrinking venti arena files -OR- arena file format

[smiley]

Hello,

I'm wondering if anyone can shed some light on growing and/or shrinking
arena files (i.e., disk partitions).  With the growing popularity of
logical volume management, vitrualization, etc., resizing partitions is
becoming more and more common, and many file systems already have
"resize" tools or options to grow/shrink file system structures
according to changes in the size of the underlying device.  I'm
wondering what capacities (if any) venti has for dealing with inceases
or decresases in the size of its arena files.

Yes, I know that the canonical way to add more storage to a venti server
is to format and add an ADDITIONAL arena file with venti/fmtindex -a.
But I'd like to know, specifically, if it's possible to grow/shrink an
arena file without breaking anything.  If an arena file is extended,
will venti automatically make use of the extra space?  Would the
additional arenas need to be formatted?  And, if so, how?  If empty
arenas at the end of an arena file are truncated (on an arena boundary,
ideally), would venti continue to function properly, fill up the
remaining space, and then continue on to the next arena file?  Or, would
venti crash and burn in a pile of flaming bits and bytes?

Another way of stating this question might be to ask for information
about the arena file format.  Knowing the file format, I could deduce
whether any inconsistency would be introduced by a change in file size.
Yes, I read the venti paper, including its descripion of arena files and
their contents.  However, it is rather vague on such points, more like
an academic lecture or advertisement than a technical specification.  Is
there a TECHNICAL SPECIFICATION for the arena file format?  Getting my
eyeballs on one would probably allow me to figure this all out, for
myself.  For that matter, any formal specifications for 9P2000 and the
venti protocol would be very helpful, too (for other purposes).

Any pointers (dereferenced or not :) would be greatly appeciated!

--
+---------------------------------------------------------------+
|   human     <smiley@icebubble.org>    PGP key ID:    BC549F8B |
|Fingerprint: 9329 DB4A 30F5 6EDA D2BA  3489 DAB7 555A BC54 9F8B|
+---------------------------------------------------------------+

Re: [9fans] growing/shrinking venti arena files -OR- arena file format

[lucio]

> Is
> there a TECHNICAL SPECIFICATION for the arena file format?

The source code is eminently readable and the "words" directory (in
/sys/src/cmd/venti) contains quite a bit of details, at a glance.  I'd
say there's enough there to make an educated guess or two.

In my experience, the most sensible way to resize disk capacity for
venti is to back up the arenas and then rewrite them to the new
format.  I didn't expect it when I did it by accident, but it seems
practical, if not particularly fast.  That, of course, may be a good
reason not to mess with arena sizes, it is hard to restore them
without spending a lot of time on them.

++L

Re: [9fans] growing/shrinking venti arena files -OR- arena file format

[erik quanstrom]

> I'm wondering if anyone can shed some light on growing and/or shrinking
> arena files (i.e., disk partitions).  With the growing popularity of
> logical volume management, vitrualization, etc., resizing partitions is
> becoming more and more common, and many file systems already have
> "resize" tools or options to grow/shrink file system structures
> according to changes in the size of the underlying device.  I'm
> wondering what capacities (if any) venti has for dealing with inceases
> or decresases in the size of its arena files.

i'll be honest, i don't understand this statement of the problem.  the
venti abstraction has some answers for this problem.  so i would be disinclined
to look "under the covers" to start.

venti arenas are very small (~650mb) and fill in order.  so taking away empty
arenas should be no problem, while taking away partial arenas will lead
to data loss.  650mb seems like rounding error with today's storage, so
personally i would ignore the problem of resizing an arena.  alternatively,
if one has truely tiny storage, i would be tempted to start with very small
arenas, and still add or remove full arenas.  worst case, i think venti/copy
can be used to recover a deleted last arena to a smaller one.  but the temporary
storage requirements are a bit greater.

- erik

Re: [9fans] growing/shrinking venti arena files -OR- arena file format

[Brian L. Stuart]

> I'm wondering if anyone can shed some light on growing
> and/or shrinking arena files (i.e., disk partitions). 
> With the growing popularity of logical volume management,
> vitrualization, etc., resizing partitions is becoming
> more and more common, and many file systems already
> have "resize" tools or options to grow/shrink file
> system structures according to changes in the size of
> the underlying device.  I'm wondering what capacities
> (if any) venti has for dealing with inceases or
> decresases in the size of its arena files.

To make sure I understand the question I'll attempt
to rephrase it.  If I have a logical volume containing
an arena partition and I resize that volume to a
larger size, is there a way to make venti aware of
the larger partition and format the additional space
as arenas in the same partition?  If I have understood
the question correctly, I don't think any of the existing
utilities will do that.  In principle, it should be
possible, assuming that you don't add so much that
the arena map won't fit into the fixed 512K map that
fmtarenas creates.  Fmtarenas just writes a partition
header and then loops through all the arenas calling
newarena.  You could probably update the partition
header and call newarena for each arena in the new
space.  There are probably some gotchas that I'm overlooking
at the moment, but that would be the place I'd start.

> Yes, I know that the canonical way to add more storage to a
> venti server is to format and add an ADDITIONAL arena file
> with venti/fmtindex -a.

I'd be inclined to think this is easier and safer
than munging around with an existing arena partition.

> Is there a TECHNICAL SPECIFICATION for the arena file
> format?

The source code would the be most accurate spec on the
format.  I'd suggest starting with src/venti/srv/dat.h
and move on to look at fmtarenas.c.  Calls in it will
lead you to the other files that will be interesting.

> For that matter, any formal specifications for 9P2000

Section 5 of the man pages is the best reference for
this.

> venti protocol would be very helpful, too (for other
> purposes).

The sources are the best place to look for this one
too.

BLS