[9fans] FS choice

[9fans] FS choice

[Ethan Grammatikidis]

Is Fossil (with Venti) the only filesystem which provides daily
snapshots? I've been meaning to ask this for some time.

--
Simplicity does not precede complexity, but follows it. -- Alan Perlis

Re: [9fans] FS choice

[erik quanstrom]

On Fri Apr 16 08:35:29 EDT 2010, eekee57@fastmail.fm wrote:
> Is Fossil (with Venti) the only filesystem which provides daily
> snapshots? I've been meaning to ask this for some time.

cwfs/kenfs provide daily snapshots, which they call dumps.
see /sys/doc/fs/fs.ps

- erik

Re: [9fans] FS choice

[cinap_lenrek]

[-- Attachment #1: Type: text/plain, Size: 2223 bytes --]

if you want to use cwfs as your boot filesystem.  i made the program
arguments in a way compatible to kfs so you can put it in your kernel
image as kfs and boot from it.  it also doesnt need special
boot/local.c and a loopback network to mount it anymore.  for
convinience, i hacked the device string parser to understand file
names as file mapped wren devices so you can directly specify the
plan9 partition path of your configuration device without needing a
map file:

root is from: local!/dev/sdC0/w0

for configuration mode type:

root is from: local!/dev/sdC0/w0 -c

(-c previously ment "use new cache format"... this is reassigned to -C)

my configuration on the thinkpad looks like this:

service cwfs.rei2
filsys main c(/dev/sdC0/w1)f(/dev/sdC0/w0)
filsys dump o
filsys other /dev/sdC0/w2

# w0 - config and fakeworm for main
# w1 - main cache
# w2 - other

alternative syntax for file mapped wren devices is:

w"/foo/bar/baz"

your kernel configuration shoud look like this:

boot
	tcp
	local

bootdir
	bootpcc.out boot
	/386/bin/ip/ipconfig
	/386/bin/auth/factotum
	/386/bin/cwfs kfs

the disadvantage is using cwfs vs fossil is that you cant have
themporary snapshots and here is no temporary flag (chmod -t) per
file.  so you have to create your users /tmp in the "other" fs wich
does not get dumped.

/lib/namespace:

# other
mount -qC /srv/boot /n/other other

bind -qc /n/other/usr/$user/tmp /usr/$user/tmp
bind -c /usr/$user/tmp /tmp

also, kenfs/cwfs does a copy on write when changing a already dumped
block in your main filesystem, but new dumped blocks are not
dedublicated by content like in fossil/venti. this means that your worm
may fill up faster than your venti arenas would.

filesystem limitations like filename length and maximum file size is
fixed once you created your filesystem.  the maximum filename length
is smaller than fossils, so you may want change that in the
configuration and recompile before creating your filesystem.

but on the good side kenfs/cwfs is a lot simpler in its design and
implementation than fossil/venti.  its working pretty robust for me
since i switched.  no bad surprises yet.

--
cinap

[-- Attachment #2: Type: message/rfc822, Size: 3385 bytes --]

From: Ethan Grammatikidis <eekee57@fastmail.fm>
To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net>
Subject: [9fans] FS choice
Date: Fri, 16 Apr 2010 13:34:33 +0100
Message-ID: <8B818FCD-3694-4475-B287-7161EB6FCAB3@fastmail.fm>

Is Fossil (with Venti) the only filesystem which provides daily
snapshots? I've been meaning to ask this for some time.

--
Simplicity does not precede complexity, but follows it. -- Alan Perlis

Re: [9fans] FS choice

[erik quanstrom]

> if you want to use cwfs as your boot filesystem.

could you send me the diffs offline.  this may be a
good solution for laptops.  kfs is nice, but history
is a lot to give up.  venti+fossil seems too dangerous.

> the disadvantage is using cwfs vs fossil is that you cant have
> themporary snapshots and here is no temporary flag (chmod -t) per
> file.  so you have to create your users /tmp in the "other" fs wich
> does not get dumped.

using ken's fs directly, this has not been a problem
at coraid.  we have 75 users on the worm and have
been doing our best to fill it up since 2004 and yet
have <500gb on the worm.  80% of the worm is
email from before the migration to nupas.  10%
is log files (we never delete or rotate them). the
balance is Actual Work.

in fact, the real advantage of venti is the ability to
back up non-plan 9 systems with no native dump
reasonablly efficiently.  unfortunately this advantage
costs a fair bit of complexity.

> filesystem limitations like filename length and maximum file size is
> fixed once you created your filesystem.  the maximum filename length
> is smaller than fossils, so you may want change that in the
> configuration and recompile before creating your filesystem.

we've set our filename length to 56 characters.
we don't allow spaces.  neither of these limitations
has been a problem yet.  even in hosting apt-get
repos and other stuff.

the maximum filesize is now argued in the moot court.
the minium maximum file size (4k blocks) is ~275TB.
i just don't see that as being a problem as that's 68750x
bigger than our largest file at coraid.  (/sys/log/ipboot)
we use 8k blocks, so we run the risk of having trouble in
6*2*68750 years.  i suppose by then i'll just break down
and use 16k blocks.

supposing you have made a mistake (seems unlikely) or
an old 32-bit fs, you can use the history mechanism
outlined in “A Portable History,”
http://www.quanstro.net/plan9/history.pdf.  i have used
this to migrate from 32-bit to 64-bit filesystems twice.
and once to migrate between compatable systems.

i can send anyone interested all the scripts.

in fact, i've had some trouble with venti and large
storage arrays (like 10-20TB).  venti seems to get a
little confused in the set up of various filters and
caches.

- erik