Re: [9fans] Changelogs & Patches?

[Roman Shaposhnik <rvs@sun.com>]

Hi Andrey!

Sorry, it took me a longer time to dig through the code than
I hoped to. So, if you're still game...

On Jan 6, 2009, at 6:22 AM, andrey mirtchovski wrote:
> i'm using zfs right now for a project storing a few terabytes worth of
> data and vm images.

Is it how it was from the get go, or did you use venti-based solutions
before?

> i have two zfs servers and about 10 pools of
> different sizes with several hundred different zfs filesystems and
> volumes of raw disk exported via iscsi.

What kind of clients are on the other side of iscsi?

> clones play a vital part in the whole set up (they number in the
> thousands).
> for what it's worth, zfs is the best thing in linux-world (sorry,
> solaris and *bsd too)

You're using it on Linux?

>> Fair enough. But YourTextGoesHere then becomes a transient property
>> of my namespace, where in case of ZFS it is truly a tag for a
>> snapshot.
>
> all snapshots have tags: their top-level sha1 score. what i supplied
> was simply a way to translate that to any random text. you don't need
> to, nor do you have to do this (by the way, do you get the irony of
> forcing snapshots to contain the '@' character in their name? sounds a
> lot like '#' to me ;)

Ok. Fair enough. I think I'm convinced on that point.

> snapshots are generally accessible via fossil as a directory with the
> date of the snapshot as its name. this starts making more sense when
> you take into consideration that snapshots are global per fossil, but
> then you can run several fossils without having them step on their
> toes when it comes to venti. at least until you get a collision in
> blocks' hashes.

Aha! And here are my first questions: you say that I can run multiple
fossils
off of the same venti and thus have a setup that is very close to zfs
clones:
    1. how do you do that exactly? fossil -f doesn't work for me (nor
should it
        according to the docs)
    2. how do you work around the fact that each fossil needs its own
         partition (unlike ZFS where all the clones can share the same
pool
         of blocks)?

> venti is write-once. if you instantiate a fossil from a venti score it
> is, by definition, read-only, as all changes to the current fossil
> will not appear to another fossil instantiated from the same venti
> score. changes are committed to venti once you do a fossil snap,
> however that automatically generates a new snapshot score (not
> modifying the old one). it should be clear from the paper.

I think I understand it now (except for the fossil -f part), but how do
you promote (zfs promote) such a clone?

>> where the second choice becomes a nuisance for me is in the case
>> where
> one has thousands of clones and needs to keep track of thousands of
> names in order to ensure that when the right one has finished the
> right clone disappears.

I see what you mean, but in case of venti -- nothing disappears, really.
 From that perspective you can sort of make those zfs clones linger.
The storage consumption won't be any different, right?

>>> - none of this can be done remotely
>>
>> Meaning?
>
> from machine X in the datacentre i want to be able to say "please
> create me a clone of the latest snapshot of this filesystem" without
> having to ssh to the solaris node running zfs.

Well, if its the protocol you don't like -- writing your own daemon
that will respond to such requests sounds like a trivial task
to me.

> i couldn't find the source for libzfs either, without having to
> register to the opensolaris developers' site.
[...]

> and i think i'm using a pretty new version of zfs and my experiences
> are, in fact, quite recent :)

well, the fact that you had to register in order to access the code
suggest a pretty dated experience ;-)
     http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/lib/libzfs/

> instead of reverse engineering a library that i have not much faith
> in, i wrote a python 9p server that uses local zfs/zpool commands to
> do what i could've done with C and libzfs. it's a hack but it gets the
> job done. now i can access block X of zfs volume Y remotely via 9p (at
> one third the speed, to be fair).

Well, Solaris desperately wanted to enter the Open Source geekdom
and from your experience it seems like it was a success ;-) Seriously
though, I personally found reading source code of zdb to be
absolutely illuminating about all sorts of things ZFS:
     http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/cmd/zdb/zdb.c

But yes -- just like with any unruly OS project you have to really
invest
your time if you want to tag along. I think it was Russ who made a
comment
that the Free Software is only free if your time has no value :-(

> i would be glad to help you understand the differences between zfs and
> fossil/venti with my limited knowledge of both.

Great! I tired to do as much homework as possible (hence the delay) but
I still have some questions left:
     0. A dumb one: what's the proper way of cleanly shutting down
fossil
     and venti?

    1. What's the use of copying arenas to CD/DVD? Is it purely back up,
     since they have to stay on-line forever?

    2. Would fossil/venti notice silent data corruptions in blocks?

    3. Do you think its a good idea to have volume management be
    part of filesystems, since that way you can try to heal the data
    on-the-fly?

    4. If I have a venti server and a bunch of sha1 codes, can I somehow
    instantiate a single fossil serving all of them under /archive?


Thanks,
Roman.