[9fans] Update on Fossil+Venti Stuff

[9fans] Update on Fossil+Venti Stuff

[Devon H. O'Dell]

I think everybody is a little confused about Fossil and Venti
interaction. When I started this project, it was under the idea that
Fossil was essentially a cache. It stored stuff, and when it either
got too full or something else, it would start writing files out to
Venti.

As I've been copying files over to my machine, I've noticed Fossil get
fuller and fuller, but not sync anything over to Venti. Upon
attempting to find information about this, we came up with:

http://www.mail-archive.com/9fans@cse.psu.edu/msg03210.html

Bruce doesn't sound too happy here, and since Plan 9 doesn't have any
version control or any commit logs (that I'm aware of), I can't tell
whether any of these bugs have been fixed. Or, if so, whether they've
been released. As my Fossil fills from 15 to 30 GB out of 35GB (with
180GB in my current Venti arena), I don't want to take this chance.

My ideal usage of Fossil+Venti was to use it as expandable, persistent
storage for my music. I'm finding out that this idea won't work
because Venti exists as expandable archival storage. Which implies
that files have to be deleted from Fossil before they'll end up in
Venti. This would be fine with me, however, it seems that, by doing
this, there's no way to get all of my mp3s listed in a single
directory -- they would be archived by snapshot date / number, and I'd
have to search through all of those to get a full list of music. This
isn't really what I want.

I could have probably saved myself a lot of suffering had I realized
this sooner. Though, I just seem to recall a couple people doing this
type of set-up, so I wonder what I'm doing wrong.

Either way, I rarely delete files, and that seems to be the only way
to get them into Venti. Or maybe I'm just missing things altogether.

If there is a way to do what I'm wanting with Venti and Fossil, I'd
love to see it. This isn't really a bitching and moaning and whining
session, because this realization is probably more my fault for not
researching enough, but regardless, I'd like to see something that can
do what I'm looking for in the future, whether I have to write it
myself or not. I guess this is sort of a probe as to how many people
would find this useful versus not (versus already possible and I'm
just dense).

--Devon

Re: [9fans] Update on Fossil+Venti Stuff

[Martin Neubauer]

Fossil is more like a write _buffer_ for venti. The interaction with venti
takes place during the nightly dumps (at 5am for a default install; can be
configured according to your needs). Fossil also regularly takes snapshots
that aren't archived to venti. (Again, for a default setup snapshots every
hour, kept for 8 days; can be configured otherwise.) The snapshots are taken
for files that changed since the last snaptime; dumps are taken from files
where the current epoch is greater than the last written to venti. Those
procedures are quite well documented.

The problems seem to arise when you write lots of new data between dumps
(more than the fossil size; can be less when taking into account
intermediate snapshots). I've been thinking about ways to cope with that
lately, but wasn't too eager to post here until I'd actually have time 
to really delve into it (next month, or so). At the moment those are little
more than some elaborations of possibilities already mentioned in the
archives. I could do a write-up of what I could think of so far later
tonight when I'm back home if there is wider interest in exploring possible
enhancements (I'm not thinking of those asfixes) to fossil. I think it
could be worthwhile, though.

	Martin

P.S. I'm sure one could describe the current workings in more detail, but
I'm too much in a hurry now.

Re: [9fans] Update on Fossil+Venti Stuff

[erik quanstrom]

On Wed Mar 21 16:24:59 EDT 2007, devon.odell@gmail.com wrote:
> I think everybody is a little confused about Fossil and Venti
> interaction. When I started this project, it was under the idea that
> Fossil was essentially a cache. It stored stuff, and when it either
> got too full or something else, it would start writing files out to
> Venti.

that "something else" is a snapshot archive which forces a flush to venti.
read the section on setting up snapshots (expecially the -a option) of
fossilcons(8).  also, make sure fossil is really talking to venti.

> 
> As I've been copying files over to my machine, I've noticed Fossil get
> fuller and fuller, but not sync anything over to Venti. Upon
> attempting to find information about this, we came up with:
> 
> http://www.mail-archive.com/9fans@cse.psu.edu/msg03210.html
> 
> Bruce doesn't sound too happy here, 

if fossil fills between archival snapshots you are hosed.  to the best of 
my knowledge, fossil still behaves this way.

> and since Plan 9 doesn't have any
> version control or any commit logs (that I'm aware of), I can't tell
> whether any of these bugs have been fixed. Or, if so, whether they've
> been released. 

while this is prima facia correct, sources keeps a archive of the sources
tree for every day.  after "9fs sourcesdump; cd /n/sourcesdump/2005/0612"
would put you at the root of sources as it existed on 12 jun 2005.
if you want to see the changes in a particular file on sources over time
you can "9fs sources; cd /n/sources/plan9/sys/src/cmd/fossil; history 9srv.c".
you will get

Jan 28 11:32:21 EST 2006 9srv.c 3956 [rsc]
Jan 28 11:32:21 EST 2006 /n/sourcesdump/2007/0321/plan9/sys/src/cmd/fossil/9srv.c 3956 [rsc]
Feb 27 10:39:06 EST 2004 /n/sourcesdump/2006/0128/plan9/sys/src/cmd/fossil/9srv.c 3879 [jmk]
Oct 13 22:21:37 EDT 2003 /n/sourcesdump/2004/0227/plan9/sys/src/cmd/fossil/9srv.c 3682 [jmk]
Jun 15 15:02:14 EDT 2003 /n/sourcesdump/2003/1013/plan9/sys/src/cmd/fossil/9srv.c 3589 [rsc]
Feb 18 15:26:58 EST 2003 /n/sourcesdump/2003/0615/plan9/sys/src/cmd/fossil/9srv.c 3291 [rsc]
Jan  8 00:58:24 EST 2003 /n/sourcesdump/2003/0218/plan9/sys/src/cmd/fossil/9srv.c 3215 [rsc]

all patches applied from external sources are also recorded in /n/sources/patch/applied.
all pending external patches are in /n/sources/patch.  internal labs changes
don't have a patch directory associated with them, but history will still show the
changes.

- erik

Re: [9fans] Update on Fossil+Venti Stuff

[Bakul Shah]

Sounds to me like you want fossil to be a true cache --
lazily writing to venti as more space is needed (or at
regular intervals) and reading back from venti if something
is needed and not in cache.

I wonder if doing so would simplify fossil....  its size
would become a function of peformance (to control data
spill/fill rates).  And one would have to always use
fossil+venti and never just fossil.  Not to mention venti
then becomes the true file server and one can imagine
building other front end cache filesystems.

>        This would be fine with me, however, it seems that, by doing
> this, there's no way to get all of my mp3s listed in a single
> directory -- they would be archived by snapshot date / number, and I'd
> have to search through all of those to get a full list of music.

You can always bind(1) all the snapshots under your music
directory!

Re: [9fans] Update on Fossil+Venti Stuff

[Richard Miller]

Devon - don't panic ;)

> that files have to be deleted from Fossil before they'll end up in
> Venti.

No, it's the other way round.  A file's blocks can be deleted from
fossil (automatically) after they have been archived to Venti,
provided they haven't been modified since.  The file's metadata
will remain on fossil.

The caveat is that blocks are archived to venti only when a snapshot
is made (either periodically as scheduled by the snaptime command
in fossilcons(8), or on request by the 'snap -a' command).  So if
the fossil partition fills with dirty (new or modified) blocks in
between snapshots, bad things happen.

Say you have a 4GB fossil partition.  If you have snaptime set to
take archival snapshots once a day, you can go on copying new mp3s
to your fossil fs, without deleting anything, provided you never copy
in more than 4GB per day.  If you're impatient, you can copy 4GB,
do a 'snap -a', copy another 4GB and so on.  (The copying of snapshot
blocks will go on in the background, so the 'snap' only freezes the
fs very briefly.)

If your fossil is getting "fuller and fuller", are you sure you
have set it up to take periodic snapshots?  You can check like this:

% con -l /srv/fscons
prompt: fsys main snaptime
    snaptime -a 0000 -s 60 -t 1440

-- Richard

Re: [9fans] Update on Fossil+Venti Stuff

[Devon H. O'Dell]

2007/3/22, Richard Miller <9fans@hamnavoe.com>:
> Devon - don't panic ;)

Trying not to. I'm just really confused, and the sentiment seems to be
``WTF is wrong with you, it's plainly obvious how this works.'' But
from what everyone says, it's not working that way for me.

> The caveat is that blocks are archived to venti only when a snapshot
> is made (either periodically as scheduled by the snaptime command
> in fossilcons(8), or on request by the 'snap -a' command).  So if
> the fossil partition fills with dirty (new or modified) blocks in
> between snapshots, bad things happen.
>
> Say you have a 4GB fossil partition.  If you have snaptime set to
> take archival snapshots once a day, you can go on copying new mp3s
> to your fossil fs, without deleting anything, provided you never copy
> in more than 4GB per day.  If you're impatient, you can copy 4GB,
> do a 'snap -a', copy another 4GB and so on.  (The copying of snapshot
> blocks will go on in the background, so the 'snap' only freezes the
> fs very briefly.)

I'm confused then. When I type snap -a, and then run fsys main df in
fossilcons, I'm only noticing the usage going down by about 2GB/day.
If I run snap -a, wait a bit, and then run fsys main df, I notice no
change in disk usage.

> If your fossil is getting "fuller and fuller", are you sure you
> have set it up to take periodic snapshots?  You can check like this:
>
> % con -l /srv/fscons
> prompt: fsys main snaptime
>     snaptime -a 0000 -s 60 -t 1440

Yep:

10.0.0.10# con -l /srv/fscons
prompt: fsys main snaptime
    snaptime -a 0500 -s 60 -t 2880

I should note that I've manually modified this to happen a few times a
day. Each time, my usage goes down a couple gigs. Why doesn't it just
go ahead and sync everything?

--dho

> -- Richard
>
>

Re: [9fans] Update on Fossil+Venti Stuff

[Martin Neubauer]

* Devon H. O'Dell (devon.odell@gmail.com) wrote:
> 10.0.0.10# con -l /srv/fscons
> prompt: fsys main snaptime
>    snaptime -a 0500 -s 60 -t 2880
> 
> I should note that I've manually modified this to happen a few times a
> day. Each time, my usage goes down a couple gigs. Why doesn't it just
> go ahead and sync everything?

It does sync everything active. There are probably still some intermediate
snapshots there that are not written to venti. Those stay there until they
expire after 2 days (2880 minutes). You can always do an explicit snapclean
or try lowering the time snapshots are kept.

	Martin

Re: [9fans] Update on Fossil+Venti Stuff

[Devon H. O'Dell]

2007/3/23, Martin Neubauer <m.ne@gmx.net>:
> * Devon H. O'Dell (devon.odell@gmail.com) wrote:
> > 10.0.0.10# con -l /srv/fscons
> > prompt: fsys main snaptime
> >    snaptime -a 0500 -s 60 -t 2880
> >
> > I should note that I've manually modified this to happen a few times a
> > day. Each time, my usage goes down a couple gigs. Why doesn't it just
> > go ahead and sync everything?
>
> It does sync everything active. There are probably still some intermediate
> snapshots there that are not written to venti. Those stay there until they
> expire after 2 days (2880 minutes). You can always do an explicit snapclean
> or try lowering the time snapshots are kept.

I'm not disagreeing that this is supposed to happen, and I hate saying
it's wrong, but it's not what I'm seeing, and I don't see anything
invalid in my configuration. I can snap -a, and it won't go to venti.
I ran snapclean and nothing went down. After that, I changed the
snaptime so that it would run a Venti dump in 5 minutes. 10 minutes
later, I see no difference.

So I'm either missing something fundamental here, something's really
broken, or I'm just dumb. I'm willing to concede the latter of the 3,
but it would be really nice to be enlightened. Is there anything I can
show you guys to help diagnose the issue further?

>         Martin

-dho

Re: [9fans] Update on Fossil+Venti Stuff

[Martin Neubauer]

* Devon H. O'Dell (devon.odell@gmail.com) wrote:
> 2007/3/23, Martin Neubauer <m.ne@gmx.net>:
> >* Devon H. O'Dell (devon.odell@gmail.com) wrote:
> >> 10.0.0.10# con -l /srv/fscons
> >> prompt: fsys main snaptime
> >>    snaptime -a 0500 -s 60 -t 2880
> >>
> >> I should note that I've manually modified this to happen a few times a
> >> day. Each time, my usage goes down a couple gigs. Why doesn't it just
> >> go ahead and sync everything?
> >
> >It does sync everything active. There are probably still some intermediate
> >snapshots there that are not written to venti. Those stay there until they
> >expire after 2 days (2880 minutes). You can always do an explicit snapclean
> >or try lowering the time snapshots are kept.
> 
> I'm not disagreeing that this is supposed to happen, and I hate saying
> it's wrong, but it's not what I'm seeing, and I don't see anything
> invalid in my configuration. I can snap -a, and it won't go to venti.
> I ran snapclean and nothing went down. After that, I changed the
> snaptime so that it would run a Venti dump in 5 minutes. 10 minutes
> later, I see no difference.

For snaptime, -a means ``time when to take archival dump.'' So you probably
just made fossil dump 5 minutes after midnight. -s sets the interval (in
minutes) between intermediate snapshots, -t the time (in minutes) after
which old snapshots are discarded. If a snapshot of a file is referenced as
the current version but isn't yet written to venti it is kept until the next
dump.

> 
> So I'm either missing something fundamental here, something's really
> broken, or I'm just dumb. I'm willing to concede the latter of the 3,
> but it would be really nice to be enlightened. Is there anything I can
> show you guys to help diagnose the issue further?

If it helps, it took some time for me to get a grasp of the workings, too.
(And I'm sure there are still plenty of pitfalls for me to run into...)

	Martin

Re: [9fans] Update on Fossil+Venti Stuff

[David Leimbach]

> If it helps, it took some time for me to get a grasp of the workings, too.
> (And I'm sure there are still plenty of pitfalls for me to run into...)

Each new user finds their own sharp edges to bump into I guess, the
wiki and documentation, as well as this mailing list, serve the
purpose of adding a bit of padding so it's a bit more comfortable from
time to time, but no one is perfect, and people don't like to repeat
themselves once it's been written.

I know I appreciate all the help I've gotten with various issues here
over the years.  Sometimes I could have done more to self-educate,
other times I'm just confused, and sometimes I've felt as frustrated
as Devon with the whole thing.

But for some silly reason I keep coming back to Plan 9, and keep
re-loading it on *something*.   Even when I only have time to boot
that machine maybe 1 time a week to play around with it, it's still
interesting :-)

Re: [9fans] Update on Fossil+Venti Stuff

[Devon H. O'Dell]

2007/3/23, David Leimbach <leimy2k@gmail.com>:
> > If it helps, it took some time for me to get a grasp of the workings, too.
> > (And I'm sure there are still plenty of pitfalls for me to run into...)
>
> Each new user finds their own sharp edges to bump into I guess, the
> wiki and documentation, as well as this mailing list, serve the
> purpose of adding a bit of padding so it's a bit more comfortable from
> time to time, but no one is perfect, and people don't like to repeat
> themselves once it's been written.
>
> I know I appreciate all the help I've gotten with various issues here
> over the years.  Sometimes I could have done more to self-educate,
> other times I'm just confused, and sometimes I've felt as frustrated
> as Devon with the whole thing.

Yeah, and I still haven't found the issue. Contrary to Martin's
suggestion, I did actually set the -a time correctly to N minutes
after the current system time, though I guess I didn't imply this.
This morning I'm still at 14GB. I mean, I should note that I'm not
actually removing any files or anything, and I still have the 25GB of
MP3s that were on there at the beginning. So. Still confused and
frustrated.

> But for some silly reason I keep coming back to Plan 9, and keep
> re-loading it on *something*.   Even when I only have time to boot
> that machine maybe 1 time a week to play around with it, it's still
> interesting :-)

I'll give you that. It'll be nice when I figure out what it is and it
gets fixed. But until then, *grrrr*

--dho

Re: [9fans] Update on Fossil+Venti Stuff

[Richard Miller]

Devon -

> This morning I'm still at 14GB. I mean, I should note that I'm not
> actually removing any files or anything, and I still have the 25GB of
> MP3s that were on there at the beginning. So. Still confused and
> frustrated.

Why are you expecting your fossil partition to become more empty?
It's a cache - so even when a block has been copied to venti, it
is still useful to keep a copy in fossil as well, in case it's referenced
again.  It will be marked 'clean', so that it can be evicted from the
cache instantly if fossil runs out of space.

Are you still worried that blocks aren't being copied to venti at all?
You can check by looking at the archive score which is printed
on fossilcons right after a 'snap -a' is done.  You can use vacfs
to mount this score as a fs, and then explore that to see if everything
is there.

If you *really* want to empty your fossil partition, you can reformat
it immediately after a 'snap -a', using the '-v score' option with the
score of the snap you just made.  (Take the usual precautions first,,
of course.)

-- Richard

Re: [9fans] Update on Fossil+Venti Stuff

[Devon H. O'Dell]

2007/3/23, Richard Miller <9fans@hamnavoe.com>:
> Devon -
>
> > This morning I'm still at 14GB. I mean, I should note that I'm not
> > actually removing any files or anything, and I still have the 25GB of
> > MP3s that were on there at the beginning. So. Still confused and
> > frustrated.
>
> Why are you expecting your fossil partition to become more empty?
> It's a cache - so even when a block has been copied to venti, it
> is still useful to keep a copy in fossil as well, in case it's referenced
> again.  It will be marked 'clean', so that it can be evicted from the
> cache instantly if fossil runs out of space.
>
> Are you still worried that blocks aren't being copied to venti at all?
> You can check by looking at the archive score which is printed
> on fossilcons right after a 'snap -a' is done.  You can use vacfs
> to mount this score as a fs, and then explore that to see if everything
> is there.

I'm not sure what you're referring to here -- when I type snap -a, I
get no output back.

> If you *really* want to empty your fossil partition, you can reformat
> it immediately after a 'snap -a', using the '-v score' option with the
> score of the snap you just made.  (Take the usual precautions first,,
> of course.)

I'll read the fossilcons manpage on this again and see what the usage
of this is. I guess I am somewhat worried, simply because I don't want
to fill my fossil partition (I still have > 35GB data to copy over)
and have it blow up.

At this point, I think I've probably munged what's in Venti enough
that I'm going to go ahead and reinstall with this `newfound'
knowledge.

> -- Richard

Thanks for the information -- I'll definitely try it out this time.

--dho

Re: [9fans] Update on Fossil+Venti Stuff

[Richard Miller]

> I'm not sure what you're referring to here -- when I type snap -a, I
> get no output back.

Wait a while before disconnecting from fscons.  It prints the score
eventually after it finishes flushing blocks to disk.  

% con -l /srv/fscons
prompt: fsys main
main: snap -a
main: archive vac:adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
main: >>> q
%

(actual score edited for security reasons!)

Re: [9fans] Update on Fossil+Venti Stuff

[Richard Miller]

> I'll read the fossilcons manpage

You might want to read /sys/doc/fossil.pdf as well.

Re: [9fans] Update on Fossil+Venti Stuff

[Devon H. O'Dell]

2007/3/23, Richard Miller <9fans@hamnavoe.com>:
> > I'm not sure what you're referring to here -- when I type snap -a, I
> > get no output back.
>
> Wait a while before disconnecting from fscons.  It prints the score
> eventually after it finishes flushing blocks to disk.
>
> % con -l /srv/fscons
> prompt: fsys main
> main: snap -a
> main: archive vac:adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
> main: >>> q
> %
>
> (actual score edited for security reasons!)

Aha! Ok. Thanks!

--dho

Re: [9fans] Update on Fossil+Venti Stuff

[Steve Simon]

> (actual score edited for security reasons!)

Dam, and there was me ready to extrapolate your source tree, your bank
account and shoe size from that score - Instead I seem to have ended up
with the music collection of somone from the planet zarg...

-Steve