[9fans] xen port

[9fans] xen port

[ron minnich]

Well, I had some time off but I'm back on the Xen port. And it's getting
pretty far. I hit a few weird issues but worked them out. All I have left
to do is get a virtual enet working and I'll be fully booting Plan 9. It's
mostly there.

I'm off for vacation now. At some point in sep. I will be handing the code
out and will probably be looking for some brave souls to help me turn my
code into something worthwhile, as well as find all the things I got
wrong. So, if you think you might want to help, check out the Xen web
page, set it up on your machine, and let me know.

thanks

ron

Re: [9fans] xen port

[boyd, rounin]

> So, if you think you might want to help, check out the Xen web
> page, set it up on your machine, and let me know.

ahh, "crazy" ron minnich [blame f2f] :)

Re: [9fans] xen port

[andrey mirtchovski]

>
> ahh, "crazy" ron minnich [blame f2f] :)

wouldn't i love to be called 'crazy' in the same context!

[9fans] venti question

[Tim Newsham]

This "paper" implies feasible attacks against SHA-0:
  http://eprint.iacr.org/2004/199.pdf

and there are (unconfirmed) rumors that the attack has been
successfully extended to SHA-1.

I was wondering what the implications of this would be to venti.
It would seem that it may open the door to malicious corruption
of venti data.  Of the top of my head -- if you could anticipate
some data that will be stored in the near future and have enough
time to calculate a collision for it, you could submit bogus data
with the same hash.  When the data is stored later, it will be
discarded because venti will already think it has the data.

Tim N.

Re: [9fans] venti question

[Russ Cox]

> I was wondering what the implications of this would be to venti.

It would not be hard to move a Venti system to a new hash function,
especially if the hash function has 160 bits of output.

Russ

Re: [9fans] venti question

[hugo rivera]

2009/5/13, Steve Simon <steve@quintile.net>:
>  I would suggest you try again from scratch.

I did, and now everything is working now with twice as many arenas :-)
I just extracted some files from march 15, and everything seems to be
working just fine.
I reformatted the indexes and called fmtindex with the new venti.conf.
Then I tried to extract something but then it couldn't find an entry
with my vac key, so then I built the index again with buildindex and
my venti.conf, and now it works again.
I saw that fmtindex has a -a flag to add more arenas to the index, but
I wasn't sure to use it from the beginning since it looks it's just
useful to add arenas and not indexes.
Thanks for the help


--
Hugo

Re: [9fans] venti question

[hugo rivera]

2009/5/13, erik quanstrom <quanstro@quanstro.net>:
> perhaps your man pages our out-of-sync.  the old
>  (and delted) ventiaux(8) man page has the first option
>  description and the second signature is found in
>  venti-fmt(8).

yes, they are inconsistent with the actual version of buildindex. The
man pages I am using are those that come with p9p, from a few months
ago. I checked the last p9p version and they still have the same
inconsistency between buildindex usage message and venti-fmt(8).

--
Hugo

Re: [9fans] venti question

[Steve Simon]

> so I removed the isect1 line from my venti.conf and executed
>
> venti/buildindex -i isect1 $home/lib/venti.conf
>
> and got
>
> 1,731,717 clumps, 63,903 buckets
> 2009/0513 10:35:35 read index
> warning: did not find index section isect1


Are you suffering from an off by one error? you removed
isect1 and then built the index on isect1 only, this sounds wrong.

I would suggest you try again from scratch.

Reformat your both your indexes  and your bloom filter and
then reinitialise your fossil using a venti.conf of:

	index main
	isect /apollo/hugo/isect0
	isect /apollo/hugo/isect1
	arenas /apollo/hugo/arenas0
	arenas /apollo/hugo/arenas1

i.e.

	venti/fmtisect isect1 /dev/sdC0/isect1
	venti/fmtisect isect2 /dev/sdC0/isect2
	venti/fmtbloom /dev/sdC0/bloom
	venti/fmtindex venti.conf
	fossil/flfmt -v 87237...2345345 /dev/sdC0/fossil

I think your problems where due to you using old
isect1 which was formatted to think there was only one index
whilst your venti.conf said there where two.

I don't see how any of the above could damage your arenas0
however do take care and check what I have said carefully,
I don't want to be responsible for you losing data.

Also beware, I don't use p9p so these instructions relate to
what I would do on native plan9. I don't believe there is any
difference these days but I cannot be sure.

Caveat Emptor.

-Steve

Re: [9fans] venti question

[erik quanstrom]

> venti/buildindex [ -B blockcachesize ] [ -Z ] venti.conf tmp
>
> but execute buildindex with no arguments and you get
>
> usage: buildindex [-bd] [-i isect]... [-M imem] venti.conf
>
> options -b -d -i -M are not described in the man page and I have no
> idea what they are supposed to do (although I can imagine the use of
> -i and -b), and there's no tmp file required anymore.
> 2. I edited my venti.conf, so now it looks like

perhaps your man pages our out-of-sync.  the old
(and delted) ventiaux(8) man page has the first option
description and the second signature is found in
venti-fmt(8).

- erik

Re: [9fans] venti question

[hugo rivera]

I am trying to use buildindex but I run into trouble:
1.- the man page and the usage displayed by the command are
inconsistent. According to the man page, buildindex takes two
arguments and may take two options

venti/buildindex [ -B blockcachesize ] [ -Z ] venti.conf tmp

but execute buildindex with no arguments and you get

usage: buildindex [-bd] [-i isect]... [-M imem] venti.conf

options -b -d -i -M are not described in the man page and I have no
idea what they are supposed to do (although I can imagine the use of
-i and -b), and there's no tmp file required anymore.
2. I edited my venti.conf, so now it looks like

index main
isect /apollo/hugo/isect0
isect /apollo/hugo/isect1     # --> new line, not found in the
original venti.conf
arenas /apollo/hugo/arenas0
arenas /apollo/hugo/arenas1     # --> new line, not found in the
original venti.conf

but when I ran

venti/buildindex -i isect1 $home/lib/venti.conf

I got

err 2: mismatched number index sections: found 2 expected 1
venti/buildindex: can't init venti: mismatched number index sections:
found 2 expected 1

so I removed the isect1 line from my venti.conf and executed

venti/buildindex -i isect1 $home/lib/venti.conf

and got

1,731,717 clumps, 63,903 buckets
2009/0513 10:35:35 read index
warning: did not find index section isect1
2009/0513 10:35:35 arena arenas00: 236950 entries
2009/0513 10:35:36 arena arenas01: 317534 entries
2009/0513 10:35:36 arena arenas02: 69564 entries
2009/0513 10:35:36 arena arenas03: 231390 entries
2009/0513 10:35:37 arena arenas04: 246845 entries
2009/0513 10:35:37 arena arenas05: 65202 entries
2009/0513 10:35:38 arena arenas06: 65204 entries
2009/0513 10:35:38 arena arenas07: 77445 entries
2009/0513 10:35:38 arena arenas08: 65197 entries
2009/0513 10:35:38 arena arenas09: 65189 entries
2009/0513 10:35:38 arena arenas010: 126699 entries
2009/0513 10:35:39 arena arenas011: 137570 entries
2009/0513 10:35:39 arena arenas012: 26928 entries
2009/0513 10:35:39 done arenaentries=1,731,717 indexed=0 (nskip=0)

which complains about not finding isect1, so I am quite confused now.
I also tried some other combinations, but I always got the "mismatched
number of index sections" error. I saw Steve's article, but he uses
the syntax described in the man page which, if given to my buildindex
version, produces the usage output on screen. Note that I am using
p9p.
Any help appreciated.

--
Hugo

Re: [9fans] venti question

[Steve Simon]

I wrote this some time ago, not sure if this
represents a large enough installation for you.

http://www.quintile.net/papers/Venti-rescue.pdf

-Steve

Re: [9fans] venti question

[erik quanstrom]

On Tue May 12 11:20:33 EDT 2009, lucho@ionkov.net wrote:
> If you add more index sections, you have to rebuild the index using
> venti/buildindex.

does buildindex revisit all the data?

are there any pactice/experience-style writups out there from
folks with large venti setups?

- erik

Re: [9fans] venti question

[hugo rivera]

OK, thanks

2009/5/12 Latchesar Ionkov <lucho@ionkov.net>:
> If you add more index sections, you have to rebuild the index using
> venti/buildindex.
>
>    Lucho
>
> On Tue, May 12, 2009 at 7:20 AM, hugo rivera <uair00@gmail.com> wrote:
>> Hi,
>> I've been running a venti server for a couple of months now, and since
>> the arenas I am using are not very big, I need to add a couple more.
>> According to the man pages it's safe to add more arenas and the run
>> venti/fmtindex -a, but it isn't clear to me if this also applies to
>> the index section files. I mean, can I add more isect and arenas to
>> venti.conf, run venti/fmtindex -a and run venti again? if not, how do
>> I add more isect files without rebuilding the index from scratch?
>>
>>
>> --
>> Hugo
>>
>>
>
>



-- 
Hugo

Re: [9fans] venti question

[Latchesar Ionkov]

If you add more index sections, you have to rebuild the index using
venti/buildindex.

    Lucho

On Tue, May 12, 2009 at 7:20 AM, hugo rivera <uair00@gmail.com> wrote:
> Hi,
> I've been running a venti server for a couple of months now, and since
> the arenas I am using are not very big, I need to add a couple more.
> According to the man pages it's safe to add more arenas and the run
> venti/fmtindex -a, but it isn't clear to me if this also applies to
> the index section files. I mean, can I add more isect and arenas to
> venti.conf, run venti/fmtindex -a and run venti again? if not, how do
> I add more isect files without rebuilding the index from scratch?
>
>
> --
> Hugo
>
>

[9fans] venti question

[hugo rivera]

Hi,
I've been running a venti server for a couple of months now, and since
the arenas I am using are not very big, I need to add a couple more.
According to the man pages it's safe to add more arenas and the run
venti/fmtindex -a, but it isn't clear to me if this also applies to
the index section files. I mean, can I add more isect and arenas to
venti.conf, run venti/fmtindex -a and run venti again? if not, how do
I add more isect files without rebuilding the index from scratch?


--
Hugo