[9fans] venti/fmtarenas approximate duration?

[9fans] venti/fmtarenas approximate duration?

[andrey mirtchovski]

what's the normal duration for venti/fmtarenas?

i am running it on a 60gb arenas partition (created with "disk/prep -n arenas
-n isect" -- very useful options!) and it's taking the better part of three
hours already (2.6ghz pentium4).

stats reports ~7000 interrupts per second, 96% of the time spent in
interrupts.

andrey

Re: [9fans] venti/fmtarenas approximate duration?

[Russ Cox]

echo dma on >/dev/sdC0/ctl

Re: [9fans] venti/fmtarenas approximate duration?

[andrey mirtchovski]

On Mon, 23 Jun 2003, Russ Cox wrote:

> echo dma on >/dev/sdC0/ctl
>

that didn't make any difference, however doing:

	echo 'rmw on' > /dev/sdC0/ctl

caused the interrupts to drop significantly (from ~7000 to ~1000) and the
load to increase somewhat substantially (though still not maxed out)...

honestly, i can't predict whether that's going to decrease or increase the
time it takes to complete the command.

from other sources i understand that venti/fmtarenas has been observed to
take the better part of a night to complete :) how are you solving this
for the future plan9 install disk, which will have the option to choose
between kfs and fossil?

andrey

Re: [9fans] venti/fmtarenas approximate duration?

[jmk]

[-- Attachment #1: Type: text/plain, Size: 173 bytes --]

i did a fmtarenas on a 500GB raid last week, it took over
6 hours at about 22MB/s.

what is the output of the 'pci' command? it really sounds like
you don't have dma on.

[-- Attachment #2: Type: message/rfc822, Size: 2454 bytes --]

From: andrey mirtchovski <mirtchov@cpsc.ucalgary.ca>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] venti/fmtarenas approximate duration?
Date: Mon, 23 Jun 2003 19:10:29 -0600 (MDT)
Message-ID: <Pine.LNX.4.44.0306231903350.17223-100000@fbsd.cpsc.ucalgary.ca>

On Mon, 23 Jun 2003, Russ Cox wrote:

> echo dma on >/dev/sdC0/ctl
>

that didn't make any difference, however doing:

	echo 'rmw on' > /dev/sdC0/ctl

caused the interrupts to drop significantly (from ~7000 to ~1000) and the
load to increase somewhat substantially (though still not maxed out)...

honestly, i can't predict whether that's going to decrease or increase the
time it takes to complete the command.

from other sources i understand that venti/fmtarenas has been observed to
take the better part of a night to complete :) how are you solving this
for the future plan9 install disk, which will have the option to choose
between kfs and fossil?

andrey

Re: [9fans] venti/fmtarenas approximate duration?

[andrey mirtchovski]

On Mon, 23 Jun 2003 jmk@plan9.bell-labs.com wrote:

> i did a fmtarenas on a 500GB raid last week, it took over
> 6 hours at about 22MB/s.
>
> what is the output of the 'pci' command? it really sounds like
> you don't have dma on.

the disk with the arenas is sdD0 (i converted the commands to sdC0 before
posting to the list). i've ran 'echo dma on' on both disks, and 'echo rmw
on' on the second one only...

# pci
0.0.0:  06.00.00 8086/2578   0
0.1.0:  06.04.00 8086/2579   0
0.29.0: 0c.03.00 8086/24d2  11 4:0000ef01 32
0.29.1: 0c.03.00 8086/24d4  10 4:0000ef21 32
0.29.2: 0c.03.00 8086/24d7   5 4:0000ef41 32
0.29.3: 0c.03.00 8086/24de  11 4:0000ef81 32
0.29.7: 0c.03.20 8086/24dd  10 0:f7fffc00 1024
0.30.0: 06.04.00 8086/244e   0
0.31.0: 06.01.00 8086/24d0   0
0.31.1: 01.01.8a 8086/24db   0 0:00000001 16 1:00000001 16 2:00000001 16 3:00000001 16 4:0000fc01 16 5:00000000 1024
0.31.3: 0c.05.00 8086/24d3   5 4:00000401 32
0.31.5: 04.01.00 8086/24d5   5 0:0000e801 256 1:0000ee81 64 2:f7fff800 512 3:f7fff400 256
2.11.0: 03.00.00 5333/8a01  10 0:f0000000 67108864
2.13.0: 02.00.00 1186/1300   5 0:0000d401 256 1:f7eef400 256
2.3.0:  0c.00.10 1106/3044   5 0:f7eef800 2048 1:0000dc01 128
2.5.0:  02.00.00 10b7/1700   5 0:f7ee8000 16384 1:0000d801 256
#
# cat /dev/sdC0/ctl
inquiry Maxtor 6Y080L0
config 0040 capabilities 2F00 dma 00550020 dmactl 00000000 rwm 16 rwmctl 0
geometry 160086528 512 16383 16 63
part data 0 160086528
part plan9 63 160071660
part 9fat 63 20545
part fs 20545 159445204
part swap 159445204 160071659
part nvram 160071659 160071660
#
# cat /dev/sdD0/ctl
inquiry Maxtor 6Y080L0
config 0040 capabilities 2F00 dma 00550020 dmactl 00000000 rwm 16 rwmctl 16
geometry 160086528 512 16383 16 63
part data 0 160086528
part plan9 63 160071660
part 9fat 63 204863
part nvram 204863 204864
part fossil 204864 25783551
part arenas 25783551 153676988
part isect 153676988 160071660
#

Re: [9fans] venti/fmtarenas approximate duration?

[jmk]

try adding

		case (0x24DB<<16)|0x8086:	/* 82801EB (ICH5) */

to the switch in /sys/src/9/pc/sdata.c:atapnp(), then try
doing the

	echo dma on >/dev/sd[CD][01]/ctl

after you have done some i/o, the config line when catting
the dev ctl file should have something other than 0 in the
dmactl field.

Re: [9fans] venti/fmtarenas approximate duration?

[andrey mirtchovski]

we'll have to wait until tomorrow for the results -- i did a
'disk/kfscmd halt' and now i can't 'echo reboot > /dev/reboot' anymore. i'm
not at the machine, so i guess i'm stuck...

andrey

ps: i added it to the bottom of the switch statement, where the other
0x8086's are...

On Mon, 23 Jun 2003 jmk@plan9.bell-labs.com wrote:

> try adding
>
> 		case (0x24DB<<16)|0x8086:	/* 82801EB (ICH5) */
>
> to the switch in /sys/src/9/pc/sdata.c:atapnp(), then try
> doing the
>
> 	echo dma on >/dev/sd[CD][01]/ctl
>
> after you have done some i/o, the config line when catting
> the dev ctl file should have something other than 0 in the
> dmactl field.
>

Re: [9fans] venti/fmtarenas approximate duration?

[Russ Cox]

> from other sources i understand that venti/fmtarenas has been observed to
> take the better part of a night to complete :) how are you solving this
> for the future plan9 install disk, which will have the option to choose
> between kfs and fossil?

I turned DMA on by default.  That won't help if you've got
an unknown controller, but otherwise it will help quite a bit.
The fact is that zeroing the disk takes time either way
(though it takes a lot longer without DMA), so if you want
to do that you have to be ready for it.

Maybe we should change Venti to zero the disk as it goes
along, keeping a ``have-zeroed-this-far'' field in the
various super blocks.

Russ

Re: [9fans] venti/fmtarenas approximate duration?

[Fco.J.Ballesteros, nemo]

> Maybe we should change Venti to zero the disk as it goes
> along, keeping a ``have-zeroed-this-far'' field in the
> various super blocks.

That would be great.

Re: [9fans] venti/fmtarenas approximate duration?

[Lucio De Re]

On Wed, Jun 25, 2003 at 09:55:56AM +0200, Fco.J.Ballesteros wrote:
>
> > Maybe we should change Venti to zero the disk as it goes
> > along, keeping a ``have-zeroed-this-far'' field in the
> > various super blocks.
>
> That would be great.

Sorry to disagree, but I'd prefer not to carry state unnecessarily, it
strikes me as being more fragile, this way.  Wasting a little time
initially I'm sure pays off over a long period.

++L

Re: [9fans] venti/fmtarenas approximate duration?

[Fco.J.Ballesteros, nemo]

> Sorry to disagree, but I'd prefer not to carry state unnecessarily, it
> strikes me as being more fragile, this way.  Wasting a little time

Why would it be more fragile?

Re: [9fans] venti/fmtarenas approximate duration?

[Lucio De Re]

On Wed, Jun 25, 2003 at 10:07:05AM +0200, Fco.J.Ballesteros wrote:
>
> > Sorry to disagree, but I'd prefer not to carry state unnecessarily, it
> > strikes me as being more fragile, this way.  Wasting a little time
>
> Why would it be more fragile?

Well, variables are just that: variable :-)  Having zeroed the
entire Venti, you know it's zeroed.  Relying on a state variable
opens the door to inconsistencies.  In this case, even though
failure is unlikely, there's an alternative that is a lot less
likely to fail.

++L

Re: [9fans] venti/fmtarenas approximate duration?

[Fco.J.Ballesteros, nemo]

> entire Venti, you know it's zeroed.  Relying on a state variable
> opens the door to inconsistencies.  In this case, even though
> failure is unlikely, there's an alternative that is a lot less
> likely to fail.

On the other hand, pre-zeroing forbids you to just copy the venti
partition to a larger drive. the `zeros' are also state :-)

Re: [9fans] venti/fmtarenas approximate duration?

[Lucio De Re]

On Wed, Jun 25, 2003 at 11:38:24AM +0200, Fco.J.Ballesteros wrote:
>
> > entire Venti, you know it's zeroed.  Relying on a state variable
> > opens the door to inconsistencies.  In this case, even though
> > failure is unlikely, there's an alternative that is a lot less
> > likely to fail.
>
> On the other hand, pre-zeroing forbids you to just copy the venti
> partition to a larger drive. the `zeros' are also state :-)

Hm, I wasn't aware of that, I hadn't considered that level of detail.
There are lots of subtleties in Venti that I haven't picked up (yet).

++L

Re: [9fans] venti/fmtarenas approximate duration?

[Sape Mullender]

> what's the normal duration for venti/fmtarenas?
>
> i am running it on a 60gb arenas partition (created with "disk/prep -n arenas
> -n isect" -- very useful options!) and it's taking the better part of three
> hours already (2.6ghz pentium4).
>
> stats reports ~7000 interrupts per second, 96% of the time spent in
> interrupts.

echo dma on >/dev/sdCx/ctl

will help a lot.

Having said that, the initial snapshot of a 35 GB file system still took 40 hours
(but everything -- fossil, arenas, index -- was on one disk (a temporary
necessity to free up disk space elsewhere)).

	Sape

Re: [9fans] venti/fmtarenas approximate duration?

[andrey mirtchovski]

On Tue, 24 Jun 2003, Sape Mullender wrote:

> echo dma on >/dev/sdCx/ctl
>
> will help a lot.
>
> Having said that, the initial snapshot of a 35 GB file system still took 40 hours
> (but everything -- fossil, arenas, index -- was on one disk (a temporary
> necessity to free up disk space elsewhere)).
>

following jmk's fix a fmtarenas on a 60gb arena took less than 30 minutes
(more like 15, don't have exact numbers). fmtisect on a 3.04Gb index
partition took about two minutes. this one i actually measured with iostats,
but the numbers are strange:


# iostats venti/fmtisect isect /dev/sdD0/isect
clearing the partition
configuring index section isect with space for index config bytes=65536

read      80863 bytes, 7.896778e+07 Kb/sec
write     3273810527 bytes, 3.197081e+12 Kb/sec
protocol  3287480890 bytes, 3.210431e+12 Kb/sec
rpc       399684 count

Message    Count   Low  High  Time Averg          in      out
version        1     0     0     0     0 ms       19       19 bytes
attach         1     0     0     0     0 ms       25       20 bytes
walk           9     0     0     0     0 ms      259      293 bytes
open           5     0     0     0     0 ms       60      120 bytes
read          21     0     0     0     0 ms      483    81094 bytes
write      399638     0     0     0     0 ms 3283002201  4396018 bytes
clunk          8     0     0     0     0 ms       88       56 bytes
stat           1     0     0     0     0 ms       11       81 bytes

Opens    Reads  (bytes)   Writes  (bytes) File
    1        0        0        2       95 (stderr)
    1        0        0   399636 3273810432 /dev/sdD0/isect
    1       21    80863        0        0 /bin/venti/fmtisect
#