[9fans] SCSI or IDE?

[9fans] SCSI or IDE?

[okamoto]

I'm now considering to build a new venti + fossil file server.
I can choose both of SCSI and IDE HDDs, because I have to 
buy one anyway.

Which one do you recommend?
Does the IDE driver use faster one, oh well, I forgot the name 
Ultra DMA?

Kenji

Re: [9fans] SCSI or IDE?

[Dan Cross]

> I'm now considering to build a new venti + fossil file server.
> I can choose both of SCSI and IDE HDDs, because I have to 
> buy one anyway.
> 
> Which one do you recommend?
> Does the IDE driver use faster one, oh well, I forgot the name 
> Ultra DMA?

I'm wondering myself.  Three weeks ago, I said I was going to build a
file server and asked whether I should go with Fossil+Venti or the
standalone server.  Unfortunately, I got side tracked and haven't built
anything.

If I were going to build a standalone file server, I would put a cached
pseudo-worm, with the cache on a set of four 9GB SCSI disks that I have
sitting in another room, with another 9GB disk for other (all hanging
off of an LSI logic SCSI controller [which, btw, can be ordered from
the LSI Logic web site]) and the pseudo-worm on a pair of mirrored
120GB IDE drives (with the kernel and 9load on an IDE flash device).
I'd probably put on a gig or so of RAM and let the kernel use that for
buffers.

With fossil, I'm just not sure yet how to configure things, and I have
little room for experimentation right now, so some guidance would be
most welcome.  I'd like something that performs as well as the
standalone file server (particular for reading, which is what I'm going
to be doing a lot more than writing), but seems kind of weird using all
those disks for a large write cache (though if a read cache is in the
works, it might make sense to put them all on the machine in
anticipation of that).  What's the advice on such things?

Also, I have an older file server that's running the standalone code
that I'd like to convert into an archive for my Unix junk.  What's the
best way to convert an old, standalone fileserver into a fossil server
(ie, preserving the dump)?  In particular, I want to replace the drives
in the old machine with something bigger that's mirrored.  Fossil for
this is ideal because I can run an NFS server off of the machine in
addition to fossil/venti.  Maybe I could put MP3's on it, too.
However, it strikes me that a lot of this is quite literally, read-only
access.  Does fossil make sense in those environments?  Or would I be
better off with a standalone FS+CPU server combination?

	- Dan C.

Re: [9fans] SCSI or IDE?

[Fco.J.Ballesteros]

I'm pretty happy using ide disks. They're fast enough and
you always have a spare one hanging around just in case one of your
disks fails. 

Regarding fossil, I'm using two 40G ide disks where I made partitions
for venti and fossil. I make the same partitions on both and use fs(3)
to mirror them. both venti and fossil run on the mirror (see the example
below).

I copied all the dumps in our worm fs to venti. Our 9fs script uses
vacfs to mount pre-fossil dumps and binds them along with fossil archive.

We keep the old worm disks inside the old file server but disconnected
(no bus), just to preserve them in case of a cataclism. The old file
server is actually a terminal now.

For us, our authentication server is also our file server (and services
httpd and smtpd as well). Since everything else is a terminal we dont need
to maintain separate configurations. 

Just to make an example, right now I have these partitions for
venti arenas, index, fossil, and a fossil partition not backed up by
venti:

# arenas and index
part other 63 20723850
part other1 20723850 22972950
# fossil (backed up by venti)
part fossil 22973000 27792450
# fossil (unbacked)
part once 27792515 36146250
# there's more room in the disk, to become arenas in the future.

Then in /boot we mirror them all like in:

echo mirror once /dev/sdC0/once /dev/sdD0/once >/dev/fs/ctl

and start venti and fossil using them:
# venti.conf
index oldworm
isect /dev/fs/other1
arenas /dev/fs/other

# flproto
...
fsys main config /dev/fs/fossil
fsys once config /dev/fs/once

Very long message, but hth.

RE: [9fans] SCSI or IDE?

[Nigel Roles]

How long did it take to copy the old dump filesystem into venti?
I know venti will reduce the data nicely, but I have about 1000
dumps, and this mornings holds 2.8Gb (the first ever is 170Mb).
A rough guesstimate is that I have to send Venti about 1.5Tb.

I'd assumed that copying the dump into Venti would have taken
way too long.

What is the typical throughput to a Venti server when writing?
Presumably the hash computation dominates?

-----Original Message-----
From: 9fans-admin@cse.psu.edu [mailto:9fans-admin@cse.psu.edu]On Behalf
Of Fco.J.Ballesteros
Sent: 31 March 2003 08:33
To: 9fans@cse.psu.edu
Subject: Re: [9fans] SCSI or IDE? 


I'm pretty happy using ide disks. They're fast enough and
you always have a spare one hanging around just in case one of your
disks fails. 

Regarding fossil, I'm using two 40G ide disks where I made partitions
for venti and fossil. I make the same partitions on both and use fs(3)
to mirror them. both venti and fossil run on the mirror (see the example
below).

I copied all the dumps in our worm fs to venti. Our 9fs script uses
vacfs to mount pre-fossil dumps and binds them along with fossil archive.

We keep the old worm disks inside the old file server but disconnected
(no bus), just to preserve them in case of a cataclism. The old file
server is actually a terminal now.

For us, our authentication server is also our file server (and services
httpd and smtpd as well). Since everything else is a terminal we dont need
to maintain separate configurations. 

Just to make an example, right now I have these partitions for
venti arenas, index, fossil, and a fossil partition not backed up by
venti:

# arenas and index
part other 63 20723850
part other1 20723850 22972950
# fossil (backed up by venti)
part fossil 22973000 27792450
# fossil (unbacked)
part once 27792515 36146250
# there's more room in the disk, to become arenas in the future.

Then in /boot we mirror them all like in:

echo mirror once /dev/sdC0/once /dev/sdD0/once >/dev/fs/ctl

and start venti and fossil using them:
# venti.conf
index oldworm
isect /dev/fs/other1
arenas /dev/fs/other

# flproto
...
fsys main config /dev/fs/fossil
fsys once config /dev/fs/once

Very long message, but hth.

RE: [9fans] SCSI or IDE?

[Fco.J.Ballesteros]

> How long did it take to copy the old dump filesystem into venti?
> I know venti will reduce the data nicely, but I have about 1000
> dumps, and this mornings holds 2.8Gb (the first ever is 170Mb).

We had 625 dumps of about the same size. It took 1000s aprox each,
but our network is slow. I think you could do it much faster.
BTW, we used -d option to avoid the write of files that vac considered
already in venti.

Re: [9fans] SCSI or IDE?

[PB]

<okamoto@granite.cias.osakafu-u.ac.jp> wrote in message
news:f8cfc6b90c51b1da2114e4624c9ae3a7@granite.cias.osakafu-u.ac.jp...
> I'm now considering to build a new venti + fossil file server.
> I can choose both of SCSI and IDE HDDs, because I have to
> buy one anyway.
>
> Which one do you recommend?

Really the only issue is whether you want to spend the extra money for SCSI
to get the extra speed.

PB

Re: [9fans] SCSI or IDE?

[Russ Cox]

> Wiki is an English page, and I don't want to make it
> dirty by my broken Japanglish. ☺

Edit it anyway.  Someone else can fix it up.
That's the point of the wiki!

(Also, I read the diffs each morning and often correct,
expand, or otherwise edit what has been added.)

Russ

Re: [9fans] SCSI or IDE?

[okamoto]

> Always feel free to edit the wiki, anyone.

Wiki is an English page, and I don't want to make it
dirty by my broken Japanglish. ☺

Kenji

Re: [9fans] SCSI or IDE?

[okamoto]

Thanks a lot, Russ.

Now, I'm confident to go with an IDE 120GB drive and a 6GB drive
for fossil+venti.

How about to put your message to Plan 9 wiki?

Kenji

Re: [9fans] SCSI or IDE?

[Russ Cox]

Always feel free to edit the wiki, anyone.

Re: [9fans] SCSI or IDE?

[C H Forsyth]

>>Also, Forsyth found a debugging statement I'd left
>>in the code that was making the free block allocator
>>much slower than it needed to be.  This was about
>>a week ago now.  It speeds things up a fair amount.

removing it changed the time considerably for
an initial load of a fossil system from a mkfs archive.
it went from over 6 hours (until i stopped it),
to about 11 minutes to finish.

Re: [9fans] SCSI or IDE?

[Russ Cox]

> In addtion, I'm anxious of what rog said here before.
> Something like venti+fossil is slower than kfs system on his notebook. ☺
> 
> I know kfs is very slow when compared with my real file server with SCSI
> disks.   So, if venti+fossil is slower than kfs, hmmm...

Kfs and the real file server are effectively the same.
The difference is that kfs typically uses 2MB of file
system buffer while the real file server uses all
available memory, so of course the real file server
will outperform kfs.  I think that if you gave kfs as
much memory as the real server uses, you'd see
they were comparable.  Also, your kfs might be slow
because you haven't enabled dma -- it's turned off by
default in the IDE driver.  echo dma on >/dev/sdC0/ctl
and your system might get a lot faster.

There's nothing inherently slower about kfs vs fs.

Fossil+venti is slow at uncached reads, because venti
is slow at doing reads.  

> By the way, in venti document, there is a point to make it faster by
> separating index and log.

The Venti index is a giant on-disk hash table.
Read performance is dominated by random disk
seeks -- a random seek to load the right hash
bucket followed by a random seek to get to the
right place in the arenas.  Write performance is
also dominated by random disk seeks -- one random
seek to read the right hash bucket, then a write
to the arena, then a write to the hash bucket.

If you are trying to set up a production Venti server,
then you would stripe the index across ten small
disks so that there could be ten index operations
going on at once.  Even just keeping the index
on a separate disk from the arenas is a big win
for writes: if they're on the same disk, then you have
to seek away from the hash bucket to update the arena,
and then back again, so that's three big seeks instead
of one. 

At the moment, if you want blazingly fast performance,
then fossil+venti isn't the way to go.  But if you want
respectable performance for supporting a bunch of
users running Plan 9, then it's still a plenty good option.

Again, if you use IDE disks and don't turn dma on, 
your fossil will be very very slow.  Turn on dma,
give it some memory to cache with (I'm using 24MB
on my laptop, as I suggested in a previous post,
and I like it much better), and you'll be happy.

Also, Forsyth found a debugging statement I'd left
in the code that was making the free block allocator
much slower than it needed to be.  This was about
a week ago now.  It speeds things up a fair amount.

Russ

Re: [9fans] SCSI or IDE?

[okamoto]

> I know kfs is very slow when compared with my real file server with SCSI
> disks.

s/kfs/IDE kfs/

Kenji

Re: [9fans] SCSI or IDE?

[okamoto]

> However, it strikes me that a lot of this is quite literally, read-only
> access.  Does fossil make sense in those environments?  Or would I be
> better off with a standalone FS+CPU server combination?

In addtion, I'm anxious of what rog said here before.
Something like venti+fossil is slower than kfs system on his notebook. ☺

I know kfs is very slow when compared with my real file server with SCSI
disks.   So, if venti+fossil is slower than kfs, hmmm...

By the way, in venti document, there is a point to make it faster by
separating index and log.

Kenji

Re: [9fans] SCSI or IDE?

[okamoto]

> Does the IDE driver use faster one, oh well, I forgot the name 
> Ultra DMA?

ATA100 or ATA133?

kenji