Re: [9fans] Petabytes on a budget: JBODs + Linux + JFS

[Jack Norton <jack@0x6a.com>]

erik quanstrom wrote:
>>> i think the lesson here is don't by cheep drives; if you
>>> have enterprise drives at 1e-15 error rate, the fail rate
>>> will be 0.8%.  of course if you don't have a raid, the fail
>>> rate is 100%.
>>>
>>> if that's not acceptable, then use raid 6.
>>>
>> Hopefully Raid 6 or zfs's raidz2 works well enough with cheap
>> drives!
>>
>
> don't hope.  do the calculations.  or simulate it.
>
> this is a pain in the neck as it's a function of ber,
> mtbf, rebuild window and number of drives.
>
> i found that not having a hot spare can increase
> your chances of a double failure by an order of
> magnitude.  the birthday paradox never ceases to
> amaze.
>
> - erik
>
>
While we are on the topic:
How many RAID cards have we failed lately?  I ask because I am about to
hit a fork in the road with my work-a-like of your diskless fs.  I was
originally going to use linux soft raid and vblade, but I am considering
using some raid cards that just so happen to be included in the piece of
hardware I will be getting soon...
At work, we recently had a massive failure of our RAID array.  After
much brown noseing, I come to find that after many harddrives being
shipped to our IT guy and him scratching his head, it was in fact the
RAID card itself that had failed (which takes out the whole array, plus
can take out any new drives you throw at it apparently).

So I ask you all this (especially those in the 'biz): all this
redundancy on the drive side, why no redundancy of controller cards (or
should I say, the driver infrastructure needed)?

It is appealing to me to try and get some plan 9 supported raid card and
have plan 9 throughout (like the coraid setup as far as I can tell), but
this little issue bothers me.

Speaking of birthday, I mentioned to our IT dep (all two people...) that
they should try and spread out the drives used among different mfg dates
and batches.  It shocked me to know that this was news to them...

-Jack