Re: [9fans] Petabytes on a budget: JBODs + Linux + JFS

[Jack Norton <jack@0x6a.com>]

erik quanstrom wrote:
>> I think what he means is:
>> You are given an inordinate amount of harddrives and some computers to
>> house them.
>> If plan9 is your only software, how would it be configured overall,
>> given that it has to perform as well, or better.
>>
>> Or put another way: your boss wants you to compete with backblaze using
>> only plan9 and (let's say) a _large_ budget.  Go!
>>
>
> forgive me for thinking in ruts ...
>
> i wouldn't ask the question just like that.  the original
> plan 9 fileserver had a practically-infinite storage system.
> it was a jukebox.  the jukebox ran some firmware that wasn't
> plan 9.  (in fact the fileserver itself wasn't running plan 9.)
>
> today, jukeboxes are still ideal in some ways, but they're too
> expensive.  i personally think you  can replace the juke
> with a set of aoe shelves.  you can treat the shelves as if
> they were jukebox platters.  add as necessary.  this gives
> you an solid, redundant foundation.
>
> for a naive first implementation targeting plan 9 clients,
> i would probablly start with ken's fs.  for coraid's modest
> requirements (10e2 users 10e2 terminals 10e1 cpu servers
> 10e2 mb/s), i built this http://www.quanstro.net/plan9/disklessfs.pdf
>
> i don't see any fundamental reasons why it would not
> scale up to petabytes.  i would put work into enabling
> multiple cpus. i would imagine it wouldn't be hard to
> saturate 2x10gbe with such a setup.  of course, there is
> no reason one would need to limit oneself to a single
> file server, other than simplicity.
>
> of course this is all a bunch of hand waving without any
> money or specific requirements.
>
> - erik
>
>
Erik,

I read the paper you wrote and I have some (probably naive) questions:
The section #6 labeled "core improvements" seems to suggest that the
fileserver is basically using the CPU/fileserver hybrid kernel (both
major changes are quoted as coming from the CPU kernel).  Is this just a
one-off adjustment made by yourself, or have these changes been made
permanent?
Also, about the coraid AoE unit: am I correct in assuming that it does
some sort of RAID functionality, and then presents the resulting
device(s) as an AoE device (and nothing more)?
Also, another probably dumb question: did the the fileserver machine use
the AoE device as a kenfs volume or a fossil(+venti)?

The reason I am asking all of this is because I have a linux fileserver
machine that _just_ serves up storage, and I have a atom based machine
not doing anything at the moment (with gbE).  I would love to have the
linux machine present its goods as an AoE device and have the atom based
machine play the fileserver role.  That would be fun.
Thanks in advance for patience involving my questions :)

-Jack