Re: [9fans] file server speed

[erik quanstrom <quanstro@quanstro.net>]

> kenfs works well, but you have to be well prepared to maintain it.
> Invest in a decent UPS - preferably one that is supported by the
> auto-shutdown (ISTR support was added for that a while back). You need
> to be careful when sizing your cache - I would invest in a pair of
> decent SSDs for cache, and two or more drives for housing the WORM. Be
> prepared for failure. The last large kensfs I maintained (around 16TB
> usable, 48TB raw) worked very well but would still crash one every
> year or two. Make sure you keep hard copies of your fsconfig and get
> comfortable with scripting as erik mentioned.

the cache comment is spot on.  i've stuck with 20GB caches.  this is
because a very large cache will use too many buckets.  buckets are
stored on disk in the cache, but a bucket needs to be in memory to use
that part of the cache.  with too many buckets, most of the i/o will be
thrashing through different buckets.  i'm very familiar with doing this
wrong, as i set up one machine (kibbiee, for those who remember) with
a 750G cache.  that was a bad idea.

i'd have to add that after running ken's file server at home since 2005,
and at least half a dozen in work environments, the file system has been
pretty good with my data.  from 2006 to 2008 i ran with local scsi disks
on an old va linux box.  in that location i had no ups, and lightning
took out power about once a week.  suprisingly, this didn't cause any
trouble.  the only time i did have trouble was not the file server's
fault entirely.  the storage was disconnected during a dump, and then
the file server was rebooted.  in that case, only the corrupt dump was
lost.

clearly any file server is ideally on a ups, with generator backup and
automatic shutdown.  :-)

in my experience, using ssds for cache doesn't speed up the file
server much, since one is usually rtt limited, and by craftiness with
the cache, files are typically written sequentially to the disk.  i
have ssds in my current file server, and am a bit disappointed in the
performance.

- erik