Re: [9fans] Help with two small shared file servers

[John Floren <john@jfloren.net>]

On Wed, Aug 17, 2011 at 7:11 AM, Anthony Sorace <a@9srv.net> wrote:
> On Aug 17, 2011, at 6:09 AM, Aram Hăvărneanu wrote:
>
>> Can anyone shed some light on why I might want one and not the other?
>> Are there any other options?
>
> ken's fs is a kernel, and essentially gives you a 9p-accessible file storage
> appliance. it takes another box to be able to run it. #k is part of the standard
> cpu server kernel, and is typically used in conjunction with fossil and/or
> venti. really, you usually want to decide between Ken's fs and fossil (with or
> without venti). the basics:
>
> Ken's FS used to be the default file server in Plan 9. It's a kernel, and takes over
> a box. It is only directly accessible via 9p and il, which pretty much means via
> Plan 9 (cpu servers can share the resources as they like, of course). It does
> automatic daily archives of the whole system. It does limited de-duplicating.
> It is very fast and extremely stable.
>
> fossil is a newer file server, designed for use with venti (but usable on its own,
> as well). it can be configured to do daily archives and/or periodic ephemeral
> snapshots (you pick how often and how long they last). It can be used stand-
> alone (can't do archives that way, but can still do snapshots) or with venti, in
> which case you get venti's block-level deduplication (and direct access to
> block-level storage). fossil performs reasonably and is relatively stable, but
> less so than ken's fs (and, personally, seems more sensitive to unexpected
> shutdown). venti is very stable. typically, if fossil goes belly-up, you can
> reconstruct it from venti (in every case i've seen).
>
> personally, if you (a) have the extra box to spare, (b) can put in a little extra
> work up front, (c) don't care about direct access to block-level storage, and
> (d) can live without ephemeral snapshots, i'd suggest ken's fs; otherwise, i'd
> suggest fossil backed by venti.
>
>(note that (d) above is slightly tricky, as the ephemeral snapshots in fossil
>seem to trigger a bug that causes fossil to hang periodically; most reports
>have that happening every ~2 months or so)

Ken's FS serves only 9P and can be a hassle to set up and get going.
In my opinion, if you can live without ephemeral snapshots, just run
fossil+venti with snapshots turned off, which should eliminate the
stability complaint.

I ran Ken's FS for a while, but in the end it wasn't worth the extra
trouble. We're back with fossil + venti now, with fossil on the local
disk and Venti on a 16 TB Coraid.


>> Is 9p suitable for this? How will the 40ms latency affect 9p operation?
>
> i expect that'll be well low enough for most uses.
>

It's going to be slow. See my thesis
(https://bitbucket.org/floren/tstream/src/67c7419ad84a/documents/Thesis.pdf)
for details, but I'd expect file transfers to be at least 6 times
slower than transferring via HTTP. When I tested 9P vs. HTTP over
connections with 25 ms latency (50ms RTT), I saw a 4x slowdown versus
HTTP. Even at 15 ms RTT, transfers took twice as long.


John