Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[Steven Stallion <sstallion@gmail.com>]

Hi Jim,

It probably helps to break apart fossil and venti for the sake of the
conversation. While you can use fossil as a standalone filesystem, it
is effectively your write cache in this scenario since it will be
backed by venti. Conventional wisdom is to size your main fossil fs
based on how much you plan to write in a given day. In my personal
setup, I use a 32GB main fs, with the rest of a mirrored SSD pair
dedicated to an "other" fossil partition where I place data that
doesn't get backed up.

Sizing venti is also simple. Start with something manageable; I chose
32GB at the time. Venti behaves much like a WORM with some additional
deduplication (I'm glossing over gobs of detail here), so you'll
likely need much less than you think. You can grow your venti story
over time if you need, and it's also relatively trivial to move your
data between stores.

In short, start small and grow as needed. For reference, when I ran
Coraid's fs based on 64-bit Ken's (WORM only, no dedupe) in RWC (based
on the main fs in Athens). Over the course of a few years the entire
WORM grew to around 35GB. This was for a couple dozen people working
full time. I believe I had the LUN configured to something rather
large at the time - around 12TB. All wasted space.

HTH,

Steve

On Wed, Oct 19, 2016 at 3:41 AM, James A. Robinson
<jim.robinson@gmail.com> wrote:
> Anyone able to tell me whether or not there are
> disk size limits I should beware of given a limited
> amount of system memory in a file server?
>
> What I'm wanting to try and do is get a hardware
> RAID1+0 enclosure and put in 20TB of  disk (so
> 10TB of usable space).
>
> The board I am looking at will take up to 8 gb of
> system memory.   Is that going to place any sort
> of a limit on fossil+venti partition sizes?
>
> There are old threads that discuss a problem
> with venti and the user hitting swap (the amount
> of memory wasn't specified, at least not as
> far as I could find in my archive or via google).
>
>
> Jim
>