[9fans] Fossil+Venti system memory requirements to be aware of?

[9fans] Fossil+Venti system memory requirements to be aware of?

[James A. Robinson]

[-- Attachment #1: Type: text/plain, Size: 620 bytes --]

Anyone able to tell me whether or not there are
disk size limits I should beware of given a limited
amount of system memory in a file server?

What I'm wanting to try and do is get a hardware
RAID1+0 enclosure and put in 20TB of  disk (so
10TB of usable space).

The board I am looking at will take up to 8 gb of
system memory.   Is that going to place any sort
of a limit on fossil+venti partition sizes?

There are old threads that discuss a problem
with venti and the user hitting swap (the amount
of memory wasn't specified, at least not as
far as I could find in my archive or via google).


Jim

[-- Attachment #2: Type: text/html, Size: 2345 bytes --]

Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[Aram Hăvărneanu]

There are cheaper ways of disposing of 10TB of data.

-- 
Aram Hăvărneanu

Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[James A. Robinson]

[-- Attachment #1: Type: text/plain, Size: 393 bytes --]

On Wed, Oct 19, 2016 at 10:13 AM Aram Hăvărneanu <aram.h@mgk.ro> wrote:

> There are cheaper ways of disposing of 10TB of data.
>

If I decide the configuration is problematic
I'm sure I can repurpose the device.

Besides, the costs of spinning disk these
days is amazingly low.  As, I think, the
developers for Plan 9 expected when they
decided to move from WORM to HDD.

Jim

[-- Attachment #2: Type: text/html, Size: 763 bytes --]

Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[Steven Stallion]

Hi Jim,

It probably helps to break apart fossil and venti for the sake of the
conversation. While you can use fossil as a standalone filesystem, it
is effectively your write cache in this scenario since it will be
backed by venti. Conventional wisdom is to size your main fossil fs
based on how much you plan to write in a given day. In my personal
setup, I use a 32GB main fs, with the rest of a mirrored SSD pair
dedicated to an "other" fossil partition where I place data that
doesn't get backed up.

Sizing venti is also simple. Start with something manageable; I chose
32GB at the time. Venti behaves much like a WORM with some additional
deduplication (I'm glossing over gobs of detail here), so you'll
likely need much less than you think. You can grow your venti story
over time if you need, and it's also relatively trivial to move your
data between stores.

In short, start small and grow as needed. For reference, when I ran
Coraid's fs based on 64-bit Ken's (WORM only, no dedupe) in RWC (based
on the main fs in Athens). Over the course of a few years the entire
WORM grew to around 35GB. This was for a couple dozen people working
full time. I believe I had the LUN configured to something rather
large at the time - around 12TB. All wasted space.

HTH,

Steve

On Wed, Oct 19, 2016 at 3:41 AM, James A. Robinson
<jim.robinson@gmail.com> wrote:
> Anyone able to tell me whether or not there are
> disk size limits I should beware of given a limited
> amount of system memory in a file server?
>
> What I'm wanting to try and do is get a hardware
> RAID1+0 enclosure and put in 20TB of  disk (so
> 10TB of usable space).
>
> The board I am looking at will take up to 8 gb of
> system memory.   Is that going to place any sort
> of a limit on fossil+venti partition sizes?
>
> There are old threads that discuss a problem
> with venti and the user hitting swap (the amount
> of memory wasn't specified, at least not as
> far as I could find in my archive or via google).
>
>
> Jim
>

Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[cigar562hfsp952fans]

Steven Stallion <sstallion@gmail.com> writes:

> Sizing venti is also simple.

I disagree with this.  The best way to configure venti depends largely
on how you plan to use it.  I have multiple venti servers configured for
different uses.  For example, I keep my DVD images on a different venti
server than I do for smaller, more frequently-used files such as mail.

Choosing a good arena size isn't trivial, either.  If you ever plan to
back-up your venti to optical media, 640 MiB is the ideal arena size.
There is a lot of math behind that determination, but its implications
can be summarized as follows: Whether you're burning to standard CD-Rs,
extended CD-Rs, DVDs, DVD DL, Blu-ray, flash media, or to another HDD,
the amount of "wasted" space will never exceed 5% of the media's total
capacity.  In other words, the archvie will always be a minimum of 95%
space-efficient.  (Caveat: when burning 640 MiB arenas to optical media,
you may need to turn off Joliet/Rock Rigde extensions in order to make
the ISO image fit.)

> and it's also relatively trivial to move your data between stores.

I don't know that to be true.  If the source and destination volumes are
the same size, yes, it's easy to just dd the arenas.  But if you're
relocating to a smaller/larger partition, what would be an easy way to
do that?

Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[Steven Stallion]

On Thu, Oct 20, 2016 at 1:15 PM,  <cigar562hfsp952fans@icebubble.org> wrote:
> Steven Stallion <sstallion@gmail.com> writes:
>
>> Sizing venti is also simple.
>
> I disagree with this.  The best way to configure venti depends largely
> on how you plan to use it.  I have multiple venti servers configured for
> different uses.  For example, I keep my DVD images on a different venti
> server than I do for smaller, more frequently-used files such as mail.

see venti(8); there's plenty of advice on configuring your store with
rough estimations for an initial conf file. Once you decide your
geometry, it's trivial to extend the number of arenas in the store,
just remember to reindex.

> I don't know that to be true.  If the source and destination volumes are
> the same size, yes, it's easy to just dd the arenas.  But if you're
> relocating to a smaller/larger partition, what would be an easy way to
> do that?

see venti-backup(8).

Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[Steve Simon]

> On 20 Oct 2016, at 19:41, Steven Stallion <sstallion@gmail.com> wrote:
> 
>> On Thu, Oct 20, 2016 at 1:15 PM,  <cigar562hfsp952fans@icebubble.org> wrote:
>> Steven Stallion <sstallion@gmail.com> writes:
>> 
>>> Sizing venti is also simple.
>> 
>> I disagree with this.  The best way to configure venti depends largely
>> on how you plan to use it.  I have multiple venti servers configured for
>> different uses.  For example, I keep my DVD images on a different venti
>> server than I do for smaller, more frequently-used files such as mail.
> 
> see venti(8); there's plenty of advice on configuring your store with
> rough estimations for an initial conf file. Once you decide your
> geometry, it's trivial to extend the number of arenas in the store,
> just remember to reindex.
> 
>> I don't know that to be true.  If the source and destination volumes are
>> the same size, yes, it's easy to just dd the arenas.  But if you're
>> relocating to a smaller/larger partition, what would be an easy way to
>> do that?
> 
> see venti-backup(8).

Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[Steve Simon]

i agree absolutely with steve here, expanding venture arena by arena is easy, the ventibackup scripts show you how. even easier is to add arenas on a different disk partition to the same venti.

personally i wouldn't keep music or videos in venti. they don't compress well using the arithmetic techniques in venti and the de duplication is unlikely to be a win either. just put them on a separate mirrored partition.

i wrote a doc called venti rescue which details replacing disks and rebuilding your venti. it may be of help.

-Steve


> On 20 Oct 2016, at 19:41, Steven Stallion <sstallion@gmail.com> wrote:
> 
>> On Thu, Oct 20, 2016 at 1:15 PM,  <cigar562hfsp952fans@icebubble.org> wrote:
>> Steven Stallion <sstallion@gmail.com> writes:
>> 
>>> Sizing venti is also simple.
>> 
>> I disagree with this.  The best way to configure venti depends largely
>> on how you plan to use it.  I have multiple venti servers configured for
>> different uses.  For example, I keep my DVD images on a different venti
>> server than I do for smaller, more frequently-used files such as mail.
> 
> see venti(8); there's plenty of advice on configuring your store with
> rough estimations for an initial conf file. Once you decide your
> geometry, it's trivial to extend the number of arenas in the store,
> just remember to reindex.
> 
>> I don't know that to be true.  If the source and destination volumes are
>> the same size, yes, it's easy to just dd the arenas.  But if you're
>> relocating to a smaller/larger partition, what would be an easy way to
>> do that?
> 
> see venti-backup(8).

Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[James A. Robinson]

[-- Attachment #1: Type: text/plain, Size: 1366 bytes --]

On Wed, Oct 19, 2016 at 9:47 AM Steven Stallion <sstallion@gmail.com> wrote:
> In short, start small and grow as needed. For reference, when I ran
> Coraid's fs based on 64-bit Ken's (WORM only, no dedupe) in RWC
> (based on the main fs in Athens). Over the course of a few years
> the entire WORM grew to around 35GB. This was for a couple dozen
> people working full time. I believe I had the LUN configured to
> something rather large at the time - around 12TB. All wasted space.

Hi,

So if you were setting up a new filesystem for the same group, and
were using venti, would you be using the default 500 mb arena size
(meaning 70+ arenas), or would you be sizing them at a larger size,
like 1 gb, 2 gb, etc?

The "setting up venti" page indicates that multiple small disks
help with performance on venti due to using more indexes, but I
can't tell if that translates to more partitions (e.g., 1 RAID 1+0
disk split into multiple smaller partitions, and divvied up into
isectX and arenaX drives).  I guess it probably comes down to
how well (or poorly) the underlying disk array performs at random
read/write ops.

I see that AWS S3 allows for up to 5 gb in a single PUT, or 5 tb
using multipart streams, and am thinking that using 1 to 5 gb
arenas might make sense as a more modern "sized for for easy
offline backup" default.

Jim

[-- Attachment #2: Type: text/html, Size: 1805 bytes --]

Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[Steven Stallion]

Hi Jim,

It's important to point out that the arena size does not have to match
the size of an arenas file. In my case, I do something similar where I
use 2GB for an arena but keep my arenas files at 2GB (I don't have
much use for keeping multiple arena files).

More indexes help to an extent. My fs isn't under exceptionally heavy
load, but I've found that 2 seem to work out nicely for my
configuration.

Steve

On Wed, Nov 2, 2016 at 1:44 PM, James A. Robinson <jimr@highwire.org> wrote:
> On Wed, Oct 19, 2016 at 9:47 AM Steven Stallion <sstallion@gmail.com> wrote:
>> In short, start small and grow as needed. For reference, when I ran
>> Coraid's fs based on 64-bit Ken's (WORM only, no dedupe) in RWC
>> (based on the main fs in Athens). Over the course of a few years
>> the entire WORM grew to around 35GB. This was for a couple dozen
>> people working full time. I believe I had the LUN configured to
>> something rather large at the time - around 12TB. All wasted space.
>
> Hi,
>
> So if you were setting up a new filesystem for the same group, and
> were using venti, would you be using the default 500 mb arena size
> (meaning 70+ arenas), or would you be sizing them at a larger size,
> like 1 gb, 2 gb, etc?
>
> The "setting up venti" page indicates that multiple small disks
> help with performance on venti due to using more indexes, but I
> can't tell if that translates to more partitions (e.g., 1 RAID 1+0
> disk split into multiple smaller partitions, and divvied up into
> isectX and arenaX drives).  I guess it probably comes down to
> how well (or poorly) the underlying disk array performs at random
> read/write ops.
>
> I see that AWS S3 allows for up to 5 gb in a single PUT, or 5 tb
> using multipart streams, and am thinking that using 1 to 5 gb
> arenas might make sense as a more modern "sized for for easy
> offline backup" default.
>
> Jim
>

Re: [9fans] Fossil+Venti system memory requirements to be aware of?

[cigar562hfsp952fans]

"James A. Robinson" <jim.robinson@gmail.com> writes:

> Anyone able to tell me whether or not there are
> disk size limits I should beware of given a limited
> amount of system memory in a file server?

Although there have been some replies on this thread, none of them have
really yet directly answered your question.  When setting up a venti
server, the size of the Bloom filter has a big impact on memory usage
and performance.  Details about computing Bloom filter size can be found
in the docs.  You could set up venti on a machine with very little
memory if you're happy with it constantly going out to check the index.