* [9fans] Fossil+Venti system memory requirements to be aware of? @ 2016-10-19 8:41 James A. Robinson 2016-10-19 10:17 ` Aram Hăvărneanu ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: James A. Robinson @ 2016-10-19 8:41 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 620 bytes --] Anyone able to tell me whether or not there are disk size limits I should beware of given a limited amount of system memory in a file server? What I'm wanting to try and do is get a hardware RAID1+0 enclosure and put in 20TB of disk (so 10TB of usable space). The board I am looking at will take up to 8 gb of system memory. Is that going to place any sort of a limit on fossil+venti partition sizes? There are old threads that discuss a problem with venti and the user hitting swap (the amount of memory wasn't specified, at least not as far as I could find in my archive or via google). Jim [-- Attachment #2: Type: text/html, Size: 2345 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Fossil+Venti system memory requirements to be aware of? 2016-10-19 8:41 [9fans] Fossil+Venti system memory requirements to be aware of? James A. Robinson @ 2016-10-19 10:17 ` Aram Hăvărneanu 2016-10-19 17:16 ` James A. Robinson 2016-10-19 16:44 ` Steven Stallion 2016-10-20 17:50 ` cigar562hfsp952fans 2 siblings, 1 reply; 11+ messages in thread From: Aram Hăvărneanu @ 2016-10-19 10:17 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs There are cheaper ways of disposing of 10TB of data. -- Aram Hăvărneanu ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Fossil+Venti system memory requirements to be aware of? 2016-10-19 10:17 ` Aram Hăvărneanu @ 2016-10-19 17:16 ` James A. Robinson 0 siblings, 0 replies; 11+ messages in thread From: James A. Robinson @ 2016-10-19 17:16 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 393 bytes --] On Wed, Oct 19, 2016 at 10:13 AM Aram Hăvărneanu <aram.h@mgk.ro> wrote: > There are cheaper ways of disposing of 10TB of data. > If I decide the configuration is problematic I'm sure I can repurpose the device. Besides, the costs of spinning disk these days is amazingly low. As, I think, the developers for Plan 9 expected when they decided to move from WORM to HDD. Jim [-- Attachment #2: Type: text/html, Size: 763 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Fossil+Venti system memory requirements to be aware of? 2016-10-19 8:41 [9fans] Fossil+Venti system memory requirements to be aware of? James A. Robinson 2016-10-19 10:17 ` Aram Hăvărneanu @ 2016-10-19 16:44 ` Steven Stallion 2016-10-20 18:15 ` cigar562hfsp952fans 2016-11-02 18:44 ` James A. Robinson 2016-10-20 17:50 ` cigar562hfsp952fans 2 siblings, 2 replies; 11+ messages in thread From: Steven Stallion @ 2016-10-19 16:44 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Hi Jim, It probably helps to break apart fossil and venti for the sake of the conversation. While you can use fossil as a standalone filesystem, it is effectively your write cache in this scenario since it will be backed by venti. Conventional wisdom is to size your main fossil fs based on how much you plan to write in a given day. In my personal setup, I use a 32GB main fs, with the rest of a mirrored SSD pair dedicated to an "other" fossil partition where I place data that doesn't get backed up. Sizing venti is also simple. Start with something manageable; I chose 32GB at the time. Venti behaves much like a WORM with some additional deduplication (I'm glossing over gobs of detail here), so you'll likely need much less than you think. You can grow your venti story over time if you need, and it's also relatively trivial to move your data between stores. In short, start small and grow as needed. For reference, when I ran Coraid's fs based on 64-bit Ken's (WORM only, no dedupe) in RWC (based on the main fs in Athens). Over the course of a few years the entire WORM grew to around 35GB. This was for a couple dozen people working full time. I believe I had the LUN configured to something rather large at the time - around 12TB. All wasted space. HTH, Steve On Wed, Oct 19, 2016 at 3:41 AM, James A. Robinson <jim.robinson@gmail.com> wrote: > Anyone able to tell me whether or not there are > disk size limits I should beware of given a limited > amount of system memory in a file server? > > What I'm wanting to try and do is get a hardware > RAID1+0 enclosure and put in 20TB of disk (so > 10TB of usable space). > > The board I am looking at will take up to 8 gb of > system memory. Is that going to place any sort > of a limit on fossil+venti partition sizes? > > There are old threads that discuss a problem > with venti and the user hitting swap (the amount > of memory wasn't specified, at least not as > far as I could find in my archive or via google). > > > Jim > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Fossil+Venti system memory requirements to be aware of? 2016-10-19 16:44 ` Steven Stallion @ 2016-10-20 18:15 ` cigar562hfsp952fans 2016-10-20 18:41 ` Steven Stallion 2016-11-02 18:44 ` James A. Robinson 1 sibling, 1 reply; 11+ messages in thread From: cigar562hfsp952fans @ 2016-10-20 18:15 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Steven Stallion <sstallion@gmail.com> writes: > Sizing venti is also simple. I disagree with this. The best way to configure venti depends largely on how you plan to use it. I have multiple venti servers configured for different uses. For example, I keep my DVD images on a different venti server than I do for smaller, more frequently-used files such as mail. Choosing a good arena size isn't trivial, either. If you ever plan to back-up your venti to optical media, 640 MiB is the ideal arena size. There is a lot of math behind that determination, but its implications can be summarized as follows: Whether you're burning to standard CD-Rs, extended CD-Rs, DVDs, DVD DL, Blu-ray, flash media, or to another HDD, the amount of "wasted" space will never exceed 5% of the media's total capacity. In other words, the archvie will always be a minimum of 95% space-efficient. (Caveat: when burning 640 MiB arenas to optical media, you may need to turn off Joliet/Rock Rigde extensions in order to make the ISO image fit.) > and it's also relatively trivial to move your data between stores. I don't know that to be true. If the source and destination volumes are the same size, yes, it's easy to just dd the arenas. But if you're relocating to a smaller/larger partition, what would be an easy way to do that? ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Fossil+Venti system memory requirements to be aware of? 2016-10-20 18:15 ` cigar562hfsp952fans @ 2016-10-20 18:41 ` Steven Stallion 2016-10-20 21:42 ` Steve Simon 2016-10-20 21:54 ` Steve Simon 0 siblings, 2 replies; 11+ messages in thread From: Steven Stallion @ 2016-10-20 18:41 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Thu, Oct 20, 2016 at 1:15 PM, <cigar562hfsp952fans@icebubble.org> wrote: > Steven Stallion <sstallion@gmail.com> writes: > >> Sizing venti is also simple. > > I disagree with this. The best way to configure venti depends largely > on how you plan to use it. I have multiple venti servers configured for > different uses. For example, I keep my DVD images on a different venti > server than I do for smaller, more frequently-used files such as mail. see venti(8); there's plenty of advice on configuring your store with rough estimations for an initial conf file. Once you decide your geometry, it's trivial to extend the number of arenas in the store, just remember to reindex. > I don't know that to be true. If the source and destination volumes are > the same size, yes, it's easy to just dd the arenas. But if you're > relocating to a smaller/larger partition, what would be an easy way to > do that? see venti-backup(8). ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Fossil+Venti system memory requirements to be aware of? 2016-10-20 18:41 ` Steven Stallion @ 2016-10-20 21:42 ` Steve Simon 2016-10-20 21:54 ` Steve Simon 1 sibling, 0 replies; 11+ messages in thread From: Steve Simon @ 2016-10-20 21:42 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs > On 20 Oct 2016, at 19:41, Steven Stallion <sstallion@gmail.com> wrote: > >> On Thu, Oct 20, 2016 at 1:15 PM, <cigar562hfsp952fans@icebubble.org> wrote: >> Steven Stallion <sstallion@gmail.com> writes: >> >>> Sizing venti is also simple. >> >> I disagree with this. The best way to configure venti depends largely >> on how you plan to use it. I have multiple venti servers configured for >> different uses. For example, I keep my DVD images on a different venti >> server than I do for smaller, more frequently-used files such as mail. > > see venti(8); there's plenty of advice on configuring your store with > rough estimations for an initial conf file. Once you decide your > geometry, it's trivial to extend the number of arenas in the store, > just remember to reindex. > >> I don't know that to be true. If the source and destination volumes are >> the same size, yes, it's easy to just dd the arenas. But if you're >> relocating to a smaller/larger partition, what would be an easy way to >> do that? > > see venti-backup(8). ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Fossil+Venti system memory requirements to be aware of? 2016-10-20 18:41 ` Steven Stallion 2016-10-20 21:42 ` Steve Simon @ 2016-10-20 21:54 ` Steve Simon 1 sibling, 0 replies; 11+ messages in thread From: Steve Simon @ 2016-10-20 21:54 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs i agree absolutely with steve here, expanding venture arena by arena is easy, the ventibackup scripts show you how. even easier is to add arenas on a different disk partition to the same venti. personally i wouldn't keep music or videos in venti. they don't compress well using the arithmetic techniques in venti and the de duplication is unlikely to be a win either. just put them on a separate mirrored partition. i wrote a doc called venti rescue which details replacing disks and rebuilding your venti. it may be of help. -Steve > On 20 Oct 2016, at 19:41, Steven Stallion <sstallion@gmail.com> wrote: > >> On Thu, Oct 20, 2016 at 1:15 PM, <cigar562hfsp952fans@icebubble.org> wrote: >> Steven Stallion <sstallion@gmail.com> writes: >> >>> Sizing venti is also simple. >> >> I disagree with this. The best way to configure venti depends largely >> on how you plan to use it. I have multiple venti servers configured for >> different uses. For example, I keep my DVD images on a different venti >> server than I do for smaller, more frequently-used files such as mail. > > see venti(8); there's plenty of advice on configuring your store with > rough estimations for an initial conf file. Once you decide your > geometry, it's trivial to extend the number of arenas in the store, > just remember to reindex. > >> I don't know that to be true. If the source and destination volumes are >> the same size, yes, it's easy to just dd the arenas. But if you're >> relocating to a smaller/larger partition, what would be an easy way to >> do that? > > see venti-backup(8). ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Fossil+Venti system memory requirements to be aware of? 2016-10-19 16:44 ` Steven Stallion 2016-10-20 18:15 ` cigar562hfsp952fans @ 2016-11-02 18:44 ` James A. Robinson 2016-11-02 19:02 ` Steven Stallion 1 sibling, 1 reply; 11+ messages in thread From: James A. Robinson @ 2016-11-02 18:44 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 1366 bytes --] On Wed, Oct 19, 2016 at 9:47 AM Steven Stallion <sstallion@gmail.com> wrote: > In short, start small and grow as needed. For reference, when I ran > Coraid's fs based on 64-bit Ken's (WORM only, no dedupe) in RWC > (based on the main fs in Athens). Over the course of a few years > the entire WORM grew to around 35GB. This was for a couple dozen > people working full time. I believe I had the LUN configured to > something rather large at the time - around 12TB. All wasted space. Hi, So if you were setting up a new filesystem for the same group, and were using venti, would you be using the default 500 mb arena size (meaning 70+ arenas), or would you be sizing them at a larger size, like 1 gb, 2 gb, etc? The "setting up venti" page indicates that multiple small disks help with performance on venti due to using more indexes, but I can't tell if that translates to more partitions (e.g., 1 RAID 1+0 disk split into multiple smaller partitions, and divvied up into isectX and arenaX drives). I guess it probably comes down to how well (or poorly) the underlying disk array performs at random read/write ops. I see that AWS S3 allows for up to 5 gb in a single PUT, or 5 tb using multipart streams, and am thinking that using 1 to 5 gb arenas might make sense as a more modern "sized for for easy offline backup" default. Jim [-- Attachment #2: Type: text/html, Size: 1805 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Fossil+Venti system memory requirements to be aware of? 2016-11-02 18:44 ` James A. Robinson @ 2016-11-02 19:02 ` Steven Stallion 0 siblings, 0 replies; 11+ messages in thread From: Steven Stallion @ 2016-11-02 19:02 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Hi Jim, It's important to point out that the arena size does not have to match the size of an arenas file. In my case, I do something similar where I use 2GB for an arena but keep my arenas files at 2GB (I don't have much use for keeping multiple arena files). More indexes help to an extent. My fs isn't under exceptionally heavy load, but I've found that 2 seem to work out nicely for my configuration. Steve On Wed, Nov 2, 2016 at 1:44 PM, James A. Robinson <jimr@highwire.org> wrote: > On Wed, Oct 19, 2016 at 9:47 AM Steven Stallion <sstallion@gmail.com> wrote: >> In short, start small and grow as needed. For reference, when I ran >> Coraid's fs based on 64-bit Ken's (WORM only, no dedupe) in RWC >> (based on the main fs in Athens). Over the course of a few years >> the entire WORM grew to around 35GB. This was for a couple dozen >> people working full time. I believe I had the LUN configured to >> something rather large at the time - around 12TB. All wasted space. > > Hi, > > So if you were setting up a new filesystem for the same group, and > were using venti, would you be using the default 500 mb arena size > (meaning 70+ arenas), or would you be sizing them at a larger size, > like 1 gb, 2 gb, etc? > > The "setting up venti" page indicates that multiple small disks > help with performance on venti due to using more indexes, but I > can't tell if that translates to more partitions (e.g., 1 RAID 1+0 > disk split into multiple smaller partitions, and divvied up into > isectX and arenaX drives). I guess it probably comes down to > how well (or poorly) the underlying disk array performs at random > read/write ops. > > I see that AWS S3 allows for up to 5 gb in a single PUT, or 5 tb > using multipart streams, and am thinking that using 1 to 5 gb > arenas might make sense as a more modern "sized for for easy > offline backup" default. > > Jim > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [9fans] Fossil+Venti system memory requirements to be aware of? 2016-10-19 8:41 [9fans] Fossil+Venti system memory requirements to be aware of? James A. Robinson 2016-10-19 10:17 ` Aram Hăvărneanu 2016-10-19 16:44 ` Steven Stallion @ 2016-10-20 17:50 ` cigar562hfsp952fans 2 siblings, 0 replies; 11+ messages in thread From: cigar562hfsp952fans @ 2016-10-20 17:50 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs "James A. Robinson" <jim.robinson@gmail.com> writes: > Anyone able to tell me whether or not there are > disk size limits I should beware of given a limited > amount of system memory in a file server? Although there have been some replies on this thread, none of them have really yet directly answered your question. When setting up a venti server, the size of the Bloom filter has a big impact on memory usage and performance. Details about computing Bloom filter size can be found in the docs. You could set up venti on a machine with very little memory if you're happy with it constantly going out to check the index. ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2016-11-02 19:02 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-10-19 8:41 [9fans] Fossil+Venti system memory requirements to be aware of? James A. Robinson 2016-10-19 10:17 ` Aram Hăvărneanu 2016-10-19 17:16 ` James A. Robinson 2016-10-19 16:44 ` Steven Stallion 2016-10-20 18:15 ` cigar562hfsp952fans 2016-10-20 18:41 ` Steven Stallion 2016-10-20 21:42 ` Steve Simon 2016-10-20 21:54 ` Steve Simon 2016-11-02 18:44 ` James A. Robinson 2016-11-02 19:02 ` Steven Stallion 2016-10-20 17:50 ` cigar562hfsp952fans
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).