From mboxrd@z Thu Jan 1 00:00:00 1970 To: 9fans@9fans.net Subject: venti survery results From: "Russ Cox" Date: Fri, 4 Jul 2008 00:18:26 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Message-Id: <20080704041619.B3D831E8C7F@holo.morphisms.net> Topicbox-Message-UUID: da986b04-ead3-11e9-9d60-3106f5b1d025 The venti surveys have stopped trickling in. As promised, here is a summary. Below is a table summarizing the venti data. Each line is one server that someone submitted. I am surprised that there are people out there with 4+ year old servers. You take very good care of your disks. I suspect that most of the large but new servers replaced older ones. Most people who replied had less than 2 million blocks, and I suspect most people who didn't reply have servers like the bottom half of the table rather than the top half. You could keep a million-block index in 50 MB of memory and do away with the index hash table completely. It might be worth having venti detect when the index cache can fit the entire index and operate entirely out of the cache, without ever needing to read from the hash table. You could get much better performance out of venti doing that, and not even need a bloom filter. In fact, in that mode you wouldn't even need to configure an index disk. Then there are the power users, with their tens or hundreds of millions of blocks. One GB of index cache only gets you thirty million index entries, so these people have essentially no choice but to maintain the hash table index with its heavy seek penalties. Maybe in a few years flash storage will save the day. Russ GB GB days clumps cclumps raw compr age 338708170 279714997 1158.6 593.4 681 most clumps 243894713 207350734 5614.7 1657.2 69 most bytes 148582718 105977246 447.2 186.5 561 45255915 23277726 135.0 87.0 169 27395797 15624078 95.9 62.9 7 22095841 12085340 78.9 53.4 66 10194765 6088883 38.8 22.7 - 7991230 6622335 34.4 17.3 1768 6618670 2068276 46.2 39.5 60 6158824 4209478 37.7 21.2 1780 5971129 4325778 40.4 23.8 361 5805550 2246660 40.2 30.4 41 4900543 3749792 23.4 12.2 132 4151139 274551 31.3 30.4 214 2465171 1065357 16.3 12.4 1044 1756289 324119 12.6 11.4 1984 oldest 1544611 17057 23.4 23.3 359 1353686 901896 2.4 1.1 - 1308981 981760 7.3 4.0 1413 1279725 1073807 7.2 3.1 1544 1197798 961554 6.8 3.3 1436 1136953 609235 6.7 4.8 220 1082686 971796 6.9 1.9 1119 837292 347263 5.5 4.2 239 834808 549286 5.3 3.2 1164 742362 609642 4.1 1.8 490 641818 457784 3.8 2.2 1631 slowest growth 279319 222898 1.6 0.8 - 250992 168172 1.4 0.8 547 238121 215040 1.4 0.5 268 213772 183795 1.2 0.5 42 151431 134091 0.9 0.4 108 141257 121809 0.8 0.3 208 122807 105275 0.6 0.3 32 70 67 0.0 0.0 0 newest