From: "Russ Cox" <rsc@swtch.com>
To: 9fans@9fans.net
Subject: venti survery results
Date: Fri, 4 Jul 2008 00:18:26 -0400 [thread overview]
Message-ID: <20080704041619.B3D831E8C7F@holo.morphisms.net> (raw)
The venti surveys have stopped trickling in. As promised,
here is a summary.
Below is a table summarizing the venti data.
Each line is one server that someone submitted.
I am surprised that there are people out there
with 4+ year old servers. You take very good care
of your disks. I suspect that most of the large but new
servers replaced older ones.
Most people who replied had less than 2 million blocks,
and I suspect most people who didn't reply have servers like
the bottom half of the table rather than the top half.
You could keep a million-block index in 50 MB of memory
and do away with the index hash table completely.
It might be worth having venti detect when the index
cache can fit the entire index and operate entirely out
of the cache, without ever needing to read from the
hash table. You could get much better performance
out of venti doing that, and not even need a bloom filter.
In fact, in that mode you wouldn't even need to configure
an index disk.
Then there are the power users, with their tens or hundreds
of millions of blocks. One GB of index cache only gets you
thirty million index entries, so these people have essentially
no choice but to maintain the hash table index with its
heavy seek penalties. Maybe in a few years flash storage will
save the day.
Russ
GB GB days
clumps cclumps raw compr age
338708170 279714997 1158.6 593.4 681 most clumps
243894713 207350734 5614.7 1657.2 69 most bytes
148582718 105977246 447.2 186.5 561
45255915 23277726 135.0 87.0 169
27395797 15624078 95.9 62.9 7
22095841 12085340 78.9 53.4 66
10194765 6088883 38.8 22.7 -
7991230 6622335 34.4 17.3 1768
6618670 2068276 46.2 39.5 60
6158824 4209478 37.7 21.2 1780
5971129 4325778 40.4 23.8 361
5805550 2246660 40.2 30.4 41
4900543 3749792 23.4 12.2 132
4151139 274551 31.3 30.4 214
2465171 1065357 16.3 12.4 1044
1756289 324119 12.6 11.4 1984 oldest
1544611 17057 23.4 23.3 359
1353686 901896 2.4 1.1 -
1308981 981760 7.3 4.0 1413
1279725 1073807 7.2 3.1 1544
1197798 961554 6.8 3.3 1436
1136953 609235 6.7 4.8 220
1082686 971796 6.9 1.9 1119
837292 347263 5.5 4.2 239
834808 549286 5.3 3.2 1164
742362 609642 4.1 1.8 490
641818 457784 3.8 2.2 1631 slowest growth
279319 222898 1.6 0.8 -
250992 168172 1.4 0.8 547
238121 215040 1.4 0.5 268
213772 183795 1.2 0.5 42
151431 134091 0.9 0.4 108
141257 121809 0.8 0.3 208
122807 105275 0.6 0.3 32
70 67 0.0 0.0 0 newest
next reply other threads:[~2008-07-04 4:18 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-07-04 4:18 Russ Cox [this message]
2008-07-04 9:07 ` [9fans] " Richard Miller
2008-07-04 9:46 ` Kernel Panic
2008-07-04 9:54 ` Richard Miller
2008-07-04 11:03 ` Kernel Panic
2008-07-04 14:04 ` erik quanstrom
2008-07-05 21:42 ` Richard Miller
2008-07-05 20:36 ` [9fans] via vt8237 erik quanstrom
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20080704041619.B3D831E8C7F@holo.morphisms.net \
--to=rsc@swtch.com \
--cc=9fans@9fans.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).