From: "Russ Cox" <rsc@swtch.com>
To: weigelt@metux.de, 9fans@cse.psu.edu
Subject: Re: [9fans] thoughs about venti+fossil
Date: Thu, 6 Mar 2008 11:58:28 -0500 [thread overview]
Message-ID: <20080306165916.7B88C1E8C22@holo.morphisms.net> (raw)
In-Reply-To: <20080306123941.GE18329@nibiru.local>
> (we couldn't use hashing for traffic reductions, safely).
yes you can. you can use hashes to build a hash table
with a collision policy. there is some company
(whose name escapes me; maybe someone else will
remember) that makes exactly this product, so that
once network A has sent a particular chunk of data to
network B once, future transmissions are replaced
transparently with a shorter name. kind of like
lempel-ziv on steroids. apparently it makes
cross-country ms exchange servers and file servers
much more bearable.
> it would be an interesting feature. Of course the fs on top then
> MUST refresh from time to time, but this can be done while the
> system is idle (good for situations with high load peaks and enough
> idle time on the other hand).
sorry, but this is just a fantastically terrible idea.
you're taking a reliable system and making it unreliable.
if you were really concerned, it would be better
to implement a garbage collector that you could
hand a root set. even that would worry me (a simple
bug would wipe out your entire archive), but it
wouldn't be as bad as relying on timeouts.
> For this I need to be *sure* that there will be
> *no* collissions, even if the system runs for a long time and
> grows really big (maybe several PB on thousands of nodes).
>
> Another interesting question: can the risk of colissions be
> reduced by combining several different hash functions in
> parallel ?
sure. use sha-256 and your probability of collision goes
down even further. but *you* (probably) still won't be *sure*.
russ
next prev parent reply other threads:[~2008-03-06 16:58 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-03-05 4:00 Enrico Weigelt
2008-03-05 4:11 ` Roman Shaposhnik
2008-03-05 4:43 ` erik quanstrom
2008-03-05 5:09 ` Roman Shaposhnik
2008-03-05 5:52 ` Enrico Weigelt
2008-03-05 6:24 ` geoff
2008-03-05 6:35 ` Taj Khattra
[not found] ` <7f575fa27b41329b9ae24f40e6e5a3cd@plan9.bell-labs.com>
2008-03-06 4:04 ` Enrico Weigelt
2008-03-06 4:13 ` Bruce Ellis
2008-03-06 4:15 ` andrey mirtchovski
2008-03-06 4:31 ` Bruce Ellis
2008-03-06 6:16 ` Enrico Weigelt
2008-03-06 18:50 ` ron minnich
2008-03-06 19:43 ` Charles Forsyth
2008-03-06 19:45 ` Paul Lalonde
2008-03-06 20:18 ` Bruce Ellis
2008-03-06 21:39 ` Paul Lalonde
2008-03-08 9:06 ` Enrico Weigelt
2008-03-06 22:10 ` Martin Harriss
2008-03-06 6:40 ` Enrico Weigelt
2008-03-06 14:35 ` erik quanstrom
2008-03-06 14:58 ` Tom Lieber
2008-03-06 15:09 ` Charles Forsyth
2008-03-06 17:09 ` Robert Raschke
2008-03-10 10:19 ` sqweek
2008-03-10 12:29 ` Gorka Guardiola
2008-03-10 13:20 ` erik quanstrom
2008-03-10 19:00 ` Wes Kussmaul
2008-03-10 19:27 ` erik quanstrom
2008-03-10 20:55 ` Bakul Shah
2008-03-11 2:04 ` Wes Kussmaul
2008-03-11 2:10 ` erik quanstrom
2008-03-11 6:03 ` Bruce Ellis
2008-03-10 16:18 ` Russ Cox
2008-03-10 18:06 ` Bruce Ellis
2008-03-10 18:31 ` Eric Van Hensbergen
2008-03-10 18:40 ` Bruce Ellis
2008-03-10 18:46 ` Geoffrey Avila
2008-03-10 20:28 ` Charles Forsyth
2008-03-10 21:35 ` Charles Forsyth
2008-03-06 9:54 ` Wilhelm B. Kloke
2008-03-08 9:37 ` Enrico Weigelt
2008-03-08 9:57 ` Bruce Ellis
2008-03-08 10:46 ` Charles Forsyth
2008-03-08 15:37 ` erik quanstrom
2008-03-06 4:40 ` cummij
2008-03-06 5:15 ` Bruce Ellis
2008-03-06 5:40 ` Uriel
2008-03-06 5:55 ` Bruce Ellis
2008-03-11 18:34 ` Uriel
2008-03-06 12:26 ` erik quanstrom
2008-03-05 5:04 ` geoff
2008-03-05 8:43 ` Charles Forsyth
2008-03-05 9:05 ` Gorka Guardiola
2008-03-05 14:33 ` Russ Cox
2008-03-06 12:39 ` Enrico Weigelt
2008-03-06 16:58 ` Russ Cox [this message]
2008-03-06 18:16 ` andrey mirtchovski
[not found] ` <a553f487750f88281db1cce3378577c7@terzarima.net>
2008-03-06 5:38 ` Enrico Weigelt
2008-03-06 9:44 ` Joel C. Salomon
2008-03-05 14:03 erik quanstrom
2008-03-05 16:00 ` Russ Cox
2008-03-06 19:09 Brian L. Stuart
2008-03-06 19:50 ` Charles Forsyth
2015-04-21 18:30 hruodr
2015-04-21 19:46 ` Russ Cox
2015-04-23 7:21 hruodr
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20080306165916.7B88C1E8C22@holo.morphisms.net \
--to=rsc@swtch.com \
--cc=9fans@cse.psu.edu \
--cc=weigelt@metux.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).