= On 26 February 2017 at 17:25, Bakul Shah <bakul@bitblocks.com> wrote:

Venti is similarly corru= ptible, right? Since the checksum is over just the content. If you download= ed=C2=A0https://shattered.io/static/shattered-1.pdf=C2=A0and=C2=A0<= a href=3D"https://shattered.io/static/shattered-2.pdf" target=3D"_blank">https://shattered.io/static/shattered-2.pdf, venti would lose t= he contents of one.

Luckily, (a) they are both = bigger than the block size usually configured, over which the hash is calcu= lated, and (b) in case someone tries it, you've actually linked to the = same file (-2.pdf) but under different names, so there won't be a colli= sion by following your links. Hurrah!

<= /div>

Venti detects a collision on the attempt to= write the second copy if that differs from the earlier one stored (error &= quot;store collision"). The earlier copy is untouched (venti anyway is= write-once per score).

Fossil doesn't = handle it well, because it turns up during archiving and ends up marking th= e archive attempt as failed, but it will try again.

Meanwhile, you've got time to change fossil to check the venti= error return for "score collision" and announce it, loudly, disc= arding the second one.

Obviously if you car= e about something, make sure your version is in venti first! Chances are th= at collisions arise from naughty people tricking you later. Probably.

--001a11489ade55040c054972f7d8-- From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> From: Charles Forsyth Date: Sun, 26 Feb 2017 18:25:34 +0000 Message-ID: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: multipart/alternative; boundary=001a11489adeb57ff20549731624 Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b57f0ae8-ead9-11e9-9d60-3106f5b1d025 --001a11489adeb57ff20549731624 Content-Type: text/plain; charset=UTF-8 It's curious that svn "corrupts" the repository, if that's really what they mean, when two leaf files collide. An index or directory colliding with a file would be more understandable. On 26 February 2017 at 18:16, Charles Forsyth wrote: > > On 26 February 2017 at 17:25, Bakul Shah wrote: > >> Venti is similarly corruptible, right? Since the checksum is over just >> the content. If you downloaded https://shattered.i >> o/static/shattered-1.pdf >> and https://shattered.io/static/shattered-2.pdf, venti would lose the >> contents of one. >> > > Luckily, (a) they are both bigger than the block size usually configured, > over which the hash is calculated, and (b) in case someone tries it, you've > actually linked to the same file (-2.pdf) but under different names, so > there won't be a collision by following your links. Hurrah! > > Venti detects a collision on the attempt to write the second copy if that > differs from the earlier one stored (error "store collision"). The earlier > copy is untouched (venti anyway is write-once per score). > Fossil doesn't handle it well, because it turns up during archiving and > ends up marking the archive attempt as failed, but it will try again. > Meanwhile, you've got time to change fossil to check the venti error > return for "score collision" and announce it, loudly, discarding the second > one. > Obviously if you care about something, make sure your version is in venti > first! Chances are that collisions arise from naughty people tricking you > later. Probably. > --001a11489adeb57ff20549731624 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

It's curious that svn "corrupts" the reposit= ory, if that's really what they mean, when two leaf files collide.

= An index or directory colliding with a file would be more understandable.

On 26 F= ebruary 2017 at 18:16, Charles Forsyth <charles.forsyth@gmail.com<= /a>> wrote:

On= 26 February 2017 at 17:25, Bakul Shah <bakul@bitblocks.com> wrote:
Venti is similarly corrupt= ible, right? Since the checksum is over just the content. If you downloaded= =C2=A0https://shattered.io/static/shattered-1.pdf=C2=A0and=C2=A0https://shattered.io/static/shattered-2.pdf, venti would lose the= contents of one.

Luckily, (a) they are = both bigger than the block size usually configured, over which the hash is = calculated, and (b) in case someone tries it, you've actually linked to= the same file (-2.pdf) but under different names, so there won't be a = collision by following your links. Hurrah!
=
Venti detects a collision on the attem= pt to write the second copy if that differs from the earlier one stored (er= ror "store collision"). The earlier copy is untouched (venti anyw= ay is write-once per score).
Fossil doesn&#= 39;t handle it well, because it turns up during archiving and ends up marki= ng the archive attempt as failed, but it will try again.
Meanwhile, you've got time to change fossil to check the = venti error return for "score collision" and announce it, loudly,= discarding the second one.
Obviously if yo= u care about something, make sure your version is in venti first! Chances a= re that collisions arise from naughty people tricking you later. Probably.<= /div>

--001a11489adeb57ff20549731624-- From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> From: Charles Forsyth Date: Sun, 26 Feb 2017 18:29:51 +0000 Message-ID: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: multipart/alternative; boundary=94eb2c0703560e1c4705497326c0 Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5831066-ead9-11e9-9d60-3106f5b1d025 --94eb2c0703560e1c4705497326c0 Content-Type: text/plain; charset=UTF-8 On 26 February 2017 at 17:30, Jules Merit wrote: > there is a backdoor when a score of 4, what data produces it i have no > idea. > where is that? I had a quick look but couldn't find it. --94eb2c0703560e1c4705497326c0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

= On 26 February 2017 at 17:30, Jules Merit <jules.merit.eur= ocorp.us@gmail.com> wrote:

=
there is a backdoor when a score of 4, what data produces = it i have no idea.

where is that= ? I had a quick look but couldn't find it.

--94eb2c0703560e1c4705497326c0-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bakul Shah Content-Type: multipart/alternative; boundary=Apple-Mail-6A28A2D7-2A22-4426-8950-8B29F558BD77 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) Date: Sun, 26 Feb 2017 10:48:19 -0800 Message-Id: References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> In-Reply-To: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5872b92-ead9-11e9-9d60-3106f5b1d025 --Apple-Mail-6A28A2D7-2A22-4426-8950-8B29F558BD77 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable The links are to different files. The pdfs look identical except for color b= ackground. The diff bytes are 193..320. The rest is the same so your first 8= k byte checksum would be the same.=20 > On Feb 26, 2017, at 10:16 AM, Charles Forsyth w= rote: >=20 >=20 >> On 26 February 2017 at 17:25, Bakul Shah wrote: >> Venti is similarly corruptible, right? Since the checksum is over just th= e content. If you downloaded https://shattered.io/static/shattered-1.pdf and= https://shattered.io/static/shattered-2.pdf, venti would lose the contents o= f one. >=20 > Luckily, (a) they are both bigger than the block size usually configured, o= ver which the hash is calculated, and (b) in case someone tries it, you've a= ctually linked to the same file (-2.pdf) but under different names, so there= won't be a collision by following your links. Hurrah! >=20 > Venti detects a collision on the attempt to write the second copy if that d= iffers from the earlier one stored (error "store collision"). The earlier co= py is untouched (venti anyway is write-once per score). > Fossil doesn't handle it well, because it turns up during archiving and en= ds up marking the archive attempt as failed, but it will try again. > Meanwhile, you've got time to change fossil to check the venti error retur= n for "score collision" and announce it, loudly, discarding the second one. > Obviously if you care about something, make sure your version is in venti f= irst! Chances are that collisions arise from naughty people tricking you lat= er. Probably. --Apple-Mail-6A28A2D7-2A22-4426-8950-8B29F558BD77 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

The links are to different f= iles. The pdfs look identical except for color background. The diff bytes ar= e 193..320. The rest is the same so your first 8k byte checksum would be the= same.

On Feb 26, 2017, at 10:16 AM, Charles Forsyth <= ;charles.forsyth@gmail.com&= gt; wrote:

On 26 February 2017 at= 17:25, Bakul Shah <bakul@bitblocks.com> wrote:
Venti is similarly corruptible, right? Since the c= hecksum is over just the content. If you downloaded https://shattered.io/static/shattered-1.pdf and https://shattered.io/static= /shattered-2.pdf, venti would lose the contents of one.

Luckily, (a) they are both bigger than the block size usual= ly configured, over which the hash is calculated, and (b) in case someone tr= ies it, you've actually linked to the same file (-2.pdf) but under different= names, so there won't be a collision by following your links. Hurrah!
=

Venti detect= s a collision on the attempt to write the second copy if that differs from t= he earlier one stored (error "store collision"). The earlier copy is untouch= ed (venti anyway is write-once per score).
Fo= ssil doesn't handle it well, because it turns up during archiving and ends u= p marking the archive attempt as failed, but it will try again.
Meanwhile, you've got time to change fossil to check the= venti error return for "score collision" and announce it, loudly, discardin= g the second one.
Obviously if you care abou= t something, make sure your version is in venti first! Chances are that coll= isions arise from naughty people tricking you later. Probably.

= --Apple-Mail-6A28A2D7-2A22-4426-8950-8B29F558BD77-- From mboxrd@z Thu Jan 1 00:00:00 1970 To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> In-reply-to: Your message of "Sun, 26 Feb 2017 18:25:34 GMT." References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> Date: Sun, 26 Feb 2017 11:46:18 -0800 From: Bakul Shah Message-Id: <20170226194618.E3E5F124AEA5@mail.bitblocks.com> Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b58b72ce-ead9-11e9-9d60-3106f5b1d025 On Sun, 26 Feb 2017 18:25:34 GMT Charles Forsyth wrote: > > It's curious that svn "corrupts" the repository, if that's really what they > mean, when two leaf files collide. > An index or directory colliding with a file would be more understandable. The only known collision is for files. I suspect this was seen as a "can't happen" event so may be dealing with the error was not done right. You can read the report: https://bugs.webkit.org/show_bug.cgi?id=168774 > > Venti detects a collision on the attempt to write the second copy if that > > differs from the earlier one stored (error "store collision"). The earlier > > copy is untouched (venti anyway is write-once per score). Good to know at least it /detects/ score collisions. The concern would be that one of two colliding files *can't* be archived and it will be lost. We only have one example so it is not a big deal right now. > > Fossil doesn't handle it well, because it turns up during archiving and > > ends up marking the archive attempt as failed, but it will try again. > > Meanwhile, you've got time to change fossil to check the venti error > > return for "score collision" and announce it, loudly, discarding the second > > one. Hopefully the two versions can co-exist on fossil? > > Obviously if you care about something, make sure your version is in venti > > first! Chances are that collisions arise from naughty people tricking you > > later. Probably. Or may be you are doing research on collsions?! From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com>

In-Reply-To: From: Charles Forsyth Date: Sun, 26 Feb 2017 19:57:53 +0000 Message-ID: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: multipart/alternative; boundary=001a11c001987f97a1054974618a Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b58f9bba-ead9-11e9-9d60-3106f5b1d025 --001a11c001987f97a1054974618a Content-Type: text/plain; charset=UTF-8 On Sun, 26 Feb 2017, 18:49 Bakul Shah, wrote: > The links are to different files. > Not on Gmail at least look to see where each link points. Both are to -2 in the message I see on Gmail. Unless it cleverly optimised the"identical" content! --001a11c001987f97a1054974618a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Sun, 26 Feb 2017, 18= :49 Bakul Shah, <bakul@bitblocks.= com> wrote:

The links are to different files.=C2=A0

--001a11c001987f97a1054974618a-- From mboxrd@z Thu Jan 1 00:00:00 1970 Date: Sun, 26 Feb 2017 12:06:21 -0800 From: Jadon Bennett To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Message-ID: <20170226200621.GA96494@jben.ca> References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com>

MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5ac0502-ead9-11e9-9d60-3106f5b1d025 On Sun, Feb 26, 2017 at 07:57:53PM +0000, Charles Forsyth wrote: > On Sun, 26 Feb 2017, 18:49 Bakul Shah, wrote: > > > The links are to different files. > > > > Not on Gmail at least look to see where each link points. Both are to -2 > in the message I see on Gmail. Unless it cleverly optimised the"identical" > content! It's a multipart email; the HTML version had two of the same link, but readers viewing the plaintext version saw the different addresses. From mboxrd@z Thu Jan 1 00:00:00 1970 To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> In-reply-to: Your message of "Sun, 26 Feb 2017 19:57:53 GMT." References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com>

Date: Sun, 26 Feb 2017 12:16:11 -0800 From: Bakul Shah Message-Id: <20170226201611.55AF3124AEA5@mail.bitblocks.com> Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5b05a6c-ead9-11e9-9d60-3106f5b1d025 On Sun, 26 Feb 2017 19:57:53 GMT Charles Forsyth wrote: > > > The links are to different files. > > > > Not on Gmail at least look to see where each link points. Both are to -2 > in the message I see on Gmail. Unless it cleverly optimised the"identical" > content! I took at a look at the raw email file and you are right. Thanks iPad + apple Mail. I can paste a URL and it gets turned into a real link in the HTML part. But if I then edit the URL, the link doesn't change. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kim Shrier Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Date: Sun, 26 Feb 2017 14:02:45 -0700 References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> <20170226194618.E3E5F124AEA5@mail.bitblocks.com> To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> In-Reply-To: <20170226194618.E3E5F124AEA5@mail.bitblocks.com> Message-Id: <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net> Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5b487fe-ead9-11e9-9d60-3106f5b1d025 I have had a personal project on my list of "things to do when I have time", is to redo venti using sha256. Does any body see any problems with doing that? Kim From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net> References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> <20170226194618.E3E5F124AEA5@mail.bitblocks.com> <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net> From: Dave MacFarlane Date: Mon, 27 Feb 2017 10:46:17 -0500 Message-ID: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: text/plain; charset=UTF-8 Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5c09a6c-ead9-11e9-9d60-3106f5b1d025 Why not skip sha-256 and go directly to Sha3? On Sun, Feb 26, 2017 at 4:02 PM, Kim Shrier wrote: > I have had a personal project on my list of "things to do > when I have time", is to redo venti using sha256. Does > any body see any problems with doing that? > > Kim > > -- - Dave From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> <20170226194618.E3E5F124AEA5@mail.bitblocks.com> <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net> From: Charles Forsyth Date: Mon, 27 Feb 2017 16:47:52 +0000 Message-ID: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: multipart/alternative; boundary=001a114e4fbc2965c1054985d779 Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5c4ac24-ead9-11e9-9d60-3106f5b1d025 --001a114e4fbc2965c1054985d779 Content-Type: text/plain; charset=UTF-8 On 27 February 2017 at 15:46, Dave MacFarlane wrote: > Why not skip sha-256 and go directly to Sha3? blake2 has also been suggested --001a114e4fbc2965c1054985d779 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

= On 27 February 2017 at 15:46, Dave MacFarlane <driusan@gmail.com> wrote:

Why not skip sha-256 and go = directly to Sha3?

blake2 has also been suggested

--001a114e4fbc2965c1054985d779-- From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> <20170226194618.E3E5F124AEA5@mail.bitblocks.com> <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net> From: Charles Forsyth Date: Mon, 27 Feb 2017 17:07:29 +0000 Message-ID: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: multipart/alternative; boundary=001a1142e2225692fd0549861d2c Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5c8e28a-ead9-11e9-9d60-3106f5b1d025 --001a1142e2225692fd0549861d2c Content-Type: text/plain; charset=UTF-8 On 27 February 2017 at 16:47, Charles Forsyth wrote: > On 27 February 2017 at 15:46, Dave MacFarlane wrote: > >> Why not skip sha-256 and go directly to Sha3? > > > blake2 has also been suggested also, it's not clear it's urgent for venti. the scam is to make a new value that produces the same hash as an earlier important value where the hash plays a part in certifying the value, or where software uses the shorthand of comparing hashes to compare values and acts on that without comparing the values. with venti, the hash is produced as a side-effect of storing a value, and it also records the value itself. when the hash is presented, the stored block is returned. the hash itself is a compact address and doesn't certify the value (ie, nothing that uses venti assumes that it also certifies the value). any attempt to store a different value with the same hash will be detected. using any hash function has a chance of collision (newer, longer hashes reduce that, but it's rare as it is). because venti is write-once, no-one can change your venti contents subtly without access to the storage device, but if they've got access to the storage they don't need to be subtle. with the collision-maker and access to the storage device, they can make a previously certain vac: mean something different, but it still needs raw access to the device, it can't be done through the venti protocol. --001a1142e2225692fd0549861d2c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

= On 27 February 2017 at 16:47, Charles Forsyth <charles.forsyth@gma= il.com> wrote:

On 27 February 2017 at 15:46, Dave MacFarl= ane <driusan@gmail.com> wrote:
Why not skip sha-256 and go directly to Sha3?

blake2 has also been suggested

also, it's no= t clear it's urgent for venti. the scam is to make a new value that pro= duces the same hash as an earlier important value where the hash plays a pa= rt in certifying the value,

or where softwa= re uses the shorthand of comparing hashes to compare values and acts on tha= t without comparing the values.

with venti,= the hash is produced as a side-effect of storing a value, and it also reco= rds the value itself.

when the hash is pres= ented, the stored block is returned. the hash itself is a compact address a= nd doesn't certify the value (ie, nothing that uses venti assumes that = it also certifies the value).

any attempt t= o store a different value with the same hash will be detected. using any ha= sh function has a chance of collision (newer, longer hashes reduce that, bu= t it's rare as it is).

because venti is= write-once, no-one can change your venti contents subtly without access to= the storage device, but if they've got access to the storage they don&= #39;t need to be subtle.

with the collision= -maker and access to the storage device, they can make a previously certain= vac: mean something different, but it still needs raw access to the device= , it can't be done through

the venti pr= otocol.

--001a1142e2225692fd0549861d2c-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bakul Shah Content-Type: multipart/alternative; boundary=Apple-Mail-266E5DB3-5BD4-4448-8C01-46F00D3C5279 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) Date: Mon, 27 Feb 2017 09:28:55 -0800 Message-Id: References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> <20170226194618.E3E5F124AEA5@mail.bitblocks.com> <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net>

In-Reply-To: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5ce00f8-ead9-11e9-9d60-3106f5b1d025 --Apple-Mail-266E5DB3-5BD4-4448-8C01-46F00D3C5279 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable My argument is that an archival system that can't store some files, no matte= r how they were generated, is not good enough. A hash collision researcher m= ay have a legitimate reason to store such files. > On Feb 27, 2017, at 9:07 AM, Charles Forsyth w= rote: >=20 >=20 >> On 27 February 2017 at 16:47, Charles Forsyth = wrote: >>> On 27 February 2017 at 15:46, Dave MacFarlane wrote:= >>> Why not skip sha-256 and go directly to Sha3? >>=20 >> blake2 has also been suggested >=20 > also, it's not clear it's urgent for venti. the scam is to make a new valu= e that produces the same hash as an earlier important value where the hash p= lays a part in certifying the value, > or where software uses the shorthand of comparing hashes to compare values= and acts on that without comparing the values. > with venti, the hash is produced as a side-effect of storing a value, and i= t also records the value itself. > when the hash is presented, the stored block is returned. the hash itself i= s a compact address and doesn't certify the value (ie, nothing that uses ven= ti assumes that it also certifies the value). > any attempt to store a different value with the same hash will be detected= . using any hash function has a chance of collision (newer, longer hashes re= duce that, but it's rare as it is). > because venti is write-once, no-one can change your venti contents subtly w= ithout access to the storage device, but if they've got access to the storag= e they don't need to be subtle. > with the collision-maker and access to the storage device, they can make a= previously certain vac: mean something different, but it still needs raw ac= cess to the device, it can't be done through > the venti protocol. --Apple-Mail-266E5DB3-5BD4-4448-8C01-46F00D3C5279 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

My argument is that an arch= ival system that can't store some files, no matter how they were generated, i= s not good enough. A hash collision researcher may have a legitimate r= eason to store such files.

On Feb 27, 2017, at 9:07 AM, Charle= s Forsyth <charles.forsyth@g= mail.com> wrote:

On 27 Feb= ruary 2017 at 16:47, Charles Forsyth <charles.forsyth@gmail.com&= gt; wrote:
On 27 February 2017 at 15:46, Dave MacFarlane <driusan@gma= il.com> wrote:
Why not skip s= ha-256 and go directly to Sha3?

blake2 has also= been suggested

also, it's not clear it's urgent for v= enti. the scam is to make a new value that produces the same hash as an earl= ier important value where the hash plays a part in certifying the value,
or where software uses the shorthand of compari= ng hashes to compare values and acts on that without comparing the values.
with venti, the hash is produced as a side-ef= fect of storing a value, and it also records the value itself.
when the hash is presented, the stored block is returned.= the hash itself is a compact address and doesn't certify the value (ie, not= hing that uses venti assumes that it also certifies the value).
any attempt to store a different value with the same has= h will be detected. using any hash function has a chance of collision (newer= , longer hashes reduce that, but it's rare as it is).
because venti is write-once, no-one can change your venti contents= subtly without access to the storage device, but if they've got access to t= he storage they don't need to be subtle.
wit= h the collision-maker and access to the storage device, they can make a prev= iously certain vac: mean something different, but it still needs raw access t= o the device, it can't be done through
the v= enti protocol.

= --Apple-Mail-266E5DB3-5BD4-4448-8C01-46F00D3C5279-- From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> <20170226194618.E3E5F124AEA5@mail.bitblocks.com> <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net>

From: hiro <23hiro@gmail.com> Date: Mon, 27 Feb 2017 19:14:41 +0100 Message-ID: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: text/plain; charset=UTF-8 Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5d2b878-ead9-11e9-9d60-3106f5b1d025 Bakul: I want to store a 1000000000 Petabyte file, can your archival system support that? I want to research big files. There's always a limit, but when does it matter? From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bakul Shah Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) Date: Mon, 27 Feb 2017 10:20:08 -0800 Message-Id: References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> <20170226194618.E3E5F124AEA5@mail.bitblocks.com> <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net>

In-Reply-To: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5d71e7c-ead9-11e9-9d60-3106f5b1d025 The two are not comparable. > On Feb 27, 2017, at 10:14 AM, hiro <23hiro@gmail.com> wrote: > > Bakul: I want to store a 1000000000 Petabyte file, can your archival > system support that? I want to research big files. > > There's always a limit, but when does it matter? > From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> <20170226194618.E3E5F124AEA5@mail.bitblocks.com> <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net>

From: Charles Forsyth Date: Mon, 27 Feb 2017 18:30:48 +0000 Message-ID: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: multipart/alternative; boundary=94eb2c124ff44d78ad0549874788 Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5dbe5b0-ead9-11e9-9d60-3106f5b1d025 --94eb2c124ff44d78ad0549874788 Content-Type: text/plain; charset=UTF-8 On 27 February 2017 at 17:28, Bakul Shah wrote: > My argument is that an archival system that can't store some files, no > matter how they were generated, is not good enough. A hash collision > researcher may have a legitimate reason to store such files. > that's a separate argument that venti would never work for you, regardless of the hash algorithm used. --94eb2c124ff44d78ad0549874788 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

= On 27 February 2017 at 17:28, Bakul Shah <bakul@bitblocks.com> wrote:

My argument is that an a= rchival system that can't store some files, no matter how they were gen= erated, is not good enough.=C2=A0 A hash collision researcher may have a le= gitimate reason to store such files.

t= hat's a separate argument that venti would never work for you, regardle= ss of the hash algorithm used.

--94eb2c124ff44d78ad0549874788-- From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> <20170226194618.E3E5F124AEA5@mail.bitblocks.com> <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net>

From: Charles Forsyth Date: Mon, 27 Feb 2017 19:02:29 +0000 Message-ID: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: multipart/alternative; boundary=001a113e98ca9b9b41054987b8d0 Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5e42c7a-ead9-11e9-9d60-3106f5b1d025 --001a113e98ca9b9b41054987b8d0 Content-Type: text/plain; charset=UTF-8 On 27 February 2017 at 18:30, Charles Forsyth wrote: > that's a separate argument that venti would never work for you, regardless > of the hash algorithm used. since venti returns the resulting score from each write, and it knows whether there's been a collision, it appears it could return a modified score (having ensured that is now unique, "and the next judge said that's a very shaggy dog") --001a113e98ca9b9b41054987b8d0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

= On 27 February 2017 at 18:30, Charles Forsyth <charles.forsyth@gma= il.com> wrote:

that's a= separate argument that venti would never work for you, regardless of the h= ash algorithm used.

since venti returns the resulting= score from each write, and it knows whether there's been a collision,<= /div>

it appears it could return a modified score= (having ensured that is now unique, "and the next judge said that'= ;s a very shaggy dog")

--001a113e98ca9b9b41054987b8d0-- From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 References: <8D987F97-4760-4243-A9E7-F2F3BA9C63E3@bitblocks.com> <20170226194618.E3E5F124AEA5@mail.bitblocks.com> <7F0C4A3F-0EC1-437B-BE8C-9BC97BE651E9@westryn.net>

In-Reply-To: From: Skip Tavakkolian Date: Mon, 27 Feb 2017 19:34:54 +0000 Message-ID: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: multipart/alternative; boundary=001a113d1b302504d90549882d71 Subject: Re: [9fans] SHA-1 collision and venti Topicbox-Message-UUID: b5e82776-ead9-11e9-9d60-3106f5b1d025 --001a113d1b302504d90549882d71 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I wondered if one could make a logical argument that says, one could use a combination of hashes that have different collision resistances (e.g. SHA1=E2=8A=95MD5) for each file, extending to any number of hashes to satis= fy that the combination is unique for all files... So I did a little research, and the short answer is NO! It turns out that the combined hash would be no better than the best of the hash functions in the combo: http://crypto.stackexchange.com/questions/270/guarding-against-cryptanalyti= c-breakthroughs-combining-multiple-hash-functions The Internet is a wonderful thing. On Mon, Feb 27, 2017 at 9:29 AM Bakul Shah wrote: > My argument is that an archival system that can't store some files, no > matter how they were generated, is not good enough. A hash collision > researcher may have a legitimate reason to store such files. > > On Feb 27, 2017, at 9:07 AM, Charles Forsyth > wrote: > > > On 27 February 2017 at 16:47, Charles Forsyth > wrote: > > On 27 February 2017 at 15:46, Dave MacFarlane wrote: > > Why not skip sha-256 and go directly to Sha3? > > > blake2 has also been suggested > > > also, it's not clear it's urgent for venti. the scam is to make a new > value that produces the same hash as an earlier important value where the > hash plays a part in certifying the value, > or where software uses the shorthand of comparing hashes to compare value= s > and acts on that without comparing the values. > with venti, the hash is produced as a side-effect of storing a value, and > it also records the value itself. > when the hash is presented, the stored block is returned. the hash itself > is a compact address and doesn't certify the value (ie, nothing that uses > venti assumes that it also certifies the value). > any attempt to store a different value with the same hash will be > detected. using any hash function has a chance of collision (newer, longe= r > hashes reduce that, but it's rare as it is). > because venti is write-once, no-one can change your venti contents subtly > without access to the storage device, but if they've got access to the > storage they don't need to be subtle. > with the collision-maker and access to the storage device, they can make = a > previously certain vac: mean something different, but it still needs raw > access to the device, it can't be done through > the venti protocol. > > --001a113d1b302504d90549882d71 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

I wondered if one could make a logical argument that says,= one could use a combination of hashes that have different collision resist= ances (e.g. SHA1=E2=8A=95MD5) for each file, extending to any number of has= hes to satisfy that the combination is unique for all files...

So I did a little research, and the short answer is NO!=C2=A0 It tur= ns out that the combined hash would be no better than the best of the hash = functions in the combo:

http://crypto.stackexchange.com= /questions/270/guarding-against-cryptanalytic-breakthroughs-combining-multi= ple-hash-functions

The Internet is a wonderful= thing.

On Mon, F= eb 27, 2017 at 9:29 AM Bakul Shah <bakul@bitblocks.com> wrote:

My argument is that an archival system that can't = store some files, no matter how they were generated, is not good enough.=C2= =A0 A hash collision researcher may have a legitimate reason to store such = files.

On Feb 27, 201= 7, at 9:07 AM, Charles Forsyth <charles.forsyth@gmail.com>= ; wrote:

On 27 February 2017 at 16:47, Ch= arles Forsyth <charles.for= syth@gmail.com> wrote:

On 27 February 2017 at 15:46, Dave MacFarlan= e <driusan@gmail.com> wrote:
Wh= y not skip sha-256 and go directly to Sha3?

<= div dir=3D"auto" class=3D"gmail_msg">

blake2 has also been suggested

also, it's not clear it's urgent f= or venti. the scam is to make a new value that produces the same hash as an= earlier important value where the hash plays a part in certifying the valu= e,
or where software uses the sho= rthand of comparing hashes to compare values and acts on that without compa= ring the values.
with venti, the = hash is produced as a side-effect of storing a value, and it also records t= he value itself.
when the hash is= presented, the stored block is returned. the hash itself is a compact addr= ess and doesn't certify the value (ie, nothing that uses venti assumes = that it also certifies the value).
because venti is write-once, no-one can change your venti contents= subtly without access to the storage device, but if they've got access= to the storage they don't need to be subtle.
with the collision-maker and access to the storage device,= they can make a previously certain vac: mean something different, but it s= till needs raw access to the device, it can't be done through
the venti protocol.