[9fans] Venti by another name

[9fans] Venti by another name

[matt]

I came across this today

http://offsystem.sourceforge.net/

It's a P2P system where data blocks are traded not files.

A file becomes a set of blocks and if requested, anyone who has the
block can supply the data, even if they don't possess the same file.

In that way no-one is sharing copyrighted material in the large, just
coincedent blocks.

I've not pushed any mpeg data into venti, though I have idly wondered if
there are any disk saving.

Re: [9fans] Venti by another name

[Devon H. O'Dell]

On this note, Marco Peereboom has been working on epitome:
http://www.peereboom.us/epitome

--dho

Re: [9fans] Venti by another name

[Joel C. Salomon]

On Sat, Feb 14, 2009 at 10:34 AM, matt <mattmobile@proweb.co.uk> wrote:
> It's a P2P system where data blocks are traded not files.
>
> A file becomes a set of blocks and if requested, anyone who has the block
> can supply the data, even if they don't possess the same file.

Sounds vaguely like Freenet.

—Joel

Re: [9fans] Venti by another name

[Eris Discordia]

BitTorrent protocol chops the upload into same-sized blocks and hashes each
block. Then the blocks are traded. It predates other P2P protocols that do
so. I don't know about anonymity/privacy guards (FreeNet was mentioned). In
BT these hashes are contained in the torrent file and used for integrity
checking so it can't be correlated with venti.

OFF's idea seems okay but it's rather trite. You can always represent
copyrighted content in different ways, pass it on, and claim you haven't
exchanged the "copyrighted bytes." That's why you are barred by the DMCA
from transcoding even for personal use the content for which you have an
"only personal home use" license (of course, you abide the law, don't you
;-). The person who did the transform/transcoding is still legally liable.

Also, you can always create a lookup table of all possible bit sequences of
a certain length and represent your data (padded to a multiple of that
length) as a chain of such sequences. That's no wonderwork.

> I've not pushed any mpeg data into venti, though I have idly wondered if
> there are any disk saving.

Can any lossless compression scheme improve on {J,M}PEG? If multiple
streams are stored in venti then there may be some common blocks but even
that should not significantly surpass RAR algorithm's "solid" archiving of
the same streams. Not that I understand these things.

--On Saturday, February 14, 2009 3:34 PM +0000 matt
<mattmobile@proweb.co.uk> wrote:

> I came across this today
>
> http://offsystem.sourceforge.net/
>
> It's a P2P system where data blocks are traded not files.
>
> A file becomes a set of blocks and if requested, anyone who has the block
> can supply the data, even if they don't possess the same file.
>
> In that way no-one is sharing copyrighted material in the large, just
> coincedent blocks.
>
> I've not pushed any mpeg data into venti, though I have idly wondered if
> there are any disk saving.
>
>
>
>

Re: [9fans] Venti by another name

[matt]

>
> Can any lossless compression scheme improve on {J,M}PEG? If multiple
> streams are stored in venti then there may be some common blocks but
> even that should not significantly surpass RAR algorithm's "solid"
> archiving of the same streams. Not that I understand these things.
>
Only if there are shared blocks. Macro blocks in MPEG are 8x8 pixels
which makes them < 32k in size. If you're lucky you might just align
them when written to Venti!

I'll know soon enough as I'm in the process of building a Venti store
for our video files. I just wondered if anyone had done it already.

I doubt there's much in it.

Matt

Re: [9fans] Venti by another name

[Eris Discordia]

> I'll know soon enough as I'm in the process of building a Venti store for
> our video files. I just wondered if anyone had done it already.

That kindles in me the bystander's interest. It'd be nice of you to report
back on results.

--On Monday, February 16, 2009 12:13 PM +0000 matt
<mattmobile@proweb.co.uk> wrote:

>
>>
>> Can any lossless compression scheme improve on {J,M}PEG? If multiple
>> streams are stored in venti then there may be some common blocks but
>> even that should not significantly surpass RAR algorithm's "solid"
>> archiving of the same streams. Not that I understand these things.
>>
> Only if there are shared blocks. Macro blocks in MPEG are 8x8 pixels
> which makes them < 32k in size. If you're lucky you might just align them
> when written to Venti!
>
> I'll know soon enough as I'm in the process of building a Venti store for
> our video files. I just wondered if anyone had done it already.
>
> I doubt there's much in it.
>
> Matt
>

Re: [9fans] Venti by another name

[Enrico Weigelt]

* matt <mattmobile@proweb.co.uk> wrote:

Hi,

> I'll know soon enough as I'm in the process of building a Venti store
> for our video files. I just wondered if anyone had done it already.

I'm actually using venti for storing media files in my MCloud
platform, which is used for several of my customer's portals,
(eg: http://www.television-europe.eu/).

But I really can't tell anything about the space saving.
(one of the primary decision for venti is that it will be
clustered some day ... ;-o)


cu
--
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [9fans] Venti by another name

[Roman V. Shaposhnik]

On Wed, 2009-02-25 at 01:26 +0100, Enrico Weigelt wrote:
> But I really can't tell anything about the space saving.
> (one of the primary decision for venti is that it will be
> clustered some day ... ;-o)

Clustered?

Thanks,
Roman.

Re: [9fans] Venti by another name

[Enrico Weigelt]

* Roman V. Shaposhnik <rvs@sun.com> wrote:
> On Wed, 2009-02-25 at 01:26 +0100, Enrico Weigelt wrote:
> > But I really can't tell anything about the space saving.
> > (one of the primary decision for venti is that it will be
> > clustered some day ... ;-o)
>
> Clustered?

Yeah, one of the projects I'm working on: an clustered version of
venti, which distributes the data amongst several nodes (multiple
copies per block for safety as well as faster access).


cu
--
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [9fans] Venti by another name

[lucio]

> On Wed, 2009-02-25 at 01:26 +0100, Enrico Weigelt wrote:
>> But I really can't tell anything about the space saving.
>> (one of the primary decision for venti is that it will be
>> clustered some day ... ;-o)
>
> Clustered?
>
Cloudstered?

++L