[9fans] de-duplication/archival software

[9fans] de-duplication/archival software

[Tharaneedharan Vilwanathan]

hi,

about a year back, someone mentioned about a tool or utility for
archiving/storage like venti. it has de-duplication and uses hashing
mechanism. i think it runs on linux mainly. can someone give me some
details on this if you remember?

sorry, i tried to dig as much as i can but couldnt locate that thread.

regards
dharani

Re: [9fans] de-duplication/archival software

[Bakul Shah]

On Thu, 14 Oct 2010 22:35:57 PDT Tharaneedharan Vilwanathan <vdharani@gmail.com>  wrote:
> hi,
>
> about a year back, someone mentioned about a tool or utility for
> archiving/storage like venti. it has de-duplication and uses hashing
> mechanism. i think it runs on linux mainly. can someone give me some
> details on this if you remember?

May be one of things mentioned on the the wikipedia article
on data duplication?

Note that ZFS now has de-duplication. It uses a crypto hash
as a checksum but not as a block id as in venti. ZFS runs on
(Open)Solaris and FreeBSD (not on the released version
though) and on top of FUSE on Linux.

Re: [9fans] de-duplication/archival software

[Tharaneedharan Vilwanathan]

hi bakul,

thanks for the quick response.

i did see the thread on zfs but what i remember is a tool rather than
a file system. i even remember green letter over black background
webpage that talked about this tool (manpage?). i will try for a
little more time and then probably give up.

regards
dharani

On Thu, Oct 14, 2010 at 11:06 PM, Bakul Shah <bakul+plan9@bitblocks.com> wrote:
> On Thu, 14 Oct 2010 22:35:57 PDT Tharaneedharan Vilwanathan <vdharani@gmail.com>  wrote:
>> hi,
>>
>> about a year back, someone mentioned about a tool or utility for
>> archiving/storage like venti. it has de-duplication and uses hashing
>> mechanism. i think it runs on linux mainly. can someone give me some
>> details on this if you remember?
>
> May be one of things mentioned on the the wikipedia article
> on data duplication?
>
> Note that ZFS now has de-duplication. It uses a crypto hash
> as a checksum but not as a block id as in venti. ZFS runs on
> (Open)Solaris and FreeBSD (not on the released version
> though) and on top of FUSE on Linux.
>
>

Re: [9fans] de-duplication/archival software

[Thomas Östreicher]

On Fri, Oct 15, 2010 at 7:35 AM, Tharaneedharan Vilwanathan
<vdharani@gmail.com> wrote:
> hi,
>
> about a year back, someone mentioned about a tool or utility for
> archiving/storage like venti. it has de-duplication and uses hashing
> mechanism. i think it runs on linux mainly. can someone give me some
> details on this if you remember?

Do you mean epitome?

Homepage: http://www.peereboom.us/epitome/
Man page:  http://www.peereboom.us/epitome/man/epitome.1.html

- thomas

Re: [9fans] de-duplication/archival software

[Enrico Weigelt]

* Tharaneedharan Vilwanathan <vdharani@gmail.com> wrote:
> hi,
>
> about a year back, someone mentioned about a tool or utility for
> archiving/storage like venti. it has de-duplication and uses hashing
> mechanism. i think it runs on linux mainly. can someone give me some
> details on this if you remember?

I'm currently working on such a thing (called "nebulon") - it
will be a kind of grid-venti which also supports strong
encryption. But still in design phase (yet lacking time :(),


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] de-duplication/archival software

[Tharaneedharan Vilwanathan]

hi thomas,

yes, this is the one. thanks a lot! i couldnt find it because my
search keywords didnt match.

regards
dharani

On Fri, Oct 15, 2010 at 3:20 AM, Thomas Östreicher
<thomas.oestreicher@gmail.com> wrote:
> On Fri, Oct 15, 2010 at 7:35 AM, Tharaneedharan Vilwanathan
> <vdharani@gmail.com> wrote:
>> hi,
>>
>> about a year back, someone mentioned about a tool or utility for
>> archiving/storage like venti. it has de-duplication and uses hashing
>> mechanism. i think it runs on linux mainly. can someone give me some
>> details on this if you remember?
>
> Do you mean epitome?
>
> Homepage: http://www.peereboom.us/epitome/
> Man page:  http://www.peereboom.us/epitome/man/epitome.1.html
>
> - thomas
>
>

Re: [9fans] de-duplication/archival software

[Nathaniel W Filardo]

[-- Attachment #1: Type: text/plain, Size: 659 bytes --]

On Fri, Oct 15, 2010 at 01:49:58PM +0200, Enrico Weigelt wrote:
> * Tharaneedharan Vilwanathan <vdharani@gmail.com> wrote:
> > hi,
> > 
> > about a year back, someone mentioned about a tool or utility for
> > archiving/storage like venti. it has de-duplication and uses hashing
> > mechanism. i think it runs on linux mainly. can someone give me some
> > details on this if you remember?
> 
> I'm currently working on such a thing (called "nebulon") - it
> will be a kind of grid-venti which also supports strong
> encryption. But still in design phase (yet lacking time :(),

You might look at http://tahoe-lafs.org/trac/tahoe-lafs ?

--nwf;

[-- Attachment #2: Type: application/pgp-signature, Size: 205 bytes --]