From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 1F67D82355 for ; Thu, 8 Feb 2018 16:43:01 +0100 (CET) Authentication-Results: mail3-smtp-sop.national.inria.fr; spf=None smtp.pra=zhenya1007@gmail.com; spf=Pass smtp.mailfrom=zhenya1007@gmail.com; spf=None smtp.helo=postmaster@mail-yw0-f173.google.com Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of zhenya1007@gmail.com) identity=pra; client-ip=209.85.161.173; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="zhenya1007@gmail.com"; x-sender="zhenya1007@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of zhenya1007@gmail.com designates 209.85.161.173 as permitted sender) identity=mailfrom; client-ip=209.85.161.173; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="zhenya1007@gmail.com"; x-sender="zhenya1007@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-yw0-f173.google.com) identity=helo; client-ip=209.85.161.173; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="zhenya1007@gmail.com"; x-sender="postmaster@mail-yw0-f173.google.com"; x-conformance=sidf_compatible IronPort-PHdr: =?us-ascii?q?9a23=3AfZSlWB9LBUKqEP9uRHKM819IXTAuvvDOBiVQ1KB2?= =?us-ascii?q?0egcTK2v8tzYMVDF4r011RmVBdyds6oMotGVmpioYXYH75eFvSJKW713fDhBt/?= =?us-ascii?q?8rmRc9CtWOE0zxIa2iRSU7GMNfSA0tpCnjYgBaF8nkelLdvGC54yIMFRXjLwp1?= =?us-ascii?q?Ifn+FpLPg8it2O2+54Dfbx9UiDahfLh/MAi4oQLNu8cMnIBsMLwxyhzHontJf+?= =?us-ascii?q?RZ22ZlLk+Nkhj/+8m94odt/zxftPw9+cFAV776f7kjQrxDEDsmKWE169b1uhTF?= =?us-ascii?q?UACC+2ETUmQSkhpPHgjF8BT3VYr/vyfmquZw3jSRMNboRr4oRzut86ZrSAfpiC?= =?us-ascii?q?gZMT457HrXgdF0gK5CvR6tuwBzz4vSbY6SKfR+Y7jdfcsESmVdQsZfWStBAoam?= =?us-ascii?q?YIsOCeoKIOJUoob5qlcLqxa1GAuiC/71yjJQhHD206003eoiHw/bwgIvA8kDv2?= =?us-ascii?q?7IoNjvLqoeTfy5wavOwD7eb/1WwzD96I3Qfx4uv/GMUqx/cczRyEIyCw3FiUiQ?= =?us-ascii?q?ppfkPzOTyusNs3Sb4PRhVeKplmUqrABwojixyccqiojGnJ8ZxkzY+Sh724s1Kt?= =?us-ascii?q?i4R1R6Yd6gCpdfqyaaN45vT84kXmpmtiE6yrgctp66eigH0JUnxxjFa/yGaYeE?= =?us-ascii?q?+BzjVPyJLTZ4nn1leLW/hxGo/Ue8ze38U9G4301LripZidbMq2wC1x/N5cibUP?= =?us-ascii?q?d9+V2h2TmX2wDL7eFEJUA1mbDFJJE83749kIcYv0fbHiLuhkn6kKubel8n9+Wo?= =?us-ascii?q?8ejrf7TrqoKGO4NpiAzzPKIjkdGlD+siKAgBRW2b9Py81LL9+U35R61Hjvgsna?= =?us-ascii?q?nYtJDWPMQap6ClDwNM3IYv9hSyAjm83NQXmnkHK11FeBaZgITzJ17OJ/X4Ae++?= =?us-ascii?q?g1Sqjjhr2+jLMqP9DpjJNHTOk7fscaxg50Nd1QY/181T6pBaB70ZJfL8QE7xtN?= =?us-ascii?q?jWDh8jNAy0xv7qCdR91owAX2KOArWWPL7OvVOU5O8iOOaMZIoPtzb8L/gp/eLh?= =?us-ascii?q?jXg8mVMFZ6mmwYMXaGykHvRhO0iWfWDjgtIFEWsTugo+TffqiEGZXD5IZ3eyWr?= =?us-ascii?q?o86SshBIKnC4fDXIGtj6ab0Ce1BJ0FLlxBX3enGHLsP6CNWvMNbi3aBs56jnRQ?= =?us-ascii?q?XrGkT8ol1AqynA780btuaOTOrH42r5XmgfVr6ODVhFkI8iF+DsKW032ATmc8yn?= =?us-ascii?q?EISjkn1fkn+RNVxVKK0Kw+iPtdQ48Ar8hVWxs3YMaPh9dxDMr/D0ecJ47YGmbj?= =?us-ascii?q?ec2vBHQKdvx0xtYPZ0hnHND710LM2iOrB/kekLnZXcVooJKZ5GD4IoNG81iDzL?= =?us-ascii?q?Mo1gB0Tc5GNGngjal6pVCKWtz51n6BnqPvTpwymS7A8GDZkziLtUBcFR9vCeDL?= =?us-ascii?q?BC9EIETRqtv96wXJSLr8Ubk=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0AeAwC4bnxah62hVdFdHAEBAQQBAQoBA?= =?us-ascii?q?YMkgQgLcCgKg1uBOYhrjiaCAoJngRqFDYEAhGKFJ4M+ghgKI4UYAoIoBxkHBDA?= =?us-ascii?q?YAQIBAQEBAQEBAQESAQEBCA0JCCgvgjgkgkcBAQEBAQEBIx0BGxIMAwELBgULD?= =?us-ascii?q?Q0dAgIiAREBBQEKEgYTEooKAQMIBQgQokVAjBeCBQUBHIMLBYNfChknAwpZWIF?= =?us-ascii?q?6AQEIAQEBARwCBhKEZ4IVgQ+FXoMkCwMBghyCaoJFIAWBLQEBAYdEDIphkEkBB?= =?us-ascii?q?wEBgXIKhiKNXoIeZ5E5ixOCaYl8FAUggRcPEIIJcFIyboEmCYITKh+BFgEIAYE?= =?us-ascii?q?TIDcBjVABAQE?= X-IPAS-Result: =?us-ascii?q?A0AeAwC4bnxah62hVdFdHAEBAQQBAQoBAYMkgQgLcCgKg1u?= =?us-ascii?q?BOYhrjiaCAoJngRqFDYEAhGKFJ4M+ghgKI4UYAoIoBxkHBDAYAQIBAQEBAQEBA?= =?us-ascii?q?QESAQEBCA0JCCgvgjgkgkcBAQEBAQEBIx0BGxIMAwELBgULDQ0dAgIiAREBBQE?= =?us-ascii?q?KEgYTEooKAQMIBQgQokVAjBeCBQUBHIMLBYNfChknAwpZWIF6AQEIAQEBARwCB?= =?us-ascii?q?hKEZ4IVgQ+FXoMkCwMBghyCaoJFIAWBLQEBAYdEDIphkEkBBwEBgXIKhiKNXoI?= =?us-ascii?q?eZ5E5ixOCaYl8FAUggRcPEIIJcFIyboEmCYITKh+BFgEIAYETIDcBjVABAQE?= X-IronPort-AV: E=Sophos;i="5.46,479,1511823600"; d="scan'208,217";a="254101750" Received: from mail-yw0-f173.google.com ([209.85.161.173]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 08 Feb 2018 16:42:59 +0100 Received: by mail-yw0-f173.google.com with SMTP id t129so2876301ywc.3 for ; Thu, 08 Feb 2018 07:42:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=djk2yoMX/bTnljVTEtMxJWaaItY26122lYfJUDbxdQg=; b=S8G9GtPuLgJ/c7kj3Vkpsct+JEFlMP030QJStS3fGtHJVCGmXaxHWBiLkvCSE/cHpZ yw1eMzt1AKDIho6SBKQ1Y21xSpVe8YzSCKh294k4+LGdA+6lfMkGHdVPfI1Y/k4HB4I8 ixIbPgV/b/fVTE7RdfYHuQTOrXaP2Lavo0dVs3qtnkvQvW8QWRo88spaEiiv4NTwYA87 DLuSFgr9CPxyccLiNFIaQzJYwvBJLJ8nhgD/WK5ZiCX+83x2zE50mJUjgzmzuxXYrOmR abeEbpLOR5xxuWtg51ZXpl29LPvWLn1KPIVH7494dRipRrbnoHzBnkhxizVpLNCJFuRa ntCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=djk2yoMX/bTnljVTEtMxJWaaItY26122lYfJUDbxdQg=; b=WiCj/V6eVkpB3YoKfybyliwRNcYWZTA4mKQViFjxUhrPsgj6125ANI9h+RJZ1mqXc0 U6zt62v+enur2pa5n0hx8Dh+4Rdvft8gqEnC46Yt0WAJ4vVJypKDQX6UGqVvur3KSTTn hloK+jjqtifAiLzrG4aN5vt2ZhTU5kR4tUXB/Bdgj0BI8wvfdVVBp2gdcLrE2v6+HMK+ PEzzU85WeX+EXcnzdSip4gW/SLXkE3Pl/65ZNkK/+oS2N6shgCGIbcx9Gwh2GZu41TiH qkMvSBRFjORBzcHNqjYE7C6VYq6h4OIE9HopIupF9Q35vtyJGiDqqJG2GmnhqdEDIZpa dTcw== X-Gm-Message-State: APf1xPAwP/NmBMeZ8noRiJ1uZ2Jype74hC00XrfB2O5MnuMgYHf7pH5t kc6YvBkUv3PUHk9gBc3veehEmxTnnRRmbqOj5siyM++n X-Google-Smtp-Source: AH8x225ZjVu/4H3/RAuzVC5q15Nl95yzgwEByNI5rCQZplrbQh1tEMxiMLnvlHyHwAPwRa3LIDvjHGPDz5OJ3OBfW/0= X-Received: by 10.37.106.130 with SMTP id f124mr841514ybc.445.1518104577711; Thu, 08 Feb 2018 07:42:57 -0800 (PST) MIME-Version: 1.0 Received: by 10.37.178.147 with HTTP; Thu, 8 Feb 2018 07:42:57 -0800 (PST) In-Reply-To: References: <20180208131150.GA81347@rxdmac.local> From: Evgeny Roubinchtein Date: Thu, 8 Feb 2018 07:42:57 -0800 Message-ID: To: OCaml Mailing List Content-Type: multipart/alternative; boundary="001a113fd1241623cd0564b544b8" Subject: Re: [Caml-list] File synchronization implementation(s) in OCaml? --001a113fd1241623cd0564b544b8 Content-Type: text/plain; charset="UTF-8" Thank you everyone for all the responses and pointers to artefacts and papers. Cedric, that is an excellent point; thank you. My current intended use is having the user edit file(s) on host A using a text editor, and, when the user saves a file, having those edits reflected as quickly as possible in the corresponding file on host B. So, my primary measure of performance is the speed of update: I want the time between the events "the user, who runs the editor on host A, has issued a command to the editor to save the file" and "the contents of the file the user has just saved on host A and the corresponding file on host B is identical" to be as low as feasible (there is a mapping between file paths on hosts A and B). Sometimes, the user will create a new file on host A; then the file's contents needs to be put as quickly as possible into the corresponding location on host B. One other consideration is that occasionally the user may have their text editor save a number of files in quick (at least in human scale) succession: for example, the user issues "M-x compile" in Emacs, and Emacs offers to save all modified buffers before running the compilation: in that case, the total time to propagate the changes to all modified file(s) should be as low as possible. So, to summarize: 1. One-way synchronization is acceptable (I may find out otherwise with experience, but, for now, I am willing to make that assumption) 2. It is always known which file(s) have been modified. 3. It is probably feasible to plumb through the information about what part(s) of of a file were modified for each file from the text editor to the updater, if that helps with the speed of the update. The editor usually "knows" what has changed, because it needs to be able to undo the changes. 4. The relevant metric is the speed of the update. 5. Sometimes changes to more than one file may be saved in quick (human scale) succession. The relevant metric is still the speed of update to all files. I apologize for the somewhat long-winded description of the metric I care about. I do hope that clarifies it. -- Best, Evgeny ("Zhenya") On Thu, Feb 8, 2018 at 6:44 AM, Yaron Minsky wrote: > Ha! The paper you linked builds off of an old paper I wrote: > > http://cis.poly.edu/westlab/papers/ref/practical.pdf > > There is in fact a full implementation of the algorithms in this paper > in OCaml, as part of the SKS system that I wrote many years ago. > > https://bitbucket.org/skskeyserver/sks-keyserver/wiki/Home > > I'm not especially proud of the code, but it does work... > > y > > On Thu, Feb 8, 2018 at 8:11 AM, Cedric Cellier > wrote: > > -[ Wed, Feb 07, 2018 at 07:05:15PM -0800, Evgeny Roubinchtein ]---- > >> I don't want anything that performs worse than rsync in practice, > > > > It would be interesting to know how you measure performance as one could > > think of many metrics: > > > > - speed on different data > > - speed on similar data > > - reliability in face of simultaneous synchronizations > > - reliability in case of bad network > > - usage of resources > > - confidentiality > > - ...? > > > > > > -- > > Caml-list mailing list. Subscription management and archives: > > https://sympa.inria.fr/sympa/arc/caml-list > > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > > Bug reports: http://caml.inria.fr/bin/caml-bugs > > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > --001a113fd1241623cd0564b544b8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thank you eve= ryone for all the responses and pointers to artefacts and papers.

Cedric, that is an excellent point; thank you.=C2=A0 My current intende= d use is having the user edit file(s) on host A using a text editor, and, w= hen the user saves a file, having those edits reflected as quickly as possi= ble in the corresponding file on host B.=C2=A0 So, my primary measure of pe= rformance is the speed of update: I want the time between the events "= the user, who runs the editor on host A, has issued a command to the editor= to save the file" and "the contents of the file the user has jus= t saved on host A and the corresponding file on host B is identical" = to be as low as feasible (there is a mapping between file paths on hosts A = and B).=C2=A0 Sometimes, the user will create a new file on host A; then th= e file's contents needs to be put as quickly as possible into the corre= sponding location on host B.=C2=A0 One other consideration is that occasion= ally =C2=A0 the user may have their text editor save a number of files in q= uick (at least in human scale) succession: for example, the user issues &qu= ot;M-x compile" in Emacs, and Emacs offers to save all modified buffer= s before running the compilation: in that case, the total time to propagate= the changes to all modified file(s) should be as low as possible.=C2=A0 So= , to summarize:

1. One-way synchronization is acceptable (I ma= y find out otherwise with experience, but, for now, I am willing to make th= at assumption)
2. It is always known which file(s) have been modif= ied.
3. It is probably feasible to plumb through the information a= bout what part(s) of of a file were modified for each file from the text ed= itor to the updater, if that helps with the speed of the update.=C2=A0 The = editor usually "knows" what has changed, because it needs to be a= ble to undo the changes.
4. The relevant metric is the speed of th= e update.
5. Sometimes changes to more than one file may be saved = in quick (human scale) succession.=C2=A0 The relevant metric is still the s= peed of update to all files.

I apologize for the somewhat long= -winded description of the metric I care about.=C2=A0 I do hope that clarif= ies it.

--
Best,
Evgeny ("Zhenya")
=

On Thu, Feb= 8, 2018 at 6:44 AM, Yaron Minsky <yminsky@janestreet.com> wrote:
Ha! The paper you linked build= s off of an old paper I wrote:

http://cis.poly.edu/westlab/papers/ref/prac= tical.pdf

There is in fact a full implementation of the algorithms in this paper
in OCaml, as part of the SKS system that I wrote many years ago.

https://bitbucket.org/skskeyserver/s= ks-keyserver/wiki/Home

I'm not especially proud of the code, but it does work...

y

On Thu, Feb 8, 2018 at 8:11 AM, Cedric Cellier <rixed@happyleptic.org> wrote:
> -[ Wed, Feb 07, 2018 at 07:05:15PM -0800, Evgeny Roubinchtein ]----
>> I don't want anything that performs worse than rsync in practi= ce,
>
> It would be interesting to know how you measure performance as one cou= ld
> think of many metrics:
>
> - speed on different data
> - speed on similar data
> - reliability in face of simultaneous synchronizations
> - reliability in case of bad network
> - usage of resources
> - confidentiality
> - ...?
>
>
> --
> Caml-list mailing list.=C2=A0 Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group= /ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

--
Caml-list mailing list.=C2=A0 Subscription management and archives:
https://sympa.inria.fr/sympa/arc/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

--001a113fd1241623cd0564b544b8--