From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 8CA9C7FA87 for ; Wed, 18 May 2016 19:42:08 +0200 (CEST) IronPort-PHdr: 9a23:NAooTB0q/ijMMUcusmDT+DRfVm0co7zxezQtwd8ZsegQKPad9pjvdHbS+e9qxAeQG96LurQf06GP6/CocFdDyKjCmUhKSIZLWR4BhJdetC0bK+nBN3fGKuX3ZTcxBsVIWQwt1Xi6NU9IBJS2PAWK8TWM5DIfUi/yKRBybrysXNWC3oLsj6vro82bSj4LrQT+SIs6FA+xowTVu5teqqpZAYF19CH0pGBVcf9d32JiKAHbtR/94sCt4MwrqHwI6LoXzNNbSajxY4g/SLVZCnxmbzF0t4XXskzZTBuC6lMcW30Ogx5FGxTE51f9RJiinDH9s79W2CSGdejrV6w0SHz26rpxVB/5ziwHOiIR/2Tei8g2h6Ve9kHy7ydjypLZNdnGfMF1ebnQKI9LSA== Authentication-Results: mail3-smtp-sop.national.inria.fr; spf=None smtp.pra=murthy.chet@gmail.com; spf=Pass smtp.mailfrom=murthy.chet@gmail.com; spf=None smtp.helo=postmaster@mail-ig0-f177.google.com Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of murthy.chet@gmail.com) identity=pra; client-ip=209.85.213.177; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="murthy.chet@gmail.com"; x-sender="murthy.chet@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of murthy.chet@gmail.com designates 209.85.213.177 as permitted sender) identity=mailfrom; client-ip=209.85.213.177; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="murthy.chet@gmail.com"; x-sender="murthy.chet@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ig0-f177.google.com) identity=helo; client-ip=209.85.213.177; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="murthy.chet@gmail.com"; x-sender="postmaster@mail-ig0-f177.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0CEAABLqDxXf7HVVdFeg1U4fgapA4Vjiw8BDYF1IoVvAoE3BzgUAQEBAQEBAQERAQEJCwsJIS+CLYIWAQEDARIRHQEbEgsBAwELBgULGh0CAiIBEQEFAQoSGRIQh3IBAw8IDqRZgTE+MYs7gWqCWAWIFQoZJwMKUoNNAQEBAQEFAQEBAQEBAQEXAgYQiV+BA4QrQYJTglkFhjwMiAyEXYR6hgCIIGWBBE6EAYhihjGHWRIegQ8PDwEBgjuCFBwyhkmBPQEBAQ X-IPAS-Result: A0CEAABLqDxXf7HVVdFeg1U4fgapA4Vjiw8BDYF1IoVvAoE3BzgUAQEBAQEBAQERAQEJCwsJIS+CLYIWAQEDARIRHQEbEgsBAwELBgULGh0CAiIBEQEFAQoSGRIQh3IBAw8IDqRZgTE+MYs7gWqCWAWIFQoZJwMKUoNNAQEBAQEFAQEBAQEBAQEXAgYQiV+BA4QrQYJTglkFhjwMiAyEXYR6hgCIIGWBBE6EAYhihjGHWRIegQ8PDwEBgjuCFBwyhkmBPQEBAQ X-IronPort-AV: E=Sophos;i="5.26,329,1459807200"; d="scan'208,217";a="178227292" Received: from mail-ig0-f177.google.com ([209.85.213.177]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES128-GCM-SHA256; 18 May 2016 19:42:07 +0200 Received: by mail-ig0-f177.google.com with SMTP id s8so31422589ign.0 for ; Wed, 18 May 2016 10:42:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=tW7e9NJIg7m0FcurRj1h7FuXDT86z7dAF48LHDkyr94=; b=0atIM1/skPOOZwUYuY6LbMm0/pYEjfeS6H/bmsEPnKABRqOK8tddEeXo3I4o4jIyv6 C1jeeQeskVs+wIbflYhijosbl5+5y1dDMfo46uNJRmTouH3NqzyHGTHIQd9s8IYcpnT0 MAjW86JPckU+Q/Hagi1v8JMrENUuy0S3uSCmF2aNGY2xgKmAaGHqXR+VLuMR/HI+QLcz FsJ2HyD8bx7UBiJmJ4CH5bm9TMtrWEEw+XSr4TbPr3P1+w4kmOQwjqw7rHqJJSTZOJir f5Vuy3dijyGd/b/hF9ZLPb0Zzy9HhoO1FAV4PHNo+IPYvHa4UIY3IlhGvXfkA/XizNqo hnUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=tW7e9NJIg7m0FcurRj1h7FuXDT86z7dAF48LHDkyr94=; b=Lnyk0RDHGVC98s+eAYGosxW9cyKmp8Gfwz7pc5bnx7OIsA5BAz7Y/yhhr0qjZ83fkC 30PuO7WD7JzjwKQHGfzQs0Eptm6M3NG6Ql1ZKcDNNWkK6CYc3U6E04sZT1c4B8rM750E OL8RgRTUMwKJIqv0kbqhcAth6aNJX0kMHhPUW8R8HvHBh7uLw3l85cmbkC/qAGma8XvW 7YtTLaEQ+7CkuibuT9B6uFxx9keikvkLGc2I7Ve5dKFGnaxtEFzKZ5kPYx14n5qmccpO ySf7l4FCvtYXXhPN2qNW4eZELAGL1pt60PnfU6r1mAR9RZvsDdBp809rwfKQ4Blk2XCw aD9Q== X-Gm-Message-State: AOPr4FXnWkTsv8DK52PdEPzd9VeXu4WuoWE7rZ2T/KpDOOSeQe0vLVVJrIr7nZPXWMYVnh0UmrV66EMcrvoc/w== MIME-Version: 1.0 X-Received: by 10.50.40.101 with SMTP id w5mr6432878igk.40.1463593325816; Wed, 18 May 2016 10:42:05 -0700 (PDT) Received: by 10.107.35.212 with HTTP; Wed, 18 May 2016 10:42:05 -0700 (PDT) In-Reply-To: <20160518134349.GA10232@pim.happyleptic.org> References: <20160518134349.GA10232@pim.happyleptic.org> Date: Wed, 18 May 2016 10:42:05 -0700 Message-ID: From: Chet Murthy To: rixed@happyleptic.org Cc: caml-list Content-Type: multipart/alternative; boundary=089e01184e4847f282053321603e Subject: Re: [Caml-list] RPC for OCaml? --089e01184e4847f282053321603e Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, I'm in the middle of building a nontrivial WAN-distributed system in Ocaml, and have built a few distributed systems as part of research, in the past. I'd like to urge you to try to use and improve one of the existing multiplatform RPC systems that exist -- specifically Thrift or Protobuf3. My last system used Thrift, and consisted in a Chubby clone, storage servers and clients, and RPC over TCP and RoCEE. Some parts were in C++ for efficiency, and some in Ocaml (for programmer-efficiency). It mattered to me, that I could get really, really efficient (microseconds counted) RPC implementations for C++, that were wireline compatible with Ocaml (so I could convert an Ocaml prototype into a C++ program for speed). The system I'm working on right now will require (in addition to Ocaml) Java, Javascript, and Golang compatibility for the RPC substrate. I think it's also important that you have stub-compilers and "standard" RPC transports for Ocaml -- you don't want to be rolling your own stubs&skeletons that you have to update as you change your IDL, when all the other languages' stubs/skeletons get automatically generated. If I had my druthers, I'd go with protobuf3. I've used protobuf2 at my previous employer, and it was quite performant. But ocaml isn't supported. Someday, when I have time, I'll fix that. But right now, that isn't high-priority. Secondly, I don't know what the story is, on thrift-vs-protobuf3 performance. Maybe it's great, GREAT! But it might not be, and there are at least reasons why it could be worse. Specificaly protobuf3 was designed for use on the open internet, not in more-controlled settings. These are issues that really involve the transports, not the upper-layers of the RPC stack, but still, it's something to be checked-out and verified carefully. OK: so my own decision was: "use Thrift for now, but be ready to switch to protobuf3 once it gets ocaml support (e.g., I write it) and I verify performance parity". My 2c, --chet-- On Wed, May 18, 2016 at 6:43 AM, wrote: > Hello. > > I'm thinking about implementing a library for doing RPC with OCaml, with > large > scale environments in mind (=C3=A0 la Stubby but with better type checkin= g of > course). I'm wondering what are the related libs I should make myself > familiar > with before starting. I've seen a few interesting things for serialization > (piqi come to mind), some interesting event engines (LWT, Core), some > protocol > implementations suitable for transport but no HTTP2, nothing to interface > with > monitoring subsystems or TSDBs, nothing related to load balancing, routin= g, > DDoS detection, etc, some crypto, an interesting TLS implementation from > MirageOs, no OAuth or similar. > > What other related projects should I look at? > > Also, if anyone would be interested in contributing ideas, experience or > code > please let me know. > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > --089e01184e4847f282053321603e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi, I'm in the middle of building a nontrivial WAN-dis= tributed system in Ocaml, and have built a few distributed systems as part = of research, in the past.=C2=A0 I'd like to urge you to try to use and = improve one of the existing multiplatform RPC systems that exist -- specifi= cally Thrift or Protobuf3.

My last system used Thrift, a= nd consisted in a Chubby clone, storage servers and clients, and RPC over T= CP and RoCEE.=C2=A0 Some parts were in C++ for efficiency, and some in Ocam= l (for programmer-efficiency).=C2=A0 It mattered to me, that I could get re= ally, really efficient (microseconds counted) RPC implementations for C++, = that were wireline compatible with Ocaml (so I could convert an Ocaml proto= type into a C++ program for speed).

The system I&#= 39;m working on right now will require (in addition to Ocaml) Java, Javascr= ipt, and Golang compatibility for the RPC substrate.

I think it's also important that you have stub-compilers and "s= tandard" RPC transports for Ocaml -- you don't want to be rolling = your own stubs&skeletons that you have to update as you change your IDL= , when all the other languages' stubs/skeletons get automatically gener= ated.

If I had my druthers, I'd go with protob= uf3.=C2=A0 I've used protobuf2 at my previous employer, and it was quit= e performant.=C2=A0 But ocaml isn't supported.=C2=A0 Someday, when I ha= ve time, I'll fix that.=C2=A0 But right now, that isn't high-priori= ty.

Secondly, I don't know what the story is, = on thrift-vs-protobuf3 performance.=C2=A0 Maybe it's great, GREAT!=C2= =A0 But it might not be, and there are at least reasons why it could be wor= se.=C2=A0 Specificaly protobuf3 was designed for use on the open internet, = not in more-controlled settings.=C2=A0 These are issues that really involve= the transports, not the upper-layers of the RPC stack, but still, it's= something to be checked-out and verified carefully.

OK: so my own decision was: "use Thrift for now, but be ready to sw= itch to protobuf3 once it gets ocaml support (e.g., I write it) and I verif= y performance parity".

My 2c,
--che= t--


On Wed, May 18, 2016 at 6:43 AM, <rixed@happyleptic.org<= /a>> wrote:
Hello.

I'm thinking about implementing a library for doing RPC with OCaml, wit= h large
scale environments in mind (=C3=A0 la Stubby but with better type checking = of
course). I'm wondering what are the related libs I should make myself f= amiliar
with before starting. I've seen a few interesting things for serializat= ion
(piqi come to mind), some interesting event engines (LWT, Core), some proto= col
implementations suitable for transport but no HTTP2, nothing to interface w= ith
monitoring subsystems or TSDBs, nothing related to load balancing, routing,=
DDoS detection, etc, some crypto, an interesting TLS implementation from
MirageOs, no OAuth or similar.

What other related projects should I look at?

Also, if anyone would be interested in contributing ideas, experience or co= de
please let me know.


--
Caml-list mailing list.=C2=A0 Subscription management and archives:
https://sympa.inria.fr/sympa/arc/caml-list
Beginner's list: http://groups.yahoo.com/group/ocam= l_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

--089e01184e4847f282053321603e--