Hi, I'm in the middle of building a nontrivial WAN-distributed system in
Ocaml, and have built a few distributed systems as part of research, in the
past.  I'd like to urge you to try to use and improve one of the existing
multiplatform RPC systems that exist -- specifically Thrift or Protobuf3.

My last system used Thrift, and consisted in a Chubby clone, storage
servers and clients, and RPC over TCP and RoCEE.  Some parts were in C++
for efficiency, and some in Ocaml (for programmer-efficiency).  It mattered
to me, that I could get really, really efficient (microseconds counted) RPC
implementations for C++, that were wireline compatible with Ocaml (so I
could convert an Ocaml prototype into a C++ program for speed).

The system I'm working on right now will require (in addition to Ocaml)
Java, Javascript, and Golang compatibility for the RPC substrate.

I think it's also important that you have stub-compilers and "standard" RPC
transports for Ocaml -- you don't want to be rolling your own
stubs&skeletons that you have to update as you change your IDL, when all
the other languages' stubs/skeletons get automatically generated.

If I had my druthers, I'd go with protobuf3.  I've used protobuf2 at my
previous employer, and it was quite performant.  But ocaml isn't
supported.  Someday, when I have time, I'll fix that.  But right now, that
isn't high-priority.

Secondly, I don't know what the story is, on thrift-vs-protobuf3
performance.  Maybe it's great, GREAT!  But it might not be, and there are
at least reasons why it could be worse.  Specificaly protobuf3 was designed
for use on the open internet, not in more-controlled settings.  These are
issues that really involve the transports, not the upper-layers of the RPC
stack, but still, it's something to be checked-out and verified carefully.

OK: so my own decision was: "use Thrift for now, but be ready to switch to
protobuf3 once it gets ocaml support (e.g., I write it) and I verify
performance parity".

My 2c,
--chet--


On Wed, May 18, 2016 at 6:43 AM, <rixed@happyleptic.org> wrote:

> Hello.
>
> I'm thinking about implementing a library for doing RPC with OCaml, with
> large
> scale environments in mind (à la Stubby but with better type checking of
> course). I'm wondering what are the related libs I should make myself
> familiar
> with before starting. I've seen a few interesting things for serialization
> (piqi come to mind), some interesting event engines (LWT, Core), some
> protocol
> implementations suitable for transport but no HTTP2, nothing to interface
> with
> monitoring subsystems or TSDBs, nothing related to load balancing, routing,
> DDoS detection, etc, some crypto, an interesting TLS implementation from
> MirageOs, no OAuth or similar.
>
> What other related projects should I look at?
>
> Also, if anyone would be interested in contributing ideas, experience or
> code
> please let me know.
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>