[Caml-list] RPC for OCaml?

[Caml-list] RPC for OCaml?

[rixed]

Hello.

I'm thinking about implementing a library for doing RPC with OCaml, with large
scale environments in mind (à la Stubby but with better type checking of
course). I'm wondering what are the related libs I should make myself familiar
with before starting. I've seen a few interesting things for serialization
(piqi come to mind), some interesting event engines (LWT, Core), some protocol
implementations suitable for transport but no HTTP2, nothing to interface with
monitoring subsystems or TSDBs, nothing related to load balancing, routing,
DDoS detection, etc, some crypto, an interesting TLS implementation from
MirageOs, no OAuth or similar.

What other related projects should I look at?

Also, if anyone would be interested in contributing ideas, experience or code
please let me know.

Re: [Caml-list] RPC for OCaml?

[Yaron Minsky]

Async-RPC is perhaps worth looking at, though I agree it doesn't give
you much of what you want --- certainly, we don't to RPC over HTTP, we
do it over bog-standard TCP, and the protocol is very much
OCaml-specific, being based on bin-io.  That said, it might be useful
to look at for inspiration, in particular for how versioning is
handled in Versioned_rpc.  We do also have some kerberos support in
there as well, though I'm not sure that's in the open source release.

On Wed, May 18, 2016 at 9:43 AM,  <rixed@happyleptic.org> wrote:
> Hello.
>
> I'm thinking about implementing a library for doing RPC with OCaml, with large
> scale environments in mind (à la Stubby but with better type checking of
> course). I'm wondering what are the related libs I should make myself familiar
> with before starting. I've seen a few interesting things for serialization
> (piqi come to mind), some interesting event engines (LWT, Core), some protocol
> implementations suitable for transport but no HTTP2, nothing to interface with
> monitoring subsystems or TSDBs, nothing related to load balancing, routing,
> DDoS detection, etc, some crypto, an interesting TLS implementation from
> MirageOs, no OAuth or similar.
>
> What other related projects should I look at?
>
> Also, if anyone would be interested in contributing ideas, experience or code
> please let me know.
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] RPC for OCaml?

[Jon Ludlam]

Mirage has a simple RPC generator that fits the same hole as Async-RPC,
which we use quite heavily in XenServer. It's camlp4 dependent right now,
which I'm keen to fix in the near future.

    github.com/mirage/ocaml-rpc 

I'll certainly be casting a careful eye over the Async-RPC versioning
support as our current versioning story is somewhat primitive.

Jon

On Wed, May 18, 2016 at 09:52:13AM -0400, Yaron Minsky wrote:
> Async-RPC is perhaps worth looking at, though I agree it doesn't give
> you much of what you want --- certainly, we don't to RPC over HTTP, we
> do it over bog-standard TCP, and the protocol is very much
> OCaml-specific, being based on bin-io.  That said, it might be useful
> to look at for inspiration, in particular for how versioning is
> handled in Versioned_rpc.  We do also have some kerberos support in
> there as well, though I'm not sure that's in the open source release.
> 
> On Wed, May 18, 2016 at 9:43 AM,  <rixed@happyleptic.org> wrote:
> > Hello.
> >
> > I'm thinking about implementing a library for doing RPC with OCaml, with large
> > scale environments in mind (à la Stubby but with better type checking of
> > course). I'm wondering what are the related libs I should make myself familiar
> > with before starting. I've seen a few interesting things for serialization
> > (piqi come to mind), some interesting event engines (LWT, Core), some protocol
> > implementations suitable for transport but no HTTP2, nothing to interface with
> > monitoring subsystems or TSDBs, nothing related to load balancing, routing,
> > DDoS detection, etc, some crypto, an interesting TLS implementation from
> > MirageOs, no OAuth or similar.
> >
> > What other related projects should I look at?
> >
> > Also, if anyone would be interested in contributing ideas, experience or code
> > please let me know.
> >
> >
> > --
> > Caml-list mailing list.  Subscription management and archives:
> > https://sympa.inria.fr/sympa/arc/caml-list
> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] RPC for OCaml?

[rixed]

-[ Wed, May 18, 2016 at 09:52:13AM -0400, Yaron Minsky ]----
> Async-RPC is perhaps worth looking at, though I agree it doesn't give
> you much of what you want --- certainly, we don't to RPC over HTTP, we
> do it over bog-standard TCP, and the protocol is very much
> OCaml-specific, being based on bin-io.

Indeed it's interesting.
That's only the lowest level part though (the actual communication between two
machines).  I guess you have some higher layers doing load balancing, service
discovery, and so on, but those are private?

I'm not actualy looking for RPC over HTTP, I was just mentionning HTTP2 because
that's a good middleground between efficient RPC and legacy infrastructures. Indeed,
I'd like an RPC library that's oblivious of the transport layer because there
can be some mandatory proxy/authentication scheme implemented at this layer
already and one may not have other choice than to implement them.

> That said, it might be useful
> to look at for inspiration, in particular for how versioning is
> handled in Versioned_rpc. 

Versionning is indeed one of the concern I forgot to mention.
Although it
may be more important to you because of the serialization engine you use. Many
serialization formats allow for extensions which in practice aleviate the
issue; although it's also a form of manual runtime type checking that can
quickly become a source of bug and boilerplate ("moving data from one format to
another as a living" kind of feeling).  My idea so far was to use whatever is
available in the serialization layer for minor additions (as for the transport
I'd like the RPC to be adaptable with any serializer, again for interfacing
with legacy), and for anything fancier just use plain variant types and let
clients and servers deal with them (GADT can be used to make sure we answer a
response-v2 to a query-v2, can't they?) or deploy entirely distinct servers and
solve this issue at the service-discovery layer.

Having an explicit adapter such as versioned_rpc is a good idea. Maybe as part
of the adapter to the underlying serializer so that implementers are strongly
encouraged to code unserializer in such a way that the service itself deals
with only one version.

> We do also have some kerberos support in
> there as well, though I'm not sure that's in the open source release.

https://github.com/janestreet-alpha/krb5 ?


Will read all this and maybe come back with more questions if you don't mind.

Re: [Caml-list] RPC for OCaml?

[rixed]

-[ Wed, May 18, 2016 at 03:17:15PM +0100, Jon Ludlam ]----
>     github.com/mirage/ocaml-rpc 

A very quick look at this a while ago gave me the impression that this was only
a serializer.  If so, then I must say I've always considered only two families
of serializers:

- legacy ones (json, protobuf...) for real situations
- standard lib Marshal for tests

But indeed there might be room for an ocaml-centric serializer with
best-in-town type checking but that does not crash programs compiled with
different versions of the compiler :-)

Re: [Caml-list] RPC for OCaml?

[Chet Murthy]

[-- Attachment #1: Type: text/plain, Size: 3215 bytes --]

Hi, I'm in the middle of building a nontrivial WAN-distributed system in
Ocaml, and have built a few distributed systems as part of research, in the
past.  I'd like to urge you to try to use and improve one of the existing
multiplatform RPC systems that exist -- specifically Thrift or Protobuf3.

My last system used Thrift, and consisted in a Chubby clone, storage
servers and clients, and RPC over TCP and RoCEE.  Some parts were in C++
for efficiency, and some in Ocaml (for programmer-efficiency).  It mattered
to me, that I could get really, really efficient (microseconds counted) RPC
implementations for C++, that were wireline compatible with Ocaml (so I
could convert an Ocaml prototype into a C++ program for speed).

The system I'm working on right now will require (in addition to Ocaml)
Java, Javascript, and Golang compatibility for the RPC substrate.

I think it's also important that you have stub-compilers and "standard" RPC
transports for Ocaml -- you don't want to be rolling your own
stubs&skeletons that you have to update as you change your IDL, when all
the other languages' stubs/skeletons get automatically generated.

If I had my druthers, I'd go with protobuf3.  I've used protobuf2 at my
previous employer, and it was quite performant.  But ocaml isn't
supported.  Someday, when I have time, I'll fix that.  But right now, that
isn't high-priority.

Secondly, I don't know what the story is, on thrift-vs-protobuf3
performance.  Maybe it's great, GREAT!  But it might not be, and there are
at least reasons why it could be worse.  Specificaly protobuf3 was designed
for use on the open internet, not in more-controlled settings.  These are
issues that really involve the transports, not the upper-layers of the RPC
stack, but still, it's something to be checked-out and verified carefully.

OK: so my own decision was: "use Thrift for now, but be ready to switch to
protobuf3 once it gets ocaml support (e.g., I write it) and I verify
performance parity".

My 2c,
--chet--


On Wed, May 18, 2016 at 6:43 AM, <rixed@happyleptic.org> wrote:

> Hello.
>
> I'm thinking about implementing a library for doing RPC with OCaml, with
> large
> scale environments in mind (à la Stubby but with better type checking of
> course). I'm wondering what are the related libs I should make myself
> familiar
> with before starting. I've seen a few interesting things for serialization
> (piqi come to mind), some interesting event engines (LWT, Core), some
> protocol
> implementations suitable for transport but no HTTP2, nothing to interface
> with
> monitoring subsystems or TSDBs, nothing related to load balancing, routing,
> DDoS detection, etc, some crypto, an interesting TLS implementation from
> MirageOs, no OAuth or similar.
>
> What other related projects should I look at?
>
> Also, if anyone would be interested in contributing ideas, experience or
> code
> please let me know.
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 4125 bytes --]

Re: [Caml-list] RPC for OCaml?

[Yaron Minsky]

Yeah, we have some service-discovery stuff and the like that's
internal, and probably not useful outside of our context.  Async RPC
is relatively independent of the transport; Async_rpc_kernel lets you
sub in any transport you want, and we've used that flexibility to use
it over websockets.

y

On Wed, May 18, 2016 at 10:26 AM,  <rixed@happyleptic.org> wrote:
> -[ Wed, May 18, 2016 at 09:52:13AM -0400, Yaron Minsky ]----
>> Async-RPC is perhaps worth looking at, though I agree it doesn't give
>> you much of what you want --- certainly, we don't to RPC over HTTP, we
>> do it over bog-standard TCP, and the protocol is very much
>> OCaml-specific, being based on bin-io.
>
> Indeed it's interesting.
> That's only the lowest level part though (the actual communication between two
> machines).  I guess you have some higher layers doing load balancing, service
> discovery, and so on, but those are private?
>
> I'm not actualy looking for RPC over HTTP, I was just mentionning HTTP2 because
> that's a good middleground between efficient RPC and legacy infrastructures. Indeed,
> I'd like an RPC library that's oblivious of the transport layer because there
> can be some mandatory proxy/authentication scheme implemented at this layer
> already and one may not have other choice than to implement them.
>
>> That said, it might be useful
>> to look at for inspiration, in particular for how versioning is
>> handled in Versioned_rpc.
>
> Versionning is indeed one of the concern I forgot to mention.
> Although it
> may be more important to you because of the serialization engine you use. Many
> serialization formats allow for extensions which in practice aleviate the
> issue; although it's also a form of manual runtime type checking that can
> quickly become a source of bug and boilerplate ("moving data from one format to
> another as a living" kind of feeling).  My idea so far was to use whatever is
> available in the serialization layer for minor additions (as for the transport
> I'd like the RPC to be adaptable with any serializer, again for interfacing
> with legacy), and for anything fancier just use plain variant types and let
> clients and servers deal with them (GADT can be used to make sure we answer a
> response-v2 to a query-v2, can't they?) or deploy entirely distinct servers and
> solve this issue at the service-discovery layer.
>
> Having an explicit adapter such as versioned_rpc is a good idea. Maybe as part
> of the adapter to the underlying serializer so that implementers are strongly
> encouraged to code unserializer in such a way that the service itself deals
> with only one version.
>
>> We do also have some kerberos support in
>> there as well, though I'm not sure that's in the open source release.
>
> https://github.com/janestreet-alpha/krb5 ?
>
>
> Will read all this and maybe come back with more questions if you don't mind.

Re: [Caml-list] RPC for OCaml?

[rixed]

Hi!

I'm not very opinionated as far as which brick should be used but for the fact
that I'd like the low level bricks to be interchangeable (transport,
serialization, RPC).

I'm more opinionated about the bigger picture, which I think is more
specificaly were we (community and industry) are missing something, probably
because of how large infrastructure is build (ie incrementaly from a small
one). We commonly consider RPC, monitoring, load balancing, authentication and
service discovery as independant things, but whenever a company has to build a
large infra from scratch they tends to prefer to integrate (at least some of)
the above, for this "economy of scale" in complexity brings many benefits:

- proxyless routing
- proxyless load balancing
- seamless authentication
- seamless quota
- instrumentation of all RPC for free
- consequently, no need for independant network monitoring
- consequently, reliable detection of problems and reliable system to route
  around or drain the faulty nodes/clusters
- uniform behavior across the whole infrastructure inducing ease to diagnosing
  issues and predict behavior
- any improvement of the infrastructure benefits every services

Compare with the industry "standard" consisting of throwing some unrelated
services together and spending an enormous amount of time interconnecting them
together with little visibility.

I do not care about speed so much as to consider C++ or RDMA, but more about
reliability and type safety.  That's why I'm envisaging this for
OCaml/MirageOs, interacting with other "legacy" stuff only at the perimeter.

In this ideal world or strongly typed microservices, stub compilers or IDL
would be superfluous, for you want the authoritative types to be defined in
OCaml rather than in, say, protobuf (esp. version 3 which weaken types even
more to pander to the json crowd, or so I like to imagine). But then it would
make it much harder to interface with the outer world, as you rightfully
described.

The only way to reconcile those conflicting views, it seems, would be to use
some IDL that's as strongly typed as OCaml, supporting various underlying
serializers, and that would compile additional runtime type checks for weaker
serializer and for softer languages.  Isn't that why piqi was invented?  (By
the way piqi supports proto2 and OCaml).

Re: [Caml-list] RPC for OCaml?

[David MENTRÉ]

Hello,

Le 2016-05-18 à 15:43, rixed@happyleptic.org a écrit :
> I'm thinking about implementing a library for doing RPC with OCaml, with large
> scale environments in mind (à la Stubby but with better type checking of
> course). I'm wondering what are the related libs I should make myself familiar
> with before starting.

[ Old thread, maybe not what you are looking for, anyway... ]

I would look at Good Old RPC protocols like ONC RPC and CORBA. Regarding
network encoding, I would consider standardized encoding like ANS.1, XDR
or CORBA's IIOP. I'm always surprised how people are forgetting history
and re-invent the wheel.

Regarding ONC RPC, I think Gerd implemented a library, with additional
bell and whistle, in OCamlNet.

Best regards,
david

Re: [Caml-list] RPC for OCaml?

[Maxime Ransan (BLOOMBERG/ 731 LEX)]

[-- Attachment #1: Type: text/plain, Size: 3614 bytes --]

Regarding protobuf you can use [1] if you want to generate OCaml from `.proto` file or [2] if you want to use ppx extension to serialize OCaml types to protobuf. 

Regaring [1], protobuf3 format is not fully supported but it could be improve. Feel free to raise issues. 


[1] https://github.com/mransan/ocaml-protoc 
[2] https://github.com/whitequark/ppx_deriving_protobuf/


From: murthy.chet@gmail.com At: May 18 2016 13:42:22
To: rixed@happyleptic.org
Cc: caml-list@inria.fr
Subject: Re: [Caml-list] RPC for OCaml?

Hi, I'm in the middle of building a nontrivial WAN-distributed system in Ocaml, and have built a few distributed systems as part of research, in the past.  I'd like to urge you to try to use and improve one of the existing multiplatform RPC systems that exist -- specifically Thrift or Protobuf3.

My last system used Thrift, and consisted in a Chubby clone, storage servers and clients, and RPC over TCP and RoCEE.  Some parts were in C++ for efficiency, and some in Ocaml (for programmer-efficiency).  It mattered to me, that I could get really, really efficient (microseconds counted) RPC implementations for C++, that were wireline compatible with Ocaml (so I could convert an Ocaml prototype into a C++ program for speed).

The system I'm working on right now will require (in addition to Ocaml) Java, Javascript, and Golang compatibility for the RPC substrate.

I think it's also important that you have stub-compilers and "standard" RPC transports for Ocaml -- you don't want to be rolling your own stubs&skeletons that you have to update as you change your IDL, when all the other languages' stubs/skeletons get automatically generated.

If I had my druthers, I'd go with protobuf3.  I've used protobuf2 at my previous employer, and it was quite performant.  But ocaml isn't supported.  Someday, when I have time, I'll fix that.  But right now, that isn't high-priority.

Secondly, I don't know what the story is, on thrift-vs-protobuf3 performance.  Maybe it's great, GREAT!  But it might not be, and there are at least reasons why it could be worse.  Specificaly protobuf3 was designed for use on the open internet, not in more-controlled settings.  These are issues that really involve the transports, not the upper-layers of the RPC stack, but still, it's something to be checked-out and verified carefully.

OK: so my own decision was: "use Thrift for now, but be ready to switch to protobuf3 once it gets ocaml support (e.g., I write it) and I verify performance parity".

My 2c,
--chet--


On Wed, May 18, 2016 at 6:43 AM,  <rixed@happyleptic.org> wrote:

Hello.

I'm thinking about implementing a library for doing RPC with OCaml, with large
scale environments in mind (à la Stubby but with better type checking of
course). I'm wondering what are the related libs I should make myself familiar
with before starting. I've seen a few interesting things for serialization
(piqi come to mind), some interesting event engines (LWT, Core), some protocol
implementations suitable for transport but no HTTP2, nothing to interface with
monitoring subsystems or TSDBs, nothing related to load balancing, routing,
DDoS detection, etc, some crypto, an interesting TLS implementation from
MirageOs, no OAuth or similar.

What other related projects should I look at?

Also, if anyone would be interested in contributing ideas, experience or code
please let me know.


--
Caml-list mailing list.  Subscription management and archives:
https://sympa.inria.fr/sympa/arc/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs



[-- Attachment #2: Type: text/html, Size: 6729 bytes --]