Re: [Caml-list] RPC for OCaml?

[rixed@happyleptic.org]

Hi!

I'm not very opinionated as far as which brick should be used but for the fact
that I'd like the low level bricks to be interchangeable (transport,
serialization, RPC).

I'm more opinionated about the bigger picture, which I think is more
specificaly were we (community and industry) are missing something, probably
because of how large infrastructure is build (ie incrementaly from a small
one). We commonly consider RPC, monitoring, load balancing, authentication and
service discovery as independant things, but whenever a company has to build a
large infra from scratch they tends to prefer to integrate (at least some of)
the above, for this "economy of scale" in complexity brings many benefits:

- proxyless routing
- proxyless load balancing
- seamless authentication
- seamless quota
- instrumentation of all RPC for free
- consequently, no need for independant network monitoring
- consequently, reliable detection of problems and reliable system to route
  around or drain the faulty nodes/clusters
- uniform behavior across the whole infrastructure inducing ease to diagnosing
  issues and predict behavior
- any improvement of the infrastructure benefits every services

Compare with the industry "standard" consisting of throwing some unrelated
services together and spending an enormous amount of time interconnecting them
together with little visibility.

I do not care about speed so much as to consider C++ or RDMA, but more about
reliability and type safety.  That's why I'm envisaging this for
OCaml/MirageOs, interacting with other "legacy" stuff only at the perimeter.

In this ideal world or strongly typed microservices, stub compilers or IDL
would be superfluous, for you want the authoritative types to be defined in
OCaml rather than in, say, protobuf (esp. version 3 which weaken types even
more to pander to the json crowd, or so I like to imagine). But then it would
make it much harder to interface with the outer world, as you rightfully
described.

The only way to reconcile those conflicting views, it seems, would be to use
some IDL that's as strongly typed as OCaml, supporting various underlying
serializers, and that would compile additional runtime type checks for weaker
serializer and for softer languages.  Isn't that why piqi was invented?  (By
the way piqi supports proto2 and OCaml).