* [Caml-list] What are people using when writing distributed applications in OCaml?
@ 2014-12-04 9:33 Francois Berenger
2014-12-04 9:47 ` Kenneth Adam Miller
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Francois Berenger @ 2014-12-04 9:33 UTC (permalink / raw)
To: OCaml List
Hello,
If you know about some distributed application that went into production
and that can happily withstand the fire of production and real world
loads, I would be happy to here about which library the system was using.
I have heard about MPI and zmq for example, which both have
ocaml bindings. However, I also know that there are several RPC
implementations out there (one in ocamlnet, one in mirage, one in core),
but I am not sure about which one is the most mature and production-tested.
Thanks a lot,
Francois.
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [Caml-list] What are people using when writing distributed applications in OCaml? 2014-12-04 9:33 [Caml-list] What are people using when writing distributed applications in OCaml? Francois Berenger @ 2014-12-04 9:47 ` Kenneth Adam Miller 2014-12-04 9:55 ` Anders Fugmann 2014-12-04 12:00 ` Gerd Stolpmann 2014-12-04 15:37 ` Malcolm Matalka 2 siblings, 1 reply; 6+ messages in thread From: Kenneth Adam Miller @ 2014-12-04 9:47 UTC (permalink / raw) To: caml users [-- Attachment #1: Type: text/plain, Size: 1006 bytes --] I personally use ocaml-zmq bindings, and I am working with the piqi author (and my work) to try and release my ocaml RPC implementation. On Thu, Dec 4, 2014 at 4:33 AM, Francois Berenger < francois.berenger@inria.fr> wrote: > Hello, > > If you know about some distributed application that went into production > and that can happily withstand the fire of production and real world > loads, I would be happy to here about which library the system was using. > > I have heard about MPI and zmq for example, which both have > ocaml bindings. However, I also know that there are several RPC > implementations out there (one in ocamlnet, one in mirage, one in core), > but I am not sure about which one is the most mature and production-tested. > > Thanks a lot, > Francois. > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > [-- Attachment #2: Type: text/html, Size: 1653 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Caml-list] What are people using when writing distributed applications in OCaml? 2014-12-04 9:47 ` Kenneth Adam Miller @ 2014-12-04 9:55 ` Anders Fugmann 0 siblings, 0 replies; 6+ messages in thread From: Anders Fugmann @ 2014-12-04 9:55 UTC (permalink / raw) To: Kenneth Adam Miller, caml users At Issuu, we started with an ZMQ / Piqi based messaging (RPC and broadcast). However we are leaning more and more to using AMQP for messaging and RPC where throughput and latency is not as critical (we still handle 50.000 messages/sec through AMQP though). Amqp handles RPC style messages nicely, and handles client disconnects and retransmissions in a useful way. Having a central message broker also decouples the services even more as the micro-services only need to agree on which Amqp to use and does not need to know the name of each other. /Anders On 12/04/2014 10:47 AM, Kenneth Adam Miller wrote: > I personally use ocaml-zmq bindings, and I am working with the piqi > author (and my work) to try and release my ocaml RPC implementation. > > On Thu, Dec 4, 2014 at 4:33 AM, Francois Berenger > <francois.berenger@inria.fr <mailto:francois.berenger@inria.fr>> wrote: > > Hello, > > If you know about some distributed application that went into production > and that can happily withstand the fire of production and real world > loads, I would be happy to here about which library the system was > using. > > I have heard about MPI and zmq for example, which both have > ocaml bindings. However, I also know that there are several RPC > implementations out there (one in ocamlnet, one in mirage, one in > core), but I am not sure about which one is the most mature and > production-tested. > > Thanks a lot, > Francois. > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/__arc/caml-list > <https://sympa.inria.fr/sympa/arc/caml-list> > Beginner's list: http://groups.yahoo.com/group/__ocaml_beginners > <http://groups.yahoo.com/group/ocaml_beginners> > Bug reports: http://caml.inria.fr/bin/caml-__bugs > <http://caml.inria.fr/bin/caml-bugs> > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Caml-list] What are people using when writing distributed applications in OCaml? 2014-12-04 9:33 [Caml-list] What are people using when writing distributed applications in OCaml? Francois Berenger 2014-12-04 9:47 ` Kenneth Adam Miller @ 2014-12-04 12:00 ` Gerd Stolpmann 2014-12-04 15:37 ` Malcolm Matalka 2 siblings, 0 replies; 6+ messages in thread From: Gerd Stolpmann @ 2014-12-04 12:00 UTC (permalink / raw) To: Francois Berenger; +Cc: OCaml List [-- Attachment #1: Type: text/plain, Size: 1350 bytes --] Am Donnerstag, den 04.12.2014, 10:33 +0100 schrieb Francois Berenger: > Hello, > > If you know about some distributed application that went into production > and that can happily withstand the fire of production and real world > loads, I would be happy to here about which library the system was using. > > I have heard about MPI and zmq for example, which both have > ocaml bindings. However, I also know that there are several RPC > implementations out there (one in ocamlnet, one in mirage, one in core), > but I am not sure about which one is the most mature and production-tested. The one in OCamlnet was extensively tested at Mylife.com (around 2009), where we built up a 24/7 distributed service over several hundred machines. Part of that were also some HA services with failover possibility, which is worth to mention because it means that the error reporting and timeout handling was working. At present, I'm not aware of a mass deployment, though. Gerd -- ------------------------------------------------------------ Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de My OCaml site: http://www.camlcity.org Contact details: http://www.camlcity.org/contact.html Company homepage: http://www.gerd-stolpmann.de ------------------------------------------------------------ [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Caml-list] What are people using when writing distributed applications in OCaml? 2014-12-04 9:33 [Caml-list] What are people using when writing distributed applications in OCaml? Francois Berenger 2014-12-04 9:47 ` Kenneth Adam Miller 2014-12-04 12:00 ` Gerd Stolpmann @ 2014-12-04 15:37 ` Malcolm Matalka 2014-12-04 19:54 ` Gerd Stolpmann 2 siblings, 1 reply; 6+ messages in thread From: Malcolm Matalka @ 2014-12-04 15:37 UTC (permalink / raw) To: Francois Berenger; +Cc: OCaml List It depends on what level you're talking. My distributed systems apps tend to be functorized over their transport, so I can switch out what works and what doesn't pretty easily. I tend to avoid RPC frameworks, I find they focus more on the computation than on the data, but depending on what you're doing they could be appropriate. Unless you're under heavy load, simple HTTP calls with some JSON is probably sufficient. Francois Berenger <francois.berenger@inria.fr> writes: > Hello, > > If you know about some distributed application that went into production > and that can happily withstand the fire of production and real world loads, I would be happy to here about which library the system was > using. > > I have heard about MPI and zmq for example, which both have > ocaml bindings. However, I also know that there are several RPC > implementations out there (one in ocamlnet, one in mirage, one in core), but I am not sure about which one is the most mature and > production-tested. > > Thanks a lot, > Francois. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Caml-list] What are people using when writing distributed applications in OCaml? 2014-12-04 15:37 ` Malcolm Matalka @ 2014-12-04 19:54 ` Gerd Stolpmann 0 siblings, 0 replies; 6+ messages in thread From: Gerd Stolpmann @ 2014-12-04 19:54 UTC (permalink / raw) To: Malcolm Matalka; +Cc: Francois Berenger, OCaml List [-- Attachment #1: Type: text/plain, Size: 2782 bytes --] Am Donnerstag, den 04.12.2014, 15:37 +0000 schrieb Malcolm Matalka: > It depends on what level you're talking. My distributed systems apps > tend to be functorized over their transport, so I can switch out what > works and what doesn't pretty easily. > > I tend to avoid RPC frameworks, I find they focus more on the > computation than on the data, but depending on what you're doing they > could be appropriate. > > Unless you're under heavy load, simple HTTP calls with some JSON is > probably sufficient. I guess Francois is asking because for some reason HTTP is inappropriate. I can understand that because it is a really bloated protocol, and you usually only use it when you need to (e.g. you have to call it from Javascript or the like, or have to go through firewalls). Also, there are several aspects you cannot control well, e.g. when TCP connections are created and kept alive (or not). Multicast messaging is completely impossible. I recommend to not to see RPC systems under their framework aspect, but simply as transport libraries. In particular, if you don't like the serialization scheme, there is usually the alternative that you take your own serializer and just send the data as (binary) strings on the RPC level (which is always possible and simple to do). From my experience it is essential that you have good control over the low-level things (e.g. connection control, timeouts, retrials, error handling etc), because this is what the distributed system keeps running. Sure it is good when an RPC library implements some automatisms (e.g. for error handling), but it is even better when there is also a manual mode, as distributed systems sometimes require unusual ways of dealing with that stuff. Gerd > > Francois Berenger <francois.berenger@inria.fr> writes: > > > Hello, > > > > If you know about some distributed application that went into production > > and that can happily withstand the fire of production and real world loads, I would be happy to here about which library the system was > > using. > > > > I have heard about MPI and zmq for example, which both have > > ocaml bindings. However, I also know that there are several RPC > > implementations out there (one in ocamlnet, one in mirage, one in core), but I am not sure about which one is the most mature and > > production-tested. > > > > Thanks a lot, > > Francois. > -- ------------------------------------------------------------ Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de My OCaml site: http://www.camlcity.org Contact details: http://www.camlcity.org/contact.html Company homepage: http://www.gerd-stolpmann.de ------------------------------------------------------------ [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-12-04 19:55 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-12-04 9:33 [Caml-list] What are people using when writing distributed applications in OCaml? Francois Berenger 2014-12-04 9:47 ` Kenneth Adam Miller 2014-12-04 9:55 ` Anders Fugmann 2014-12-04 12:00 ` Gerd Stolpmann 2014-12-04 15:37 ` Malcolm Matalka 2014-12-04 19:54 ` Gerd Stolpmann
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).