From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 4EE9A7EE7C for ; Thu, 28 Mar 2013 10:24:19 +0100 (CET) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of philippe.veber@gmail.com) identity=pra; client-ip=209.85.210.173; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="philippe.veber@gmail.com"; x-sender="philippe.veber@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of philippe.veber@gmail.com designates 209.85.210.173 as permitted sender) identity=mailfrom; client-ip=209.85.210.173; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="philippe.veber@gmail.com"; x-sender="philippe.veber@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ia0-f173.google.com) identity=helo; client-ip=209.85.210.173; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="philippe.veber@gmail.com"; x-sender="postmaster@mail-ia0-f173.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ai4BAAcMVFHRVdKtm2dsb2JhbABDgzqsWolgAYgqewgWDgEBAQEBBgsLCRQogh8BAQQBQAEbEAIIAwEDAQsGBQsDBBMhIQEBEQEFAQoEDgYTCQmHbwEDCQYMoFOMMoJ7hDEKGScDClmIfAEFDIw8gioiBAeDQAOTIYFmgWCBH4pNgzkWKYQvOw X-IPAS-Result: Ai4BAAcMVFHRVdKtm2dsb2JhbABDgzqsWolgAYgqewgWDgEBAQEBBgsLCRQogh8BAQQBQAEbEAIIAwEDAQsGBQsDBBMhIQEBEQEFAQoEDgYTCQmHbwEDCQYMoFOMMoJ7hDEKGScDClmIfAEFDIw8gioiBAeDQAOTIYFmgWCBH4pNgzkWKYQvOw X-IronPort-AV: E=Sophos;i="4.84,924,1355094000"; d="scan'208";a="10813072" Received: from mail-ia0-f173.google.com ([209.85.210.173]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 28 Mar 2013 10:24:14 +0100 Received: by mail-ia0-f173.google.com with SMTP id h37so8125164iak.4 for ; Thu, 28 Mar 2013 02:24:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=h7K+ndruUKvfxZ5jOArxYl9o9G6sfDe52QnW8FkxezA=; b=EwnH4YGvx0xB4JSI0eLYi6dyRRdz/PSDaP1PliCBze/Vy1c+zOLeexTRDjy6MMEbqj yomz+PtWTUtG60Mhzn8+x70ttQznRa/6NQ6b5ts3IVcHsS/fJI83HDI78+U6aLSPszKd D2vlLNqBugVm8YDL7cOaT2AbRGFydgXO/s4femGWpx4PaaLns/zfyVmvsctRmWSniIyz Rx9MFi0w5cavcUS6uXrC0jTB72Ne+VjhtujwYMF9b1JlMKuwbmoxvveEW4mRmnIiN1TM d/Cw03vP7UWGOq1r36a5vf7WBxIZqlwPOEHMfHP/VqujItt8bKRsF/BIQ5pCbGIDKDEw DJvA== X-Received: by 10.50.66.133 with SMTP id f5mr6700049igt.38.1364462653056; Thu, 28 Mar 2013 02:24:13 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.136.8 with HTTP; Thu, 28 Mar 2013 02:23:52 -0700 (PDT) In-Reply-To: <4A6314AA-0C59-4E35-9EA4-F465C0A5AF3A@gmail.com> References: <1364424571.14693.1@samsung> <4A6314AA-0C59-4E35-9EA4-F465C0A5AF3A@gmail.com> From: Philippe Veber Date: Thu, 28 Mar 2013 10:23:52 +0100 Message-ID: To: Denis Berthod Cc: Gerd Stolpmann , caml users Content-Type: multipart/alternative; boundary=047d7bdc0faebec02a04d8f8b7cc X-Validation-by: philippe.veber@gmail.com Subject: Re: AW: [Caml-list] Master-slave architecture behind an ocsigen server. --047d7bdc0faebec02a04d8f8b7cc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Denis, As Gerd kindly explained, using threads wouldn't help for scalability. It turns out that it wouldn't help for stability either, because the workers have to deal with R's (as in statistics softw'R) interpreter, and the communication between OCaml and R is not bullet-proof right now. So I'd rather have the computation isolated in its process, not to harm the web-server. Cheers, ph. 2013/3/28 Denis Berthod > > > > Le 27 mars 2013 =E0 23:49, Gerd Stolpmann a =E9crit : > > > Am 27.03.2013 23:42:14 schrieb(en) Denis Berthod: > >> Hello, > >> Isn't the Lwt_preemptive module exactly what you want? > >> http://ocsigen.org/lwt/api/Lwt_preemptive > >> But maybe, you want to use real processes and not threads. > > > > In OCaml, only one thread may access runtime functions at a time, and > because of this, only one thread at a time can run OCaml code. So you > exploit only one core with a multi-threaded program. I guess this does not > meet Philippe's scalability requirement, and he wants to use processes > instead. > > > > Gerd > > > > > Indead, I have only considered the stability problem that can be tackle by > real threads as they don't block the lwt runtimes during computation. > Howerver, you are right concerning the scalability. > > Cheers, > > Denis > > > > > > >> Cheers, > >> Denis > >> Le 26 mars 2013 =E0 15:29, Philippe Veber a =E9crit : > >> > Dear all, > >> > > >> > I'm developping an ocsigen website doing some scientific > calculations. Up to now, the calculations were done in the same process > that runs the server. In order to gain in scalability (and maybe stability > too), I would like to run those calculations in a separate (pool of) > process(es). As this is a pretty typical setup, I guess quite a few people > have already done that. So I'd like to hear some suggestions on what > library to use in this particular context. It seems to me that the release > library [1] should do the job and is lwt-friendly, but there are maybe > other good options? > >> > > >> > Thanks for any hint, cheers! > >> > > >> > Philippe. > >> > > >> > [1] https://github.com/andrenth/release > >> -- > >> Caml-list mailing list. Subscription management and archives: > >> https://sympa.inria.fr/sympa/arc/caml-list > >> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > >> Bug reports: http://caml.inria.fr/bin/caml-bugs > > > > -- > > ------------------------------------------------------------ > > Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de > > Creator of GODI and camlcity.org. > > Contact details: http://www.camlcity.org/contact.html > > Company homepage: http://www.gerd-stolpmann.de > > ------------------------------------------------------------ > > -- > > Caml-list mailing list. Subscription management and archives: > > https://sympa.inria.fr/sympa/arc/caml-list > > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > > Bug reports: http://caml.inria.fr/bin/caml-bugs > > --047d7bdc0faebec02a04d8f8b7cc Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi Denis,

As Gerd kindly explained, using threads w= ouldn't help for scalability. It turns out that it wouldn't help fo= r stability either, because the workers have to deal with R's (as in st= atistics softw'R) interpreter, and the communication between OCaml and = R is not bullet-proof right now. So I'd rather have the computation iso= lated in its process, not to harm the web-server.

Cheers,
=A0 ph.
=A0


<= div class=3D"gmail_quote">2013/3/28 Denis Berthod <casa.berthod@gmail= .com>



Le 27 mars 2013 =E0 23:49, Gerd Stolpmann a =E9crit :

> Am 27.03.2013 2= 3:42:14 schrieb(en) Denis Berthod:
>> Hello,
>> Isn't the Lwt_preemptive module exactly what you want?
>> http://ocsigen.org/lwt/api/Lwt_preemptive
>> But maybe, you want to use real processes and not threads.
>
> In OCaml, only one thread may access runtime functions at a time, and = because of this, only one thread at a time can run OCaml code. So you explo= it only one core with a multi-threaded program. I guess this does not meet = Philippe's scalability requirement, and he wants to use processes inste= ad.
>
> Gerd
>


Indead, I have only considered the stability problem that can be tack= le by real threads as they don't block the lwt runtimes during computat= ion.
Howerver, you are right concerning the scalability.

Cheers,

Denis



>
>> Cheers,
>> Denis
>> Le 26 mars 2013 =E0 15:29, Philippe Veber a =E9crit :
>> > Dear all,
>> >
>> > I'm developping an ocsigen website doing some scientific = calculations. Up to now, the calculations were done in the same process tha= t runs the server. In order to gain in scalability (and maybe stability too= ), I would like to run those calculations in a separate (pool of) process(e= s). As this is a pretty typical setup, I guess quite a few people have alre= ady done that. So I'd like to hear some suggestions on what library to = use in this particular context. It seems to me that the release library [1]= should do the job and is lwt-friendly, but there are maybe other good opti= ons?
>> >
>> > Thanks for any hint, cheers!
>> >
>> > Philippe.
>> >
>> > [1] https://github.com/andrenth/release
>> --
>> Caml-list mailing list. =A0Subscription management and archives: >> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginner= s
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
> --
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany =A0 =A0gerd@gerd-stolpmann.de
> Creator of GODI and = camlcity.org.
> Contact details: =A0 =A0 =A0 =A0http://www.camlcity.org/contact.html
> Company homepage: =A0 =A0 =A0 http://www.gerd-stolpmann.de
> ------------------------------------------------------------
> --
> Caml-list mailing list. =A0Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs


--047d7bdc0faebec02a04d8f8b7cc--