From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 874727EE7A for ; Thu, 28 Mar 2013 08:37:27 +0100 (CET) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of philippe.veber@gmail.com) identity=pra; client-ip=209.85.223.177; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="philippe.veber@gmail.com"; x-sender="philippe.veber@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of philippe.veber@gmail.com designates 209.85.223.177 as permitted sender) identity=mailfrom; client-ip=209.85.223.177; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="philippe.veber@gmail.com"; x-sender="philippe.veber@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ie0-f177.google.com) identity=helo; client-ip=209.85.223.177; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="philippe.veber@gmail.com"; x-sender="postmaster@mail-ie0-f177.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ai4BANrxU1HRVd+xm2dsb2JhbABDgzqsWolgAYgqewgWDgEBAQEBBgsLCRQogh8BAQQBQAEbEAIIAwEDAQsGBQsHBg0hIgERAQUBCgQOBhMICodvAQMJBgygM4wygnuEMQoZJwMKWYh8AQUMjwgEB4NAA5Mhg0aBH44GFimCVIFbOw X-IPAS-Result: Ai4BANrxU1HRVd+xm2dsb2JhbABDgzqsWolgAYgqewgWDgEBAQEBBgsLCRQogh8BAQQBQAEbEAIIAwEDAQsGBQsHBg0hIgERAQUBCgQOBhMICodvAQMJBgygM4wygnuEMQoZJwMKWYh8AQUMjwgEB4NAA5Mhg0aBH44GFimCVIFbOw X-IronPort-AV: E=Sophos;i="4.84,924,1355094000"; d="scan'208";a="9006853" Received: from mail-ie0-f177.google.com ([209.85.223.177]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 28 Mar 2013 08:37:26 +0100 Received: by mail-ie0-f177.google.com with SMTP id tp5so6896772ieb.22 for ; Thu, 28 Mar 2013 00:37:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=XlK5Q6GZRyEfQprUeVpkdBEZwzx7KPLYyItBBYneXUE=; b=ZOHwHlD2Amn8v40/59wtefSULNGLoQVyioeIOp2WRtQz87rWoVaM8E47XtJuXYcKPp LSUdFnoTdzNCvz1GDz5KVt5HDGzbHx6+1JL1kmueOUrEVxzJuLo9so1gfLAhCURVo7wx wKhxaAEJwm8raNIK2J6PGKRg9XNC5EN9ZZHjKs2ZMT5oOV7lqD4pjjFQmpBs6s8lcQyX 17CnGH7vELq4zHik6emTWV1tew22fuLZ4B+Myi2FDhObMvwbQo4MGM/mCmYQRWg1MeU0 1umz+IApzbPFk8FMk3U9eWOAU+tJAmVRFP9+CSV0/wv8UyzD5YEkeoXk4O4tBVuBQ0WU 7E8g== X-Received: by 10.43.88.134 with SMTP id ba6mr13543969icc.18.1364456244874; Thu, 28 Mar 2013 00:37:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.136.8 with HTTP; Thu, 28 Mar 2013 00:37:04 -0700 (PDT) In-Reply-To: <51520CAE.6020009@ens-lyon.org> References: <51520CAE.6020009@ens-lyon.org> From: Philippe Veber Date: Thu, 28 Mar 2013 08:37:04 +0100 Message-ID: To: Martin Jambon Cc: caml users Content-Type: multipart/alternative; boundary=bcaec517c9a2c992cc04d8f73931 X-Validation-by: philippe.veber@gmail.com Subject: Re: [Caml-list] Master-slave architecture behind an ocsigen server. --bcaec517c9a2c992cc04d8f73931 Content-Type: text/plain; charset=ISO-8859-1 Hi Martin, nproc meets exactly my needs: a simple lwt-friendly interface to dispatch function calls on a pool of processes that run on the same machine. I have only one concern, that should probably be discussed on the ocsigen list, that is I wonder if it is okay to fork the process running the ocsigen server. I think I remember warnings on having parent and children processes sharing connections/channels but it's really not clear to me. Thanks a lot for your answer! ph. [Off-topic] PS I understand your remark on the use of the word slave, be assured that I did not mean to hurt anyone's feelings. However I'm affraid replacing slave by worker does not make the whole expression sound better, as it implies calling a worker's boss 'master'. This is also a bit creepy to me. Maybe it's enough to recall that we are talking about processes. Or go for a more peaceful context, music, and call this a conductor/performer architecture :o)? 2013/3/26 Martin Jambon > On 03/26/2013 07:29 AM, Philippe Veber wrote: > >> Dear all, >> >> I'm developping an ocsigen website doing some scientific calculations. >> Up to now, the calculations were done in the same process that runs the >> server. In order to gain in scalability (and maybe stability too), I >> would like to run those calculations in a separate (pool of) >> process(es). As this is a pretty typical setup, I guess quite a few >> people have already done that. So I'd like to hear some suggestions on >> what library to use in this particular context. It seems to me that the >> release library [1] should do the job and is lwt-friendly, but there are >> maybe other good options? >> > > I wrote and used a library called Nproc about a year ago. It lets you > create (Nproc.create) a pool of N processes, to which you can submit > (Nproc.submit) computations of any type quasi-magically - just make sure > any big environment required for the computation is not copied with each > closure that you send to the workers. The submodule Nproc.Full provides a > more advanced interface that lets each worker process have its own local > environment. > > https://github.com/MyLifeLabs/**nproc > > I haven't used Nproc in a while but it was working fine and should still > work. > > > Martin > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/**arc/caml-list > Beginner's list: http://groups.yahoo.com/group/**ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-**bugs > --bcaec517c9a2c992cc04d8f73931 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi Martin,
nproc meets exactl= y my needs: a simple lwt-friendly interface to dispatch function calls on a= pool of processes that run on the same machine. I have only one concern, t= hat should probably be discussed on the ocsigen list, that is I wonder if i= t is okay to fork the process running the ocsigen server. I think I remembe= r warnings on having parent and children processes sharing connections/chan= nels but it's really not clear to me.
Thanks a lot for your answer!
ph.

[Off-topic] P= S I understand your remark on the use of the word slave, be assured that I = did not mean to hurt anyone's feelings. However I'm affraid replaci= ng slave by worker does not make the whole expression sound better, as it i= mplies calling a worker's boss 'master'. This is also a bit cre= epy to me. Maybe it's enough to recall that we are talking about proces= ses. Or go for a more peaceful context, music, and call this a conductor/pe= rformer architecture :o)?



201= 3/3/26 Martin Jambon <martin.jambon@ens-lyon.org>
On 03/26/2013 07:29 AM, Philippe Veber wrote:
Dear all,

I'm developping an ocsigen website doing some scientific calculations.<= br> Up to now, the calculations were done in the same process that runs the
server. In order to gain in scalability (and maybe stability too), I
would like to run those calculations in a separate (pool of)
process(es). As this is a pretty typical setup, I guess quite a few
people have already done that. So I'd like to hear some suggestions on<= br> what library to use in this particular context. It seems to me that the
release library [1] should do the job and is lwt-friendly, but there are
maybe other good options?

I wrote and used a library called Nproc about a year ago. It lets you creat= e (Nproc.create) a pool of N processes, to which you can submit (Nproc.subm= it) computations of any type quasi-magically - just make sure any big envir= onment required for the computation is not copied with each closure that yo= u send to the workers. The submodule Nproc.Full provides a more advanced in= terface that lets each worker process have its own local environment.

=A0 https= ://github.com/MyLifeLabs/nproc

I haven't used Nproc in a while but it was working fine and should stil= l work.


Martin


--
Caml-list mailing list. =A0Subscription management and archives:
ht= tps://sympa.inria.fr/sympa/arc/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners<= /a>
Bug reports: http://caml.inria.fr/bin/caml-bugs

--bcaec517c9a2c992cc04d8f73931--