From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id D01F97EE7A for ; Thu, 28 Mar 2013 09:34:51 +0100 (CET) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of philippe.veber@gmail.com) identity=pra; client-ip=209.85.223.170; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="philippe.veber@gmail.com"; x-sender="philippe.veber@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of philippe.veber@gmail.com designates 209.85.223.170 as permitted sender) identity=mailfrom; client-ip=209.85.223.170; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="philippe.veber@gmail.com"; x-sender="philippe.veber@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ie0-f170.google.com) identity=helo; client-ip=209.85.223.170; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="philippe.veber@gmail.com"; x-sender="postmaster@mail-ie0-f170.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjYBAGn/U1HRVd+qm2dsb2JhbABDgzqsWjKJLgGIKnsIFg4BAQEBAQYLCwkUKIIfAQEEAUABGxACCAMBAwELBgUEBwcTISIBEQEFAQoEDgYTEodvAQMJBgygQIwygnuELwoZJwMKWYh8AQUMjwgEB4NAA5Mhg0aBH44GFimCVIFbOw X-IPAS-Result: AjYBAGn/U1HRVd+qm2dsb2JhbABDgzqsWjKJLgGIKnsIFg4BAQEBAQYLCwkUKIIfAQEEAUABGxACCAMBAwELBgUEBwcTISIBEQEFAQoEDgYTEodvAQMJBgygQIwygnuELwoZJwMKWYh8AQUMjwgEB4NAA5Mhg0aBH44GFimCVIFbOw X-IronPort-AV: E=Sophos;i="4.84,924,1355094000"; d="scan'208";a="10803399" Received: from mail-ie0-f170.google.com ([209.85.223.170]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 28 Mar 2013 09:34:51 +0100 Received: by mail-ie0-f170.google.com with SMTP id c11so11409454ieb.15 for ; Thu, 28 Mar 2013 01:34:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=EYRrIVffgv6PspXxJCkiiXPTsYFsMd/yVvbkuP967qA=; b=tB5dTp6uUs2xOqNrQwPaRACYhZVs/VKIwtsP3ojAiFo6qS8WEjHOaptPofI62M05cr 7IKJjaCrT6DaskJQsKhXH5vg4T9ZdnlfFBgOYc4ITlhe2CGCgqv/r4t7QtGgSVCEBADs cYeBXHmFiPXmLawVIUPGos141xNTQdfJ/ft1R1ll73mLkr5Jt9uf0C4HcgpaFg6qmFQV NMs3UnkL2p5M/BxflzR/nAsKV8rBLf5kGQgFUXqFc2FQ3vavUMSEmNZRGnbNd/T2/uzN B2CHMGMZkUj92Ut5eL61X0jm3vRA5BMsUeMdB/N7ksfnb28rwUR1gyxwq9BO+jOLad1n lS6Q== X-Received: by 10.42.133.74 with SMTP id g10mr13358896ict.32.1364459689847; Thu, 28 Mar 2013 01:34:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.136.8 with HTTP; Thu, 28 Mar 2013 01:34:29 -0700 (PDT) In-Reply-To: <5152C354.7060708@chimrod.com> References: <5152C354.7060708@chimrod.com> From: Philippe Veber Date: Thu, 28 Mar 2013 09:34:29 +0100 Message-ID: To: =?ISO-8859-1?Q?S=E9bastien_Dailly?= Cc: caml users Content-Type: multipart/alternative; boundary=90e6ba6149981fb66a04d8f807c3 X-Validation-by: philippe.veber@gmail.com Subject: Re: [Caml-list] Master-slave architecture behind an ocsigen server. --90e6ba6149981fb66a04d8f807c3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Thanks S=E9bastien, a message broker layer could indeed simplify communication between the server and the workers. I'll think of it if I cannot use an higher-level API (dealing with a worker pool). 2013/3/27 S=E9bastien Dailly > Le 26/03/2013 15:29, Philippe Veber a =E9crit : > > Dear all, >> >> I'm developping an ocsigen website doing some scientific calculations. >> Up to now, the calculations were done in the same process that runs the >> server. In order to gain in scalability (and maybe stability too), I >> would like to run those calculations in a separate (pool of) >> process(es). As this is a pretty typical setup, I guess quite a few >> people have already done that. So I'd like to hear some suggestions on >> what library to use in this particular context. It seems to me that the >> release library [1] should do the job and is lwt-friendly, but there are >> maybe other good options? >> >> > This make me think of the python celery project[1]. It rely on rabbitMQ > for storing the task to execute and ensure persistance. The project just > handle the message and give them to the registered worker. > > I found ocamlmq[2] wich could match, but didn't test it. > > [1] http://celeryproject.org/ > [2] https://github.com/mfp/ocamlmq > > -- > S=E9bastien Dailly > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/**arc/caml-list > Beginner's list: http://groups.yahoo.com/group/**ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-**bugs > --90e6ba6149981fb66a04d8f807c3 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Thanks S=E9bastien, a message broker layer could indeed si= mplify communication between the server and the workers. I'll think of = it if I cannot use an higher-level API (dealing with a worker pool).


2013/3/27 S= =E9bastien Dailly <sebastien-ocaml@chimrod.com>
Le 26/03/2013 15:29, Philippe Veber a =E9crit :

Dear all,

I'm developping an ocsigen website doing some scientific calculations.<= br> Up to now, the calculations were done in the same process that runs the
server. In order to gain in scalability (and maybe stability too), I
would like to run those calculations in a separate (pool of)
process(es). As this is a pretty typical setup, I guess quite a few
people have already done that. So I'd like to hear some suggestions on<= br> what library to use in this particular context. It seems to me that the
release library [1] should do the job and is lwt-friendly, but there are
maybe other good options?


This make me think of the python celery project[1]. It rely on rabbitMQ for= storing the task to execute and ensure persistance. The project just handl= e the message and give them to the registered worker.

I found ocamlmq[2] wich could match, but didn't test it.

[1] http://celerypr= oject.org/
[2] https://gi= thub.com/mfp/ocamlmq

--
S=E9bastien Dailly


--
Caml-list mailing list. =A0Subscription management and archives:
ht= tps://sympa.inria.fr/sympa/arc/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners<= /a>
Bug reports: http://caml.inria.fr/bin/caml-bugs

--90e6ba6149981fb66a04d8f807c3--