[Caml-list] Master-slave architecture behind an ocsigen server.

[Caml-list] Master-slave architecture behind an ocsigen server.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 676 bytes --]

Dear all,

I'm developping an ocsigen website doing some scientific calculations. Up
to now, the calculations were done in the same process that runs the
server. In order to gain in scalability (and maybe stability too), I would
like to run those calculations in a separate (pool of) process(es). As this
is a pretty typical setup, I guess quite a few people have already done
that. So I'd like to hear some suggestions on what library to use in this
particular context. It seems to me that the release library [1] should do
the job and is lwt-friendly, but there are maybe other good options?

Thanks for any hint, cheers!

Philippe.

[1] https://github.com/andrenth/release

[-- Attachment #2: Type: text/html, Size: 788 bytes --]

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Martin Jambon]

I know it's off-topic but I suggest not using the word "slave" but 
"worker" instead. I find it unnecessarily offensive.

Martin

On Tue 26 Mar 2013 07:29:10 AM PDT, Philippe Veber wrote:
> Dear all,
>
> I'm developping an ocsigen website doing some scientific calculations.
> Up to now, the calculations were done in the same process that runs
> the server. In order to gain in scalability (and maybe stability too),
> I would like to run those calculations in a separate (pool of)
> process(es). As this is a pretty typical setup, I guess quite a few
> people have already done that. So I'd like to hear some suggestions on
> what library to use in this particular context. It seems to me that
> the release library [1] should do the job and is lwt-friendly, but
> there are maybe other good options?
>
> Thanks for any hint, cheers!
>
> Philippe.
>
> [1] https://github.com/andrenth/release

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Martin Jambon]

On 03/26/2013 07:29 AM, Philippe Veber wrote:
> Dear all,
>
> I'm developping an ocsigen website doing some scientific calculations.
> Up to now, the calculations were done in the same process that runs the
> server. In order to gain in scalability (and maybe stability too), I
> would like to run those calculations in a separate (pool of)
> process(es). As this is a pretty typical setup, I guess quite a few
> people have already done that. So I'd like to hear some suggestions on
> what library to use in this particular context. It seems to me that the
> release library [1] should do the job and is lwt-friendly, but there are
> maybe other good options?

I wrote and used a library called Nproc about a year ago. It lets you 
create (Nproc.create) a pool of N processes, to which you can submit 
(Nproc.submit) computations of any type quasi-magically - just make sure 
any big environment required for the computation is not copied with each 
closure that you send to the workers. The submodule Nproc.Full provides 
a more advanced interface that lets each worker process have its own 
local environment.

   https://github.com/MyLifeLabs/nproc

I haven't used Nproc in a while but it was working fine and should still 
work.


Martin

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Francois Berenger]

On 03/27/2013 06:01 AM, Martin Jambon wrote:
> On 03/26/2013 07:29 AM, Philippe Veber wrote:
>> Dear all,
>>
>> I'm developping an ocsigen website doing some scientific calculations.
>> Up to now, the calculations were done in the same process that runs the
>> server. In order to gain in scalability (and maybe stability too), I
>> would like to run those calculations in a separate (pool of)
>> process(es). As this is a pretty typical setup, I guess quite a few
>> people have already done that. So I'd like to hear some suggestions on
>> what library to use in this particular context. It seems to me that the
>> release library [1] should do the job and is lwt-friendly, but there are
>> maybe other good options?
>
> I wrote and used a library called Nproc about a year ago.

Not in OPAM. :(

 > It lets you
> create (Nproc.create) a pool of N processes, to which you can submit
> (Nproc.submit) computations of any type quasi-magically - just make sure
> any big environment required for the computation is not copied with each
> closure that you send to the workers. The submodule Nproc.Full provides
> a more advanced interface that lets each worker process have its own
> local environment.
>
>    https://github.com/MyLifeLabs/nproc
>
> I haven't used Nproc in a while but it was working fine and should still
> work.
>
>
> Martin
>
>

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Sébastien Dailly]

Le 26/03/2013 15:29, Philippe Veber a écrit :
> Dear all,
>
> I'm developping an ocsigen website doing some scientific calculations.
> Up to now, the calculations were done in the same process that runs the
> server. In order to gain in scalability (and maybe stability too), I
> would like to run those calculations in a separate (pool of)
> process(es). As this is a pretty typical setup, I guess quite a few
> people have already done that. So I'd like to hear some suggestions on
> what library to use in this particular context. It seems to me that the
> release library [1] should do the job and is lwt-friendly, but there are
> maybe other good options?
>

This make me think of the python celery project[1]. It rely on rabbitMQ 
for storing the task to execute and ensure persistance. The project just 
handle the message and give them to the registered worker.

I found ocamlmq[2] wich could match, but didn't test it.

[1] http://celeryproject.org/
[2] https://github.com/mfp/ocamlmq

-- 
Sébastien Dailly

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Gerd Stolpmann]

Am Dienstag, den 26.03.2013, 15:29 +0100 schrieb Philippe Veber:
> Dear all,
> 
> I'm developping an ocsigen website doing some scientific calculations.
> Up to now, the calculations were done in the same process that runs
> the server. In order to gain in scalability (and maybe stability too),
> I would like to run those calculations in a separate (pool of)
> process(es). As this is a pretty typical setup, I guess quite a few
> people have already done that. So I'd like to hear some suggestions on
> what library to use in this particular context. It seems to me that
> the release library [1] should do the job and is lwt-friendly, but
> there are maybe other good options?

Well, I don't know whether this is an option for Ocsigen users, but
Ocamlnet includes fairly good multiprocessing support. You can run
servers that dynamically start subprocesses on demand. Look for Netplex:

http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Intro.html#netplex

I've no good recipe, though, how to plug in service processors that base
on lwt (well, there is an adaptor in Ocamlnet for lwt - Uw_lwt - but I
wouldn't know what to do on the Ocsigen side, but maybe worth
exploring).

Ocamlnet also includes other mechanisms that are generally interesting
for compute stuff, namely Netmulticore for exploiting several cores on
the same machine with fast shared memory architecture, and RPC for
distributing computations in a network. Both are extensions of Netplex,
so it is easy to integrate into a single program.

Gerd

> Thanks for any hint, cheers!
> 
> Philippe.
> 
> [1] https://github.com/andrenth/release
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
*** Searching for new projects! Need consulting for system
*** programming in Ocaml? Gerd Stolpmann can help you.
------------------------------------------------------------

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Denis Berthod]

[-- Attachment #1: Type: text/plain, Size: 954 bytes --]

 Hello,

Isn't the Lwt_preemptive module exactly what you want?

http://ocsigen.org/lwt/api/Lwt_preemptive

But maybe, you want to use real processes and not threads.

Cheers,

Denis

Le 26 mars 2013 à 15:29, Philippe Veber a écrit :

> Dear all,
> 
> I'm developping an ocsigen website doing some scientific calculations. Up to now, the calculations were done in the same process that runs the server. In order to gain in scalability (and maybe stability too), I would like to run those calculations in a separate (pool of) process(es). As this is a pretty typical setup, I guess quite a few people have already done that. So I'd like to hear some suggestions on what library to use in this particular context. It seems to me that the release library [1] should do the job and is lwt-friendly, but there are maybe other good options?
> 
> Thanks for any hint, cheers!
> 
> Philippe.
> 
> [1] https://github.com/andrenth/release


[-- Attachment #2: Type: text/html, Size: 1444 bytes --]

AW: [Caml-list] Master-slave architecture behind an ocsigen server.

[Gerd Stolpmann]

Am 27.03.2013 23:42:14 schrieb(en) Denis Berthod:
>  Hello,
> 
> Isn't the Lwt_preemptive module exactly what you want?
> 
> http://ocsigen.org/lwt/api/Lwt_preemptive
> 
> But maybe, you want to use real processes and not threads.

In OCaml, only one thread may access runtime functions at a time, and  
because of this, only one thread at a time can run OCaml code. So you  
exploit only one core with a multi-threaded program. I guess this does  
not meet Philippe's scalability requirement, and he wants to use  
processes instead.

Gerd


> Cheers,
> 
> Denis
> 
> Le 26 mars 2013 à 15:29, Philippe Veber a écrit :
> 
> > Dear all,
> >
> > I'm developping an ocsigen website doing some scientific  
> calculations. Up to now, the calculations were done in the same  
> process that runs the server. In order to gain in scalability (and  
> maybe stability too), I would like to run those calculations in a  
> separate (pool of) process(es). As this is a pretty typical setup, I  
> guess quite a few people have already done that. So I'd like to hear  
> some suggestions on what library to use in this particular context.  
> It seems to me that the release library [1] should do the job and is  
> lwt-friendly, but there are maybe other good options?
> >
> > Thanks for any hint, cheers!
> >
> > Philippe.
> >
> > [1] https://github.com/andrenth/release
> 
> 
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 2687 bytes --]

Hi Martin,
nproc meets exactly my needs: a simple lwt-friendly interface to dispatch
function calls on a pool of processes that run on the same machine. I have
only one concern, that should probably be discussed on the ocsigen list,
that is I wonder if it is okay to fork the process running the ocsigen
server. I think I remember warnings on having parent and children processes
sharing connections/channels but it's really not clear to me.
Thanks a lot for your answer!
ph.

[Off-topic] PS I understand your remark on the use of the word slave, be
assured that I did not mean to hurt anyone's feelings. However I'm affraid
replacing slave by worker does not make the whole expression sound better,
as it implies calling a worker's boss 'master'. This is also a bit creepy
to me. Maybe it's enough to recall that we are talking about processes. Or
go for a more peaceful context, music, and call this a conductor/performer
architecture :o)?



2013/3/26 Martin Jambon <martin.jambon@ens-lyon.org>

> On 03/26/2013 07:29 AM, Philippe Veber wrote:
>
>> Dear all,
>>
>> I'm developping an ocsigen website doing some scientific calculations.
>> Up to now, the calculations were done in the same process that runs the
>> server. In order to gain in scalability (and maybe stability too), I
>> would like to run those calculations in a separate (pool of)
>> process(es). As this is a pretty typical setup, I guess quite a few
>> people have already done that. So I'd like to hear some suggestions on
>> what library to use in this particular context. It seems to me that the
>> release library [1] should do the job and is lwt-friendly, but there are
>> maybe other good options?
>>
>
> I wrote and used a library called Nproc about a year ago. It lets you
> create (Nproc.create) a pool of N processes, to which you can submit
> (Nproc.submit) computations of any type quasi-magically - just make sure
> any big environment required for the computation is not copied with each
> closure that you send to the workers. The submodule Nproc.Full provides a
> more advanced interface that lets each worker process have its own local
> environment.
>
>   https://github.com/MyLifeLabs/**nproc<https://github.com/MyLifeLabs/nproc>
>
> I haven't used Nproc in a while but it was working fine and should still
> work.
>
>
> Martin
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/**arc/caml-list<https://sympa.inria.fr/sympa/arc/caml-list>
> Beginner's list: http://groups.yahoo.com/group/**ocaml_beginners<http://groups.yahoo.com/group/ocaml_beginners>
> Bug reports: http://caml.inria.fr/bin/caml-**bugs<http://caml.inria.fr/bin/caml-bugs>
>

[-- Attachment #2: Type: text/html, Size: 3512 bytes --]

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 1716 bytes --]

Thanks Sébastien, a message broker layer could indeed simplify
communication between the server and the workers. I'll think of it if I
cannot use an higher-level API (dealing with a worker pool).


2013/3/27 Sébastien Dailly <sebastien-ocaml@chimrod.com>

> Le 26/03/2013 15:29, Philippe Veber a écrit :
>
>  Dear all,
>>
>> I'm developping an ocsigen website doing some scientific calculations.
>> Up to now, the calculations were done in the same process that runs the
>> server. In order to gain in scalability (and maybe stability too), I
>> would like to run those calculations in a separate (pool of)
>> process(es). As this is a pretty typical setup, I guess quite a few
>> people have already done that. So I'd like to hear some suggestions on
>> what library to use in this particular context. It seems to me that the
>> release library [1] should do the job and is lwt-friendly, but there are
>> maybe other good options?
>>
>>
> This make me think of the python celery project[1]. It rely on rabbitMQ
> for storing the task to execute and ensure persistance. The project just
> handle the message and give them to the registered worker.
>
> I found ocamlmq[2] wich could match, but didn't test it.
>
> [1] http://celeryproject.org/
> [2] https://github.com/mfp/ocamlmq
>
> --
> Sébastien Dailly
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/**arc/caml-list<https://sympa.inria.fr/sympa/arc/caml-list>
> Beginner's list: http://groups.yahoo.com/group/**ocaml_beginners<http://groups.yahoo.com/group/ocaml_beginners>
> Bug reports: http://caml.inria.fr/bin/caml-**bugs<http://caml.inria.fr/bin/caml-bugs>
>

[-- Attachment #2: Type: text/html, Size: 2550 bytes --]

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Alain Frisch]

On 03/28/2013 08:37 AM, Philippe Veber wrote:
> Hi Martin,
> nproc meets exactly my needs: a simple lwt-friendly interface to
> dispatch function calls on a pool of processes that run on the same
> machine. I have only one concern, that should probably be discussed on
> the ocsigen list, that is I wonder if it is okay to fork the process
> running the ocsigen server. I think I remember warnings on having parent
> and children processes sharing connections/channels but it's really not
> clear to me.

FWIW, LexiFi uses an architecture quite close to this for our 
application.  The main process manages the GUI and dispatches 
computations tasks to external processes.  Some points to be noted:

- Since this is a Windows application, we cannot rely on fork.  Instead, 
we restart the application (Sys.argv.(0)), with specific command-line 
flag, captured by the library in charge of managing computations.  This 
is done by calling a special function in this library; the function does 
nothing in the main process and in the sub-processes, it starts the 
special mode and never returns.  This gives a chance to the main 
application to do some global initialization common to the main and sub 
processes (for instance, we dynlink external plugins in this 
initialization phase).

- Computation functions are registered as global values.  Registration 
returns an opaque handle which can be used to call such a function.  We 
don't rely on marshaling closures.

- The GUI process actually spawns a single sub-process (the Scheduler), 
which itself manages more worker sub-sub-processes (with a maximal 
number of workers).  Currently, we don't do very clever scheduling based 
on task priorities, but this could easily be added.

- An external computation can spawn sub-computations (by applying a 
parallel "map" to a list) either synchronously (direct style) or 
asynchronously (by providing a continuation function, which will be 
applied to the list of results, maybe in a different process).  In both 
cases,  this is done by sending those tasks to the Scheduler.  The 
Scheduler dispatches computation tasks to available workers.  In the 
synchronous parallel map, the caller runs an inner event loop to 
communicate with the Scheduler (and it only accepts sub-tasks created by 
itself or one of its descendants).

- Top-level external computations can be stopped by the main process 
(e.g. on user request).  Concretely, this kills all workers currently 
working on that task or one of its sub-tasks.

- In addition to sending back the final results, computations can report 
progress to their caller and more intermediate results.  This is useful 
to show a progress bar/status and partial results in the GUI before the 
end of the entire computation.

- Communication between processes is done by exchanging marshaled 
"variants" (a tagged representation of OCaml values, generated 
automatically using our runtime types).  Since we can attach special 
variantizers/devariantizers to specific types, this gives a chance to 
customize how some values have to be exchanged between processes (e.g. 
values relying on internal hash-consing are treated specially to 
recreate the maximal sharing in the sub-process).

- Concretely, the communication between processes is done through queues 
of messages implemented with shared memory.  (This component was 
developed by Fabrice Le Fessant and OCamlPro.)   Large computation 
arguments or results (above a certain size) are stored on the file 
system, to avoid having to keep them in RAM for too long (if all workers 
are busy, the computation might wait for some time being started).

- The API supports easily distributing computation tasks to several 
machines.  We have done some experiments with using our application's 
database to dispatch computations, but we don't use it in production.





Alain

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 2945 bytes --]

Thanks for your input Gerd! As I understand it, your suggestion is to have
an RPC server (based on netplex) doing the actual calculations. That RPC
server would be called by the ocsigen server when needed (the ocsigen
server is the client in that scheme). So in that schema, only the RPC call
should be lwt-friendly. Digging in ocamlnet documentation, it seems that
could be achieved using [Rpc_simple_client.call] wrapped inside a
[Lwt_preemptive.detach].

This would be more complex than using nproc, but if it is effectively
dangerous/impossible to fork the ocsigen server process (as required to use
nproc), that seems a feasible solution for me --> I might ask a couple more
questions later then.

Danke sehr!
ph.



2013/3/27 Gerd Stolpmann <info@gerd-stolpmann.de>

> Am Dienstag, den 26.03.2013, 15:29 +0100 schrieb Philippe Veber:
> > Dear all,
> >
> > I'm developping an ocsigen website doing some scientific calculations.
> > Up to now, the calculations were done in the same process that runs
> > the server. In order to gain in scalability (and maybe stability too),
> > I would like to run those calculations in a separate (pool of)
> > process(es). As this is a pretty typical setup, I guess quite a few
> > people have already done that. So I'd like to hear some suggestions on
> > what library to use in this particular context. It seems to me that
> > the release library [1] should do the job and is lwt-friendly, but
> > there are maybe other good options?
>
> Well, I don't know whether this is an option for Ocsigen users, but
> Ocamlnet includes fairly good multiprocessing support. You can run
> servers that dynamically start subprocesses on demand. Look for Netplex:
>
>
> http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Intro.html#netplex
>
> I've no good recipe, though, how to plug in service processors that base
> on lwt (well, there is an adaptor in Ocamlnet for lwt - Uw_lwt - but I
> wouldn't know what to do on the Ocsigen side, but maybe worth
> exploring).
>
> Ocamlnet also includes other mechanisms that are generally interesting
> for compute stuff, namely Netmulticore for exploiting several cores on
> the same machine with fast shared memory architecture, and RPC for
> distributing computations in a network. Both are extensions of Netplex,
> so it is easy to integrate into a single program.
>
> Gerd
>
> > Thanks for any hint, cheers!
> >
> > Philippe.
> >
> > [1] https://github.com/andrenth/release
> >
>
> --
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
> Creator of GODI and camlcity.org.
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
> *** Searching for new projects! Need consulting for system
> *** programming in Ocaml? Gerd Stolpmann can help you.
> ------------------------------------------------------------
>
>

[-- Attachment #2: Type: text/html, Size: 4036 bytes --]

Re: AW: [Caml-list] Master-slave architecture behind an ocsigen server.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 3087 bytes --]

Hi Denis,

As Gerd kindly explained, using threads wouldn't help for scalability. It
turns out that it wouldn't help for stability either, because the workers
have to deal with R's (as in statistics softw'R) interpreter, and the
communication between OCaml and R is not bullet-proof right now. So I'd
rather have the computation isolated in its process, not to harm the
web-server.

Cheers,
  ph.



2013/3/28 Denis Berthod <casa.berthod@gmail.com>

>
>
>
> Le 27 mars 2013 à 23:49, Gerd Stolpmann a écrit :
>
> > Am 27.03.2013 23:42:14 schrieb(en) Denis Berthod:
> >> Hello,
> >> Isn't the Lwt_preemptive module exactly what you want?
> >> http://ocsigen.org/lwt/api/Lwt_preemptive
> >> But maybe, you want to use real processes and not threads.
> >
> > In OCaml, only one thread may access runtime functions at a time, and
> because of this, only one thread at a time can run OCaml code. So you
> exploit only one core with a multi-threaded program. I guess this does not
> meet Philippe's scalability requirement, and he wants to use processes
> instead.
> >
> > Gerd
> >
>
>
> Indead, I have only considered the stability problem that can be tackle by
> real threads as they don't block the lwt runtimes during computation.
> Howerver, you are right concerning the scalability.
>
> Cheers,
>
> Denis
>
>
>
> >
> >> Cheers,
> >> Denis
> >> Le 26 mars 2013 à 15:29, Philippe Veber a écrit :
> >> > Dear all,
> >> >
> >> > I'm developping an ocsigen website doing some scientific
> calculations. Up to now, the calculations were done in the same process
> that runs the server. In order to gain in scalability (and maybe stability
> too), I would like to run those calculations in a separate (pool of)
> process(es). As this is a pretty typical setup, I guess quite a few people
> have already done that. So I'd like to hear some suggestions on what
> library to use in this particular context. It seems to me that the release
> library [1] should do the job and is lwt-friendly, but there are maybe
> other good options?
> >> >
> >> > Thanks for any hint, cheers!
> >> >
> >> > Philippe.
> >> >
> >> > [1] https://github.com/andrenth/release
> >> --
> >> Caml-list mailing list.  Subscription management and archives:
> >> https://sympa.inria.fr/sympa/arc/caml-list
> >> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> >> Bug reports: http://caml.inria.fr/bin/caml-bugs
> >
> > --
> > ------------------------------------------------------------
> > Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
> > Creator of GODI and camlcity.org.
> > Contact details:        http://www.camlcity.org/contact.html
> > Company homepage:       http://www.gerd-stolpmann.de
> > ------------------------------------------------------------
> > --
> > Caml-list mailing list.  Subscription management and archives:
> > https://sympa.inria.fr/sympa/arc/caml-list
> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>

[-- Attachment #2: Type: text/html, Size: 4668 bytes --]

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 4561 bytes --]

Thanks Alain for this detailed description. I did not get why you do not
use marshalling for computation functions: this should be safe given the
same code is run in the GUI process and in the calculation sub-processes,
right?

Compared to your setting, I'm affraid I cannot use the same trick for
running the sub-processes, as the ocsigen server is in charge here. Maybe
there are some hooks that can help?

Also I like this ability to send partial results, this would be a nice
feature in my case. I'll have to think how to achieve this ...

Cheers
  ph.



2013/3/28 Alain Frisch <alain@frisch.fr>

> On 03/28/2013 08:37 AM, Philippe Veber wrote:
>
>> Hi Martin,
>> nproc meets exactly my needs: a simple lwt-friendly interface to
>> dispatch function calls on a pool of processes that run on the same
>> machine. I have only one concern, that should probably be discussed on
>> the ocsigen list, that is I wonder if it is okay to fork the process
>> running the ocsigen server. I think I remember warnings on having parent
>> and children processes sharing connections/channels but it's really not
>> clear to me.
>>
>
> FWIW, LexiFi uses an architecture quite close to this for our application.
>  The main process manages the GUI and dispatches computations tasks to
> external processes.  Some points to be noted:
>
> - Since this is a Windows application, we cannot rely on fork.  Instead,
> we restart the application (Sys.argv.(0)), with specific command-line flag,
> captured by the library in charge of managing computations.  This is done
> by calling a special function in this library; the function does nothing in
> the main process and in the sub-processes, it starts the special mode and
> never returns.  This gives a chance to the main application to do some
> global initialization common to the main and sub processes (for instance,
> we dynlink external plugins in this initialization phase).
>
> - Computation functions are registered as global values.  Registration
> returns an opaque handle which can be used to call such a function.  We
> don't rely on marshaling closures.
>
> - The GUI process actually spawns a single sub-process (the Scheduler),
> which itself manages more worker sub-sub-processes (with a maximal number
> of workers).  Currently, we don't do very clever scheduling based on task
> priorities, but this could easily be added.
>
> - An external computation can spawn sub-computations (by applying a
> parallel "map" to a list) either synchronously (direct style) or
> asynchronously (by providing a continuation function, which will be applied
> to the list of results, maybe in a different process).  In both cases,
>  this is done by sending those tasks to the Scheduler.  The Scheduler
> dispatches computation tasks to available workers.  In the synchronous
> parallel map, the caller runs an inner event loop to communicate with the
> Scheduler (and it only accepts sub-tasks created by itself or one of its
> descendants).
>
> - Top-level external computations can be stopped by the main process (e.g.
> on user request).  Concretely, this kills all workers currently working on
> that task or one of its sub-tasks.
>
> - In addition to sending back the final results, computations can report
> progress to their caller and more intermediate results.  This is useful to
> show a progress bar/status and partial results in the GUI before the end of
> the entire computation.
>
> - Communication between processes is done by exchanging marshaled
> "variants" (a tagged representation of OCaml values, generated
> automatically using our runtime types).  Since we can attach special
> variantizers/devariantizers to specific types, this gives a chance to
> customize how some values have to be exchanged between processes (e.g.
> values relying on internal hash-consing are treated specially to recreate
> the maximal sharing in the sub-process).
>
> - Concretely, the communication between processes is done through queues
> of messages implemented with shared memory.  (This component was developed
> by Fabrice Le Fessant and OCamlPro.)   Large computation arguments or
> results (above a certain size) are stored on the file system, to avoid
> having to keep them in RAM for too long (if all workers are busy, the
> computation might wait for some time being started).
>
> - The API supports easily distributing computation tasks to several
> machines.  We have done some experiments with using our application's
> database to dispatch computations, but we don't use it in production.
>
>
>
>
>
> Alain
>

[-- Attachment #2: Type: text/html, Size: 5193 bytes --]

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Alain Frisch]

On 03/28/2013 10:39 AM, Philippe Veber wrote:
> Thanks Alain for this detailed description. I did not get why you do not
> use marshalling for computation functions: this should be safe given the
> same code is run in the GUI process and in the calculation
> sub-processes, right?

I believe marshaling of functions did not always work nicely with 
dynlinked code.  More fundamentally, we cannot rely on the generic 
marshaling of data, for the reason I mentioned (some data types require 
special handling), and it's very difficult to know exactly what is 
captured by a closure.  In practice, the computation functions capture 
some values such as global values (global memoization hashtables, etc), 
and it would be wrong to marshal them.


Alain

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Anil Madhavapeddy]

On 28 Mar 2013, at 08:47, Alain Frisch <alain@frisch.fr> wrote:

> On 03/28/2013 08:37 AM, Philippe Veber wrote:
>> Hi Martin,
>> nproc meets exactly my needs: a simple lwt-friendly interface to
>> dispatch function calls on a pool of processes that run on the same
>> machine. I have only one concern, that should probably be discussed on
>> the ocsigen list, that is I wonder if it is okay to fork the process
>> running the ocsigen server. I think I remember warnings on having parent
>> and children processes sharing connections/channels but it's really not
>> clear to me.
> 
> FWIW, LexiFi uses an architecture quite close to this for our application.  The main process manages the GUI and dispatches computations tasks to external processes.  Some points to be noted:
> 
> - Since this is a Windows application, we cannot rely on fork.  Instead, we restart the application (Sys.argv.(0)), with specific command-line flag, captured by the library in charge of managing computations.  This is done by calling a special function in this library; the function does nothing in the main process and in the sub-processes, it starts the special mode and never returns.  This gives a chance to the main application to do some global initialization common to the main and sub processes (for instance, we dynlink external plugins in this initialization phase).
> 
> - Computation functions are registered as global values.  Registration returns an opaque handle which can be used to call such a function.  We don't rely on marshaling closures.
> 
> - The GUI process actually spawns a single sub-process (the Scheduler), which itself manages more worker sub-sub-processes (with a maximal number of workers).  Currently, we don't do very clever scheduling based on task priorities, but this could easily be added.
> 
> - An external computation can spawn sub-computations (by applying a parallel "map" to a list) either synchronously (direct style) or asynchronously (by providing a continuation function, which will be applied to the list of results, maybe in a different process).  In both cases,  this is done by sending those tasks to the Scheduler.  The Scheduler dispatches computation tasks to available workers.  In the synchronous parallel map, the caller runs an inner event loop to communicate with the Scheduler (and it only accepts sub-tasks created by itself or one of its descendants).
> 
> - Top-level external computations can be stopped by the main process (e.g. on user request).  Concretely, this kills all workers currently working on that task or one of its sub-tasks.
> 
> - In addition to sending back the final results, computations can report progress to their caller and more intermediate results.  This is useful to show a progress bar/status and partial results in the GUI before the end of the entire computation.
> 
> - Communication between processes is done by exchanging marshaled "variants" (a tagged representation of OCaml values, generated automatically using our runtime types).  Since we can attach special variantizers/devariantizers to specific types, this gives a chance to customize how some values have to be exchanged between processes (e.g. values relying on internal hash-consing are treated specially to recreate the maximal sharing in the sub-process).
> 
> - Concretely, the communication between processes is done through queues of messages implemented with shared memory.  (This component was developed by Fabrice Le Fessant and OCamlPro.)   Large computation arguments or results (above a certain size) are stored on the file system, to avoid having to keep them in RAM for too long (if all workers are busy, the computation might wait for some time being started).

Are all of the messages through these queues persistent, or just the larger ones that are too big to fit in the shared memory segment, and are they always point-to-point streams?

We've got a similar need in Xen/Mirage for shared memory communication and queues, and have been breaking them out into standalone libs such as:

https://github.com/djs55/shared-memory-ring

...which is ABI-compatible with the existing Xen shared memory interfaces, and also an OCaml version of the transport-agnostic API sketched out in:
http://anil.recoil.org/papers/2012-resolve-fable.pdf

The missing link currently is the persistent queuing service, but we're investigating the options here (ocamlmq looks rather nice).

-anil

Re: [Caml-list] Master-slave architecture behind an ocsigen server.

[Alain Frisch]

On 03/28/2013 12:02 PM, Anil Madhavapeddy wrote:
> Are all of the messages through these queues persistent, or just the larger ones that are too big to fit in the shared memory segment, and are they always point-to-point streams?

The messages always go through shared memory queues (non persistent), 
but their payload is offloaded to the file system (in temporary files) 
when it is too large.  There is no real persistence, though, because the 
temporary file is not self-describing (it is not sufficient to restart a 
computation after a process failure, for instance).  (The distribution 
of computations through a database is closer to real persistence.)

Queues are unidirectional point-to-point streams between two processes 
(we use a pair of such queues for between the main process and the 
scheduler, and between the scheduler and each worker process).

The relevant part of the API is:

========================================================
type t
       (** The type of point-to-point shared memory queues. *)

val create: ?max_size:int -> unit -> t
     (** A queue is created in one process with [create], passed by
         name (using [id]) to a single another process which can call
         [from_id]. *)

val from_id:  ?max_size:int -> string -> t
     (** Access a queue created by another process given its unique
         name. *)

val id: t -> string
     (** Globally (system-wide) unique name attached to the queue. *)

val close: t -> unit

exception CannotGrow
exception BrokenPipe

val send: t -> string -> unit
     (** Non-blocking operation.
         Raises [BrokenPipe] if the reader has disconnected.
         Raises [CannotGrow] if the maximum size of the buffer has been 
reached. *)
val read: t -> string option
     (** Non-blocking operation.
         Returns [None] if no message is available.
         Raises [BrokenPipe] if the writer has disconnected. *)
========================================================

If there is some interest for it, maybe Fabrice could release the code 
with an open source license?

One thing which is not supported by this library is a notification 
mechanism to inform the other side of the queue that messages are 
available.  For now, we simulate that by pinging the processes 
stdin/stdout descriptors.  In the scheduler and the worker, we use 
select to monitor them (there is probably a big cost of doing so, 
especially considering how select is emulated under Windows).  In the 
GUI process, we use standard .Net process monitoring facilities to 
inject callbacks in the main GUI thread.  (The OCaml side of our 
application in mono-threaded, and we use a few external native or .Net 
threads to monitor system conditions.)


Alain

AW: [Caml-list] Master-slave architecture behind an ocsigen server.

[Gerd Stolpmann]

Am 28.03.2013 12:02:46 schrieb(en) Anil Madhavapeddy:
> On 28 Mar 2013, at 08:47, Alain Frisch <alain@frisch.fr> wrote:
> 
> > On 03/28/2013 08:37 AM, Philippe Veber wrote:
> >> Hi Martin,
> >> nproc meets exactly my needs: a simple lwt-friendly interface to
> >> dispatch function calls on a pool of processes that run on the same
> >> machine. I have only one concern, that should probably be  
> discussed on
> >> the ocsigen list, that is I wonder if it is okay to fork the  
> process
> >> running the ocsigen server. I think I remember warnings on having  
> parent
> >> and children processes sharing connections/channels but it's  
> really not
> >> clear to me.
> >
> > FWIW, LexiFi uses an architecture quite close to this for our  
> application.  The main process manages the GUI and dispatches  
> computations tasks to external processes.  Some points to be noted:
> >
> > - Since this is a Windows application, we cannot rely on fork.   
> Instead, we restart the application (Sys.argv.(0)), with specific  
> command-line flag, captured by the library in charge of managing  
> computations.  This is done by calling a special function in this  
> library; the function does nothing in the main process and in the  
> sub-processes, it starts the special mode and never returns.  This  
> gives a chance to the main application to do some global  
> initialization common to the main and sub processes (for instance, we  
> dynlink external plugins in this initialization phase).
> >
> > - Computation functions are registered as global values.   
> Registration returns an opaque handle which can be used to call such  
> a function.  We don't rely on marshaling closures.
> >
> > - The GUI process actually spawns a single sub-process (the  
> Scheduler), which itself manages more worker sub-sub-processes (with  
> a maximal number of workers).  Currently, we don't do very clever  
> scheduling based on task priorities, but this could easily be added.
> >
> > - An external computation can spawn sub-computations (by applying a  
> parallel "map" to a list) either synchronously (direct style) or  
> asynchronously (by providing a continuation function, which will be  
> applied to the list of results, maybe in a different process).  In  
> both cases,  this is done by sending those tasks to the Scheduler.   
> The Scheduler dispatches computation tasks to available workers.  In  
> the synchronous parallel map, the caller runs an inner event loop to  
> communicate with the Scheduler (and it only accepts sub-tasks created  
> by itself or one of its descendants).
> >
> > - Top-level external computations can be stopped by the main  
> process (e.g. on user request).  Concretely, this kills all workers  
> currently working on that task or one of its sub-tasks.
> >
> > - In addition to sending back the final results, computations can  
> report progress to their caller and more intermediate results.  This  
> is useful to show a progress bar/status and partial results in the  
> GUI before the end of the entire computation.
> >
> > - Communication between processes is done by exchanging marshaled  
> "variants" (a tagged representation of OCaml values, generated  
> automatically using our runtime types).  Since we can attach special  
> variantizers/devariantizers to specific types, this gives a chance to  
> customize how some values have to be exchanged between processes  
> (e.g. values relying on internal hash-consing are treated specially  
> to recreate the maximal sharing in the sub-process).
> >
> > - Concretely, the communication between processes is done through  
> queues of messages implemented with shared memory.  (This component  
> was developed by Fabrice Le Fessant and OCamlPro.)   Large  
> computation arguments or results (above a certain size) are stored on  
> the file system, to avoid having to keep them in RAM for too long (if  
> all workers are busy, the computation might wait for some time being  
> started).
> 
> Are all of the messages through these queues persistent, or just the  
> larger ones that are too big to fit in the shared memory segment, and  
> are they always point-to-point streams?
> 
> We've got a similar need in Xen/Mirage for shared memory  
> communication and queues, and have been breaking them out into  
> standalone libs such as:
> 
> https://github.com/djs55/shared-memory-ring
> 
> ...which is ABI-compatible with the existing Xen shared memory  
> interfaces, and also an OCaml version of the transport-agnostic API  
> sketched out in:
> http://anil.recoil.org/papers/2012-resolve-fable.pdf

Interesting that there are now other shared memory implementations for  
OCaml. Note that there are a number of them in Ocamlnet, with some  
specialities not yet mentioned. There is the Netcamlbox library  
providing message boxes of limited size for exchanging OCaml values  
directly. That means the value is copied to the shared memory block by  
the sender, and the receiver can pick it up there without copying it  
again. Sender and receiver can map the memory at different addresses  
(the copy procedure invoked by the sender takes care of possible  
offsets, so that that Netcamlbox also allows the communication between  
processes that don't have a fork relation). There is no need for  
marshalling the value.

http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Netcamlbox.html

Going even beyond that, Netmulticore implements an "ancient" heap in  
shared memory (like Richard's Ancient lib, but with more options). This  
heap is organized like OCaml's major heap, and there is even a GC  
implementation for it. There are a number of data structures (arrays,  
hash tables, queues, buffers) which are aware of residing in shared  
memory. For synchronization there are mutexes, semaphores and condition  
variables. So far the values to manipulate are already in shared  
memory, programming with Netmulticore feels a lot like programming with  
multi-threading. In practice, however, you need to frequently copy  
values in and out, so it is not exactly as convenient. For  
Netmulticore, all processes must map the shared memory to the same  
address (easy with "fork").

http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Intro.html#netmulticore
http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Netmcore_tut.html

> The missing link currently is the persistent queuing service, but  
> we're investigating the options here (ocamlmq looks rather nice).

There is also Netamqp, which can be used together with RabbitMQ.

http://projects.camlcity.org/projects/netamqp.html

Gerd


> -anil
> 
> 
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

AW: [Caml-list] Master-slave architecture behind an ocsigen server.

[Gerd Stolpmann]

Am 28.03.2013 10:18:28 schrieb(en) Philippe Veber:
> Thanks for your input Gerd! As I understand it, your suggestion is to  
> have
> an RPC server (based on netplex) doing the actual calculations. That  
> RPC
> server would be called by the ocsigen server when needed (the ocsigen
> server is the client in that scheme). So in that schema, only the RPC  
> call
> should be lwt-friendly. Digging in ocamlnet documentation, it seems  
> that
> could be achieved using [Rpc_simple_client.call] wrapped inside a
> [Lwt_preemptive.detach].

I guess RPC would be most convenient here - it supports a server mode  
where the child processes accept the new connections (btw, if you don't  
want to deal with the RPC encoding stuff (i.e. XDR), just marshal the  
OCaml value as string, and use RPC functions that are declared as  
string->string).

A sample program would the "finder" service here:

http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/examples/rpc/finder

Gerd

> This would be more complex than using nproc, but if it is effectively
> dangerous/impossible to fork the ocsigen server process (as required  
> to use
> nproc), that seems a feasible solution for me --> I might ask a  
> couple more
> questions later then.
> 
> Danke sehr!
> ph.
> 
> 
> 
> 2013/3/27 Gerd Stolpmann <info@gerd-stolpmann.de>
> 
> > Am Dienstag, den 26.03.2013, 15:29 +0100 schrieb Philippe Veber:
> > > Dear all,
> > >
> > > I'm developping an ocsigen website doing some scientific  
> calculations.
> > > Up to now, the calculations were done in the same process that  
> runs
> > > the server. In order to gain in scalability (and maybe stability  
> too),
> > > I would like to run those calculations in a separate (pool of)
> > > process(es). As this is a pretty typical setup, I guess quite a  
> few
> > > people have already done that. So I'd like to hear some  
> suggestions on
> > > what library to use in this particular context. It seems to me  
> that
> > > the release library [1] should do the job and is lwt-friendly, but
> > > there are maybe other good options?
> >
> > Well, I don't know whether this is an option for Ocsigen users, but
> > Ocamlnet includes fairly good multiprocessing support. You can run
> > servers that dynamically start subprocesses on demand. Look for  
> Netplex:
> >
> >
> >  
> http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Intro.html#netplex
> >
> > I've no good recipe, though, how to plug in service processors that  
> base
> > on lwt (well, there is an adaptor in Ocamlnet for lwt - Uw_lwt -  
> but I
> > wouldn't know what to do on the Ocsigen side, but maybe worth
> > exploring).
> >
> > Ocamlnet also includes other mechanisms that are generally  
> interesting
> > for compute stuff, namely Netmulticore for exploiting several cores  
> on
> > the same machine with fast shared memory architecture, and RPC for
> > distributing computations in a network. Both are extensions of  
> Netplex,
> > so it is easy to integrate into a single program.
> >
> > Gerd
> >
> > > Thanks for any hint, cheers!
> > >
> > > Philippe.
> > >
> > > [1] https://github.com/andrenth/release
> > >
> >
> > --
> > ------------------------------------------------------------
> > Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
> > Creator of GODI and camlcity.org.
> > Contact details:        http://www.camlcity.org/contact.html
> > Company homepage:       http://www.gerd-stolpmann.de
> > *** Searching for new projects! Need consulting for system
> > *** programming in Ocaml? Gerd Stolpmann can help you.
> > ------------------------------------------------------------
> >
> >
> 
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------