From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id E4E167EE7A for ; Thu, 28 Mar 2013 13:19:12 +0100 (CET) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of info@gerd-stolpmann.de) identity=pra; client-ip=212.227.17.10; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="info@gerd-stolpmann.de"; x-sender="info@gerd-stolpmann.de"; x-conformance=sidf_compatible Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of info@gerd-stolpmann.de) identity=mailfrom; client-ip=212.227.17.10; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="info@gerd-stolpmann.de"; x-sender="info@gerd-stolpmann.de"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of postmaster@moutng.kundenserver.de designates 212.227.17.10 as permitted sender) identity=helo; client-ip=212.227.17.10; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="info@gerd-stolpmann.de"; x-sender="postmaster@moutng.kundenserver.de"; x-conformance=sidf_compatible; x-record-type="v=spf1" X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArYAAFs0VFHU4xEKk2dsb2JhbABDgzqsWZILgQMWDgEBAQEHCwsJFAMlgh8BAQMBATo0BAcFKAYNXQkEAQ0GEwkJh3ADCQoItQMDiWUViQGESoESJgeDQAOOGYltknSBaQ X-IPAS-Result: ArYAAFs0VFHU4xEKk2dsb2JhbABDgzqsWZILgQMWDgEBAQEHCwsJFAMlgh8BAQMBATo0BAcFKAYNXQkEAQ0GEwkJh3ADCQoItQMDiWUViQGESoESJgeDQAOOGYltknSBaQ X-IronPort-AV: E=Sophos;i="4.84,925,1355094000"; d="scan'208";a="9045619" Received: from moutng.kundenserver.de ([212.227.17.10]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 28 Mar 2013 13:19:12 +0100 Received: from office1.lan.sumadev.de (dslb-094-219-220-119.pools.arcor-ip.net [94.219.220.119]) by mrelayeu.kundenserver.de (node=mreu4) with ESMTP (Nemesis) id 0LbCQQ-1V63CA2DdK-00koDW; Thu, 28 Mar 2013 13:19:00 +0100 Received: from samsung (ip-5-146-55-186.unitymediagroup.de [5.146.55.186]) by office1.lan.sumadev.de (Postfix) with ESMTPSA id 25BBBC00D0; Thu, 28 Mar 2013 13:19:00 +0100 (CET) Date: Thu, 28 Mar 2013 13:18:59 +0100 From: Gerd Stolpmann To: Anil Madhavapeddy Cc: Alain Frisch , "cl-mirage@lists.cam.ac.uk List" , Philippe Veber , Martin Jambon , caml users In-Reply-To: (from anil@recoil.org on Thu Mar 28 12:02:46 2013) X-Mailer: Balsa 2.4.11 Message-Id: <1364473139.14693.2@samsung> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; DelSp=Yes; Format=Flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable X-Provags-ID: V02:K0:PaO70jxg63qLsi2nqv8Gcz5My+Z5IJSZfcmvOgB1na8 KCMozN6R3/pB9nVqiCjbbbEMfm3iMTSRTDl4FVdR2OA+GQxIoZ C7syaXoHvc5cM8TO4ARaURaLkJCYz8AbgIHUju8SUcg6I0Gh7R 94njrTESDlSHJVV/72RXnzVLNGhjU4gZEoWRmlUzeqni5zaNRF s3CBKm6mw9sZSo9Csl2jzHaHY9CZP+cPjGLrvStKyQNQBS0D2F nr8aVg2EzmQnL/CS5+8sMXOlzhDlWK/9QpH8YnDkHC3xJad0k3 0akLz0IJsvC4dKA7QOXd63/l0cEVSa3sxGjzCfVwgY/5E0bQwM O7iLTO5MIjq9FJuLJGBc= Subject: AW: [Caml-list] Master-slave architecture behind an ocsigen server. Am 28.03.2013 12:02:46 schrieb(en) Anil Madhavapeddy: > On 28 Mar 2013, at 08:47, Alain Frisch wrote: >=20 > > On 03/28/2013 08:37 AM, Philippe Veber wrote: > >> Hi Martin, > >> nproc meets exactly my needs: a simple lwt-friendly interface to > >> dispatch function calls on a pool of processes that run on the same > >> machine. I have only one concern, that should probably be=20=20 > discussed on > >> the ocsigen list, that is I wonder if it is okay to fork the=20=20 > process > >> running the ocsigen server. I think I remember warnings on having=20= =20 > parent > >> and children processes sharing connections/channels but it's=20=20 > really not > >> clear to me. > > > > FWIW, LexiFi uses an architecture quite close to this for our=20=20 > application. The main process manages the GUI and dispatches=20=20 > computations tasks to external processes. Some points to be noted: > > > > - Since this is a Windows application, we cannot rely on fork.=20=20=20 > Instead, we restart the application (Sys.argv.(0)), with specific=20=20 > command-line flag, captured by the library in charge of managing=20=20 > computations. This is done by calling a special function in this=20=20 > library; the function does nothing in the main process and in the=20=20 > sub-processes, it starts the special mode and never returns. This=20=20 > gives a chance to the main application to do some global=20=20 > initialization common to the main and sub processes (for instance, we=20= =20 > dynlink external plugins in this initialization phase). > > > > - Computation functions are registered as global values.=20=20=20 > Registration returns an opaque handle which can be used to call such=20= =20 > a function. We don't rely on marshaling closures. > > > > - The GUI process actually spawns a single sub-process (the=20=20 > Scheduler), which itself manages more worker sub-sub-processes (with=20= =20 > a maximal number of workers). Currently, we don't do very clever=20=20 > scheduling based on task priorities, but this could easily be added. > > > > - An external computation can spawn sub-computations (by applying a=20= =20 > parallel "map" to a list) either synchronously (direct style) or=20=20 > asynchronously (by providing a continuation function, which will be=20=20 > applied to the list of results, maybe in a different process). In=20=20 > both cases, this is done by sending those tasks to the Scheduler.=20=20= =20 > The Scheduler dispatches computation tasks to available workers. In=20= =20 > the synchronous parallel map, the caller runs an inner event loop to=20= =20 > communicate with the Scheduler (and it only accepts sub-tasks created=20= =20 > by itself or one of its descendants). > > > > - Top-level external computations can be stopped by the main=20=20 > process (e.g. on user request). Concretely, this kills all workers=20=20 > currently working on that task or one of its sub-tasks. > > > > - In addition to sending back the final results, computations can=20=20 > report progress to their caller and more intermediate results. This=20= =20 > is useful to show a progress bar/status and partial results in the=20=20 > GUI before the end of the entire computation. > > > > - Communication between processes is done by exchanging marshaled=20=20 > "variants" (a tagged representation of OCaml values, generated=20=20 > automatically using our runtime types). Since we can attach special=20= =20 > variantizers/devariantizers to specific types, this gives a chance to=20= =20 > customize how some values have to be exchanged between processes=20=20 > (e.g. values relying on internal hash-consing are treated specially=20=20 > to recreate the maximal sharing in the sub-process). > > > > - Concretely, the communication between processes is done through=20=20 > queues of messages implemented with shared memory. (This component=20=20 > was developed by Fabrice Le Fessant and OCamlPro.) Large=20=20 > computation arguments or results (above a certain size) are stored on=20= =20 > the file system, to avoid having to keep them in RAM for too long (if=20= =20 > all workers are busy, the computation might wait for some time being=20= =20 > started). >=20 > Are all of the messages through these queues persistent, or just the=20= =20 > larger ones that are too big to fit in the shared memory segment, and=20= =20 > are they always point-to-point streams? >=20 > We've got a similar need in Xen/Mirage for shared memory=20=20 > communication and queues, and have been breaking them out into=20=20 > standalone libs such as: >=20 > https://github.com/djs55/shared-memory-ring >=20 > ...which is ABI-compatible with the existing Xen shared memory=20=20 > interfaces, and also an OCaml version of the transport-agnostic API=20=20 > sketched out in: > http://anil.recoil.org/papers/2012-resolve-fable.pdf Interesting that there are now other shared memory implementations for=20= =20 OCaml. Note that there are a number of them in Ocamlnet, with some=20=20 specialities not yet mentioned. There is the Netcamlbox library=20=20 providing message boxes of limited size for exchanging OCaml values=20=20 directly. That means the value is copied to the shared memory block by=20= =20 the sender, and the receiver can pick it up there without copying it=20=20 again. Sender and receiver can map the memory at different addresses=20=20 (the copy procedure invoked by the sender takes care of possible=20=20 offsets, so that that Netcamlbox also allows the communication between=20= =20 processes that don't have a fork relation). There is no need for=20=20 marshalling the value. http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Netca= mlbox.html Going even beyond that, Netmulticore implements an "ancient" heap in=20=20 shared memory (like Richard's Ancient lib, but with more options). This=20= =20 heap is organized like OCaml's major heap, and there is even a GC=20=20 implementation for it. There are a number of data structures (arrays,=20=20 hash tables, queues, buffers) which are aware of residing in shared=20=20 memory. For synchronization there are mutexes, semaphores and condition=20= =20 variables. So far the values to manipulate are already in shared=20=20 memory, programming with Netmulticore feels a lot like programming with=20= =20 multi-threading. In practice, however, you need to frequently copy=20=20 values in and out, so it is not exactly as convenient. For=20=20 Netmulticore, all processes must map the shared memory to the same=20=20 address (easy with "fork"). http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Intro= .html#netmulticore http://projects.camlcity.org/projects/dl/ocamlnet-3.6.3/doc/html-main/Netmc= ore_tut.html > The missing link currently is the persistent queuing service, but=20=20 > we're investigating the options here (ocamlmq looks rather nice). There is also Netamqp, which can be used together with RabbitMQ. http://projects.camlcity.org/projects/netamqp.html Gerd > -anil >=20 >=20 > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs >=20 --=20 ------------------------------------------------------------ Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de Creator of GODI and camlcity.org. Contact details: http://www.camlcity.org/contact.html Company homepage: http://www.gerd-stolpmann.de ------------------------------------------------------------=