From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.2 required=5.0 tests=AWL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from discorde.inria.fr (discorde.inria.fr [192.93.2.38]) by yquem.inria.fr (Postfix) with ESMTP id 3D241BBE1 for ; Tue, 10 Oct 2006 14:38:41 +0200 (CEST) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by discorde.inria.fr (8.13.6/8.13.6) with ESMTP id k9ACcWUn029961 for ; Tue, 10 Oct 2006 14:38:41 +0200 Received: from [84.58.187.102] (helo=gate.lan.gerd-stolpmann.de) by mrelayeu.kundenserver.de (node=mrelayeu5) with ESMTP (Nemesis), id 0ML25U-1GXGsQ1TJj-0007hc; Tue, 10 Oct 2006 14:38:31 +0200 Received: from flakew.lan.gerd-stolpmann.de (fw.lan.gerd-stolpmann.de [192.168.1.1]) by gate.lan.gerd-stolpmann.de (Postfix) with ESMTP id 1179EC0AD; Tue, 10 Oct 2006 14:38:30 +0200 (CEST) Subject: Re: [Caml-list] forking, threads and events From: Gerd Stolpmann To: Erik de Castro Lopo Cc: caml-list@inria.fr In-Reply-To: <20061010221607.7ecc5cba.mle+ocaml@mega-nerd.com> References: <20061010221607.7ecc5cba.mle+ocaml@mega-nerd.com> Content-Type: text/plain Date: Tue, 10 Oct 2006 14:38:28 +0200 Message-Id: <1160483909.32409.35.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.6.1 Content-Transfer-Encoding: 7bit X-Provags-ID: kundenserver.de abuse@kundenserver.de login:a6865a839c0178d9aa0ce41878507ea2 X-Miltered: at discorde with ID 452B9448.001 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; threads:01 gerd:01 stolpmann:01 compiler:01 unix:01 threads:01 threading:01 ocamlnet:01 ocamlnet-:01 ocamlnet:01 gerd:01 stolpmann:01 viktoriastr:01 64293:01 darmstadt:01 Am Dienstag, den 10.10.2006, 22:16 +1000 schrieb Erik de Castro Lopo: > Hi all, > > I'm thinking about to start a new project which has some rather > critical requirements. It will be compiled with the native compiler > initially targeting Linux/Unix but eventually also windoze. > > The app consists of a main engine which spawns many short lived > child threads or processes. The children go away, do their work and > then pass their results back to the main engine. Many of the children > will spawn another process and read the child process output via a > pipe and many of the children will block on I/O. A small portion of > the children may be I/O bound, but there is not way of telling which > beforehand. > > Since I would like to maximize the throughput on multi-core and > multi-processor machines I am thinking of using a mix of forking and > threading. You know that there is no fork on Windows? > For communications, I was thinking of using the Event > module for communication between threads, but I don't think that > works for forked process (pipes maybe?). > > Anybody have any advice for this project? Any war stories from similar > projects? Any readings they can recommend? If there wasn't the Windows requirement I could recommend this: You may have a look at ocamlnet2. It includes the netplex library that manages parent/children relationships between either forked processes or threads. You can use SunRPC for communication (or whatever you like, but SunRPC support is included). The bad news is that it does not run on the native Windows port, not even in the threaded variant (because there's no socketpair...). Ah yes, I have a war story, but cannot tell it now. I can only say it is used for a really big commercial project. Download it here: http://www.ocaml-programming.de/packages/ocamlnet-2.2test13.tar.gz An online manual is here: http://ocamlnet.sourceforge.net/manual-2.2/ Gerd -- ------------------------------------------------------------ Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ------------------------------------------------------------