From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 0BEDA7ED25 for ; Fri, 19 Jul 2013 16:35:39 +0200 (CEST) X-IronPort-AV: E=Sophos;i="4.89,702,1367964000"; d="scan'208";a="26643568" Received: from dhcp-rocq-253.inria.fr (HELO [128.93.62.253]) ([128.93.62.253]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-CAMELLIA256-SHA; 19 Jul 2013 16:35:38 +0200 Message-ID: <51E94EBA.609@inria.fr> Date: Fri, 19 Jul 2013 16:35:38 +0200 From: Romain Bardou User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130116 Icedove/10.0.12 MIME-Version: 1.0 To: caml-list@inria.fr References: <51E9398A.9010402@inria.fr> <51E94B76.6070207@etorok.net> In-Reply-To: <51E94B76.6070207@etorok.net> X-Enigmail-Version: 1.4.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Validation-by: romain.bardou@inria.fr Subject: Re: [Caml-list] Request for feedback: Procord, a library to delegate tasks to other processes Le 19/07/2013 16:21, Török Edwin a écrit : > On 07/19/2013 04:05 PM, Romain Bardou wrote: >> Hello, >> >> I plan on writing yet another library to help with concurrency in OCaml. >> The motivations for this library, and the interface I have in mind, are >> available here: >> >> http://romain.bardou.fr/procord/api/Procord.html >> >> Before actually implementing the library, I would be very happy to >> receive feedback. I am interested to know, among others: >> - whether I miss important information which would make the very >> existence of this library stupid (such as, it already exists); >> - whether I forgot some important use case; > > Processing streams of data in parallel without having to read all the data in memory first. Good point. I'm not sure how to deal with this and I actually think it calls for a slightly different paradigm. Although maybe one could eventually add send/receive functions to send/receive to/from an 'a process, and the same for the worker. It would allow to transmit more than one input and more than one output. > You could sort of do this with your current interface but only on Unix (i.e. mkfifo and pass filename). > Don't know what'd work on Windows, perhaps creating a (named) pipe, and sending the file descriptor to a newly spawned child? I don't think there are named pipes on Windows but I may be wrong. Maybe it can be emulated using files anyway. I don't think sending a file descriptor is a good idea but sending a file name is of course possible. > Also it'd be nice to have something similar to the reduce in map-reduce. That's definitely something I want to be able to add easily. It won't be in the initial version, as I have no need for it right now, but it is important for me that it can easily be added, maybe as a new layer on top of Procord itself. >> - whether the names I chose have better ubiquitous equivalents; >> - whether you believe I should choose another interface entirely, for >> instance, if you don't like functors. > > I'd prefer if there was also a Lwt-like monadic interface, but it is not absolutely required (it could probably be done on top of your already existing interface). That is also something I want to be able to add easily, but I don't want Procord to depend on anything and to force one library. The user should be able to choose Async or Lwt or whatever. This is why I propose get_file_descriptors and process_status. It should be easy to write a Lwt using that. It would be possible to provide separate modules Procord_lwt and Procord_async as interfaces though. Thanks for your inputs! Best regards, -- Romain Bardou