From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Delivered-To: caml-list@yquem.inria.fr Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by yquem.inria.fr (Postfix) with ESMTP id 38CCBBCAB for ; Wed, 25 May 2005 12:30:53 +0200 (CEST) Received: from pauillac.inria.fr (pauillac.inria.fr [128.93.11.35]) by nez-perce.inria.fr (8.13.0/8.13.0) with ESMTP id j4PAUqca025087 for ; Wed, 25 May 2005 12:30:52 +0200 Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id MAA20706 for ; Wed, 25 May 2005 12:30:52 +0200 (MET DST) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.189]) by concorde.inria.fr (8.13.0/8.13.0) with ESMTP id j4PAUphN016471 for ; Wed, 25 May 2005 12:30:52 +0200 Received: from [212.227.126.160] (helo=mrelayng.kundenserver.de) by moutng.kundenserver.de with esmtp (Exim 3.35 #1) id 1Dat49-0004SD-00; Wed, 25 May 2005 12:24:45 +0200 Received: from [84.167.130.127] (helo=gate.lan.gerd-stolpmann.de) by mrelayng.kundenserver.de with asmtp (Exim 3.35 #1) id 1Dat49-0007Hg-00; Wed, 25 May 2005 12:24:45 +0200 Received: from flakew.lan.gerd-stolpmann.de (flakew.lan.gerd-stolpmann.de [192.168.0.32]) by gate.lan.gerd-stolpmann.de (Postfix) with ESMTP id 7CD1DC10A; Wed, 25 May 2005 12:24:45 +0200 (CEST) Subject: Re: [Caml-list] Re: Efficient I/O with threads From: Gerd Stolpmann To: yminsky@cs.cornell.edu Cc: Caml Mailing List In-Reply-To: <891bd339050524184221de71b8@mail.gmail.com> References: <891bd33905052415145bb1818f@mail.gmail.com> <891bd339050524184221de71b8@mail.gmail.com> Content-Type: text/plain Date: Wed, 25 May 2005 12:24:45 +0200 Message-Id: <1117016685.6326.44.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.0.2 Content-Transfer-Encoding: 7bit X-Provags-ID: kundenserver.de abuse@kundenserver.de auth:a6865a839c0178d9aa0ce41878507ea2 X-Miltered: at nez-perce with ID 429453DC.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Miltered: at concorde with ID 429453DC.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; caml-list:01 threads:01 gerd:01 stolpmann:01 yaron:01 minsky:01 buffering:01 ocaml:01 buffering:01 reimplement:01 buffer:01 ocaml:01 runtime:01 ocamlnet:01 netchannels:01 X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on yquem.inria.fr X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.0.2 X-Spam-Level: Am Dienstag, den 24.05.2005, 21:42 -0400 schrieb Yaron Minsky: > An addendum. One thing that was pointed out to me in some private > emails was that buffering could solve the problem on the reading side > as well. That is true, as far as it goes --- that's why I said that > I can't think of a _clean_ way of handling it. One of the nice things > about ocaml IO channels is that they handle buffering, and it seems a > shame to have to reimplement buffering on top of them. > > Put another way, the problem with input/output channels appears to be > that the buffering is done on the wrong side of the lock. You > shouldn't have to do any locking to do IO when the request can be > satisfied from the buffer. The fact that IO channels always require > you to acquire the lock means that the performance is crappy unless > you bundle up writes by yourself. > > Fixing this is perhaps too deep of a change to drive into the OCaml > system at this point. Is this a problem that is addressed by the I/O > channels provided by any other library such as extlib? I just looked into the sources of the OCaml runtime. The additional work to lock/unlock the I/O channels is very, very small, just a pthread_mutex_lock and a pthread_mutex_unlock for every operation. What counts more is the general overhead for the multi-threading machinery. For every blocking system call a lot of additional overhead is necessary. As an alternative, you can try the object channels of Ocamlnet. With let in_ch = Netchannels.lift_in (`Raw (new Netchannels.input_descr in_fd)) and let out_ch = Netchannels.lift_out (`Raw (new Netchannels.output_descr out_fd)) you get object channels over the file descriptors in_fd, out_fd that implement buffers by O'Caml code and work much like the built-in channels. These channels aren't protected against concurrent usage, and may be more light-weight because of this. Gerd -- ------------------------------------------------------------ Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de ------------------------------------------------------------