From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.9 required=5.0 tests=AWL,SPF_FAIL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id 8C5B6BB84 for ; Mon, 16 Feb 2009 17:33:12 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlECAIQkmUlQW+UCe2dsb2JhbACUYQEBFiIEvCOEHAY X-IronPort-AV: E=Sophos;i="4.38,217,1233529200"; d="scan'208";a="23037335" Received: from main.gmane.org (HELO ciao.gmane.org) ([80.91.229.2]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/AES256-SHA; 16 Feb 2009 17:33:11 +0100 Received: from list by ciao.gmane.org with local (Exim 4.43) id 1LZ6P5-0007DT-Bq for caml-list@inria.fr; Mon, 16 Feb 2009 16:33:07 +0000 Received: from ks300734.kimsufi.com ([91.121.65.225]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 16 Feb 2009 16:33:07 +0000 Received: from sylvain by ks300734.kimsufi.com with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 16 Feb 2009 16:33:07 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: caml-list@inria.fr From: Sylvain Le Gall Subject: Re: Threads performance issue. Date: Mon, 16 Feb 2009 16:32:58 +0000 (UTC) Message-ID: References: <2184b2340902160715y1f935b5ehc0e6195b3f75b66b@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: ks300734.kimsufi.com User-Agent: slrn/pre0.9.9-102 (Linux) Sender: news X-Spam: no; 0.00; le-gall:01 ocaml:01 pcre:01 ocaml:01 pointers:01 buffer:01 buffer:01 low-level:01 threads:01 threads:01 wrote:01 unix:01 unix:01 remi:01 inefficient:02 Hello, On 16-02-2009, Rémi Dewitte wrote: > --===============0282778124== > Content-Type: multipart/alternative; boundary=00504502b0791d7c5b04630aa761 > > --00504502b0791d7c5b04630aa761 > Content-Type: text/plain; charset=UTF-8 > Content-Transfer-Encoding: quoted-printable > > Hello, > > I would like to read two files in two different threads. > > I have made a first version reading the first then the second and it takes > 2.8s (native). > > I decided to make a threaded version and before any use of thread I realize= > d > that just linking no even using it to the threads library makes my first > version of the program to run in 12s ! > There is a small function call to handle thread (caml_(enter|leave)_blocking_section). I don't know how much it cost in term of performance but I am under the impression that it cost more time than you win. These function calls can be found in many files all around the OCaml source distribution... > I use pcre, extlib, csv libraries as well. > Some of this library can have a high cost for thread synchronisation on global variable. You need to investigate. > I guess it might come from GC slowing down thinks here, doesn't it ? Where > can it come from otherwise ? Is there a workaround or something I should > know ? Maybe... You need to look at external library and to benchmark your own code... This is not an easy task. > > Can ocaml use multiple cores ? > As advertised in the OCaml documentation: http://caml.inria.fr/pub/docs/manual-ocaml/manual038.html The threads library is implemented by time-sharing on a single processor. It will not take advantage of multi-processor machines. Using this library will therefore never make programs run faster. However, many programs are easier to write when structured as several communicating processes. One of the point is that the GC doesn't take advantage of multiple-core. Current GC that support this feature are slower than OCaml single-threaded GC... > Do you have few pointers on libraries to make parallel I/Os ? > Since you are running a fairly recent Linux kernel, I recommend you: https://forge.ocamlcore.org/projects/libaio-ocaml/ which should allow you to use AIO (asynchronous IO in the kernel, see "man aio_read"). Now on a more "ask-yourself" tone: I have tried using thread to speed up IO on multiple core (in C code). It is really tricky to get something that really work faster. In fact, for reading you don't get performance at all when using threaded IO. I am still asking myself why. I think it as something todo with the fact that when you generate too much read request, the OS begin to do inefficient I/O seek all around (almost no effect on Linux, timex4 on Windows). As a matter of fact (for now), using non-threaded Unix.read with 16k buffer and threaded Unix.write with 4M buffer is the most efficient I/O scheme. All in all, I think you should not try to use thread to improve your software performance in OCaml - or rely on low-level asynchronous IO (aio). Regards, Sylvain Le Gall