From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=5.0 tests=AWL,DNS_FROM_RFC_POST, HTML_MESSAGE,SPF_NEUTRAL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail1-relais-roc.national.inria.fr (mail1-relais-roc.national.inria.fr [192.134.164.82]) by yquem.inria.fr (Postfix) with ESMTP id D8C16BBB7 for ; Tue, 17 Feb 2009 11:50:44 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AtIBALIlmknRVdrYmWdsb2JhbACCQDGRFT8BAQEBAQgLCgcRrQqPGgEDAQOEEAaENA X-IronPort-AV: E=Sophos;i="4.38,222,1233529200"; d="scan'208";a="24195062" Received: from mail-bw0-f216.google.com ([209.85.218.216]) by mail1-smtp-roc.national.inria.fr with ESMTP; 17 Feb 2009 11:50:44 +0100 Received: by bwz12 with SMTP id 12so8338674bwz.9 for ; Tue, 17 Feb 2009 02:50:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=uKZJ34liCxNjZrjKvnVw+oX0cTD5xT8wvTsTa/nqaqg=; b=hQLf6nuYKh+wkMuMi2he/8WgY6MPqmV1mEp1Rw0MQJbexjXIvDZYWJ8RnxCyQ5c85k 4rDIpd1dkUBnvcZGNPxg97IgxmeBS0LrZ/T1zl4z9mGCVgLia2XjIOg64NP/dX9nWo1c vJ08ruA7SD9Q1Tlv2bgChB1py32LAQfNFILIQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=M0mEr5v0bufaNY424AMniOYaxTFPJfuTrK6ImSbWQtiUDL5+vbo0VECh1vcN90nlSm wassGVoonhtwyQOrpGTscXYB5aQqjUoH+lED/t0xBGDmFdI+gPjZ2yvAaj5BN5nE8sNJ Qx2HDkzX+NdXfA/ZWXB5eTyqdfLhybWSUsuJM= MIME-Version: 1.0 Sender: remidewitte@gmail.com Received: by 10.223.127.8 with SMTP id e8mr5006170fas.95.1234867844110; Tue, 17 Feb 2009 02:50:44 -0800 (PST) In-Reply-To: <20090217102659.GD29651@janestcapital.com> References: <2184b2340902160715y1f935b5ehc0e6195b3f75b66b@mail.gmail.com> <891bd3390902160847p25ad3bf1pe59da620dfc667f2@mail.gmail.com> <2184b2340902160937i53b8f3fbga01eaf14ed829f8f@mail.gmail.com> <2184b2340902162340s540c5ac7g9f42b59d03f643cb@mail.gmail.com> <20090217102659.GD29651@janestcapital.com> Date: Tue, 17 Feb 2009 11:50:44 +0100 X-Google-Sender-Auth: 85c03105f0b6f9cb Message-ID: <2184b2340902170250g75d4482bt1d49e0e295c2e6d7@mail.gmail.com> Subject: Re: [Caml-list] Re: Threads performance issue. From: =?UTF-8?Q?R=C3=A9mi_Dewitte?= To: Mark Shinwell Cc: caml-list@inria.fr Content-Type: multipart/alternative; boundary=001636c5b6da3157da04631b13db X-Spam: no; 0.00; bigarray:01 shinwell:01 buffer:01 hackery:01 runtime:01 bigarray:01 shinwell:01 buffer:01 hackery:01 runtime:01 2009:98 26,:98 2009:98 deceiving:98 26,:98 X-Attachments: cset="UTF-8" cset="UTF-8" --001636c5b6da3157da04631b13db Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello ! Many thanks all for your answers ! Managing to have the almost same performance whether in mutithreaded environment or not (even not using threads for this particular task) is something I would like to have anyway. I'll give a try to big buffers using Using.read. Any example code around ? And then why not try iao ! Memory mapping of the file could be done using BigArray or do I have to write C code ? R=C3=A9mi On Tue, Feb 17, 2009 at 11:26, Mark Shinwell w= rote: > On Tue, Feb 17, 2009 at 10:07:05AM +0000, Sylvain Le Gall wrote: > > On 17-02-2009, R=C3=A9mi Dewitte wrote: > > You are using input_char and standard IO channel. This is a good choice > > for non-threaded program. But in your case, I will use Unix.read with a > > big buffer (32KB to 4MB) and change your program to use it. As > > benchmarked by John Harrop, you are spending most of your time in > > caml_enter|leave_blocking section. > > This isn't quite right actually -- the profile is deceiving. It is true > that there are a lot of calls to enter/leave_blocking_section, but you're > actually being killed by the overhead of an independent locking strategy > in the channel-based I/O calls. I've measured this using some hackery > with a hex editor. When you call input_char, you acquire and then releas= e > another lock which is specific to these calls (the global runtime lock is > often not released here). This process isn't especially cheap, so it wou= ld > be better to use one of the other channel calls to read data in larger > blocks. > > Mark > --001636c5b6da3157da04631b13db Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello !

Many thanks all for your answers !

Managing to have t= he almost same performance whether in mutithreaded environment or not (even= not using threads for this particular task) is something I would like to h= ave anyway.

I'll give a try to big buffers using Using.read. Any example code a= round ? And then why not try iao !

Memory mapping of the file could = be done using BigArray or do I have to write C code ?

R=C3=A9mi
<= br>
On Tue, Feb 17, 2009 at 11:26, Mark Shinwell <mshinwell= @janestcapital.com> wrote:
On Tue, Feb 17, 2009 at 10:07:05AM +0000, Sylvain Le = Gall wrote:
> On 17-02-2009, R=C3=A9mi Dewitte <= remi@gide.net> wrote:
> You are using input_char and standard IO c= hannel. This is a good choice
> for non-threaded program. But in your case, I will use Unix.read with = a
> big buffer (32KB to 4MB) and change your program to use it. As
> benchmarked by John Harrop, you are spending most of your time in
> caml_enter|leave_blocking section.

This isn't quite right actually -- the profile is deceiving. &nbs= p;It is true
that there are a lot of calls to enter/leave_blocking_section, but you'= re
actually being killed by the overhead of an independent locking strategy in the channel-based I/O calls.  I've measured this using some hac= kery
with a hex editor.  When you call input_char, you acquire and then rel= ease
another lock which is specific to these calls (the global runtime lock is often not released here).  This process isn't especially cheap, so= it would
be better to use one of the other channel calls to read data in larger bloc= ks.

Mark

--001636c5b6da3157da04631b13db--