From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=AWL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by yquem.inria.fr (Postfix) with ESMTP id 228F7BC37 for ; Wed, 29 Apr 2009 16:48:09 +0200 (CEST) X-IronPort-AV: E=Sophos;i="4.40,266,1238968800"; d="scan'208";a="25368883" Received: from macabane.inria.fr ([128.93.8.160]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/AES128-SHA; 29 Apr 2009 16:48:08 +0200 Cc: Brighten Godfrey Message-Id: <0FDD87FB-2461-4BBC-BDB1-5316A5AE4D23@inria.fr> From: Damien Doligez To: OCaml List In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Subject: Re: [Caml-list] Strange performance bug Date: Wed, 29 Apr 2009 16:48:08 +0200 References: <324B24CA-9671-42C0-B722-C7710C0C45C7@cs.berkeley.edu> <59BD1B3A-B449-4963-9910-ED5E755D00E6@cs.berkeley.edu> <49F7F135.5080504@gmail.com> <08C6A0FC-90F2-43A4-AB78-4A3B68291FAA@cs.berkeley.edu> <49F7F59B.7070204@frisch.fr> <6D9C5A68-1874-4BBC-AE3D-9CCC3614AF7C@cs.berkeley.edu> X-Mailer: Apple Mail (2.930.3) X-Spam: no; 0.00; damien:01 damien:01 bug:01 markus:01 mottl:01 vastly:01 pcre:01 markus:01 alain's:01 pcre:01 2009:98 steadily:98 steadily:98 doligez:01 doligez:01 On 2009-04-29, at 15:58, Markus Mottl wrote: > Note that the effect of not precompiling the regular expressions is > not just the overhead of this computation, but also vastly greater > GC-pressure. > > The current GC-settings in Pcre will trigger a full GC-cycle every 500 > regular expressions allocated, i.e. would perform a full major > collection every 500 lines in your case. This setting works fine for > just about any application I've seen, because virtually nobody has to > create patterns dynamically at rates so high that this matters. Markus, you put your finger right on the problem. That program doesn't suddenly start to get slow, it gets steadily slower as it runs. The heap also gets steadily bigger, and the major GC does way too much work. Alain's explanation (about why changing the last line changes the speed of the rest of the program) looks right to me. When you remove that last line, the GC still does too much work, but the heap doesn't grow, so it doesn't get slower. To see the problem, you should play with the GC verbosity settings, for example, see the difference between OCAMLRUNPARAM=v=1 ./problem slow and OCAMLRUNPARAM=v=1 ./problem fast Maybe PCRE should change its settings to trigger GCs less often but, as Markus said, this doesn't look really important. -- Damien