From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by yquem.inria.fr (Postfix) with ESMTP id 76EFCBCD9 for ; Tue, 8 Dec 2009 09:45:34 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AusBAH9XHUvRVdi2kWdsb2JhbACQIYQAhnM/AQEBAQkLCgcTA65EMYRrI4hHAQIDBYQuBIsp X-IronPort-AV: E=Sophos;i="4.47,358,1257116400"; d="scan'208";a="38185150" Received: from mail-px0-f182.google.com ([209.85.216.182]) by mail2-smtp-roc.national.inria.fr with ESMTP; 08 Dec 2009 04:30:52 +0100 Received: by pxi12 with SMTP id 12so2617637pxi.3 for ; Mon, 07 Dec 2009 19:30:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=woFYR/pj9gi+wLJ4/nbU3OUn1ZKrfJHKaK4N1RDx0IU=; b=kP5DmtjFMY+0KQIkFR+Kqs51HVHCdf9Fc/lbJYb9wdizyznHbe4vv0AXW5BZPXN35V rPhDP+7nO91JdF99qytP4X8op/oDzU7zuFM8Li9fYz108MSugYrW5BcbrTFmlIe2Y/xE szIVczIrPWKjiOtkr5vsQcfPdVukNgbCip9dA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=Sk4KoKkRIh+5GAZECN6Kr0r/pNk23jOgzoP+iSp2rvKe9wb+8hdxrk+6YFaAnahwns D4fsR0JA2Da3n4wnFLHDRyApd4pJtEPHv+g15G6aMFXpM88BQyj93YdVdLNFeHG772oS 3fq9+QOSk2hlWvmFAgP7aYFTYy2kNQAtDyzrY= MIME-Version: 1.0 Sender: aaron678@gmail.com Received: by 10.142.9.16 with SMTP id 16mr835530wfi.92.1260243050870; Mon, 07 Dec 2009 19:30:50 -0800 (PST) In-Reply-To: <95594484-7A9F-4ECA-B7A8-079D433BF4FB@inria.fr> References: <95594484-7A9F-4ECA-B7A8-079D433BF4FB@inria.fr> Date: Mon, 7 Dec 2009 22:30:50 -0500 X-Google-Sender-Auth: c5c2f6568060b252 Message-ID: Subject: Re: [Caml-list] Gc.compact surprisingly helpful From: Aaron Bohannon To: Damien Doligez Cc: OCaml List Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam: no; 0.00; damien:01 damien:01 compactions:01 compaction:01 compactions:01 2009:98 2009:98 doligez:01 doligez:01 wrote:01 wrote:01 upenn:01 heap:01 heap:01 caml-list:01 On Sat, Dec 5, 2009 at 12:06 PM, Damien Doligez w= rote: > > On 2009-12-04, at 20:09, Aaron Bohannon wrote: > >> So in a 15 second run (with no >> idle time, as I said), it now does about 130 heap compactions instead >> of 3 and gets better total performance because of it, utterly defying >> my GC intuition. > > What is the size of your heap? =A0Have you tried compacting only once > every 2, 3, 5, or 10 loops? In a 15 sec run, the total allocated memory is about 8GB (from Gc.allocated_bytes()). The max heap size reached using one compaction per loop is about 7MB (on a 64-bit machine, based on the "top_heap_words" field). The max heap size reached with no compactions at all is only about 10MB. These numbers may seem wierd, but they are really not too surprising since the program is a signal-processing pipeline that does a huge number of (purely functional) Array.map and Array.init operations to build short-lived intermediate results, and it does the same operatations in each (sub-)cycle regardless of the input. Yeah, I can squeeze out a bit more performance by using calling Gc.compact slightly less often, as suggested, but since we're only talking about around 5%, it doesn't make much difference. In theory, I could also optimize away some allocation by replacing functional Array ops with imperative Array updates, but the throughput seems good enough right now---I'm actually just concerned about how to keep the the maximum GC pause time as low as possible. - Aaron