From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by yquem.inria.fr (Postfix) with ESMTP id BB89FBC57 for ; Sun, 15 Aug 2010 12:45:49 +0200 (CEST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApsCANRhZ0zRVdQ0kGdsb2JhbACTOo0JCBUBAQEBCQkMBxEDH55GiRCCEoUfLohUAQEDBYU2BIli X-IronPort-AV: E=Sophos;i="4.55,371,1278280800"; d="scan'208";a="67625754" Received: from mail-vw0-f52.google.com ([209.85.212.52]) by mail4-smtp-sop.national.inria.fr with ESMTP; 15 Aug 2010 12:45:49 +0200 Received: by vws14 with SMTP id 14so2925720vws.39 for ; Sun, 15 Aug 2010 03:45:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=TDEZ1q4kqLKgpMPq7OjOatV505F+W4W2oXXAAYvi710=; b=Tjdbcv+qyuQNO32xZaJZcUCJPPJxsFHa2ZQTFGuzJVjEoF4OZDOjrKH2TDqXneIxy8 JJEJFl3qhpbEl5yPIsYHgGrUMACf/8Y7vPyxDryQQEFv+dYLz/3r8xOBPzfZflSeIBKV JTngLE0RzXL/HxGrWhrR0Kjh9Oip5oFRxYn6M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=vN6JD3DZXy/f/hNaeMULPnoRcKHV2ni7B84JP1nl+4xQHHuTrSSkkFPmjLHgK6UV6i VMvulFZzNWySdNDkVzgMAHn0JFzHRLwMsxXqrY5ZfsFVHPxGa5cYjmfV1HR5SNElk9hV tf8SHwV9nc2lNHl8migz9LN+ZfS8/LLlwet/E= MIME-Version: 1.0 Received: by 10.220.93.17 with SMTP id t17mr2230529vcm.266.1281869148055; Sun, 15 Aug 2010 03:45:48 -0700 (PDT) Received: by 10.220.179.200 with HTTP; Sun, 15 Aug 2010 03:45:47 -0700 (PDT) In-Reply-To: <1281863810.2115.11.camel@glinka> References: <1281855821.2115.4.camel@glinka> <1281863810.2115.11.camel@glinka> Date: Sun, 15 Aug 2010 12:45:47 +0200 Message-ID: Subject: Re: [Caml-list] More re GC hanging From: Adrien To: basile@starynkevitch.net Cc: Paul Steckler , caml-list@yquem.inria.fr Content-Type: text/plain; charset=ISO-8859-1 X-Spam: no; 0.00; iirc:01 memoized:01 basile:01 bindings:01 bindings:01 'print:01 endline:01 zsh:01 git:98 garbage:01 garbage:01 caml-list:01 functions:01 functions:01 fragments:01 Hi, I recently had similar output from the GC (huge count of words) which I noticed after my program started to exit with an out-of-memory error. It doesn't seem to be happening anymore but I'm not sure I "fixed" it. There are three things I thought of to get rid of it. (btw, I'm on 64bit linux) First, remove all non-tail-rec functions: no more List.map, @ or List.concat. All lists were pretty short (definitely less than 1000 elements) but maybe the amount of calls generated garbage or something like that: I couldn't get much infos about the problem so I tried what I could think of and being tail-rec couldn't be a bad thing anyway. The idea was to create less values since I first suspected a memory fragmentation issue (iirc I had thousands of fragments), so I also memoized some functions. Then, as Basile mentionned, call something like Gc.compact () regularly. The overhead is actually pretty small, especially if ran regularly. Finally, C bindings. I created a few while not having access to the internet and they are quite dirty. I highly doubt they play perfectly well with the garbage collector: they seem ok but probably aren't perfect. That's definitely something to check, even if the bindings were written by someone else because working nicely with the GC can be quite hard. Now, the problem seems to be gone but I can't say for sure. One really annoying thing was that adding a line like 'print_endline "pouet";' would make the out-of-memory problem go away! Same when getting stats from the GC. As for the problem with randomize_va_space on 64bit, I thought it had been fixed in 3.11 so I haven't looked at it (and in the absence of internet access, I was unable to get details for that problem). I just tried about a hundred run with VA-space-randomization on and without Gc.compact calls and ran without problem. Hopefully everything is tracked in git so I can get the older and non-working code if needed. Also, on my computer, I have the following behaviour: 11:44 ~ % sudo echo 0 > /proc/sys/kernel/randomize_va_space zsh: permission denied: /proc/sys/kernel/randomize_va_space root@jarjar:~# echo 0 > /proc/sys/kernel/randomize_va_space root@jarjar:~# I can't use sudo to write to most files in /proc or /sys: I have to log in as root ('su -' does the job just fine). Hope this helps. --- Adrien Nader