From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: ** X-Spam-Status: No, score=2.8 required=5.0 tests=DNS_FROM_RFC_POST,SPF_NEUTRAL autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by yquem.inria.fr (Postfix) with ESMTP id 39C5DBB84 for ; Sun, 13 Jul 2008 16:03:15 +0200 (CEST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApsEAGuoeUjAXQIn/2dsb2JhbACRapEahC0 X-IronPort-AV: E=Sophos;i="4.30,353,1212357600"; d="scan'208";a="13077396" Received: from concorde.inria.fr ([192.93.2.39]) by mail2-smtp-roc.national.inria.fr with ESMTP; 13 Jul 2008 16:03:08 +0200 Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by concorde.inria.fr (8.13.6/8.13.6) with ESMTP id m6DE38dU030348 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=OK) for ; Sun, 13 Jul 2008 16:03:08 +0200 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AowIAHuneUhKfS4eY2dsb2JhbACRajkaBB4DkBeELQ X-IronPort-AV: E=Sophos;i="4.30,353,1212357600"; d="scan'208";a="15072814" Received: from yw-out-2324.google.com ([74.125.46.30]) by mail3-smtp-sop.national.inria.fr with ESMTP; 13 Jul 2008 16:03:07 +0200 Received: by yw-out-2324.google.com with SMTP id 5so2063366ywh.27 for ; Sun, 13 Jul 2008 07:03:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=zuC6r1QuzzG79mrxjcUaCKTM4DxHViZgm50uPx+l5Mc=; b=oxRssRXaJWqJ7j3oe8In1ya/q5VhK3sxzuXhSo0OxrCOCypW1DhHijrySlLvOR906Q Q45rwgG9k3MbjzChGTwwBKrurcENKgPwchy7lqaBE0T/RGpo9UbhZ9cOd7yjI8JkKfGL iP5TH6NOzr2d8pvitqfH24k3Fm7n9Mt7NueUA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=XvLuftG6Pe+9uQalWydDWhe2eIjF/OhSbO/m2bVtu6aS/iUHPQGCXZF0ZzpGPTvQU7 YHyZyxl3UZ4f87Gp1NhtSc5fMi/R+0QfmzqySW2+qJ4SBs9UvnAbTvOSgH+HuGD71nUT bmYU4FjW0FqiN2m7LwPValQLZsq+GQjmdByV8= Received: by 10.150.49.1 with SMTP id w1mr18753002ybw.23.1215957786868; Sun, 13 Jul 2008 07:03:06 -0700 (PDT) Received: from ?10.0.0.183? ( [216.254.79.182]) by mx.google.com with ESMTPS id 30sm3976876yxk.4.2008.07.13.07.03.05 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 13 Jul 2008 07:03:05 -0700 (PDT) Cc: caml-list@inria.fr Message-Id: <3898DD39-12CD-4957-9165-CF437685E3F2@gmail.com> From: Thomas Crimi To: Sean Seefried In-Reply-To: <50710496-7007-4A9C-BFA4-DB98EEAD360E@nicta.com.au> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v926) Subject: Re: [Caml-list] Heaps size problems with "caml_alloc_small" in foreign function interfaces Date: Sun, 13 Jul 2008 10:03:04 -0400 References: <97BDCFD7-C714-440D-A04E-2C141E4AD3E0@nicta.com.au> <20080712085016.GA430@annexia.org> <50710496-7007-4A9C-BFA4-DB98EEAD360E@nicta.com.au> X-Mailer: Apple Mail (2.926) X-Miltered: at concorde with ID 487A0B1C.000 by Joe's j-chkmail (http://j-chkmail . ensmp . fr)! X-Spam: no; 0.00; alloc:01 pointers:01 ocaml:01 run-time:01 pointers:01 hashtable:01 ocaml:01 pointer:01 pointer:01 camlidl:01 tcrimi:98 garbage:01 wrote:01 integer:01 integer:01 On Jul 13, 2008, at 9:25 AM, Sean Seefried wrote: > > So now I have a question. Is there any way that I can find out what > address the garbage collector moves these values to? I need to > update the table when this happens. You have to register each of your pointers as global roots, but beware that doing so for a large number of objects can dramatically slow down the GC since all the roots must be scanned on each minor collection. The main issue here is that the OCaml run-time can't tell if you're modifying root pointers behind its back; pointing into the young generation, so each scan of the young generation needs to first check all root pointers. It should be possible to better optimize this by having the C code promise not to do this without telling the GC, but such functions currently don't exist. A solution I've used in the past (surely obtained from this list) is to keep the Hashtable in OCaml, and register some functions that the C code can use to access it. The C code can refer to objects long-term by integer ID which can be used for fast lookups. When you need the object, look it up by integer id to get the pointer. Then, of course, register the pointer as a global root. Once your done with the object pointer you release the global root again (or just keep a single always-registered global root around and re-point it to each object you need). This keeps the number of global roots to a minimum but still allows fast access to the pointers. Generally in my code I've created C++ wrapper classes to wrap a 'val' and provide C++-object-like access. It's very easy to have a base class that properly manages root allocation / release and then never have to think about it again. GC errors like this are insidious since your program can run for weeks without issue then one particular data-set causes a GC at the inopportune time and you have a mysterious crash. I'm not familiar with CamlIDL, so I'm not sure how amenable these tricks are. Regards, Tom