From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: * X-Spam-Status: No, score=1.2 required=5.0 tests=AWL,HTML_10_20,HTML_MESSAGE, SPF_NEUTRAL,UNPARSEABLE_RELAY autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by yquem.inria.fr (Postfix) with ESMTP id 345DCBB84; Sun, 13 Jul 2008 15:26:18 +0200 (CEST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlwCAIufeUiBXvI7gWdsb2JhbACCco87AQEQIEqUCQ X-IronPort-AV: E=Sophos;i="4.30,353,1212357600"; d="scan'208,217";a="27282943" Received: from concorde.inria.fr ([192.93.2.39]) by mail4-smtp-sop.national.inria.fr with ESMTP; 13 Jul 2008 15:26:18 +0200 Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by concorde.inria.fr (8.13.6/8.13.6) with ESMTP id m6DDQGd3029631 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=OK); Sun, 13 Jul 2008 15:26:17 +0200 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlwCAIufeUiBXvI7gWdsb2JhbACCco87AQEQIEqUCQ X-IronPort-AV: E=Sophos;i="4.30,353,1212357600"; d="scan'208,217";a="27282933" Received: from tone.orchestra.cse.unsw.edu.au ([129.94.242.59]) by mail4-smtp-sop.national.inria.fr with ESMTP; 13 Jul 2008 15:25:42 +0200 Received: From [IPv6???1] ([129.94.242.49] == weill.orchestra.cse.unsw.EDU.AU) (ident-user sseefried) (cse-authentic-sender sseefried) By tone With Smtp ; Sun, 13 Jul 2008 23:25:39 +1000 From: Sean Seefried To: Xavier Leroy Date: Sun, 13 Jul 2008 23:25:38 +1000 Cc: caml-list@inria.fr Message-Id: <50710496-7007-4A9C-BFA4-DB98EEAD360E@nicta.com.au> In-Reply-To: <20080712085016.GA430@annexia.org> Content-Type: multipart/alternative; boundary=Apple-Mail-22--722345891 Mime-Version: 1.0 (Apple Message framework v926) Subject: Re: [Caml-list] Heaps size problems with "caml_alloc_small" in foreign function interfaces References: <97BDCFD7-C714-440D-A04E-2C141E4AD3E0@nicta.com.au> <20080712085016.GA430@annexia.org> X-Mailer: Apple Mail (2.926) X-Miltered: at concorde with ID 487A0278.000 by Joe's j-chkmail (http://j-chkmail . ensmp . fr)! X-Spam: no; 0.00; alloc:01 bug:01 pointer:01 camlidl:01 translating:01 ocaml:01 translating:01 structs:01 prev:01 pointers:01 recursive:01 hashtable:01 pointer:01 ocaml:01 hash:01 --Apple-Mail-22--722345891 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit After a weekend's worth of ruminating I have finally found the bug in my code. It didn't turn out to be a root registration problem but your pointer to this possible cause led me to read a lot of interesting things about the garbage collector which, indirectly, helped me find the problem. Xavier, I ran into a problem with CamlIDL when translating C data structures to OCaml ones. The problem was that I was translating a really large data structure with a lot of sharing. Most structs had next, prev, and parent pointers which meant that many, many recursive calls were made. This eventually blew the stack. So what I decided to do was create a hashtable. Whenever I encountered a C pointer type I first allocated memory for the OCaml value filled it's fields with dummy values and then associated the address of the C structure (the key) with the address of the OCaml value (the value). This was great because whenever I came across the same C pointer again I would just look up the value and use that. However, I failed to think about garbage collection! Naturally, whenever garbage collection is done the OCaml values are moved in memory, but the addresses of the OCaml data structures in my hash table stay the same. This is no good since those very same addresses become available again to be allocated by camlidl_alloc_small. This can lead to all sorts of unpleasantness! So now I have a question. Is there any way that I can find out what address the garbage collector moves these values to? I need to update the table when this happens. Sean --Apple-Mail-22--722345891 Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: quoted-printable
After a weekend's = worth of ruminating I have finally found the bug in my code. It didn't = turn out to be a root registration problem but your pointer to this = possible cause led me to read a lot of interesting things about the = garbage collector which, indirectly, helped me find the = problem.

Xavier, I ran into a problem with = CamlIDL when translating C data structures to OCaml ones. The problem = was that I was translating a really large data structure with a lot of = sharing. Most structs had next, prev, and parent pointers which meant = that many, many recursive calls were made. This eventually blew the = stack.

So what I decided to do was create a = hashtable.  Whenever I encountered a C pointer type I first = allocated memory for the OCaml value filled it's fields with dummy = values and then associated the address of the C structure (the key) with = the address of the OCaml value (the value).  This was great because = whenever I came across the same C pointer again I would just look up the = value and use that.

However, I failed to = think about garbage = collection! 

Naturally, whenever = garbage collection is done the OCaml values are moved in memory, but the = addresses of the OCaml data structures in my hash table stay the same. =  This is no good since those very same addresses become available = again to be allocated by camlidl_alloc_small. This can lead to all sorts = of unpleasantness! 

So now I have a = question. Is there any way that I can find out what address the garbage = collector moves these values to? I need to update the table when this = happens.

Sean
= --Apple-Mail-22--722345891--