From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id 23705BC57 for ; Mon, 20 Sep 2010 20:53:08 +0200 (CEST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlYBAKZKl0zU4xEJkGdsb2JhbACDHJ5/FQEBAQEJCQwHEQMfsGKSBQKBIIMscwSGcoZE X-IronPort-AV: E=Sophos;i="4.56,395,1280700000"; d="scan'208";a="57590146" Received: from moutng.kundenserver.de ([212.227.17.9]) by mail3-smtp-sop.national.inria.fr with ESMTP; 20 Sep 2010 20:53:07 +0200 Received: from office1.lan.sumadev.de (dslb-188-097-001-161.pools.arcor-ip.net [188.97.1.161]) by mrelayeu.kundenserver.de (node=mreu1) with ESMTP (Nemesis) id 0LyyMm-1OkTnr057g-014gs1; Mon, 20 Sep 2010 20:52:55 +0200 Received: from [192.168.0.29] (dslb-084-058-057-237.pools.arcor-ip.net [84.58.57.237]) by office1.lan.sumadev.de (Postfix) with ESMTPA id 6E8215F702; Mon, 20 Sep 2010 20:52:54 +0200 (CEST) Subject: Re: [Caml-list] Re: ancient module From: Gerd Stolpmann To: Richard Jones Cc: Yoann Padioleau , caml-list@inria.fr In-Reply-To: <20100914204624.GA1246@annexia.org> References: <7366F08F-88A4-40BA-95EE-1E682BEDBEFA@facebook.com> <20100914204624.GA1246@annexia.org> Content-Type: text/plain; charset="UTF-8" Date: Mon, 20 Sep 2010 20:52:52 +0200 Message-ID: <1285008772.20388.12.camel@thinkpad> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Provags-ID: V02:K0:zlZXSyOXvJanMTdRWJFdrtm9U2WAUrz0GBs+F6LuI8O q+jyB3H0qkeMsYtI2aRsJcPs3j2Iui/5xaoIzuUNqB+71zphyh 0fJnJt5pc9OQfMXUKQqWbethWgrodS8SfxC4bO6mKowkrMOsSb nyvPCz2o8jR3Un1ZQBlFUtNvbRf11wCabQuWqIV1uVyo6rt6zl rzgX3v/arUkh4tHtSrkQg== X-Spam: no; 0.00; gerd:01 stolpmann:01 iterating:01 arrays:01 hashtbl:01 hashtbl:01 ad-hoc:01 marshalling:01 hashing:01 hashing:01 ceb:01 ocaml:01 runtime:01 pointers:01 pointer:01 Am Dienstag, den 14.09.2010, 21:46 +0100 schrieb Richard Jones: > On Tue, Sep 14, 2010 at 08:19:49PM +0000, Yoann Padioleau wrote: > > Hi, > > > > I am trying to use your Ancient module to avoid having the garbage > > collector spends lots of time iterating over huge data in memory. It > > works quite well for arrays but for hashtbl I have some problems > > where I am not able to find back keys that were clearly in the > > original hashtbl (before Ancient.mark it). > > > > In the doc it says: > > > > (1) Ad-hoc polymorphic primitives (structural equality, marshalling > > and hashing) do not work on ancient data structures, meaning that you > > will need to provide your own comparison and hashing functions. > > The issue is described by Xavier Leroy: > http://caml.inria.fr/pub/ml-archives/caml-list/2006/09/977818689f4ceb2178c592453df7a343.en.html > > As far as my understanding goes, what happens is that the OCaml > compare function (or some C equivalent in the runtime) looks at the > two string pointers and decides that since both are out of the normal > heap they are just opaque objects. Thus it won't compare the content > of the strings, but will just do pointer equality. This massively > breaks assumptions in some ordinary OCaml code, in this instance in > Hashtbl. There is now a way to change this. You can call caml_page_table_add (since 3.11) to explicitly declare a memory region as containing Ocaml values. The polymorphic comparison, the hash primitive, and marshalling work then. There is support for this in Ocamlnet-3: http://projects.camlcity.org/projects/dl/ocamlnet-3.0.3/doc/html-main/Netsys_mem.html#VALvalue_area Gerd > > > which mean I have to transform my code using Hashtbl.xxx into one > > using the functorized version of hashtbl ? I have hashtbl of strings > > to complex data type. What would be a good hash function for > > strings ? > > It may be that Map also has the same problems. You wouldn't really > know except by examining the code. > > Later you wrote: > > Actually it seems I have the problem only with Hashtbl from strings > > to whatever. I also have some Hashtbl from int to whatever and they > > work fine after the Ancient.mark. > > ints aren't compared in the same way. They are always compared using > pointer equality, so there's no issue. > > I've only used ancient to store simple arrays, and when we needed to > do string equality I remember writing a function which was aware of > the above issue (you can compare them byte for byte just fine, even > from OCaml code). > > Rich. > -- ------------------------------------------------------------ Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ------------------------------------------------------------