From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail1-relais-roc.national.inria.fr (mail1-relais-roc.national.inria.fr [192.134.164.82]) by yquem.inria.fr (Postfix) with ESMTP id E9655BC58 for ; Tue, 14 Sep 2010 22:46:25 +0200 (CEST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AuoEAEJ8j0xQRFuwXmdsb2JhbAChYwsXFQY0xhiFQAQ X-IronPort-AV: E=Sophos;i="4.56,366,1280700000"; d="scan'208";a="67588997" Received: from furbychan.cocan.org ([80.68.91.176]) by mail1-smtp-roc.national.inria.fr with ESMTP; 14 Sep 2010 22:46:25 +0200 Received: from rich by furbychan.cocan.org with local (Exim 4.63) (envelope-from ) id 1OvcOW-0000PX-AL; Tue, 14 Sep 2010 21:46:24 +0100 Date: Tue, 14 Sep 2010 21:46:24 +0100 To: Yoann Padioleau Cc: caml-list@inria.fr Subject: Re: ancient module Message-ID: <20100914204624.GA1246@annexia.org> References: <7366F08F-88A4-40BA-95EE-1E682BEDBEFA@facebook.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <7366F08F-88A4-40BA-95EE-1E682BEDBEFA@facebook.com> User-Agent: Mutt/1.5.13 (2006-08-11) From: Richard Jones X-Spam: no; 0.00; iterating:01 arrays:01 hashtbl:01 hashtbl:01 ad-hoc:01 marshalling:01 hashing:01 hashing:01 ceb:01 ocaml:01 runtime:01 pointers:01 pointer:01 ocaml:01 hash:01 On Tue, Sep 14, 2010 at 08:19:49PM +0000, Yoann Padioleau wrote: > Hi, > > I am trying to use your Ancient module to avoid having the garbage > collector spends lots of time iterating over huge data in memory. It > works quite well for arrays but for hashtbl I have some problems > where I am not able to find back keys that were clearly in the > original hashtbl (before Ancient.mark it). > > In the doc it says:=20 >=20 > (1) Ad-hoc polymorphic primitives (structural equality, marshalling > and hashing) do not work on ancient data structures, meaning that you > will need to provide your own comparison and hashing functions. =20 The issue is described by Xavier Leroy: http://caml.inria.fr/pub/ml-archives/caml-list/2006/09/977818689f4ceb2178c5= 92453df7a343.en.html As far as my understanding goes, what happens is that the OCaml compare function (or some C equivalent in the runtime) looks at the two string pointers and decides that since both are out of the normal heap they are just opaque objects. Thus it won't compare the content of the strings, but will just do pointer equality. This massively breaks assumptions in some ordinary OCaml code, in this instance in Hashtbl. > which mean I have to transform my code using Hashtbl.xxx into one > using the functorized version of hashtbl ? I have hashtbl of strings > to complex data type. What would be a good hash function for > strings ? It may be that Map also has the same problems. You wouldn't really know except by examining the code. Later you wrote: > Actually it seems I have the problem only with Hashtbl from strings > to whatever. I also have some Hashtbl from int to whatever and they > work fine after the Ancient.mark. ints aren't compared in the same way. They are always compared using pointer equality, so there's no issue. I've only used ancient to store simple arrays, and when we needed to do string equality I remember writing a function which was aware of the above issue (you can compare them byte for byte just fine, even =66rom OCaml code). Rich. --=20 Richard Jones Red Hat