From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by yquem.inria.fr (Postfix) with ESMTP id BEA13BC68; Tue, 26 Sep 2006 20:57:07 +0200 (CEST) Received: from furbychan.cocan.org (furbychan.cocan.org [80.68.91.176]) by concorde.inria.fr (8.13.6/8.13.6) with ESMTP id k8QIv5bp020339 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Tue, 26 Sep 2006 20:57:07 +0200 Received: from rich by furbychan.cocan.org with local (Exim 3.35 #1 (Debian)) id 1GSI6V-0000p6-00; Tue, 26 Sep 2006 19:56:27 +0100 Date: Tue, 26 Sep 2006 19:56:27 +0100 To: Markus Mottl Cc: Damien Doligez , quant@janestcapital.com, caml users Subject: Re: [Caml-list] Regarding SMP computing Message-ID: <20060926185626.GA8522@furbychan.cocan.org> References: <4517C7FE.8080102@mcmaster.ca> <20060925194153.GA20467@furbychan.cocan.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.9i From: Richard Jones X-Miltered: at concorde with ID 45197801.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Spam: no; 0.00; markus:01 mottl:01 damien:01 damien:01 ocaml:01 rework:01 runtime:01 pointers:01 ocaml-heap:01 pointers:01 ocaml-heap:01 pointer:01 ocaml-value:01 compaction:01 ocaml-values:01 On Tue, Sep 26, 2006 at 10:37:27AM -0400, Markus Mottl wrote: > On 9/26/06, Damien Doligez wrote: > >The GC doesn't work by checking whether something has become > >unreachable. It works by marking all reachable data, then deallocating > >everything else. There is no way to do what you want in OCaml, and > >I don't think it can be done without a major rework of the runtime > >system. > > But isn't it true that the GC doesn't follow pointers that point > outside the OCaml-heap? In that case it might be conceivable to copy > OCaml-data that must not be reclaimed into the C-heap. Of course, > this would mean that pointers must not point back into the OCaml-heap > from there, because there is no way the GC could know then that some > value in the OCaml-heap is still in use, or how to update the pointer > in the C-heap in case the OCaml-value gets moved around, e.g. during a > compaction. > > If the above really works, I'd be glad to know whether there is > already functionality to copy OCaml-structures around. We have some > applications that would greatly benefit from this feature, too: they > require an enormous amount of static background knowledge, and have to > use it for small jobs which can be easily distributed. We have > multi-core, multi-processor machines, and would be able to greatly > speed up our compute jobs if we could put these large OCaml-values > into a shared-memory region. It would save us both a hell lot of > memory (probably in the range of GBs per machine for some jobs), and > also make the GC work less hard marking data that is not going to be > reclaimed anyway. Really similar case to us! In theory you'd have a "MARK_ANCIENT (obj)" operation. It would copy obj and everything pointed to by obj out of the OCaml heap into malloc'd memory. It would need to find everything pointing to obj and update those pointers to the out-of-heap copy, or else have a proxy in place of obj. All done in C so that the garbage collector wouldn't be running while this happened, and in Markus's case you'd want to move out to some sort of shared memory, perhaps using something like mmalloc[1]. The problem with all this is that such an "ancient" object had really better not be changed. If it was changed such that part of it pointed to a new object allocated on the Caml heap, then that new object could never be reached by the GC, and so would quickly get collected, resulting in a dangling pointer. You'd have to ensure that the ancient object was never changed by careful programming ... Rich. [1] http://www.math.utah.edu/docs/info/mmalloc_1.html -- Richard Jones, CTO Merjis Ltd. Merjis - web marketing and technology - http://merjis.com Internet Marketing and AdWords courses - http://merjis.com/courses - NEW! Merjis blog - http://blog.merjis.com - NEW!