From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id 207A2BC57 for ; Tue, 30 Nov 2010 15:09:29 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: At8BAA+U9EzU4xEIkGdsb2JhbACDUJEDjj0VAQEBAQkJDAcRAx+yVZEbAoEfgzNzBIcQhmE X-IronPort-AV: E=Sophos;i="4.59,280,1288566000"; d="scan'208";a="68855785" Received: from moutng.kundenserver.de ([212.227.17.8]) by mail3-smtp-sop.national.inria.fr with ESMTP; 30 Nov 2010 15:09:27 +0100 Received: from office1.lan.sumadev.de (dslb-094-219-222-142.pools.arcor-ip.net [94.219.222.142]) by mrelayeu.kundenserver.de (node=mreu0) with ESMTP (Nemesis) id 0MbcUz-1P4WSo3953-00J452; Tue, 30 Nov 2010 15:09:19 +0100 Received: from [192.168.1.111] (84-107-230-64.dsl.quicknet.nl [84.107.230.64]) by office1.lan.sumadev.de (Postfix) with ESMTPA id 630615F702; Tue, 30 Nov 2010 15:09:19 +0100 (CET) Subject: Re: Threading and SharedMem (Re: [Caml-list] Re: Is OCaml fast?) From: Gerd Stolpmann To: oliver@first.in-berlin.de Cc: caml-list@yquem.inria.fr In-Reply-To: <20101130125545.GE1637@siouxsie> References: <4CF4B17C.7000703@planet.nl> <20101130125545.GE1637@siouxsie> Content-Type: text/plain; charset="UTF-8" Date: Tue, 30 Nov 2010 15:09:17 +0100 Message-ID: <1291126157.16005.567.camel@thinkpad> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Provags-ID: V02:K0:5dkFL/58VbsMr615VBTxj/GI3fMeyCsncnG+6gaMXc5 tVQ9PX7l3xkVuaQiHYRs+FjQa8F3F/Z12+oDV+YDC8jjGdtFnD 5YF7hwMPAnreLrPAQG82Hhptq46IqhC+d99i9CcDGNNn8ZrvX4 daBCKd8QD0cgmjAIon8OkBK4RNRLII3mE9b1CNDKZw5Lc9QKmL EZMKz8Wd/pNpK3ihsgdaQ== X-Spam: no; 0.00; threading:01 ocaml:01 gerd:01 stolpmann:01 in-berlin:01 0100,:01 bandel:01 gerd:01 stolpmann:01 bandel:01 libs:01 ocaml:01 libs:01 bigarray:01 genarray:01 Am Dienstag, den 30.11.2010, 13:55 +0100 schrieb oliver@first.in-berlin.de: > On Tue, Nov 30, 2010 at 09:10:36AM +0100, Stephan Houben wrote: > > On 11/29/2010 04:33 PM, Oliver Bandel wrote: > > >Zitat von "Gerd Stolpmann" : > > > > > >>Am Montag, den 29.11.2010, 17:12 +0100 schrieb Oliver Bandel: > > >>>Zitat von "Gerd Stolpmann" : > > >>> > > > > >>>You use shared mem(?), but you link only to *.ml files, > > >>>and I see no *.c there. > > > > >>>How can this be done? > > >>> > > >>>At least not via the libs that are shipped with OCaml?! > > > > Actually it can be done using the libs that ship with OCaml > > (Unix and Bigarray), although it is not 100% POSIX : > > > > let create_shared_genarray kind layout dims = > > let fd = Unix.openfile "/dev/zero" [Unix.O_RDWR] 0 > > in let ar = Bigarray.Genarray.map_file fd kind layout true dims > > in Unix.close fd; ar > > > > > > The resulting bigarray object is shared among subsequent forks. > > Hmhhh... we started talking about Threads and SharedMem. > You mean even fork.... hmhhh Independent processes are right now the only way to use several cores. You can organize shared memory between processes, but it is tricky. That's what I try to ease with my Netmulticore library. > > This relies on the fact that mmap-ing /dev/zero is equivalent > > to an anonymous mmap. > > > > http://en.wikipedia.org/wiki//dev/zero > > > > Well, at least it works on Linux. > > In APUE it's mentioned that memory mapped regions are inherited > by a child, when forking it. So it should work on all Unix-systems too. Yes, but is not defined by POSIX what mapping /dev/zero means. > There is one problem with this... when you have forked, then > you obviously have separated processes and also in each process > your own ocaml-program with it's own GC running... > > ..with such a mem-mapping trick (never used Bigarray, so I'm astouned it uses > mmap) Bigarrays can use any memory with fixed addresses. That's the essence here: Bigarrays are not moved around by the GC. > you then have independent processes, working on shared mem without > synchronisation. > > This is a good possibility to get corrupted data, and therefore unreliable behaviour. > > So, you have somehow to create a way of communicating of these processes. So you need inter-process synchronization primitives, like POSIX semaphores. > This already is easily done in the Threads-module, because synchronisation > mechanisms are bound there to the OCaml API and can be used easily. > > In the Unix module there is not much of ths IPC stuff... But in Ocamlnet's netsys module. > (A thread-specific GC for thread-specific variables would help here, > making global locks only necessary when accessing global used variables. > But I don't know if such a way would be possible without changing the GC-stuff > itself.) The global lock does not protect user variables, but the Ocaml runtime, e.g. the state of the memory manager/garbage collector. Also it eases code generation - the memory image needs not be in a consistent state all the time (i.e. all pointers meaningful), but only when the runtime gets a hand on it. Removing this lock has far-reaching consequences. The oc4mc (ocaml for multicore) project used a separate minor heap per thread, which actually eases the task a lot - memory is in most cases allocated in the minor heap anyway. Many variables keeping the state of the runtime are then thread-local. Gerd > > > Ciao, > Oliver > > _______________________________________________ > Caml-list mailing list. Subscription management: > http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list > Archives: http://caml.inria.fr > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > -- ------------------------------------------------------------ Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ------------------------------------------------------------