Re: [Caml-list] Re: multicore wish

[Gerd Stolpmann <gerd@gerd-stolpmann.de>]

Am Sonntag, den 27.12.2009, 13:45 +0100 schrieb Goswin von Brederlow:
> Jon Harrop <jon@ffconsultancy.com> writes:
> 
> > On Thursday 24 December 2009 13:19:52 Goswin von Brederlow wrote:
> >> Jon Harrop <jon@ffconsultancy.com> writes:
> >> > No, in OCaml I fork every child. That is the only transparent way to give
> >> > the child a coherent view of the heap but it is extremely slow (~1ms):
> >>
> >> So if you add a (sleep 60) to the ocaml code then ocaml gets even
> >> more worse than F#? You are comparing an implementation that is
> >> specifically optimized to use things F# is good at and ocaml is
> >> bad. To give a fair comparison you need to optimize the implementation 
> >> to the language.
> >
> > Post a better OCaml solution if you can.
> >
> >> >> Each process then has a work queue which is works through. So they will
> >> >> always use the local data. Only when a queue runs dry they steal from
> >> >> another process and ruin locality.
> >> >
> >> > You are correctly describing the efficient solution based upon
> >> > work-stealing queues that F# uses but OCaml cannot express it.
> >>
> >> You mean that you didn't implement that way.
> >
> > No, I mean OCaml cannot express it.
> >
> >> I can easily express that with closures and pre-forked worker threads.
> >
> > OCaml's threads do not run in parallel so that will not work.
> 
> >> Or just implement it properly in ocaml. The problem part is the
> >> GC. The rest is easy.
> >
> > No kidding.
> 
> There is one implementation: http://www.algo-prog.info/ocmc/web/
> But as said maybe not a verry good one.
> 
> I tried implementing parallel threads under the original GC by forking
> multiple instances of the same program and using a Bigarray to mmap
> /dev/null for shared memory between the instances. That works for
> sharing primitive types, flat records (records of primitive types) and
> even fixed (or bound) sized structures but it gets more and more
> complex to share each and needs some Obj magic, marshaling or C stubs
> (except for primitive types). It works but is a real hack.

It works with all types:

https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/src/netsys/netsys_mem.mli

look for init_value. It's non-released code yet.

However, there are some problems: Values outside the heap do not support
the polymorphic comparison and hash functions. That's a hard limitation,
e.g. you cannot even compare two strings, or build a hash table with
strings as keys. That limits the usefulness of shared memory.

Also, as we are striving for safe programs, there is the question when
shared memory can be safely released. The GC could help here, but there
are no ways to hook in, e.g. to get notified when there are no pointers
anymore to a value living in shared memory.

Gerd

> 
> I hope someone will pick up the pices and update OCaml4Multicore to
> the latest ocaml or maybe for ocaml to add it directly. If not then I
> fear ocaml will be left behind soon.
> 
> MfG
>         Goswin
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
-- 
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------