Re: [Caml-list] Re: multicore wish

[Jon Harrop <jon@ffconsultancy.com>]

On Sunday 27 December 2009 12:45:53 Goswin von Brederlow wrote:
> There is one implementation: http://www.algo-prog.info/ocmc/web/
> But as said maybe not a very good one.
>
> I tried implementing parallel threads under the original GC by forking
> multiple instances of the same program and using a Bigarray to mmap
> /dev/null for shared memory between the instances. That works for
> sharing primitive types, flat records (records of primitive types) and
> even fixed (or bound) sized structures but it gets more and more
> complex to share each and needs some Obj magic, marshaling or C stubs
> (except for primitive types). It works but is a real hack.

Once you've conceded to manual memory management (mmap of shared bigarrays) 
and low-level programming (no polymorphism, Obj.magic) you've lost the main 
advantages of OCaml and you still cannot get great performance.

> I hope someone will pick up the pices and update OCaml4Multicore to
> the latest ocaml or maybe for ocaml to add it directly. If not then I
> fear ocaml will be left behind soon.

Building upon OCaml rather than starting from scratch makes it vastly more 
difficult to implement a useful parallel GC not just because the GC and code 
gen must work in harmony together but because OCaml has been so heavily 
optimized in the wrong direction for this (e.g. a bit twiddled uniform 
representation that burdens the GC with everything from complex numbers to 
pairs).

That's why I think the best solution is to start from scratch and build a 
completely separate VM bred for shared-memory parallelism. Indeed, HLVM 
already outperforms OC4MC even though I have put a fraction of the effort 
into it and built a lot more surrounding infrastructure. For example, my 
latest GC only took 5 days to write.

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e