Re: [Caml-list] Concurrent/parallel programming

[Markus Mottl <markus.mottl@gmail.com>]

Though it may not necessarily lend itself to every problem, it is
often a good idea to use vectorization to achieve high performance.
If your machine learning code is mostly numeric, you may want to use
bigarrays and highly optimized C or Fortran routines as building
blocks for your algorithm (e.g. using Lacaml).  Even without threads
this will generally give a very significant performance boost.  The
use of bigarrays also makes it possible to release the OCaml runtime
lock while a basic routine is executing, which would allow you to run
several such routines on multiple cores without having to resort to
more complicated multi-process solutions.  Note, too, that bigarrays
can in principle be used with shared memory if multiple processes are
still required.

Regards,
Markus

On Tue, Jan 7, 2014 at 2:54 PM, Yotam Barnoy <yotambarnoy@gmail.com> wrote:
> Hi List
>
> So far, I've been programming in ocaml using only sequential programs. In my
> last project, which was an implementation of a large machine learning
> algorithm, I tried to speed up computation using a little bit of parallelism
> with ParMap, and it was a complete failure. It's possible that more time
> would have yielded better results, but I just didn't have the time to invest
> in it given how bad the initial results were.
>
> My question is, what are the options right now as far as parallelism is
> concerned? I'm not talking about cooperative multitasking, but about really
> taking advantage of multiple cores. I'm well aware of the runtime lock and
> I'm ok with message passing between processes or a shared area in memory,
> but I'd rather have something more high level than starting up several
> processes, creating a named pipe or a socket, and trying to pass messages
> through that. Also, I assume that using a shared area in memory involves
> some C code? Am I wrong about that?
>
> I was expecting Core's Async to fill this role, but realworldocaml is fuzzy
> on this topic, apparently preferring to dwell on cooperative multitasking
> (which is fine but not what I'm looking for), and I couldn't find any other
> documentation that was clearer.
>
> Thanks
> Yotam



-- 
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com