Re: [Caml-list] Efficient OCaml multicore -- roadmap?

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Dienstag, den 19.04.2011, 12:57 +0300 schrieb Eray Ozkural:
> 
> 
> On Fri, Mar 25, 2011 at 9:19 PM, Hugo Ferreira <hmf@inescporto.pt>
> wrote:
>         On 03/25/2011 06:24 PM, Martin Jambon wrote:
>                         On 03/25/2011 01:10 PM, Fabrice Le Fessant
>                         wrote:
>                                   Of course, sharing structured
>                                 mutable data between threads will not
>                                 be
>                                 possible, but actually, it is a good
>                                 thing if you want to write correct
>                                 programs ;-)
>                 
>                 On 03/25/11 08:44, Hugo Ferreira replied:
>                         I'll stick to my guns here. It simply makes
>                         solving certain problem
>                         unfeasible. Point in case: I work on machine
>                         learning algorithms. I
>                         use large data-structures that must be
>                         processed (altered)
>                         in order to learn. Because these
>                         data-structures are large it become
>                         impractical to copy this to a process every
>                         time I start off a new
>                         "thread".
>                 
>                 The solution would be to use get/set via a
>                 message-passing interface.
>                 
>         
>         
>         Cannot see how this works. Say I want to share a balanced
>         binary tree.
>         Several processes/threads each take this tree and alter it by
>         adding and
>         deleting elements. Each (new) tree is then further processed
>         by other
>         processes/threads.
>         
>         How can get/set be used in this scenario?
>         
>         
> 
> 
> I think it won't have good performance and it won't scale, and it will
> fail for truly delicate shared memory architectures of the future with
> thousands of cores....
> 
> 
> And neither will it support on-chip message passing facilities of
> those future processors.
> 
> 
> The shared memory message passing never worked too well, anyway, too
> many redundant copies. Not fitting for high performance computing. 
> 
> 
> No need at all except for embarrassingly parallel applications. I
> suppose that's the target, right?

I did experiment a bit in the meantime with Netmulticore [1], my
implementation of multi-processing with shared memory. Netmulticore
provides both alterable data structures and a quite efficient message
passing interface. The experience so far: Message passing wins if you do
it the right way. Avoid ping-pong games like get/set - shared memory is
far better for this kind of operation. The better topology for message
passing are pipelines where data flows only into one direction.

I don't know how this translates to Hugo's machine learning problem. I
could imagine a shared data structure is good here for providing the
starting point for learning. If you run several learning steps in
parallel, you want to avoid that these steps lock out each other, i.e.
try to ensure that they affect distinct parts of the matrix. These
updates would be sent (using message passing) to an update manager,
which would apply the learning results to the matrix, and compute the
next version, for which a number of new learning steps would be started.
I'm just guessing here how it could be done. In my imagination a clever
combination of both (alterable) shared memory and message passing is the
way to go.

Hugo, I'd like to do some more experiments into this direction. Is there
a simple version of machine learning algorithm I could try to
parallelize?

Gerd

[1] Netmulticore: now available in ocamlnet-3.3.0test1:
http://blog.camlcity.org/blog/multicore2.html

> 
> 
> Best,
> 
> -- 
> Eray Ozkural, PhD candidate.  Comp. Sci. Dept., Bilkent University,
> Ankara
> http://groups.yahoo.com/group/ai-philosophy
> http://myspace.com/arizanesil http://myspace.com/malfunct
> 


-- 
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------