Re: [Caml-list] Looking for information regarding use of OCaml in scientific computing and simulation

[Eray Ozkural <examachine@gmail.com>]

On Tue, Dec 22, 2009 at 6:40 AM, Linas Vepstas <linasvepstas@gmail.com> wrote:
> However, if you are  interested in merely using the system
> to do your "real" work, then writing message-passing code
> is an utter waste of time -- its difficult, time-consuming, error
> prone, hard to balance and optimize & tune, works well only
> for "embarrasingly parallel" code, etc.  Even the evil
> slow-down of NUMA is often better than trying to
> performance-tune a message-passing system.

Message passing doesn't work well only for embarrassingly parallel
code. For instance, you can implement the aforementioned parallel
quicksort rather easily, but it's true that message passing is
low-level. The really bad thing about MPI is that it assumes some
C-like environment. C has the worst semantics ever, so programs that
require/encourage such a style of writing are inherently, well,  bad
=) And yes, MPI's difficult to debug.

What message passing really is, it is the perfect match to a
distributed memory architecture. Since, as you suggest, multicore
chips have more or less a shared memory architecture, message passing
is indeed not a good match.

However, let's not forget about the new GPU architectures, which are
sort of hybrid. The newer GPUs will have more exotic on-chip
interconnection networks.

> Let me put it this way: suggesting that programmers can
> write their own message-passing system is kind of like
> telling them that they can write their own garbage-collection
> system, or design their own closures, or they can go
> create their own type system. Of course they can ... and
> if they wanted to do that, they would be programming in
> C or assembly, and would probably be designing new
> languages.  Cause by the time you get done with message
> passing, you've created a significant and rich programming
> system that resembles a poorly-designed language... been
> there, done that.

For a functional language, am I right in expecting a high-level and
clean interface for explicit parallelism?

I suppose a "spawn" directive would not be very hard to implement. Or
a parallel let block. I don't know what kind of direction the caml
researchers have in mind (except for some INRIA projects I've read
about) but I suppose it can be done (closures could be a nice
interface, I think).

Message Passing/Distributed Memory can also be accommodated I suppose.
For distributed memory, you just need an addressing scheme to denote
the processor number. You could allocate with a parallel new that
takes a processor number argument for instance. For message passing,
you need one-to-one, collective, and one-sided communications. On top
of an imperative but failsafe and debuggable interface, you could
provide neat functional blocks. For instance, you could present the
user with a shuffle/map/reduce interface like in Google's grid
computing platform.

I would also like to have some computing-abstraction (like Monads, but
more flexible) so I can easily build networks of sequential algorithms
(kind of like Communicating Sequential Processses). It would also be
nice to have functors that implement commonly occurring parallel
programming patterns (such as master-slave dynamic load balancing or
pipelining). Such things are difficult to manage in a low-level
language like C, but they would be a piece of cake in ocaml.  Or is
fate sending that my way? Oh, not again! :)

OcamlP3l looks pretty cool. Parallel combinators? Definitely what I'm
talking about, as usual the future is here with ocaml ;)

http://ocamlp3l.inria.fr/eng.htm

Best,

-- 
Eray Ozkural, PhD candidate.  Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct