Re: [Caml-list] More cores

["Mikkel Fahnøe Jørgensen" <mikkel@dvide.com>]

2008/12/19 Jon Harrop <jon@ffconsultancy.com>:
> ii) People used to choose OCaml because it was fast but lack of support for
> multicores means that OCaml is no longer competitively performant for many
> tasks on today's computers.

That is definitely an argument.

>>      If your programming paradigm for concurrency is Threads+Mutexes,

I think this model is necessary for systems programming, but I don't
think we should look broader than this.

I'm not really sure what separates threads from functions, but I'd
like to be able to chain logic sequentially and concurrently in a way
that sync primitives are available but usually not required, and in a
way that efficiently handle concurrent operations in C libraries,
including transaction operations. I believe Oleg did some work on
this, but this work would likely benefit from compiler support.

So the discussion should be

a) is there any real interest in supporting multiple cores - when we
look beyond concurrent GC and Mutexes?

b) what are the basic primitives we need from the compiler to support
a range of interesting concurrency models in an efficient manner.

c) we are probably not inventing a new vector language - like fortress
- super efficient at micro parallelism, but it should be possible to
efficiently tap into the processing power available.

d) error handling and asymmetric termination times of concurrent
operations is non-trivial - you may not want to wait for all sub-tasks
to complete before the main task does starts aggregating. And if you
embed distributed concurrency you may want to reschedule a long
running operation on a new node. This is not a compiler issue, but the
compiler may support a failure model that makes this easier to handle
in  a generic way for a distributed library. Google often mentions the
importance of abandoning failed tasks because a hard-drive is down
ever so often, or something.

The spawn multiple processes solutions we have today concerns me
slightly because I'm not convinced that processes are cleaned up
properly on failure. I may be wrong though.


> I think ML has proven its worth and
> it is time to begin developing a completely new open source ML
> implementation.

I think this sounds interesting.

But before starting such an effort, it would be nice with more road
map information from Inria. Do they want to include LLVM, or are there
good reasons not to?

More importantly, I believe Xavier is right about the non-scalability
of plain threaded garbage collection. So I really think we need to
understand how we want to obtain that parallelism efficiently. And
when we understand that, perhaps Inria is more willing to listen to
the multi-core argument?

That said, it can't hurt to have an experimental solution to try out
new ideas and prove that they work.

Mikkel