Re: [Caml-list] thousands of CPU cores

[Jon Harrop <jon@ffconsultancy.com>]

On Thursday 10 July 2008 14:44:25 Peng Zang wrote:
> On Thursday 10 July 2008 01:57:44 am J C wrote:
> > I know that Caml team wanted to see if many-core shared-memory systems
> > were going to stick around before bothering with Caml development that
> > takes advantage of them.
> >
> > Well, it looks like they are here to stay, after all:
> >
> > http://news.cnet.com/8301-13924_3-9981760-64.html
>
> This article doesn't say anything about whether the many-core system will
> be shared-memory.  Remember, a shared memory architecture has to deal with
> cache and memory coherence.  The prevailing view is that the overhead for
> such an approach does not scale.  For massively parallel computation we
> must turn to message passing or barrier/sync paradigms.
>
> I am doubtful that a thousand core machine will be shared-memory based.

Today's biggest shared-memory supercomputers already have thousands of cores.

> Also, this is a CNET article.. not exactly known for being in depth or well
> researched and this article is no exception.  It is an article based
> entirely on a few speculative comments of some Intel guys.  I wouldn't take
> it too seriously.
>
> Personally, I can see why the Caml development team opted not to put effort
> into dealing with shared-memory systems.

The OCaml development team put huge effort into their concurrent run-time.

> It is a stop-gap solution... 

That is not true. Many-core machines will always be decomposed into 
shared-memory clusters of as many cores as possible because shared memory 
parallelism will always be orders of magnitude more efficient than 
distributed parallelism.

OCaml is already ~8x slower than F# on today's eight core desktops. If OCaml's 
shortcomings are not remedied then it will become exponentially slower than 
parallelized languages like F# over the next few years until we reach the 
limit of shared memory parallelism in ~10 years time.

Consequently, the parallel GC scheduled for this summer will be the single 
most important development in OCaml world ever.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e