How INRIA people envision OCaml's parallel future?

How INRIA people envision OCaml's parallel future?

[David MENTRE]

Hello OCaml hackers,

Several people on this list have expressed concerns about the current
limitation of OCaml runtime to work efficiently on multi-processor
machines (limitation related to the inability of current GC to
simultaneously support different threads if I've understood
correctly). I'm amongst them. On the other side, it is clear that
multi-core and multi-threaded processor are going to be widely
available in a short time frame, in embedded[1], desktop and high-end
machines.

How INRIA people are considering the parallel future of OCaml? Do you
consider improving the GC to support concurrent threads? Or do you
consider this a non issue?

More generally, do you plan to introduce new language constructs to
support parallelism? I'm thinking of Jocaml and Skeleton approach for
example. Or simply some libraries?

I know that predictions are quite difficult, especially about the
future, :) but I suppose INRIA people are thinking about those issues.
OCaml is a very exciting experiment of pragmatism and theory and I'm
curious of what might appear on the parallel topic.

Sincerely yours,
david

[1] At work we are using dual-core MIPS processor and 8-core, 8-thread
per core network processor for an embedded system. I've heard of 8-
and 16-core MIPS processor for embedded boards. Very powerful, but a
nightmare to program. I wish I had an OCaml-like language on the
network processor. :)

Re: [Caml-list] How INRIA people envision OCaml's parallel future?

[Frédéric Gava]

>More generally, do you plan to introduce new language constructs to
>support parallelism? I'm thinking of Jocaml and Skeleton approach for
>example. Or simply some libraries?

Developing a new programming language is a difficult task. JoCaml is a good
example. Hard for the maintenance. Also JoCaml is not a parallel language
but a concurrent language <=> if you create two "tasks" in a concurrent
language, you can't know if there are on two different processors or
sequentials tasks; in a parallel language, when you create two tasks, you
want that those two tasks to be in two different processors.

Skeleton (OcamlP3L) is a good way but there is also Data-Parallel languages
sush as HirondML, Ocaml-Flight, BSMLlib (a library for BSP computing; BSP is
a parallel model with a cost model; no deadlock, no undeterministic results;
it is thus a good candidat for developping parallel algorithms) and many
others. An advantage of Data-Parallel languages is that you have the "same
programs" in each processors and thus a sequential GC is sufficient: the
maintenance of the libraries is "easier".

>I know that predictions are quite difficult, especially about the
>future, :) but I suppose INRIA people are thinking about those issues.
>OCaml is a very exciting experiment of pragmatism and theory and I'm
>curious of what might appear on the parallel topic.

It is hard to work on the maintenance of a parallel library (see MPI, PVM,
PUB for examples) and thus parallel peoples do not want to deal (in general)
with the sequential parts of the languages...

There is also the fact that you do not develop parallel programs for the
same things than sequential ones: what is the interest to have a parallel
language for just an application that play a MP3 ?  Using parallelism is
good for some specific cases and thus need specifics libraries. Developping
a "big language" that is good for parallel computing and sequential
computing seems (it is my opinion) to be a too much harder work.

Regards,
Frédéric Gava

Re: [Caml-list] How INRIA people envision OCaml's parallel future?

[David MENTRE]

Hello Frédéric,

Thank you for the different pointers. I've heard of BSP approach
previously but I'm not up to date with latest news.

2005/6/23, Frédéric Gava <gava@univ-paris12.fr>:
> There is also the fact that you do not develop parallel programs for the
> same things than sequential ones: what is the interest to have a parallel
> language for just an application that play a MP3 ?

Good question that I did not considered when asking my own question.

Right now, I'm writing a user program (with Lablgtk2 user interface)
and a server program (network, file and databases I/O, lot of in
memory data structures, few computation intensive parts, mostly
algorithmic issues). On the user side, I see no real need of
parallelism, except if it can improve responsiveness (e.g. user
graphical front-end and back-end that interact with the server on the
network). On the server side, it is different. If I have a dual core
machine (and *I'm going to have* a 2- or 4-core machine), I would like
my server to reply as fast as possible to clients, which implies that
some tasks must be done in the background, with all the implied
synchronization issues.

As you see, my programmer profile is quite different from the expected
usage of BSMLlib (if I have understood BSMLlib description correctly):
no data parallel programming, rather control parallel programming.

>  Using parallelism is
> good for some specific cases and thus need specifics libraries. Developping
> a "big language" that is good for parallel computing and sequential
> computing seems (it is my opinion) to be a too much harder work.

I'm not interested in THE parallel paradigm that will solve all
parallel programming issues. After 50 years of research, no consensus
have emerged yet.

That's said, it does not mean that OCaml should not provide tools to
improve the current state of affair. It is already doing that with
type inference, GC or sum types. Programming parallel code with Posix
mutexes and threads is a nightmare. I have to say that I had a very
pleasant experiment with OCaml typed channels (aka Concurrent ML
channels). I'm just wondering if INRIA people consider that unknown
(to a wide public) constructs could be offered to ease concurrent
programming. And knowing that current ocaml cannot handle real
concurrent threads is a real concern.

Yours,
david

Re: [Caml-list] How INRIA people envision OCaml's parallel future?

[Jonathan_T_Bryant]

I don't know if I'm alone in this opinion, but transparency would be
nice.  It would be nice to be able to take full advantage of SMP
machines using things like threads without necessarily writing parallel
code.  Just things like servers are easier to write in this manner (a
project I'm currently working on in OCaml).  A new language is not
necessary and would actually be overkill.  Sure there could be an
interface for manually adjusting which processor a thread runs on, etc.,
but it should not necessarily have to be used to take advantage of SMP.

As far as the GC, am I'm just kinda throwing this out off the cuff, but
couldn't the heap be a shared memory segment that all threads attach to,
thereby allowing easier collection on multiple processors.  All
allocations could be synchronized (via some kind of semaphore) and could
be locked only during a collection.  I know this is a lot harder than
that, but, like I said, it's just an idea...

--Jonathan

On Thu, 2005-06-23 at 09:21 +0200, David MENTRE wrote:
> Hello OCaml hackers,
> 
> Several people on this list have expressed concerns about the current
> limitation of OCaml runtime to work efficiently on multi-processor
> machines (limitation related to the inability of current GC to
> simultaneously support different threads if I've understood
> correctly). I'm amongst them. On the other side, it is clear that
> multi-core and multi-threaded processor are going to be widely
> available in a short time frame, in embedded[1], desktop and high-end
> machines.
> 
> How INRIA people are considering the parallel future of OCaml? Do you
> consider improving the GC to support concurrent threads? Or do you
> consider this a non issue?
> 
> More generally, do you plan to introduce new language constructs to
> support parallelism? I'm thinking of Jocaml and Skeleton approach for
> example. Or simply some libraries?
> 
> I know that predictions are quite difficult, especially about the
> future, :) but I suppose INRIA people are thinking about those issues.
> OCaml is a very exciting experiment of pragmatism and theory and I'm
> curious of what might appear on the parallel topic.
> 
> Sincerely yours,
> david
> 
> [1] At work we are using dual-core MIPS processor and 8-core, 8-thread
> per core network processor for an embedded system. I've heard of 8-
> and 16-core MIPS processor for embedded boards. Very powerful, but a
> nightmare to program. I wish I had an OCaml-like language on the
> network processor. :)
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] How INRIA people envision OCaml's parallel future?

[Jean-Marie Gaillourdet]

Hi,

Am Donnerstag, den 23.06.2005, 13:20 -0400 schrieb Jonathan_T_Bryant:
> As far as the GC, am I'm just kinda throwing this out off the cuff, but
> couldn't the heap be a shared memory segment that all threads attach to,
> thereby allowing easier collection on multiple processors.  All
> allocations could be synchronized (via some kind of semaphore) and could
> be locked only during a collection.  I know this is a lot harder than
> that, but, like I said, it's just an idea...

I am not a garbage collector specialist, too. But I guess it is much
harder than this. Your idea implies that every allocation requires to
obtain a lock. How often do you allocate memory in a functional
programming language? Quite often, I guess. Every simple tuple, every
constructor application and so on. So you would synchronize all threads
very often. Additionally, during every collection phase, all other cpus
have to wait for the garbage collector. I am pretty sure, that this
would not scale very far. Sorry ;-)

Best Regards,
  Jean-Marie

Re: [Caml-list] How INRIA people envision OCaml's parallel future?

[David MENTRE]

Hello,

2005/6/24, Jean-Marie Gaillourdet <jmg@gaillourdet.net>:
> I am not a garbage collector specialist, too. But I guess it is much
> harder than this. Your idea implies that every allocation requires to
> obtain a lock. How often do you allocate memory in a functional
> programming language? Quite often, I guess. Every simple tuple, every
> constructor application and so on. So you would synchronize all threads
> very often. Additionally, during every collection phase, all other cpus
> have to wait for the garbage collector. I am pretty sure, that this
> would not scale very far. Sorry ;-)

One simple solution to this issue would be to have a per thread minor
allocation zone, with a global, lock-protected, major zone. For newly
allocated objects and minor collection phase, the code is lock-free.

But I suspect that interleaving between minor and major phases makes
things much more difficult. Maybe dedicating a thread to major
collection could be useful (major collection would be blocked by
access to minor zone, with priority given to minor zone).

Anyway, INRIA people have already worked on those issues and had
proposals. It is just that they feel it was not worth the trouble, *at
that time*:
http://groups.google.com/groups?q=doligez+parallel+GC+caml-list&hl=fr&lr=&selm=fa.dlqoshv.1a66ho7%40ifi.uio.no&rnum=2

Yours,
d.

Re: [Caml-list] How INRIA people envision OCaml's parallel future?

[David MENTRE]

[ Last email, I swear it! ;-) ]

Hello,

2005/6/24, David MENTRE <david.mentre@gmail.com>:
> Anyway, INRIA people have already worked on those issues and had
> proposals. It is just that they feel it was not worth the trouble, *at
> that time*:
> http://groups.google.com/groups?q=doligez+parallel+GC+caml-list&hl=fr&lr=&selm=fa.dlqoshv.1a66ho7%40ifi.uio.no&rnum=2

>From private and public emails, it appears that I have fed an old troll. 

My intent is not to start another thread about GC and threading. My
intent is to know how, in a short future, I could efficiently use a
moderately SMP machine by writing pure OCaml code.

It could be with typed channels between unix processes, by modifying
the GC, by using concurrent constructs within the language, ... I
don't know so this is why I'm asking it here.

Yours,
d.

Re: [Caml-list] How INRIA people envision OCaml's parallel future?

[Jonathan_T_Bryant]

Well, one of the easiest ways is to extend the Unix library to be able
to include the shared memory and semaphores system calls (shmget, shmat,
shmctl, shmdt, semget, semctl, and a couple of others, too).  That way
heavyweight processes could simulate threads (they CAN take advantage of
multiple processors but can also share data.  That would be the easiest
way to "solve" (if only partially) the problem.

--Jonathan

On Fri, 2005-06-24 at 14:50 +0200, David MENTRE wrote:
> [ Last email, I swear it! ;-) ]
> 
> Hello,
> 
> 2005/6/24, David MENTRE <david.mentre@gmail.com>:
> > Anyway, INRIA people have already worked on those issues and had
> > proposals. It is just that they feel it was not worth the trouble, *at
> > that time*:
> > http://groups.google.com/groups?q=doligez+parallel+GC+caml-list&hl=fr&lr=&selm=fa.dlqoshv.1a66ho7%40ifi.uio.no&rnum=2
> 
> >From private and public emails, it appears that I have fed an old troll. 
> 
> My intent is not to start another thread about GC and threading. My
> intent is to know how, in a short future, I could efficiently use a
> moderately SMP machine by writing pure OCaml code.
> 
> It could be with typed channels between unix processes, by modifying
> the GC, by using concurrent constructs within the language, ... I
> don't know so this is why I'm asking it here.
> 
> Yours,
> d.
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] How INRIA people envision OCaml's parallel future?

[Eric Stokes]

On Jun 24, 2005, at 5:50 AM, David MENTRE wrote:


> [ Last email, I swear it! ;-) ]
>
> Hello,
>
> 2005/6/24, David MENTRE <david.mentre@gmail.com>:
>
>
>> Anyway, INRIA people have already worked on those issues and had
>> proposals. It is just that they feel it was not worth the trouble,  
>> *at
>> that time*:
>> http://groups.google.com/groups?q=doligez+parallel+GC+caml- 
>> list&hl=fr&lr=&selm=fa.dlqoshv.1a66ho7%40ifi.uio.no&rnum=2
>>
>>
>
> From private and public emails, it appears that I have fed an old  
> troll.
>

Don't worry about it, it really is a valid issue that needs to be  
evaluated again every so often. Its true that many people have  
noticed the new multi core trend. However it has to be asked whether  
threads are really the right answer. They can honestly become a  
nightmare quite quickly.


>
> My intent is not to start another thread about GC and threading. My
> intent is to know how, in a short future, I could efficiently use a
> moderately SMP machine by writing pure OCaml code.
>

You sound like you need something right now, as well as in the  
future. So let me suggest a few tools that you might use to scratch  
your itch right now in the vein of typed channels.

1. Gerd Stolpman has one of the best XDR RPC libraries written for  
any language, it can certainly help improve the safety of a parallel  
program a lot by enforcing that data is exchanged in a type safe  
fashion, and it is also quite fast. It has a very nice rpc compiler  
which allows one to specify fairly painlessly the structure of  
messages. For your purposes, you could use unix domain sockets for  
transport, perhaps even using socketpair to avoid book keeping on the  
filesystem. http://ocaml-programming.de/packages/

2. If you prefer ASN.1 and BER, a subset of BER is included with my  
ldap library, although it is somewhat less nice than the current XDR  
library, it is functional, and also quite fast. http:// 
ocamldap.sourceforge.net


>
> It could be with typed channels between unix processes, by modifying
> the GC, by using concurrent constructs within the language, ... I
> don't know so this is why I'm asking it here.
>
> Yours,
> d.
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>

Re: [Caml-list] How INRIA people envision OCaml's parallel future?

[Jonathan_T_Bryant]

A shared memory segment is just a void pointer (in C), but the problem
is that the static typing in OCaml does not allow that kind of lunacy,
thank God.  I guess that internally OCaml could see it as an array of
some type and do some basic memory management on its own.  Or, while
slightly more sophisticated and complicated, maybe it could serve as a
non-garbage collected static sized heap (OCaml would have to do some
memory management, though).  For example:

Shm.get *some params* : Get a segment
Shm.attatch *param*   : Attatch
Shm.detatch *params*  : Detatch
Shm.control *params*  : Control the segment
Shm.to_shared_heap *param* : Move something on the main heap to this
heap
Shm.to_gc_heap *param* : Move something from this heap to the main GC-ed
heap

Like all of my comments on this subject, I'd like to remind everyone
that I'm not a GC expert or a language expert, but a code jockey.  I
just know what would make my life easier.

On Fri, 2005-06-24 at 17:34 +0100, David Teller wrote:
> How would you handle the shared memory ?
> 
> I'm all for typed channels between processes, though. Although I don't
> know how you could typecheck this, except perhaps by doing it the Acute
> way (i.e. new names, which can be distributed).
> 
> Cheers,
>  David
> 
> On Fri, 2005-06-24 at 12:14 -0400, Jonathan_T_Bryant wrote:
> > Well, one of the easiest ways is to extend the Unix library to be able
> > to include the shared memory and semaphores system calls (shmget, shmat,
> > shmctl, shmdt, semget, semctl, and a couple of others, too).  That way
> > heavyweight processes could simulate threads (they CAN take advantage of
> > multiple processors but can also share data.  That would be the easiest
> > way to "solve" (if only partially) the problem.
> > 
> > --Jonathan
> 
> 

XDR and ASN.1 (was: Re: [Caml-list] How INRIA people envision OCaml's parallel future?)

[David MENTRE]

Hello Eric,

Eric Stokes <eric.stokes@csun.edu> writes:

> 1. Gerd Stolpman has one of the best XDR RPC libraries written for  any
> language, it can certainly help improve the safety of a parallel
> program a lot by enforcing that data is exchanged in a type safe
> fashion, and it is also quite fast. It has a very nice rpc compiler
> which allows one to specify fairly painlessly the structure of
> messages. For your purposes, you could use unix domain sockets for
> transport, perhaps even using socketpair to avoid book keeping on the
> filesystem. http://ocaml-programming.de/packages/

I'm already using Gerd's RPC code to manage communication between my
client and server. I second the praise of Gerd's code.


> 2. If you prefer ASN.1 and BER

<rant>
Well, after doing some work on H.323 and related protocols, I try to
avoid ASN.1 by all means. Speak of an overbloated protocol description
system! 
</rant>

Yours,
d.
-- 
pub  1024D/A3AD7A2A 2004-10-03 David MENTRE <dmentre@linux-france.org>
 5996 CC46 4612 9CA4 3562  D7AC 6C67 9E96 A3AD 7A2A

Re: [Caml-list] How INRIA people envision OCaml's parallel future?

[Erik de Castro Lopo]

Eric Stokes wrote:

> Don't worry about it, it really is a valid issue that needs to be  
> evaluated again every so often. Its true that many people have  
> noticed the new multi core trend. However it has to be asked whether  
> threads are really the right answer. They can honestly become a  
> nightmare quite quickly.

The problem with threads is that they were never meant as a solution
to the multi-processing problem. See here:

    http://sardes.inrialpes.fr/~aschmitt/cwn/2002.11.26.html#8

For multiprocessing you need to look at something like shared
memory of commumication over pipes or sockets.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo  nospam@mega-nerd.com (Yes it's valid)
+-----------------------------------------------------------+
C++ is a siren song. It *looks* like a HLL in which you ought 
to be able to write an application, but it really isn't."
-- Alain Picard (comp.lang.lisp)