Re: [Caml-list] Faking concurrency using Unix forks and pipes (ray tracing results)

[Jonathan Bryant <jtbryant@valdosta.edu>]

On Jun 4, 2007, at 11:50 AM, Benedikt Grundmann wrote:

> Hi Jonathan,
>
> 2007/6/4, Jonathan Bryant <jtbryant@valdosta.edu>:
>>
>> On Jun 4, 2007, at 6:56 AM, Jon Harrop wrote:
>>
>> >
>> > When you apply this map to an array, a new process is forked for
>> > each element.
>> > As forking is time consuming, you should only apply this to short
>> > arrays.
>>
>> It might be worth your while to implement a process pool.  I said
>> before that I fork of one base process from which to fork the others,
>> and it's not possible to make that process just a controller for a
>> pool of waiting processes.
>>
>> --Jonathan
>>
>
> Do you also do that in your work for the OSP?  I'm asking cause you
> mentioned previously that you want to stay compatible to the ocaml
> threads library... and I just can not figure out how you want to
> implement Thread.create using that base process...

I'm staying with the feel of the threads library, but I'm going to  
have to make it incompatible.  My API uses functors to build  
everything up and then resembles the existing API.  I know there has  
been some talk on this list about performance issues with that, but I  
prefer a clean design over speed (at least at the moment).  Mainly  
this is because Marshal is not trustworthy, so every message type  
needs to have to_ and of_ string functions:

module type Message =
	sig
		type 'a t
		val to_string : 'a t -> string
		val of_string : string -> 'a t
	end

 From there it builds up.  I'm actually trying to design it in such a  
way that channels aren't limited to  UNIX/TCP sockets, and that the  
on-wire protocol can be changed as well.  Hopefully that will go as  
planned.

   As for threads, there are actually 3 levels: concurrent (the  
existing threads), parallel (forked processes), and remote (forked  
process on another machine).  The level of concurrency is specified  
at thread creation either by using a special version of Thread.create  
(such as Thread.concurrent or Thread.parallel) or by an optional  
parameter to Thread.create.  This allows the programmer to use the  
mechanism needed (threads/processes) in the same manner.  One  
additional change is that threads can either be attached (return a  
value), or detached (like they are now and return unit), allowing for  
a future-esque usage of threads.  Also, since it's internally just  
HOFs filled in by the runtime, it allows programs written to run on a  
cluster (using remote threads) to run in a single process for testing  
(using just concurrent threads), and then moved to a multi-core /  
multi-machine environment without any recoding or recompilation.

> BTW: Maybe we should subscribe each others project's mailing list? So
> tha twe can help and learn from each other?
>

I'm already on yours :).

Mine's http://groups.google.com/group/osp2007-northpole

 From what you've said, I think we might be covering a lot of the  
same ground.

>
> Here is the link to mine:
>
> http://groups.google.com/group/osp2007-econcurrency
>
> Cheers,
>
> Bene
>
>
> -- 
> Calvin: I try to make everyone's day a little more
> surreal.
>
> (From Calvin & Hobbes)