Re: Threading and SharedMem (Re: [Caml-list] Re: Is OCaml fast?)

[Stephan Houben <stephanh@planet.nl>]

On 11/30/2010 02:22 PM, Gerd Stolpmann wrote:
> I don't think this is the reason. Many people can ignore Windows,
> actually.
>
> The problem is more that your whole program needs then to be
> restructured - multi-processing implies a process model (which is the
> master, which are the workers). With multi-threading you can start
> threads at all times without having to worry about that (i.e. supports
> "programming without design" if you want to take that as a negative
> point).
>
> This is what I want to fix with my Netmulticore library - it defines a
> framework allowing you to start new processes at any time without having
> to worry about the process hierarchy.

I have in fact read with much interest your blog at
http://blog.camlcity.org/blog/parallelmm.html .

Your approach there is to really have separate programs for
server and client. However, one nice thing about fork is that you don't
have to restructure your program; you can just call fork down somewhere
in some subroutine where you decide it is convenient, start doing some
multicore computation, finish and return, and the caller needs never know
that you did that. So you can indeed program without design using fork.

Of course, the advantage of your approach is that you can now distribute
the work over multiple machines. So I guess there is an appropriate
place for all of these techniques.

> Also, many practical problems are only O(n log n), at most. The cost for
> serialization of data through a pipe cannot be neglected here. This
> makes shared memory attractive, even if it is only available in a
> restricted form (like write once memory).

Well, the original context was one of a benchmark which had an
arbitrary rule that you can only use functions from the bundled libraries.
And my proposal was to use the pipe for synchronisation and the shared memory
for bulk communication.

If we drop the arbitrary rule there are faster options than pipes.
(e.g. POSIX semaphores in a shared memory segment).

Stephan