Re: Threading and SharedMem (Re: [Caml-list] Re: Is OCaml fast?)

[oliver@first.in-berlin.de]

On Tue, Nov 30, 2010 at 05:07:31PM +0100, Gerd Stolpmann wrote:
> Am Dienstag, den 30.11.2010, 16:30 +0100 schrieb Stephan Houben:
> > On 11/30/2010 02:22 PM, Gerd Stolpmann wrote:
> > > I don't think this is the reason. Many people can ignore Windows,
> > > actually.
> > >
> > > The problem is more that your whole program needs then to be
> > > restructured - multi-processing implies a process model (which is the
> > > master, which are the workers). With multi-threading you can start
> > > threads at all times without having to worry about that (i.e. supports
> > > "programming without design" if you want to take that as a negative
> > > point).
> > >
> > > This is what I want to fix with my Netmulticore library - it defines a
> > > framework allowing you to start new processes at any time without having
> > > to worry about the process hierarchy.
> > 
> > I have in fact read with much interest your blog at
> > http://blog.camlcity.org/blog/parallelmm.html .
> > 
> > Your approach there is to really have separate programs for
> > server and client. However, one nice thing about fork is that you don't
> > have to restructure your program; you can just call fork down somewhere
> > in some subroutine where you decide it is convenient, start doing some
> > multicore computation, finish and return, and the caller needs never know
> > that you did that. So you can indeed program without design using fork.
> 
> Well, I would not recommend that in all cases: fork duplicates all
> memory, and if this is a lot, you can end up consuming a lot of RAM.
[...]

Hence the fork-early recommendation, wich should be a known "rule".
But of course this is, what you addressed with "programming without design"
would be problematic.

But the same also holds for threaded programs, or programs in general.

Ad-hoc code is often a good way to just start something, but after a while
one should start design... and I mean... long before release. ;)
And doing the design after starting with ad-hoc code is done even better
with a language that makes refactoring easy. (And this is the reason
why we are on the Caml-list :))

Some people might even start with the design... and I think a rigid type system
like OCaml supports design, because starting with types can give a good
starting point in a design.
So I experienced that starting with types (and interface) can make things
very clear from the beginning. But of course this only helps, if you have
clear so far, what you want to achieve (a clear, distinct task).
But somehow this becomes off-topic and would be a separate discussion.



[...]
> > Of course, the advantage of your approach is that you can now distribute
> > the work over multiple machines. So I guess there is an appropriate
> > place for all of these techniques.
> 
> I was recently working on an improved fork machinery, with some
> indirection between the request for creating a new worker process, and
> the actual fork. That's this netmulticore library I'm talking about. The
> new process is not a child of the process requesting the creation, but
> always a child of a common master process.
[...]

So, you ask for new sisters and brothers....
...and the parent can be slim.



Ciao,
   Oliver