Re: [Caml-list] Re: Is OCaml fast?

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Sonntag, den 28.11.2010, 19:14 +0100 schrieb
oliver@first.in-berlin.de:
> On Thu, Nov 25, 2010 at 11:50:58PM +0100, Fabrice Le Fessant wrote:
> [...]
> >  The main problem was that other languages have bigger standard
> > libraries, whereas OCaml has a very small one (just what is needed
> > to compile the compiler, actually). In many problems, you could
> > benefit from using a very simple shared-memory library (in
> > mandelbrot, the ocaml multicore solution has to copy the image in a
> > socket between processes, whereas it could just be in a shared
> > memory segment),
> 
> 
> ...so you work on a shared-mem module?!

Don't know what Fabrice is referring to, but at least I work on a
multicore-enabling library:

https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/src/netmulticore/

This is work in progress and highly experimental. What's currently
available:

- managing processes and resources like files, shared memory objects 
  etc.
- support for message passing via Netcamlbox (another library)
- low-level only so far: shared memory, including copying Ocaml values
  to and from shm
- low-level only so far: kernel semaphores
- a Netmulticore process is also a Netplex container, so mixing with
  Netplex-managed servers is possible. Also, the Netplex plugins 
  are available (semaphores, mutexes, global variables), but these
  are relatively slow

I've also written a few examples:

https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/examples/multicore/

Don't expect wonders, though. For instance, the port of the chameneos
game is based on message passing, but this is by design slower than
direct use of shared memory (and this game mostly consists of
synchronization, and there is not any computation involved where
multicore would be an advantage).

The further direction is the addition of more primitives, especially for
managing shared memory. The difficult here is that there is not any
support in the core language, and I have to work around that. This is
based on

val Netsys_mem.init_value 
      ?targetaddr:nativeint -> 
      ?target_custom_ops:(string * custom_ops) list ->
      memory -> int -> 'a -> init_value_flag list -> (int * int)

where type memory = (char,Bigarray.int8_unsigned_elt,Bigarray.c_layout) Bigarray.Array1.t

This means shared memory is handled just as a bigarray of chars that is
mapped into the address spaces of several processes. The init_value
function copies an arbitrary Ocaml value to such a bigarray  - in the
same way as Marshal.to_string - only that it is "to_bigarray", and that
the same representation is used as for normal values (so you can access
the copied values directly). This copying is quite time-consuming: you
have to create the Ocaml value in normal memory first, and then to use a
quite expensive generic copying machinery to get it to shared memory. It
would be more elegant if there was a way to instruct ocamlopt so that
code is emitted that creates the value directly in a user-supplied
memory area.

The arguments targetaddr and target_custom_ops point to another
difficulty: For certain uses of shared memory you cannot ensure that all
processes map the area to the same address. Because of this, there is
support for filling bigarrays so that the addresses are offset for a
different final mapping address. Netcamlbox uses this feature - the
sending process can map the shared bigarray to any address, and
nevertheless format a message with addresses that are right for the
receiving process.

In general, shared memory is difficult to add in an add-on library.
However, some lessons can be learned, and maybe this results in some
"plot" for adding better support in the core language.

Gerd

> 
> 
> > and in general, many solutions could benefit from
> > specialised data structures that are provided in other languages by
> > their standard libraries, and from some system calls that are
> > currently not in the Unix library.
> [...]
> 
> During the last some releases a lot more unix syscalls were added
> and that's fine of course).
> 
> Which calls are you missing there?
> 
> Ciao,
>    Oliver
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 


-- 
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------