Re: [Caml-list] Closures serialization and hash.

[Gerd Stolpmann <info@gerd-stolpmann.de>]

Am Montag, den 15.04.2013, 11:16 +0200 schrieb Christophe Raffalli:
> Hello,
> 
> The usual limitation of serialization and hashing of closures is sometimes painful, especially with
> the current temptation of parallelisation. I am involved in two projects/sofwares that are impacted
> by this: bindlib[1] (where data structure using bound variables uses closures) and Patoline[2]
> (where documents using animation and compiled with the "Bin" driver contain closures).
> 
> There is a simple way out of this problem, at least for libraries where the function pointer
> in closure can be predicted enough :
> 
>   - use a table associating function "names" (position in the .cmo, or the lambda-tree or anything
> portable) to source code adresses. The function name should be portable across architecture and
> distinct binary using common librairies.
> 
>  - (1) fill this table by calling a function "register_code_pointer : ('a -> 'b) -> unit"
> 
>  - (2) or even better offer a linking option to register all closures from some compilation unit.
> 
> With this, serialisation and hashing functions could use those "names" instead of addresses. Thus we
> would have reproducible hash and portable serialization for registered functions in closure.
> 
> I will probably try this soon by adding custom version of Hash and Marshal to bindlib using (1) ...
> But support from real OCaml giving (2) would be much better.
> 
> So my question is : are you ennoyed by this problem, and, if yes, would you be happy with the above
> solutions.

For the Plasma map/reduce platform, a similar problem arises. It starts
the same executable on several computers, and it is required to run
functions on remote instances. I've introduced so-called "registered
functions" to address this problem:

http://plasma.camlcity.org/plasma/dl/plasma-0.6.2/doc/html/Plasmamr_toolkit.html#2_Registeredfunctions

I've added some camlp4 support to make the registration less painful, so
you can simply do

let f = <:rfun< variables -> body >>

This concept also distinguishes between local arguments and remote
arguments. The local arguments are normal function arguments, and are
not marshalled, as if you did

let f localarg = <:rfun ...>>

(only that this form would not run any registration code). The remote
arguments are the marshalled ones.

Of course, there is still the problem that we cannot ensure that the
registration of the function is done before its use, simply because we
don't know whether the registration is done in a toplevel module (I
guess this is why you'd prefer (2) ).

Gerd





> 
> Cheers,
> Christophe
> 
> [1] http://www.lama.univ-savoie.fr/~raffalli/bindlib
> [2] http://www.patoline.com
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------