[Caml-list] Callback.register equivalent before runtime is initialized?

[Caml-list] Callback.register equivalent before runtime is initialized?

[Evgeny Roubinchtein]

Hello, everyone,

I am learning about run-time compilation, and using OCaml as a base for
my experiments.  So far, I have a really simple C primitive that just
calls back into a simple OCaml wrapper around Toplevel.load_lambda (it
passes a Lambda.lambda that has been marshalled to a string constant).
Since the C primitive is in the run-time (libcamlrun) library, and the
wrapper is in the Toplevel, I am using Callback.register to pass the
address of the wrapper to the C primitive.  That set-up works pretty
well in the toplevel because I can call Callback.register when the
toplevel is being initialized, i.e., before any code is read from the
user.  

However, I am now trying to move to producing bytecode executables, so,
if I stay with my current set-up I would need to do the equivalent of
Callback.register (i.e., put the address of the wrapper into the hash
table maintained by caml_callback_register) before any of the "user
code" in the executable starts to run.  I have been looking at the
(bytecode) linker and also the caml_startup function to understand how
the running of bytecode "gets started", and I have also modified tools/objinfo
to print the contens of the SYMB section in bytecode executables and the
relocations table in .cmo's to see if inspiration strikes ;-) -- but so
far I am not quite seeing "how things work."

I'll keep looking, but I though I'd ask for advice here in case someone
has some advice to offer.

-- 
Best,
Evgeny

RE: [Caml-list] Callback.register equivalent before runtime is initialized?

[David Allsopp]

Evgeny Roubinchtein wrote:
> Hello, everyone,
> 
> I am learning about run-time compilation, and using OCaml as a base for my
> experiments.  So far, I have a really simple C primitive that just calls
> back into a simple OCaml wrapper around Toplevel.load_lambda (it passes a
> Lambda.lambda that has been marshalled to a string constant).
> Since the C primitive is in the run-time (libcamlrun) library, and the
> wrapper is in the Toplevel, I am using Callback.register to pass the
> address of the wrapper to the C primitive.  That set-up works pretty well
> in the toplevel because I can call Callback.register when the toplevel is
> being initialized, i.e., before any code is read from the user.
> 
> However, I am now trying to move to producing bytecode executables, so, if
> I stay with my current set-up I would need to do the equivalent of
> Callback.register (i.e., put the address of the wrapper into the hash
> table maintained by caml_callback_register) before any of the "user code"
> in the executable starts to run.  I have been looking at the
> (bytecode) linker and also the caml_startup function to understand how the
> running of bytecode "gets started", and I have also modified tools/objinfo
> to print the contens of the SYMB section in bytecode executables and the
> relocations table in .cmo's to see if inspiration strikes ;-) -- but so
> far I am not quite seeing "how things work."
> 
> I'll keep looking, but I though I'd ask for advice here in case someone
> has some advice to offer.

Exposing the primitives directly is in general a bad idea - and the "problem" you're looking at here is one reason why it's bad. Wrap the primitive in a module and instruct your users (even if that's just you!) to use the module - link order then takes care of your problem. So if your primitive is my_first_primitive then you could have:

MyPrimitives.mli:
external first_primitive : foo -> bar = "my_first_primitive"

MyPrimitives.ml:
external first_primitive : foo -> bar = "my_first_primitive"

let _ =
  Callback.register ...

Now when you compile a program using your primitive, you should link MyPrimitives.cmo (or MyPrimitives.cmx in native) and your initialisation code is guaranteed to be called before MyPrimitives.first_primitive is ever called by a user. This also ensures that your primitive is used with the correct OCaml type (although bear in mind that C primitives can be written to be legitimately used with multiple types - e.g. %identity).

Yes, your primitive could still be called directly - but that's now an "unsupported" route and a user doing that gets all that's coming to them! Incidentally, you don't have to use the "external" declaration in the .mli, but the manual chapter on C linking explains that there is a small benefit to doing that, if you don't mind revealing that the function is a C stub.

HTH,


David

Re: [Caml-list] Callback.register equivalent before runtime is

[oleg]

> I am learning about run-time compilation, and using OCaml as a base for
> my experiments.  So far, I have a really simple C primitive that just

> However, I am now trying to move to producing bytecode executables, so,
> if I stay with my current set-up I would need to do the equivalent of
> Callback.register (i.e., put the address of the wrapper into the hash
> table maintained by caml_callback_register) before any of the "user
> code" in the executable starts to run. 

You may want to look into how MetaOCaml does it. Briefly, you just
have to link in your own .cmo file right before user's files. Your
file will set things up. For example, metaocamlc is a shell script
that does 

     exec ocamlc -I +compiler-libs ocamlcommon.cma ocamlbytecomp.cma ocamltoplevel.cma metalib.cma berstart.cmo "$@"

The key is the file berstart.cmo placed before any user files to
compile and link. That file is trivial and enclosed for
convenience. The file mentions Toplevel because MetaOCaml relies on
top-level for executing code created at run-time. You can add your
registration code to that file.

(*
   To `run' the code we use toplevel facilities.
   If we invoke BER MetaOcaml top level, then Toplevel.topstart() will
   initalialize the top level.
   If we execute a byte-compiled executable, we link with
   the top-level library. But we need initialize it first.
   This is the job of the current file.

   This file must be linked in *before* the first user executable.

   The present code roughly do the same steps OCaml top level does
   when executing a script.
   See Toplevel.topmain
*)

let () =
  Toploop.set_paths ();
  Compile.init_path();
  Toploop.initialize_toplevel_env ()

Re: [Caml-list] Callback.register equivalent before runtime is

[Evgeny Roubinchtein]

On Thu, Aug 22, 2013 at 05:47:56AM -0000, oleg@okmij.org wrote:
> 
> > I am learning about run-time compilation, and using OCaml as a base for
> > my experiments.  So far, I have a really simple C primitive that just
> 
> > However, I am now trying to move to producing bytecode executables, so,
> > if I stay with my current set-up I would need to do the equivalent of
> > Callback.register (i.e., put the address of the wrapper into the hash
> > table maintained by caml_callback_register) before any of the "user
> > code" in the executable starts to run. 
> 
> You may want to look into how MetaOCaml does it. Briefly, you just
> have to link in your own .cmo file right before user's files. Your
> file will set things up. For example, metaocamlc is a shell script
> that does 
> 
>      exec ocamlc -I +compiler-libs ocamlcommon.cma ocamlbytecomp.cma ocamltoplevel.cma metalib.cma berstart.cmo "$@"
> 
> The key is the file berstart.cmo placed before any user files to
> compile and link. That file is trivial and enclosed for
> convenience. The file mentions Toplevel because MetaOCaml relies on
> top-level for executing code created at run-time. You can add your
> registration code to that file.
> 
> (*
>    To `run' the code we use toplevel facilities.
>    If we invoke BER MetaOcaml top level, then Toplevel.topstart() will
>    initalialize the top level.
>    If we execute a byte-compiled executable, we link with
>    the top-level library. But we need initialize it first.
>    This is the job of the current file.
> 
>    This file must be linked in *before* the first user executable.
> 
>    The present code roughly do the same steps OCaml top level does
>    when executing a script.
>    See Toplevel.topmain
> *)
> 
> let () =
>   Toploop.set_paths ();
>   Compile.init_path();
>   Toploop.initialize_toplevel_env ()

Thank you.  Yes, after experimenting with David Allsop's earlier
suggestion, I eventually realized that "module initialization" code gets
executed "early enough for my purposes", so I ended up doing exactly
what you suggested above.

-- 
Best,
Evgeny