Re: [Caml-list] Problems with printing MetaOCaml generated code

[oleg@okmij.org]

After Jeremy's thorough reply, there is little more to say. I'd like
to reinforce a few points. First of all, I was confused by

  > The problem is MetaOCaml never prints generated code, 
  > it's always failing with `Warning 22: The CSP value is a closure or
  > too deep to serialize`.

because Warning 22 is a warning, it is not an error or
failure. Second, MetaOCaml always prints the generated code, with the
proviso that non-seriealizable CSP are printed in a way that make the
code unsuitable for separate compilation. But the code is still
suitable for Runcode.run. (see below). Your concrete example helped.

> So now that I've expressed my frustration about this, I guess my main
> question is: In what cases does MetaOCaml prints generated code? In
> what cases it doesn't? Note that it fails to generate code even after
> calling `print_closed_code (close_code ...)`, so being closed or not
> doesn't seem very relevent here. Is this a bug in MetaOCaml? Are there
> any workarounds? Any ideas what am I doing wrong?

MetaOCaml always prints the generated code. However, if the generated
code contains non-serializable CSP, they are printed in a way that
makes saving the code to a file and then compiling separately
impossible.
Warning 22 was specifically introduced to tell that the generated code
contains a non-serializable CSP. That means that separately compiling
such code is impossible in principle. (I should admit that when CSP is
serializable, it may still be in many cases printed badly -- but at
least that problem is fixable in principle. Or even in practice: CSPs
used internally by MetaOCaml are serializable and are printed in a
good way.)

BTW, one should distinguish ``code with badly printed CSP'' from
``fails to generate code''. The code with badly printed CSP is still a
good code and can be run. So, seeing (* CSP x *) in the output is NOT
a failure of code generation. 

Perhaps a concrete example will help.

# let lift x = .<x>.;;
val lift : 'a -> 'a code = <fun>

# lift "aa";;
- : string code = .<"aa">. 

The generated code can be saved to a file and compiled separately.

# lift 1;;
- : int code = .<(* CSP x *) Obj.magic 1>. 

Here, the generated code looks odd but it is still a well-formed OCaml
code, which may be saved into a file and compiled separately.


# let p = lift (+);;
val p : (int -> int -> int) code = .<(* CSP x *)>. 

This code cannot be compiled separately (and it is accompanied by
Warning 22). Here, CSP is a function (closure). How do you print
closures (other than a sequence of bytes in memory, which is not very
informative)? Saving it to a file is also problematic. 

Yet this is NO failure of code generation. The code p can be used as
any other code value, for example being spliced:

# let p1 = .<.~p 1 2>.;;
val p1 : int code = .<(* CSP x *) 1 2>. 

And we can run the result:

# Runcode.run p1;;
- : int = 3

There is no problems here.

I should stress that there are two ways of using MetaOCaml: run-time
code specialization and generating code for off-line use. In the
former case, it is appropriate, and even beneficial, to share values
between the generator and the generated code, via the common
heap. This is especially good for large values or for closures, open
channels, etc. Such sharing through the common heap is bad for
off-line use, because there is no common heap any more. Warning 22 is
emitted when there is shared through the common heap. This warning is
issued at run-time of the generator: it is difficult to detect
heap-based sharing at compile time: look at the function lift above.
It is polymorphic, so we can't tell if sharing through the heap will
be required from the type of the function (or its definition). In some
uses of this function, like (lift "aa") and (lift 1), no heap sharing
occurs (or, at least, it is not essential and irrelevant). In some
cases, like (lift (+)), the heap-sharing is essential. 

That said, there is something we can do. For example, we can prohibit
polymorphic lifting, like lift above. The user will have to supply
their own function of the type t -> t code for some the concrete type
t. Perhaps that is the way to go, and I am evaluating it. Warning 22
is specifically introduced to help determine how much work the user
will have to do if this solution is implemented. Basically, Warning 22
will become a compile-time error, and the user will have to supply
their own lifting function. That's where modular implicits will be of
great help.


> and `print_code_as_ast` is printing something that definitely doesn't
> look like an AST for any imaginable programming language.(okay... I
> might have exaggerated a bit, but it's definitely not OCaml AST and it
> contains some references to generator program source, like
> "Generator.ml[248]" for example)

It may be hard to accept this, but what you see as the output of
print_code_as_ast is indeed the OCaml AST, printed by OCaml's own
Printast.implementation function.