[Caml-list] Problems with printing MetaOCaml generated code

[Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

Hi all,

I'm working on a MetaOCaml program and I want to save generated programs to
files instead of directly running them within the same program using
`Runcode.(.!)` or `Runcode.run`. The problem is MetaOCaml never prints
generated code, it's always failing with `Warning 22: The CSP value is a
closure or too deep to serialize`.

I can't see anything related in it's web page:
http://okmij.org/ftp/ML/MetaOCaml.html. What I understand from that page is
once I have a closed code, I should be able to print it no matter what.
However in my experience even the simplest realistic MetaOCaml program fails to
generate the code because of this error I mentioned above.

(One thing to note here is that my program generates the code on a couple of
inputs, but fails in most of them with this error message.)

In "Many ways to run the code" section it's said that "Since closed code is
essentially OCaml AST, after closing the generated code, the user may examine
and `run' it in many ways. One way of running the code is printing it." and
then it lists some API functions for printing. In my experience none of them
really work, I have closed code which I successfully closed using
`Runcode.close_code`, but `print_closed_code` is still failing with the error,
and `print_code_as_ast` is printing something that definitely doesn't look like
an AST for any imaginable programming language.(okay... I might have
exaggerated a bit, but it's definitely not OCaml AST and it contains some
references to generator program source, like "Generator.ml[248]" for example)

One relevant thing I could find in the documentation is this: "If the code
includes CSP only as literals or external references (and any code can be
arranged that way), the code can be stored into a file and then passed to
ocamlopt." If we remove this sentence from the documentation, I don't think it
mentions about any possible failures in code printing.

So now that I've expressed my frustration about this, I guess my main question
is: In what cases does MetaOCaml prints generated code? In what cases it
doesn't? Note that it fails to generate code even after calling
`print_closed_code (close_code ...)`, so being closed or not doesn't seem very
relevent here. Is this a bug in MetaOCaml? Are there any workarounds? Any ideas
what am I doing wrong?

Thanks in advance for any answers,

Ömer

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Jacques Carette]

You will have some difficulties printing complex closures, especially 
when they refer to values built in the generator (CSPs) which are not 
simple values (strings, integers, etc).

Sometimes, very simple changes to the generator can allow closures and 
CSPs to be printable or not -- without very specific examples, I can't 
help (it's been a few years since I dug into this deeply).

I have been able to print rather complex, large codes with metaocaml.  
But one has to structure the generator rather carefully to ensure this 
possibility.

Jacques

On 2015-04-30 14:36 , Ömer Sinan Ağacan wrote:
> Hi all,
>
> I'm working on a MetaOCaml program and I want to save generated programs to
> files instead of directly running them within the same program using
> `Runcode.(.!)` or `Runcode.run`. The problem is MetaOCaml never prints
> generated code, it's always failing with `Warning 22: The CSP value is a
> closure or too deep to serialize`.
>
> I can't see anything related in it's web page:
> http://okmij.org/ftp/ML/MetaOCaml.html. What I understand from that page is
> once I have a closed code, I should be able to print it no matter what.
> However in my experience even the simplest realistic MetaOCaml program fails to
> generate the code because of this error I mentioned above.
>
> (One thing to note here is that my program generates the code on a couple of
> inputs, but fails in most of them with this error message.)
>
> In "Many ways to run the code" section it's said that "Since closed code is
> essentially OCaml AST, after closing the generated code, the user may examine
> and `run' it in many ways. One way of running the code is printing it." and
> then it lists some API functions for printing. In my experience none of them
> really work, I have closed code which I successfully closed using
> `Runcode.close_code`, but `print_closed_code` is still failing with the error,
> and `print_code_as_ast` is printing something that definitely doesn't look like
> an AST for any imaginable programming language.(okay... I might have
> exaggerated a bit, but it's definitely not OCaml AST and it contains some
> references to generator program source, like "Generator.ml[248]" for example)
>
> One relevant thing I could find in the documentation is this: "If the code
> includes CSP only as literals or external references (and any code can be
> arranged that way), the code can be stored into a file and then passed to
> ocamlopt." If we remove this sentence from the documentation, I don't think it
> mentions about any possible failures in code printing.
>
> So now that I've expressed my frustration about this, I guess my main question
> is: In what cases does MetaOCaml prints generated code? In what cases it
> doesn't? Note that it fails to generate code even after calling
> `print_closed_code (close_code ...)`, so being closed or not doesn't seem very
> relevent here. Is this a bug in MetaOCaml? Are there any workarounds? Any ideas
> what am I doing wrong?
>
> Thanks in advance for any answers,
>
> Ömer
>

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

Thanks for the answer.

My main problem is not that some generated code are not printable -- rather,
it's not clear at all when a code is not printable. Now I have to play around
with code until suddenly it's printable...

Certainly not all code is printable even in theory, for example, what happens
if my generated code captures a file handle from code generator? Or a heap
object that's completely opaque to runtime system.(no type information etc.) In
those cases it may be possible to directly run the code but it's not possible
to print it.

But IMHO MetaOCaml should have done a better job at reporting those. Or at
least at least documenting when it's not possible.

I guess I have to play around and see if some of my random changes make it
printable.

---
Ömer Sinan Ağacan
http://osa1.net


2015-04-30 15:52 GMT-04:00 Jacques Carette <carette@mcmaster.ca>:
> You will have some difficulties printing complex closures, especially when
> they refer to values built in the generator (CSPs) which are not simple
> values (strings, integers, etc).
>
> Sometimes, very simple changes to the generator can allow closures and CSPs
> to be printable or not -- without very specific examples, I can't help (it's
> been a few years since I dug into this deeply).
>
> I have been able to print rather complex, large codes with metaocaml.  But
> one has to structure the generator rather carefully to ensure this
> possibility.
>
> Jacques
>
>
> On 2015-04-30 14:36 , Ömer Sinan Ağacan wrote:
>>
>> Hi all,
>>
>> I'm working on a MetaOCaml program and I want to save generated programs
>> to
>> files instead of directly running them within the same program using
>> `Runcode.(.!)` or `Runcode.run`. The problem is MetaOCaml never prints
>> generated code, it's always failing with `Warning 22: The CSP value is a
>> closure or too deep to serialize`.
>>
>> I can't see anything related in it's web page:
>> http://okmij.org/ftp/ML/MetaOCaml.html. What I understand from that page
>> is
>> once I have a closed code, I should be able to print it no matter what.
>> However in my experience even the simplest realistic MetaOCaml program
>> fails to
>> generate the code because of this error I mentioned above.
>>
>> (One thing to note here is that my program generates the code on a couple
>> of
>> inputs, but fails in most of them with this error message.)
>>
>> In "Many ways to run the code" section it's said that "Since closed code
>> is
>> essentially OCaml AST, after closing the generated code, the user may
>> examine
>> and `run' it in many ways. One way of running the code is printing it."
>> and
>> then it lists some API functions for printing. In my experience none of
>> them
>> really work, I have closed code which I successfully closed using
>> `Runcode.close_code`, but `print_closed_code` is still failing with the
>> error,
>> and `print_code_as_ast` is printing something that definitely doesn't look
>> like
>> an AST for any imaginable programming language.(okay... I might have
>> exaggerated a bit, but it's definitely not OCaml AST and it contains some
>> references to generator program source, like "Generator.ml[248]" for
>> example)
>>
>> One relevant thing I could find in the documentation is this: "If the code
>> includes CSP only as literals or external references (and any code can be
>> arranged that way), the code can be stored into a file and then passed to
>> ocamlopt." If we remove this sentence from the documentation, I don't
>> think it
>> mentions about any possible failures in code printing.
>>
>> So now that I've expressed my frustration about this, I guess my main
>> question
>> is: In what cases does MetaOCaml prints generated code? In what cases it
>> doesn't? Note that it fails to generate code even after calling
>> `print_closed_code (close_code ...)`, so being closed or not doesn't seem
>> very
>> relevent here. Is this a bug in MetaOCaml? Are there any workarounds? Any
>> ideas
>> what am I doing wrong?
>>
>> Thanks in advance for any answers,
>>
>> Ömer
>>
>

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

Let's talk on an example:

-- contents of TestData.ml
  type foo = Foo of int

-- contents of main
  open TestData

  let global_ref : int ref = ref 0

  let gen_inc =
    .< let _ = global_ref := !global_ref + 1 in () >.

  let gen_read =
    .< !global_ref >.

  let gen_write i =
    .< match i with
       | Foo i -> global_ref := i
     >.

  let gen_cls_wrt cls =
    .< global_ref := cls () >.

  let f_toplevel () = Random.int 10

  let _ =
    print_string "\n----------------------------\n";
    let _ = Print_code.print_code Format.std_formatter gen_inc in
    print_string "\n----------------------------\n";
    let _ = Print_code.print_closed_code Format.std_formatter
(Runcode.close_code gen_inc) in
    print_string "\n----------------------------\n";
    let _ = Print_code.print_closed_code Format.std_formatter
(Runcode.close_code (gen_write (Foo 10))) in
    print_string "\n----------------------------\n";
    let _ = Print_code.print_closed_code Format.std_formatter
(Runcode.close_code gen_read) in
    print_string "\n----------------------------\n";
    let f () = Random.int 10 in
    let _ = Print_code.print_closed_code Format.std_formatter
              (Runcode.close_code (gen_cls_wrt f)) in
    print_string "\n----------------------------\n";
    let _ = Print_code.print_closed_code Format.std_formatter
              (Runcode.close_code (gen_cls_wrt f_toplevel)) in
    print_string "\n----------------------------\n";
    ()

MetaOCaml does some very interesting/weird stuff here:

* In printed code it doesn't always show the CSP values, it just says "CSP
  global_ref" for example, but it does that only sometimes. As an example, it
  does that for global_ref, but doesn't do that for f_toplevel. I don't
  understand how are these two any different. I can refer to top level names
  but only sometimes...
  It shows the CSP values in some simple cases, for example if the value is an
  int than it just shows the literal. (not shown in this example)

* I don't know if this is MetaOCaml related, but output of this program is
  interleaved. Instead of printing a line than code it prints two lines and
  then two code etc. How is this possible??

For the reference, the output I'm getting:

  ➜  metaocaml_test git:(master) ✗ ./a.out
  File "metaocaml_test.ml", line 17, characters 19-22:
  Warning 22: The CSP value is a closure or too deep to serialize
  File "metaocaml_test.ml", line 17, characters 19-22:
  Warning 22: The CSP value is a closure or too deep to serialize

  ----------------------------

  ----------------------------
  .<let _ = (* CSP global_ref *) := ((! (* CSP global_ref *)) + 1) in ()>. .<
  let _ = (* CSP global_ref *) := ((! (* CSP global_ref *)) + 1) in ()>.
  ----------------------------
   .<
  match (* CSP i *) with | TestData.Foo i_1 -> (* CSP global_ref *) := i_1>.
  ----------------------------
   .<
  ! (* CSP global_ref *)>.
  ----------------------------
   .<
  ----------------------------
  (* CSP global_ref *) := ((* CSP cls *) ())>. .<
  (* CSP global_ref *) := ((* CSP cls *) ())>.
  ----------------------------


2015-04-30 16:25 GMT-04:00 Ömer Sinan Ağacan <omeragacan@gmail.com>:
> Thanks for the answer.
>
> My main problem is not that some generated code are not printable -- rather,
> it's not clear at all when a code is not printable. Now I have to play around
> with code until suddenly it's printable...
>
> Certainly not all code is printable even in theory, for example, what happens
> if my generated code captures a file handle from code generator? Or a heap
> object that's completely opaque to runtime system.(no type information etc.) In
> those cases it may be possible to directly run the code but it's not possible
> to print it.
>
> But IMHO MetaOCaml should have done a better job at reporting those. Or at
> least at least documenting when it's not possible.
>
> I guess I have to play around and see if some of my random changes make it
> printable.
>
> ---
> Ömer Sinan Ağacan
> http://osa1.net
>
>
> 2015-04-30 15:52 GMT-04:00 Jacques Carette <carette@mcmaster.ca>:
>> You will have some difficulties printing complex closures, especially when
>> they refer to values built in the generator (CSPs) which are not simple
>> values (strings, integers, etc).
>>
>> Sometimes, very simple changes to the generator can allow closures and CSPs
>> to be printable or not -- without very specific examples, I can't help (it's
>> been a few years since I dug into this deeply).
>>
>> I have been able to print rather complex, large codes with metaocaml.  But
>> one has to structure the generator rather carefully to ensure this
>> possibility.
>>
>> Jacques
>>
>>
>> On 2015-04-30 14:36 , Ömer Sinan Ağacan wrote:
>>>
>>> Hi all,
>>>
>>> I'm working on a MetaOCaml program and I want to save generated programs
>>> to
>>> files instead of directly running them within the same program using
>>> `Runcode.(.!)` or `Runcode.run`. The problem is MetaOCaml never prints
>>> generated code, it's always failing with `Warning 22: The CSP value is a
>>> closure or too deep to serialize`.
>>>
>>> I can't see anything related in it's web page:
>>> http://okmij.org/ftp/ML/MetaOCaml.html. What I understand from that page
>>> is
>>> once I have a closed code, I should be able to print it no matter what.
>>> However in my experience even the simplest realistic MetaOCaml program
>>> fails to
>>> generate the code because of this error I mentioned above.
>>>
>>> (One thing to note here is that my program generates the code on a couple
>>> of
>>> inputs, but fails in most of them with this error message.)
>>>
>>> In "Many ways to run the code" section it's said that "Since closed code
>>> is
>>> essentially OCaml AST, after closing the generated code, the user may
>>> examine
>>> and `run' it in many ways. One way of running the code is printing it."
>>> and
>>> then it lists some API functions for printing. In my experience none of
>>> them
>>> really work, I have closed code which I successfully closed using
>>> `Runcode.close_code`, but `print_closed_code` is still failing with the
>>> error,
>>> and `print_code_as_ast` is printing something that definitely doesn't look
>>> like
>>> an AST for any imaginable programming language.(okay... I might have
>>> exaggerated a bit, but it's definitely not OCaml AST and it contains some
>>> references to generator program source, like "Generator.ml[248]" for
>>> example)
>>>
>>> One relevant thing I could find in the documentation is this: "If the code
>>> includes CSP only as literals or external references (and any code can be
>>> arranged that way), the code can be stored into a file and then passed to
>>> ocamlopt." If we remove this sentence from the documentation, I don't
>>> think it
>>> mentions about any possible failures in code printing.
>>>
>>> So now that I've expressed my frustration about this, I guess my main
>>> question
>>> is: In what cases does MetaOCaml prints generated code? In what cases it
>>> doesn't? Note that it fails to generate code even after calling
>>> `print_closed_code (close_code ...)`, so being closed or not doesn't seem
>>> very
>>> relevent here. Is this a bug in MetaOCaml? Are there any workarounds? Any
>>> ideas
>>> what am I doing wrong?
>>>
>>> Thanks in advance for any answers,
>>>
>>> Ömer
>>>
>>

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Jeremy Yallop]

On 30 April 2015 at 21:57, Ömer Sinan Ağacan <omeragacan@gmail.com> wrote:
> Let's talk on an example:
>
> -- contents of TestData.ml
>   type foo = Foo of int
>
> -- contents of main
>   open TestData
>
>   let global_ref : int ref = ref 0
>
>   let gen_inc =
>     .< let _ = global_ref := !global_ref + 1 in () >.

You can improve the output here (and everywhere else that you use
global_ref in a quotation) by putting the global reference in a
separate file, so that MetaOCaml can persist it as a path:

   let gen_inc =
     .< let _ = Globals.global_ref := !Globals.global_ref + 1 in () >.

>   let gen_write i =
>     .< match i with
>        | Foo i -> global_ref := i
>      >.

In this case the problem is that 'i' is a local variable bound to a
current-stage value.  It's better to make it a future-stage (code)
expression, and splice that in:

   let gen_write i =
     .< match .~i with
        | Foo i -> global_ref := i
      >.

You'll also need to change the call:

    gen_write .<Foo 10>.

>   let gen_cls_wrt cls =
>     .< global_ref := cls () >.

This time there are two problems.  First, cls is local, as with 'i'
above.  Second, 'cls' is a closure (function), so it doesn't have a
printed representation.  You can address the problem by making cls
either a code value or a top-level path, e.g.

     .< Globals.global_ref := Globals.cls () >.

or

     .< Globals.global_ref := .~cls () >.

With these changes the program runs without warnings or errors and
generates compilable code without CSP comments.

> * I don't know if this is MetaOCaml related, but output of this program is
>   interleaved. Instead of printing a line than code it prints two lines and
>   then two code etc. How is this possible??

The interleaving is probably due to internal buffering in the format library.

Jeremy.

Re: [Caml-list] Problems with printing MetaOCaml generated code

[oleg]

After Jeremy's thorough reply, there is little more to say. I'd like
to reinforce a few points. First of all, I was confused by

  > The problem is MetaOCaml never prints generated code, 
  > it's always failing with `Warning 22: The CSP value is a closure or
  > too deep to serialize`.

because Warning 22 is a warning, it is not an error or
failure. Second, MetaOCaml always prints the generated code, with the
proviso that non-seriealizable CSP are printed in a way that make the
code unsuitable for separate compilation. But the code is still
suitable for Runcode.run. (see below). Your concrete example helped.

> So now that I've expressed my frustration about this, I guess my main
> question is: In what cases does MetaOCaml prints generated code? In
> what cases it doesn't? Note that it fails to generate code even after
> calling `print_closed_code (close_code ...)`, so being closed or not
> doesn't seem very relevent here. Is this a bug in MetaOCaml? Are there
> any workarounds? Any ideas what am I doing wrong?

MetaOCaml always prints the generated code. However, if the generated
code contains non-serializable CSP, they are printed in a way that
makes saving the code to a file and then compiling separately
impossible.
Warning 22 was specifically introduced to tell that the generated code
contains a non-serializable CSP. That means that separately compiling
such code is impossible in principle. (I should admit that when CSP is
serializable, it may still be in many cases printed badly -- but at
least that problem is fixable in principle. Or even in practice: CSPs
used internally by MetaOCaml are serializable and are printed in a
good way.)

BTW, one should distinguish ``code with badly printed CSP'' from
``fails to generate code''. The code with badly printed CSP is still a
good code and can be run. So, seeing (* CSP x *) in the output is NOT
a failure of code generation. 

Perhaps a concrete example will help.

# let lift x = .<x>.;;
val lift : 'a -> 'a code = <fun>

# lift "aa";;
- : string code = .<"aa">. 

The generated code can be saved to a file and compiled separately.

# lift 1;;
- : int code = .<(* CSP x *) Obj.magic 1>. 

Here, the generated code looks odd but it is still a well-formed OCaml
code, which may be saved into a file and compiled separately.


# let p = lift (+);;
val p : (int -> int -> int) code = .<(* CSP x *)>. 

This code cannot be compiled separately (and it is accompanied by
Warning 22). Here, CSP is a function (closure). How do you print
closures (other than a sequence of bytes in memory, which is not very
informative)? Saving it to a file is also problematic. 

Yet this is NO failure of code generation. The code p can be used as
any other code value, for example being spliced:

# let p1 = .<.~p 1 2>.;;
val p1 : int code = .<(* CSP x *) 1 2>. 

And we can run the result:

# Runcode.run p1;;
- : int = 3

There is no problems here.

I should stress that there are two ways of using MetaOCaml: run-time
code specialization and generating code for off-line use. In the
former case, it is appropriate, and even beneficial, to share values
between the generator and the generated code, via the common
heap. This is especially good for large values or for closures, open
channels, etc. Such sharing through the common heap is bad for
off-line use, because there is no common heap any more. Warning 22 is
emitted when there is shared through the common heap. This warning is
issued at run-time of the generator: it is difficult to detect
heap-based sharing at compile time: look at the function lift above.
It is polymorphic, so we can't tell if sharing through the heap will
be required from the type of the function (or its definition). In some
uses of this function, like (lift "aa") and (lift 1), no heap sharing
occurs (or, at least, it is not essential and irrelevant). In some
cases, like (lift (+)), the heap-sharing is essential. 

That said, there is something we can do. For example, we can prohibit
polymorphic lifting, like lift above. The user will have to supply
their own function of the type t -> t code for some the concrete type
t. Perhaps that is the way to go, and I am evaluating it. Warning 22
is specifically introduced to help determine how much work the user
will have to do if this solution is implemented. Basically, Warning 22
will become a compile-time error, and the user will have to supply
their own lifting function. That's where modular implicits will be of
great help.


> and `print_code_as_ast` is printing something that definitely doesn't
> look like an AST for any imaginable programming language.(okay... I
> might have exaggerated a bit, but it's definitely not OCaml AST and it
> contains some references to generator program source, like
> "Generator.ml[248]" for example)

It may be hard to accept this, but what you see as the output of
print_code_as_ast is indeed the OCaml AST, printed by OCaml's own
Printast.implementation function.

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

Thanks for all the answers,

> MetaOCaml always prints the generated code. However, if the generated code
> contains non-serializable CSP, they are printed in a way that makes saving
> the code to a file and then compiling separately impossible.

Can we talk about what is non-serializable here? I pushed some code to a
public repo for demonstration purposes. I have a simple interpreter for
a simple language. I'm experimenting with different levels of
interpreter specializations(aka. first Futamura projections). As a
simplest specialization, I want to eliminate parsing costs. The code
that's supposed to do this is here:

https://github.com/osa1/int-proj/blob/master/Unlambda.ml#L204

`p'` in that code is lifted, and it's parsed syntax tree of given source
file. Syntax tree is defined here:

https://github.com/osa1/int-proj/blob/master/Syntax.ml#L63

I think it doesn't have anything that's not serializable: It's just a
simple monomorphic type, doesn't have any closures etc.

But MetaOCaml is failing to print this code. To try it yourself, make
sure you have metaocamlc in $PATH, and run:

  ➜  unlambda git:(master) ✗ make unlambda_metaocaml
  metaocamlc -c Syntax.ml
  metaocamlc Syntax.cmo Unlambda.ml -o unlambda_metaocaml

  ➜  unlambda git:(master) ✗ ./unlambda_metaocaml programs/Hello.unl stage0
  File "Unlambda.ml", line 204, characters 67-69:
  Warning 22: The CSP value is a closure or too deep to serialize
  File "Unlambda.ml", line 204, characters 58-66:
  Warning 22: The CSP value is a closure or too deep to serialize
  .<(* CSP eval_ref *) (* CSP p' *) []>.

To make my point clear: I don't actually need to print the code, even
though offline compilation would be awesome, I'm fine with just (.!)ing
my code for now. But I should be able to see the transformations I'm
doing. I'm perfectly fine with just (.!)ing my code, as long as I can
see what I'm running.

But in this case MetaOCaml is failing to print a code that IMO is
completely printable.

Thanks again for very helpful responses.

2015-05-01 7:21 GMT-04:00  <oleg@okmij.org>:
>
> After Jeremy's thorough reply, there is little more to say. I'd like
> to reinforce a few points. First of all, I was confused by
>
>   > The problem is MetaOCaml never prints generated code,
>   > it's always failing with `Warning 22: The CSP value is a closure or
>   > too deep to serialize`.
>
> because Warning 22 is a warning, it is not an error or
> failure. Second, MetaOCaml always prints the generated code, with the
> proviso that non-seriealizable CSP are printed in a way that make the
> code unsuitable for separate compilation. But the code is still
> suitable for Runcode.run. (see below). Your concrete example helped.
>
>> So now that I've expressed my frustration about this, I guess my main
>> question is: In what cases does MetaOCaml prints generated code? In
>> what cases it doesn't? Note that it fails to generate code even after
>> calling `print_closed_code (close_code ...)`, so being closed or not
>> doesn't seem very relevent here. Is this a bug in MetaOCaml? Are there
>> any workarounds? Any ideas what am I doing wrong?
>
> MetaOCaml always prints the generated code. However, if the generated
> code contains non-serializable CSP, they are printed in a way that
> makes saving the code to a file and then compiling separately
> impossible.
> Warning 22 was specifically introduced to tell that the generated code
> contains a non-serializable CSP. That means that separately compiling
> such code is impossible in principle. (I should admit that when CSP is
> serializable, it may still be in many cases printed badly -- but at
> least that problem is fixable in principle. Or even in practice: CSPs
> used internally by MetaOCaml are serializable and are printed in a
> good way.)
>
> BTW, one should distinguish ``code with badly printed CSP'' from
> ``fails to generate code''. The code with badly printed CSP is still a
> good code and can be run. So, seeing (* CSP x *) in the output is NOT
> a failure of code generation.
>
> Perhaps a concrete example will help.
>
> # let lift x = .<x>.;;
> val lift : 'a -> 'a code = <fun>
>
> # lift "aa";;
> - : string code = .<"aa">.
>
> The generated code can be saved to a file and compiled separately.
>
> # lift 1;;
> - : int code = .<(* CSP x *) Obj.magic 1>.
>
> Here, the generated code looks odd but it is still a well-formed OCaml
> code, which may be saved into a file and compiled separately.
>
>
> # let p = lift (+);;
> val p : (int -> int -> int) code = .<(* CSP x *)>.
>
> This code cannot be compiled separately (and it is accompanied by
> Warning 22). Here, CSP is a function (closure). How do you print
> closures (other than a sequence of bytes in memory, which is not very
> informative)? Saving it to a file is also problematic.
>
> Yet this is NO failure of code generation. The code p can be used as
> any other code value, for example being spliced:
>
> # let p1 = .<.~p 1 2>.;;
> val p1 : int code = .<(* CSP x *) 1 2>.
>
> And we can run the result:
>
> # Runcode.run p1;;
> - : int = 3
>
> There is no problems here.
>
> I should stress that there are two ways of using MetaOCaml: run-time
> code specialization and generating code for off-line use. In the
> former case, it is appropriate, and even beneficial, to share values
> between the generator and the generated code, via the common
> heap. This is especially good for large values or for closures, open
> channels, etc. Such sharing through the common heap is bad for
> off-line use, because there is no common heap any more. Warning 22 is
> emitted when there is shared through the common heap. This warning is
> issued at run-time of the generator: it is difficult to detect
> heap-based sharing at compile time: look at the function lift above.
> It is polymorphic, so we can't tell if sharing through the heap will
> be required from the type of the function (or its definition). In some
> uses of this function, like (lift "aa") and (lift 1), no heap sharing
> occurs (or, at least, it is not essential and irrelevant). In some
> cases, like (lift (+)), the heap-sharing is essential.
>
> That said, there is something we can do. For example, we can prohibit
> polymorphic lifting, like lift above. The user will have to supply
> their own function of the type t -> t code for some the concrete type
> t. Perhaps that is the way to go, and I am evaluating it. Warning 22
> is specifically introduced to help determine how much work the user
> will have to do if this solution is implemented. Basically, Warning 22
> will become a compile-time error, and the user will have to supply
> their own lifting function. That's where modular implicits will be of
> great help.
>
>
>> and `print_code_as_ast` is printing something that definitely doesn't
>> look like an AST for any imaginable programming language.(okay... I
>> might have exaggerated a bit, but it's definitely not OCaml AST and it
>> contains some references to generator program source, like
>> "Generator.ml[248]" for example)
>
> It may be hard to accept this, but what you see as the output of
> print_code_as_ast is indeed the OCaml AST, printed by OCaml's own
> Printast.implementation function.
>

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Leo White]

> Can we talk about what is non-serializable here? I pushed some code to a
> public repo for demonstration purposes. I have a simple interpreter for
> a simple language. I'm experimenting with different levels of
> interpreter specializations(aka. first Futamura projections). As a
> simplest specialization, I want to eliminate parsing costs. The code
> that's supposed to do this is here:
> 
> https://github.com/osa1/int-proj/blob/master/Unlambda.ml#L204
> 
> `p'` in that code is lifted, and it's parsed syntax tree of given source
> file. Syntax tree is defined here:
> 
> https://github.com/osa1/int-proj/blob/master/Syntax.ml#L63
> 
> I think it doesn't have anything that's not serializable: It's just a
> simple monomorphic type, doesn't have any closures etc.
> 
> But MetaOCaml is failing to print this code. To try it yourself, make
> sure you have metaocamlc in $PATH, and run:
> 
>   ➜  unlambda git:(master) ✗ make unlambda_metaocaml
>   metaocamlc -c Syntax.ml
>   metaocamlc Syntax.cmo Unlambda.ml -o unlambda_metaocaml
> 
>   ➜  unlambda git:(master) ✗ ./unlambda_metaocaml programs/Hello.unl stage0
>   File "Unlambda.ml", line 204, characters 67-69:
>   Warning 22: The CSP value is a closure or too deep to serialize
>   File "Unlambda.ml", line 204, characters 58-66:
>   Warning 22: The CSP value is a closure or too deep to serialize
>   .<(* CSP eval_ref *) (* CSP p' *) []>.
> 

From a quick look at the code, it seems pretty clear that `eval_ref` is not serializable.
It is a local function so there is no AST fragment which could represent it globally. I
suspect that if you moved `eval_ref` into a separate module, so that it was
a global function, then this code would work.

Regards,

Leo

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

2015-05-01 12:16 GMT-04:00 Leo White <leo@lpw25.net>:
> From a quick look at the code, it seems pretty clear that `eval_ref` is not serializable.
> It is a local function so there is no AST fragment which could represent it globally. I
> suspect that if you moved `eval_ref` into a separate module, so that it was
> a global function, then this code would work.

It should serialize `p'`, not `eval_ref`, I'd expect serialization of
`eval_ref` to be the identifier `eval_ref`.

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Leo White]

On Fri, 1 May 2015, at 12:41 PM, Ömer Sinan Ağacan wrote:
> 2015-05-01 12:16 GMT-04:00 Leo White <leo@lpw25.net>:
> > From a quick look at the code, it seems pretty clear that `eval_ref` is not serializable.
> > It is a local function so there is no AST fragment which could represent it globally. I
> > suspect that if you moved `eval_ref` into a separate module, so that it was
> > a global function, then this code would work.
> 
> It should serialize `p'`, not `eval_ref`, I'd expect serialization of
> `eval_ref` to be the identifier `eval_ref`.

You can't serialize `eval_ref` as `eval_ref` because that is a local identifier. If you print
out `eval_ref` into some other ml file and compiler it, it is going to give an "Unbound
identifier eval_ref" error.

Regards,

Leo

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

> You can't serialize `eval_ref` as `eval_ref` because that is a local
> identifier. If you print out `eval_ref` into some other ml file and compiler
> it, it is going to give an "Unbound identifier eval_ref" error.

That's true. Just to make sure and make the output more clear, I moved the
relevant code to another module, and now it's printing this:

.<Unlambda.eval_ref (* CSP p' *) []>.

My main question is that it should serialize p' here, but it doesn't. I'm
trying to understand why.

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

In case anyone's still interested, I produced a very simple example that
demonstrates the issue:

  ➜  metaocaml_serialization_issue git:(master) ✗ ls
  Main.ml  Syntax.ml
  ➜  metaocaml_serialization_issue git:(master) ✗ cat Syntax.ml
  type stx =
    | A
    | B of stx
    | C of (stx * stx)
  ➜  metaocaml_serialization_issue git:(master) ✗ cat Main.ml
  open Format
  open Print_code
  open Runcode
  open Syntax

  let _ =
    let stx1 = A in
    let stx2 = B A in
    let stx3 = C (A, A) in

    print_code std_formatter .< stx1 >.;
    print_code std_formatter .< stx2 >.;
    print_code std_formatter .< stx3 >.;

    print_closed_code std_formatter (close_code .< stx1 >.);
    print_closed_code std_formatter (close_code .< stx2 >.);
    print_closed_code std_formatter (close_code .< stx3 >.);
  ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc Syntax.ml -c
  ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc
Syntax.cmo Main.ml -o main
  ➜  metaocaml_serialization_issue git:(master) ✗ ./main
  .<(* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>. .<
  (* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>.
  ➜  metaocaml_serialization_issue git:(master) ✗


2015-05-01 12:53 GMT-04:00 Ömer Sinan Ağacan <omeragacan@gmail.com>:
>> You can't serialize `eval_ref` as `eval_ref` because that is a local
>> identifier. If you print out `eval_ref` into some other ml file and compiler
>> it, it is going to give an "Unbound identifier eval_ref" error.
>
> That's true. Just to make sure and make the output more clear, I moved the
> relevant code to another module, and now it's printing this:
>
> .<Unlambda.eval_ref (* CSP p' *) []>.
>
> My main question is that it should serialize p' here, but it doesn't. I'm
> trying to understand why.

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Jacques Carette]

try instead
    let stx1 = .< A >. in
and then
    print_code std_formatter .< .~stx1 >. ;

That ought to work as you wish.

Jacques

On 2015-05-02 2:45 PM, Ömer Sinan Ağacan wrote:
> In case anyone's still interested, I produced a very simple example that
> demonstrates the issue:
>
>    ➜  metaocaml_serialization_issue git:(master) ✗ ls
>    Main.ml  Syntax.ml
>    ➜  metaocaml_serialization_issue git:(master) ✗ cat Syntax.ml
>    type stx =
>      | A
>      | B of stx
>      | C of (stx * stx)
>    ➜  metaocaml_serialization_issue git:(master) ✗ cat Main.ml
>    open Format
>    open Print_code
>    open Runcode
>    open Syntax
>
>    let _ =
>      let stx1 = A in
>      let stx2 = B A in
>      let stx3 = C (A, A) in
>
>      print_code std_formatter .< stx1 >.;
>      print_code std_formatter .< stx2 >.;
>      print_code std_formatter .< stx3 >.;
>
>      print_closed_code std_formatter (close_code .< stx1 >.);
>      print_closed_code std_formatter (close_code .< stx2 >.);
>      print_closed_code std_formatter (close_code .< stx3 >.);
>    ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc Syntax.ml -c
>    ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc
> Syntax.cmo Main.ml -o main
>    ➜  metaocaml_serialization_issue git:(master) ✗ ./main
>    .<(* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>. .<
>    (* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>.
>    ➜  metaocaml_serialization_issue git:(master) ✗
>
>
> 2015-05-01 12:53 GMT-04:00 Ömer Sinan Ağacan <omeragacan@gmail.com>:
>>> You can't serialize `eval_ref` as `eval_ref` because that is a local
>>> identifier. If you print out `eval_ref` into some other ml file and compiler
>>> it, it is going to give an "Unbound identifier eval_ref" error.
>> That's true. Just to make sure and make the output more clear, I moved the
>> relevant code to another module, and now it's printing this:
>>
>> .<Unlambda.eval_ref (* CSP p' *) []>.
>>
>> My main question is that it should serialize p' here, but it doesn't. I'm
>> trying to understand why.

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

That's not a solution. I should be able to generate some values in
code generation time and persist them in code values, that's the whole
point here.

2015-05-02 16:49 GMT-04:00 Jacques Carette <carette@mcmaster.ca>:
> try instead
>    let stx1 = .< A >. in
> and then
>    print_code std_formatter .< .~stx1 >. ;
>
> That ought to work as you wish.
>
> Jacques
>
>
> On 2015-05-02 2:45 PM, Ömer Sinan Ağacan wrote:
>>
>> In case anyone's still interested, I produced a very simple example that
>> demonstrates the issue:
>>
>>    ➜  metaocaml_serialization_issue git:(master) ✗ ls
>>    Main.ml  Syntax.ml
>>    ➜  metaocaml_serialization_issue git:(master) ✗ cat Syntax.ml
>>    type stx =
>>      | A
>>      | B of stx
>>      | C of (stx * stx)
>>    ➜  metaocaml_serialization_issue git:(master) ✗ cat Main.ml
>>    open Format
>>    open Print_code
>>    open Runcode
>>    open Syntax
>>
>>    let _ =
>>      let stx1 = A in
>>      let stx2 = B A in
>>      let stx3 = C (A, A) in
>>
>>      print_code std_formatter .< stx1 >.;
>>      print_code std_formatter .< stx2 >.;
>>      print_code std_formatter .< stx3 >.;
>>
>>      print_closed_code std_formatter (close_code .< stx1 >.);
>>      print_closed_code std_formatter (close_code .< stx2 >.);
>>      print_closed_code std_formatter (close_code .< stx3 >.);
>>    ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc Syntax.ml -c
>>    ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc
>> Syntax.cmo Main.ml -o main
>>    ➜  metaocaml_serialization_issue git:(master) ✗ ./main
>>    .<(* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>. .<
>>    (* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>.
>>    ➜  metaocaml_serialization_issue git:(master) ✗
>>
>>
>> 2015-05-01 12:53 GMT-04:00 Ömer Sinan Ağacan <omeragacan@gmail.com>:
>>>>
>>>> You can't serialize `eval_ref` as `eval_ref` because that is a local
>>>> identifier. If you print out `eval_ref` into some other ml file and
>>>> compiler
>>>> it, it is going to give an "Unbound identifier eval_ref" error.
>>>
>>> That's true. Just to make sure and make the output more clear, I moved
>>> the
>>> relevant code to another module, and now it's printing this:
>>>
>>> .<Unlambda.eval_ref (* CSP p' *) []>.
>>>
>>> My main question is that it should serialize p' here, but it doesn't. I'm
>>> trying to understand why.
>
>

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Jacques Carette]

But that's the whole point: you are not persisting a value, you are 
persisting a local variable.

In the context of staging, a variable and its value are radically 
different, unlike in the traditional functional programming context, 
where this difference can be safely and harmlessly blurred.

If you want to persist "named values", then give globally accessible 
names to these values [which I believe others have already told you].  
If you want to persist values, then use my solution or a variant of that.

Jacques

On 2015-05-02 9:56 PM, Ömer Sinan Ağacan wrote:
> That's not a solution. I should be able to generate some values in
> code generation time and persist them in code values, that's the whole
> point here.
>
> 2015-05-02 16:49 GMT-04:00 Jacques Carette <carette@mcmaster.ca>:
>> try instead
>>     let stx1 = .< A >. in
>> and then
>>     print_code std_formatter .< .~stx1 >. ;
>>
>> That ought to work as you wish.
>>
>> Jacques
>>
>>
>> On 2015-05-02 2:45 PM, Ömer Sinan Ağacan wrote:
>>> In case anyone's still interested, I produced a very simple example that
>>> demonstrates the issue:
>>>
>>>     ➜  metaocaml_serialization_issue git:(master) ✗ ls
>>>     Main.ml  Syntax.ml
>>>     ➜  metaocaml_serialization_issue git:(master) ✗ cat Syntax.ml
>>>     type stx =
>>>       | A
>>>       | B of stx
>>>       | C of (stx * stx)
>>>     ➜  metaocaml_serialization_issue git:(master) ✗ cat Main.ml
>>>     open Format
>>>     open Print_code
>>>     open Runcode
>>>     open Syntax
>>>
>>>     let _ =
>>>       let stx1 = A in
>>>       let stx2 = B A in
>>>       let stx3 = C (A, A) in
>>>
>>>       print_code std_formatter .< stx1 >.;
>>>       print_code std_formatter .< stx2 >.;
>>>       print_code std_formatter .< stx3 >.;
>>>
>>>       print_closed_code std_formatter (close_code .< stx1 >.);
>>>       print_closed_code std_formatter (close_code .< stx2 >.);
>>>       print_closed_code std_formatter (close_code .< stx3 >.);
>>>     ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc Syntax.ml -c
>>>     ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc
>>> Syntax.cmo Main.ml -o main
>>>     ➜  metaocaml_serialization_issue git:(master) ✗ ./main
>>>     .<(* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>. .<
>>>     (* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>.
>>>     ➜  metaocaml_serialization_issue git:(master) ✗
>>>
>>>
>>> 2015-05-01 12:53 GMT-04:00 Ömer Sinan Ağacan <omeragacan@gmail.com>:
>>>>> You can't serialize `eval_ref` as `eval_ref` because that is a local
>>>>> identifier. If you print out `eval_ref` into some other ml file and
>>>>> compiler
>>>>> it, it is going to give an "Unbound identifier eval_ref" error.
>>>> That's true. Just to make sure and make the output more clear, I moved
>>>> the
>>>> relevant code to another module, and now it's printing this:
>>>>
>>>> .<Unlambda.eval_ref (* CSP p' *) []>.
>>>>
>>>> My main question is that it should serialize p' here, but it doesn't. I'm
>>>> trying to understand why.
>>

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

> But that's the whole point: you are not persisting a value, you are
> persisting a local variable.

Can you explain what's wrong with that? I'm making sure that the local
variables I'm trying to persist are persistable, as in my last example:

  let stx = A in .< stx >.

(Btw, MetaML has explicit `lift` function for this operation, I'm wondering if
there are any differences between MetaML's `lift` and MetaOCaml's implicit
lifting)

> In the context of staging, a variable and its value are radically different,
> unlike in the traditional functional programming context, where this
> difference can be safely and harmlessly blurred.

I understand that code values are similar to closures in some ways, for example
they capture some variables, and they're not always serializable. My problem
here is that they're almost never serializable in the case of MetaOCaml. It
even fails to serialize a code object that just holds a single constructor for
a simple type with one constructor, for example.

(It'd be really great if you could elaborate on how they're "radically
different")

> If you want to persist "named values", then give globally accessible names to
> these values [which I believe others have already told you].

...

> If you want to persist values, then use my solution or a variant of that.

Can you explain how is this a "solution"? To rephrase my goal one more time: I'm
generating specialized code in runtime, but instead of running it, I want to
serialize it so that I can 1) inspect the generated code and make sure I'm
really doing the optimizations I'm expecting to do 2) I can compile my code
separately. But MetaOCaml is just refusing to show data values in my generated
code.

I started to feel like the story of MetaOCaml serialization as described in
manual's "Many ways to run the code" section is just wrong.

2015-05-02 22:28 GMT-04:00 Jacques Carette <carette@mcmaster.ca>:
> But that's the whole point: you are not persisting a value, you are
> persisting a local variable.
>
> In the context of staging, a variable and its value are radically different,
> unlike in the traditional functional programming context, where this
> difference can be safely and harmlessly blurred.
>
> If you want to persist "named values", then give globally accessible names
> to these values [which I believe others have already told you].  If you want
> to persist values, then use my solution or a variant of that.
>
> Jacques
>
>
> On 2015-05-02 9:56 PM, Ömer Sinan Ağacan wrote:
>>
>> That's not a solution. I should be able to generate some values in
>> code generation time and persist them in code values, that's the whole
>> point here.
>>
>> 2015-05-02 16:49 GMT-04:00 Jacques Carette <carette@mcmaster.ca>:
>>>
>>> try instead
>>>     let stx1 = .< A >. in
>>> and then
>>>     print_code std_formatter .< .~stx1 >. ;
>>>
>>> That ought to work as you wish.
>>>
>>> Jacques
>>>
>>>
>>> On 2015-05-02 2:45 PM, Ömer Sinan Ağacan wrote:
>>>>
>>>> In case anyone's still interested, I produced a very simple example that
>>>> demonstrates the issue:
>>>>
>>>>     ➜  metaocaml_serialization_issue git:(master) ✗ ls
>>>>     Main.ml  Syntax.ml
>>>>     ➜  metaocaml_serialization_issue git:(master) ✗ cat Syntax.ml
>>>>     type stx =
>>>>       | A
>>>>       | B of stx
>>>>       | C of (stx * stx)
>>>>     ➜  metaocaml_serialization_issue git:(master) ✗ cat Main.ml
>>>>     open Format
>>>>     open Print_code
>>>>     open Runcode
>>>>     open Syntax
>>>>
>>>>     let _ =
>>>>       let stx1 = A in
>>>>       let stx2 = B A in
>>>>       let stx3 = C (A, A) in
>>>>
>>>>       print_code std_formatter .< stx1 >.;
>>>>       print_code std_formatter .< stx2 >.;
>>>>       print_code std_formatter .< stx3 >.;
>>>>
>>>>       print_closed_code std_formatter (close_code .< stx1 >.);
>>>>       print_closed_code std_formatter (close_code .< stx2 >.);
>>>>       print_closed_code std_formatter (close_code .< stx3 >.);
>>>>     ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc Syntax.ml
>>>> -c
>>>>     ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc
>>>> Syntax.cmo Main.ml -o main
>>>>     ➜  metaocaml_serialization_issue git:(master) ✗ ./main
>>>>     .<(* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>.
>>>> .<
>>>>     (* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>.
>>>>     ➜  metaocaml_serialization_issue git:(master) ✗
>>>>
>>>>
>>>> 2015-05-01 12:53 GMT-04:00 Ömer Sinan Ağacan <omeragacan@gmail.com>:
>>>>>>
>>>>>> You can't serialize `eval_ref` as `eval_ref` because that is a local
>>>>>> identifier. If you print out `eval_ref` into some other ml file and
>>>>>> compiler
>>>>>> it, it is going to give an "Unbound identifier eval_ref" error.
>>>>>
>>>>> That's true. Just to make sure and make the output more clear, I moved
>>>>> the
>>>>> relevant code to another module, and now it's printing this:
>>>>>
>>>>> .<Unlambda.eval_ref (* CSP p' *) []>.
>>>>>
>>>>> My main question is that it should serialize p' here, but it doesn't.
>>>>> I'm
>>>>> trying to understand why.
>>>
>>>
>

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Gabriel Scherer]

[-- Attachment #1: Type: text/plain, Size: 7369 bytes --]

.< x >. represents a piece of code that is just the variable x. Printed to
a string, it is "x". (Regardless of the definition of x, for example (A))

.< ~x >. represents a piece of code that is contained in the value of the
variable x. For example, if the value of x is .< A >. , then the string
representation of .< ~x >. would be "A".

When running code in the same MetaOCaml program, you can use them in
closely related ways: running .< f ~x >. will run the function f with the
value of x, A, but running .< f x >. will also work because x acts here as
a reference to a previous stage, and the value of the previous stage
(because it is "serializable") can be transported to the current stage.
(One can think of this as a runtime reference to a compile-time value or
computation)

When printing the code, they are fundamentally different, because indeed
the variable .< x >. may not mean anything in printed code if it is only
meaningful in the local context. Consider:

  print_code (let x = A in .< x >.)

what could the program print here? "x" would be wrong, because the receiver
of the code has no way to know what "x" means (the declaration of x is at
the previous stage, it is not printed).

On the contrary, with:

  lib.ml:
    let x = A

  main.ml:
    print_code (let open Lib in .< x >.)

there, because x is a reference that *can be named* at the toplevel of an
OCaml program, you can print something, namely "Lib.x".

On Sun, May 3, 2015 at 5:19 AM, Ömer Sinan Ağacan <omeragacan@gmail.com>
wrote:

> > But that's the whole point: you are not persisting a value, you are
> > persisting a local variable.
>
> Can you explain what's wrong with that? I'm making sure that the local
> variables I'm trying to persist are persistable, as in my last example:
>
>   let stx = A in .< stx >.
>
> (Btw, MetaML has explicit `lift` function for this operation, I'm
> wondering if
> there are any differences between MetaML's `lift` and MetaOCaml's implicit
> lifting)
>
> > In the context of staging, a variable and its value are radically
> different,
> > unlike in the traditional functional programming context, where this
> > difference can be safely and harmlessly blurred.
>
> I understand that code values are similar to closures in some ways, for
> example
> they capture some variables, and they're not always serializable. My
> problem
> here is that they're almost never serializable in the case of MetaOCaml. It
> even fails to serialize a code object that just holds a single constructor
> for
> a simple type with one constructor, for example.
>
> (It'd be really great if you could elaborate on how they're "radically
> different")
>
> > If you want to persist "named values", then give globally accessible
> names to
> > these values [which I believe others have already told you].
>
> ...
>
> > If you want to persist values, then use my solution or a variant of that.
>
> Can you explain how is this a "solution"? To rephrase my goal one more
> time: I'm
> generating specialized code in runtime, but instead of running it, I want
> to
> serialize it so that I can 1) inspect the generated code and make sure I'm
> really doing the optimizations I'm expecting to do 2) I can compile my code
> separately. But MetaOCaml is just refusing to show data values in my
> generated
> code.
>
> I started to feel like the story of MetaOCaml serialization as described in
> manual's "Many ways to run the code" section is just wrong.
>
> 2015-05-02 22:28 GMT-04:00 Jacques Carette <carette@mcmaster.ca>:
> > But that's the whole point: you are not persisting a value, you are
> > persisting a local variable.
> >
> > In the context of staging, a variable and its value are radically
> different,
> > unlike in the traditional functional programming context, where this
> > difference can be safely and harmlessly blurred.
> >
> > If you want to persist "named values", then give globally accessible
> names
> > to these values [which I believe others have already told you].  If you
> want
> > to persist values, then use my solution or a variant of that.
> >
> > Jacques
> >
> >
> > On 2015-05-02 9:56 PM, Ömer Sinan Ağacan wrote:
> >>
> >> That's not a solution. I should be able to generate some values in
> >> code generation time and persist them in code values, that's the whole
> >> point here.
> >>
> >> 2015-05-02 16:49 GMT-04:00 Jacques Carette <carette@mcmaster.ca>:
> >>>
> >>> try instead
> >>>     let stx1 = .< A >. in
> >>> and then
> >>>     print_code std_formatter .< .~stx1 >. ;
> >>>
> >>> That ought to work as you wish.
> >>>
> >>> Jacques
> >>>
> >>>
> >>> On 2015-05-02 2:45 PM, Ömer Sinan Ağacan wrote:
> >>>>
> >>>> In case anyone's still interested, I produced a very simple example
> that
> >>>> demonstrates the issue:
> >>>>
> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ ls
> >>>>     Main.ml  Syntax.ml
> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ cat Syntax.ml
> >>>>     type stx =
> >>>>       | A
> >>>>       | B of stx
> >>>>       | C of (stx * stx)
> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ cat Main.ml
> >>>>     open Format
> >>>>     open Print_code
> >>>>     open Runcode
> >>>>     open Syntax
> >>>>
> >>>>     let _ =
> >>>>       let stx1 = A in
> >>>>       let stx2 = B A in
> >>>>       let stx3 = C (A, A) in
> >>>>
> >>>>       print_code std_formatter .< stx1 >.;
> >>>>       print_code std_formatter .< stx2 >.;
> >>>>       print_code std_formatter .< stx3 >.;
> >>>>
> >>>>       print_closed_code std_formatter (close_code .< stx1 >.);
> >>>>       print_closed_code std_formatter (close_code .< stx2 >.);
> >>>>       print_closed_code std_formatter (close_code .< stx3 >.);
> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc
> Syntax.ml
> >>>> -c
> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc
> >>>> Syntax.cmo Main.ml -o main
> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ ./main
> >>>>     .<(* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3
> *)>.
> >>>> .<
> >>>>     (* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3 *)>.
> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗
> >>>>
> >>>>
> >>>> 2015-05-01 12:53 GMT-04:00 Ömer Sinan Ağacan <omeragacan@gmail.com>:
> >>>>>>
> >>>>>> You can't serialize `eval_ref` as `eval_ref` because that is a local
> >>>>>> identifier. If you print out `eval_ref` into some other ml file and
> >>>>>> compiler
> >>>>>> it, it is going to give an "Unbound identifier eval_ref" error.
> >>>>>
> >>>>> That's true. Just to make sure and make the output more clear, I
> moved
> >>>>> the
> >>>>> relevant code to another module, and now it's printing this:
> >>>>>
> >>>>> .<Unlambda.eval_ref (* CSP p' *) []>.
> >>>>>
> >>>>> My main question is that it should serialize p' here, but it doesn't.
> >>>>> I'm
> >>>>> trying to understand why.
> >>>
> >>>
> >
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 10063 bytes --]

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

> .< x >. represents a piece of code that is just the variable x. Printed to a
> string, it is "x". (Regardless of the definition of x, for example (A))

I don't think this is true, as far as I can see MetaOCaml doesn't have open code
values that are only runnable in environments that bind free variables in the
code.

  # .<a>.;;
  Error: Unbound value a
  # let a = 1 in .<a>.;;
  - : int code = .<1>.

In this example `.<a>.` didn't mean a code that is just the variable `a`,
instead, `a` is lifted in code and code is now `.<1>.`.

I think this automatic lifting is why MetaOCaml doesn't have explicit "lift"
operation like MetaML does: variables bound in the environment automatically
lifted, so no open code values are possible.

> When printing the code, they are fundamentally different, because indeed the
> variable .< x >. may not mean anything in printed code if it is only
> meaningful in the local context. Consider:
>
>   print_code (let x = A in .< x >.)
>
> what could the program print here? "x" would be wrong, because the receiver
> of the code has no way to know what "x" means (the declaration of x is at
> the previous stage, it is not printed).

That's exactly what I'm trying to ask here... People are trying to explain that
this should be printed as "A":

  print_code (let x = .< A >. in .< .~ x >.);

But this shouldn't:

  print_code (let x = A in .< x >.);

This doesn't make sense at all. In your example you just said that "x would be
wrong, because receiver of the code has no way to know what x means". Here the
code works like a closure, it captures `x`, but when it comes to printing, it
just prints `CSP value x` instead of printing the value of it, which is
completely serializable, indeed it serializes it in the first case.

Also, note that if the code is only printed in the first case, then it's
impossible to print anything. Because at some point I have to lift some values
in my code values, but then I'm losing the ability to print anyway. Example:

  let stx = B A in
  let c   = .< stc >. in
  print_code .< .~ c >.;;

Prints .< (* CSP stx *) >. . So once you lift something to code values, you lose
the ability to print.

2015-05-03 4:40 GMT-04:00 Gabriel Scherer <gabriel.scherer@gmail.com>:
> .< x >. represents a piece of code that is just the variable x. Printed to a
> string, it is "x". (Regardless of the definition of x, for example (A))
>
> .< ~x >. represents a piece of code that is contained in the value of the
> variable x. For example, if the value of x is .< A >. , then the string
> representation of .< ~x >. would be "A".
>
> When running code in the same MetaOCaml program, you can use them in closely
> related ways: running .< f ~x >. will run the function f with the value of
> x, A, but running .< f x >. will also work because x acts here as a
> reference to a previous stage, and the value of the previous stage (because
> it is "serializable") can be transported to the current stage. (One can
> think of this as a runtime reference to a compile-time value or computation)
>
> When printing the code, they are fundamentally different, because indeed the
> variable .< x >. may not mean anything in printed code if it is only
> meaningful in the local context. Consider:
>
>   print_code (let x = A in .< x >.)
>
> what could the program print here? "x" would be wrong, because the receiver
> of the code has no way to know what "x" means (the declaration of x is at
> the previous stage, it is not printed).
>
> On the contrary, with:
>
>   lib.ml:
>     let x = A
>
>   main.ml:
>     print_code (let open Lib in .< x >.)
>
> there, because x is a reference that *can be named* at the toplevel of an
> OCaml program, you can print something, namely "Lib.x".
>
> On Sun, May 3, 2015 at 5:19 AM, Ömer Sinan Ağacan <omeragacan@gmail.com>
> wrote:
>>
>> > But that's the whole point: you are not persisting a value, you are
>> > persisting a local variable.
>>
>> Can you explain what's wrong with that? I'm making sure that the local
>> variables I'm trying to persist are persistable, as in my last example:
>>
>>   let stx = A in .< stx >.
>>
>> (Btw, MetaML has explicit `lift` function for this operation, I'm
>> wondering if
>> there are any differences between MetaML's `lift` and MetaOCaml's implicit
>> lifting)
>>
>> > In the context of staging, a variable and its value are radically
>> > different,
>> > unlike in the traditional functional programming context, where this
>> > difference can be safely and harmlessly blurred.
>>
>> I understand that code values are similar to closures in some ways, for
>> example
>> they capture some variables, and they're not always serializable. My
>> problem
>> here is that they're almost never serializable in the case of MetaOCaml.
>> It
>> even fails to serialize a code object that just holds a single constructor
>> for
>> a simple type with one constructor, for example.
>>
>> (It'd be really great if you could elaborate on how they're "radically
>> different")
>>
>> > If you want to persist "named values", then give globally accessible
>> > names to
>> > these values [which I believe others have already told you].
>>
>> ...
>>
>> > If you want to persist values, then use my solution or a variant of
>> > that.
>>
>> Can you explain how is this a "solution"? To rephrase my goal one more
>> time: I'm
>> generating specialized code in runtime, but instead of running it, I want
>> to
>> serialize it so that I can 1) inspect the generated code and make sure I'm
>> really doing the optimizations I'm expecting to do 2) I can compile my
>> code
>> separately. But MetaOCaml is just refusing to show data values in my
>> generated
>> code.
>>
>> I started to feel like the story of MetaOCaml serialization as described
>> in
>> manual's "Many ways to run the code" section is just wrong.
>>
>> 2015-05-02 22:28 GMT-04:00 Jacques Carette <carette@mcmaster.ca>:
>> > But that's the whole point: you are not persisting a value, you are
>> > persisting a local variable.
>> >
>> > In the context of staging, a variable and its value are radically
>> > different,
>> > unlike in the traditional functional programming context, where this
>> > difference can be safely and harmlessly blurred.
>> >
>> > If you want to persist "named values", then give globally accessible
>> > names
>> > to these values [which I believe others have already told you].  If you
>> > want
>> > to persist values, then use my solution or a variant of that.
>> >
>> > Jacques
>> >
>> >
>> > On 2015-05-02 9:56 PM, Ömer Sinan Ağacan wrote:
>> >>
>> >> That's not a solution. I should be able to generate some values in
>> >> code generation time and persist them in code values, that's the whole
>> >> point here.
>> >>
>> >> 2015-05-02 16:49 GMT-04:00 Jacques Carette <carette@mcmaster.ca>:
>> >>>
>> >>> try instead
>> >>>     let stx1 = .< A >. in
>> >>> and then
>> >>>     print_code std_formatter .< .~stx1 >. ;
>> >>>
>> >>> That ought to work as you wish.
>> >>>
>> >>> Jacques
>> >>>
>> >>>
>> >>> On 2015-05-02 2:45 PM, Ömer Sinan Ağacan wrote:
>> >>>>
>> >>>> In case anyone's still interested, I produced a very simple example
>> >>>> that
>> >>>> demonstrates the issue:
>> >>>>
>> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ ls
>> >>>>     Main.ml  Syntax.ml
>> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ cat Syntax.ml
>> >>>>     type stx =
>> >>>>       | A
>> >>>>       | B of stx
>> >>>>       | C of (stx * stx)
>> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ cat Main.ml
>> >>>>     open Format
>> >>>>     open Print_code
>> >>>>     open Runcode
>> >>>>     open Syntax
>> >>>>
>> >>>>     let _ =
>> >>>>       let stx1 = A in
>> >>>>       let stx2 = B A in
>> >>>>       let stx3 = C (A, A) in
>> >>>>
>> >>>>       print_code std_formatter .< stx1 >.;
>> >>>>       print_code std_formatter .< stx2 >.;
>> >>>>       print_code std_formatter .< stx3 >.;
>> >>>>
>> >>>>       print_closed_code std_formatter (close_code .< stx1 >.);
>> >>>>       print_closed_code std_formatter (close_code .< stx2 >.);
>> >>>>       print_closed_code std_formatter (close_code .< stx3 >.);
>> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc
>> >>>> Syntax.ml
>> >>>> -c
>> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ metaocamlc
>> >>>> Syntax.cmo Main.ml -o main
>> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗ ./main
>> >>>>     .<(* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3
>> >>>> *)>.
>> >>>> .<
>> >>>>     (* CSP stx1 *) Obj.magic 0>. .<(* CSP stx2 *)>. .<(* CSP stx3
>> >>>> *)>.
>> >>>>     ➜  metaocaml_serialization_issue git:(master) ✗
>> >>>>
>> >>>>
>> >>>> 2015-05-01 12:53 GMT-04:00 Ömer Sinan Ağacan <omeragacan@gmail.com>:
>> >>>>>>
>> >>>>>> You can't serialize `eval_ref` as `eval_ref` because that is a
>> >>>>>> local
>> >>>>>> identifier. If you print out `eval_ref` into some other ml file and
>> >>>>>> compiler
>> >>>>>> it, it is going to give an "Unbound identifier eval_ref" error.
>> >>>>>
>> >>>>> That's true. Just to make sure and make the output more clear, I
>> >>>>> moved
>> >>>>> the
>> >>>>> relevant code to another module, and now it's printing this:
>> >>>>>
>> >>>>> .<Unlambda.eval_ref (* CSP p' *) []>.
>> >>>>>
>> >>>>> My main question is that it should serialize p' here, but it
>> >>>>> doesn't.
>> >>>>> I'm
>> >>>>> trying to understand why.
>> >>>
>> >>>
>> >
>>
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Leo White]

> I don't think this is true, as far as I can see MetaOCaml doesn't have open code
> values that are only runnable in environments that bind free variables in the
> code.
> 
>   # .<a>.;;
>   Error: Unbound value a
>   # let a = 1 in .<a>.;;
>   - : int code = .<1>.
> 
> In this example `.<a>.` didn't mean a code that is just the variable `a`,
> instead, `a` is lifted in code and code is now `.<1>.`.
> 
> I think this automatic lifting is why MetaOCaml doesn't have explicit "lift"
> operation like MetaML does: variables bound in the environment automatically
> lifted, so no open code values are possible.

It is true that MetaOCaml tries to auto-magically lift *some* things for you
(e.g. ints), as in this example. If type information is available, then for known
types it can produce a call to a function such as `Trx.lift_constant_int` to do the
lifting. For other types it will examine the value at run-time and attempt to lift it.
For example, if the value has a "string tag" then it can be represented by a string
literal. If this fails then it will fall back to using true CSP -- which can be run but
cannot be printed.

Personally, I think it is best to avoid this auto-magic and just do your lifting by
hand. The type-based auto-magic doesn't work with polymorphism, whilst the
value-based auto-magic is not parametric.

Regards,

Leo

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Ömer Sinan Ağacan]

> Personally, I think it is best to avoid this auto-magic and just do your
> lifting by hand. The type-based auto-magic doesn't work with polymorphism,
> whilst the value-based auto-magic is not parametric.

Hm, I didn't know this was possible. I just tried and it seems to work really
well:

  type stx =
    | A
    | B of stx
    | C of (stx * stx)

  let rec lift_stx (stx : stx) : stx code =
    match stx with
    | A              -> .< A >.
    | B stx'         -> .< B .~ (lift_stx stx') >.
    | C (stx1, stx2) -> .< C ( .~ (lift_stx stx1), .~ (lift_stx stx2) ) >.

  let stx = C (C (C (A, A), C (A, A)), C (C (A, A), C (A, A))) in
  print_code std_formatter .< .~ (lift_stx stx) >.;

So it seems like I finally have a workaround now, thanks!

The story of MetaOCaml code printing is still not completely clear to me, and
this workaround means lots of boilerplace, but at least I have a workaround now
and I can make progress.

2015-05-03 11:24 GMT-04:00 Leo White <leo@lpw25.net>:
>> I don't think this is true, as far as I can see MetaOCaml doesn't have open code
>> values that are only runnable in environments that bind free variables in the
>> code.
>>
>>   # .<a>.;;
>>   Error: Unbound value a
>>   # let a = 1 in .<a>.;;
>>   - : int code = .<1>.
>>
>> In this example `.<a>.` didn't mean a code that is just the variable `a`,
>> instead, `a` is lifted in code and code is now `.<1>.`.
>>
>> I think this automatic lifting is why MetaOCaml doesn't have explicit "lift"
>> operation like MetaML does: variables bound in the environment automatically
>> lifted, so no open code values are possible.
>
> It is true that MetaOCaml tries to auto-magically lift *some* things for you
> (e.g. ints), as in this example. If type information is available, then for known
> types it can produce a call to a function such as `Trx.lift_constant_int` to do the
> lifting. For other types it will examine the value at run-time and attempt to lift it.
> For example, if the value has a "string tag" then it can be represented by a string
> literal. If this fails then it will fall back to using true CSP -- which can be run but
> cannot be printed.
>
> Personally, I think it is best to avoid this auto-magic and just do your lifting by
> hand. The type-based auto-magic doesn't work with polymorphism, whilst the
> value-based auto-magic is not parametric.
>
> Regards,
>
> Leo
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Problems with printing MetaOCaml generated code

[oleg]

Leo has given an excellent explanation. I merely wish to stress two
subtle points, which are worth repeating.

If we restrict ourselves to two stages for the sake of explanation, a
cross-stage persistent identifier is the one that is bound outside the
brackets but used within. What will appear in the generated code
depends on whether that identifier is ``global'' or not.  If it is
global, the name of that identifier will appear. If it is local,
MetaOCaml will try to lift the corresponding value one way or
another.

I should stress the special meaning of the ``global identifier''. In
the discussions about CSP, a global identifier is a top-level
identifier *in a different compilation unit*. Emphatically, the
top-level identifier in the current compilation unit is *not*
considered a global identifier.  Therefore, if you write

let foo = fun x -> x
.<foo 1>.
you will see the CSP 
- : int code = .<(* CSP foo *) 1>. 

rather than the expected .<foo 1>..

The reason top-level identifiers in the current module are not
considered global is because they may be re-defined -- even at a
different type. For example, the above program may continue with
        let foo = true
If the generated code were .<foo 1>., which particular foo it would
refer to? BTW, the global references generated by OCaml proper (which
one may refer too) do not have disambiguating timestamps. If you
examine the object file generated by OCaml, you will see just 'foo'.

MetaOCaml-specific processing is done definition-by-definition. When
MetaOCaml typechecker sees 'foo', it has no knowledge if that identifier
will be later redefined.


Suppose the identifier x is not global, and and it appears within
brackets, for example

        .<fun u ->  x>.
You can read this expression as if it were

        .<fun u -> .~(lift_it x)>.

where 'lift_it' is one of the lifting function that MetaOCaml chooses
for you. For example, if x is of the type int, then lift_it is essentially
string_of_int. If x is not one of the few types MetaOCaml knows a
priori how to lift (or if x is polymorphic and so its exact type
cannot be determined), then lift_it is chosen to be a dynquote, a
function that examines the run-time representation of the value and
tries to do something intelligent. Since the type information is
erased by that time (or was not available to start with, if x was
polymorphic), we cannot distinguish the integer 0 from None from []
or from the first 0-ary constructor of any data type. All of them are
represented internally as 0. That's why you see Obj.magic 0 in the
generated code.

Of course MetaOCaml serialization can be improved. What I'd like to
stress is that you don't have to wait for the improvement. You can
always, instead of
        .<fun u ->  x>.
write
        .<fun u -> .~(mylift x)>.
where
        mylift : t -> t code
is *your* function that does whatever _you_ like it to do at that
particular type t (it should still produce something of the type (t
code)).

If some particular mylift functions turn out popular, they can be
added to MetaOCaml, to save everyone trouble writing them.

And I generally agree with Leo that this implicit lifting is
baroque. At present I'm not sure if requiring the explicit lifting is
too much of a burden. I'm sure that with modular implicits, it won't
be.

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Jeremy Yallop]

On 6 May 2015 at 10:50,  <oleg@okmij.org> wrote:
> Of course MetaOCaml serialization can be improved. What I'd like to
> stress is that you don't have to wait for the improvement. You can
> always, instead of
>         .<fun u ->  x>.
> write
>         .<fun u -> .~(mylift x)>.
> where
>         mylift : t -> t code
> is *your* function that does whatever _you_ like it to do at that
> particular type t (it should still produce something of the type (t
> code)).
>
> If some particular mylift functions turn out popular, they can be
> added to MetaOCaml, to save everyone trouble writing them.
>
> And I generally agree with Leo that this implicit lifting is
> baroque. At present I'm not sure if requiring the explicit lifting is
> too much of a burden. I'm sure that with modular implicits, it won't
> be.

I've just pushed an OPAM switch for an OCaml compiler that combines
the MetaOCaml and modular implicits patches, making it possible to
experiment with explicit user-defined polymorphic CSP.

For example, you might define a signature, CSP, for "things that can
be persisted":

  module type CSP =
   sig
     type t
     val lift : t -> t code
  end

together with a top-level function that dispatches to the appropriate instance

  let csp (implicit C: CSP) (x : C.t) = C.lift x

and instances of CSP for each type of interest.  Here's an instance
for the stx type from earlier in the thread:

  implicit module CSP_stx : CSP with type t = stx =
  struct
    type t = stx

    let rec lift : stx -> stx code = function
      | A -> .< A >.
      | B s -> .< B .~ (lift s) >.
      | C (s1, s2) -> .< C ( .~(lift s1), .~(lift s2) ) >.
   end

and here's a parameterised instance for lists that makes it possible
to persist lists of any persistable element type:

  implicit functor CSP_list(C: CSP) : CSP with type t = C.t list =
  struct
    type t = C.t list

    let rec lift : C.t list -> C.t list code = function
        [] -> .< [] >.
      | x :: xs -> .< .~(csp x) :: .~(lift xs) >.
  end

These two instances make it possible to use the CSP function to
persist stx values, or lists of stx values, or lists of lists of stx
values (etc.):

   # let ba = B A in .< .~(csp ba) >.;;
   - : stx code = .<Stx.B Stx.A>.

  # let l = [A; B A] in .< .~(csp l) >.;;
  - : stx list code = .<[Stx.A; Stx.B Stx.A]>.

  # let ll = [[A; B A]] and ba = B A in .< .~(csp ll), .~(csp ba) >.;;
  - : (stx list list * stx) code = .<([[Stx.A; Stx.B Stx.A]], (Stx.B Stx.A))>.

It's easy to imagine having the csp function built in to MetaOCaml, so
that we could write .< x >.  (or some similarly convenient syntax) to
mean .< .~(csp x) >...

You can try out the switch with OPAM in the usual way:

   opam update
   opam switch 4.02.1+modular-implicits-ber
   eval `opam config env`

Jeremy.

Re: [Caml-list] Problems with printing MetaOCaml generated code

[Yotam Barnoy]

[-- Attachment #1: Type: text/plain, Size: 3358 bytes --]

Mind blown.

Let's just call this branch... hmm, I dunno, how about 'OCaml 5'?

Sorry for the noise.

On Wed, May 6, 2015 at 11:58 AM, Jeremy Yallop <yallop@gmail.com> wrote:

> On 6 May 2015 at 10:50,  <oleg@okmij.org> wrote:
> > Of course MetaOCaml serialization can be improved. What I'd like to
> > stress is that you don't have to wait for the improvement. You can
> > always, instead of
> >         .<fun u ->  x>.
> > write
> >         .<fun u -> .~(mylift x)>.
> > where
> >         mylift : t -> t code
> > is *your* function that does whatever _you_ like it to do at that
> > particular type t (it should still produce something of the type (t
> > code)).
> >
> > If some particular mylift functions turn out popular, they can be
> > added to MetaOCaml, to save everyone trouble writing them.
> >
> > And I generally agree with Leo that this implicit lifting is
> > baroque. At present I'm not sure if requiring the explicit lifting is
> > too much of a burden. I'm sure that with modular implicits, it won't
> > be.
>
> I've just pushed an OPAM switch for an OCaml compiler that combines
> the MetaOCaml and modular implicits patches, making it possible to
> experiment with explicit user-defined polymorphic CSP.
>
> For example, you might define a signature, CSP, for "things that can
> be persisted":
>
>   module type CSP =
>    sig
>      type t
>      val lift : t -> t code
>   end
>
> together with a top-level function that dispatches to the appropriate
> instance
>
>   let csp (implicit C: CSP) (x : C.t) = C.lift x
>
> and instances of CSP for each type of interest.  Here's an instance
> for the stx type from earlier in the thread:
>
>   implicit module CSP_stx : CSP with type t = stx =
>   struct
>     type t = stx
>
>     let rec lift : stx -> stx code = function
>       | A -> .< A >.
>       | B s -> .< B .~ (lift s) >.
>       | C (s1, s2) -> .< C ( .~(lift s1), .~(lift s2) ) >.
>    end
>
> and here's a parameterised instance for lists that makes it possible
> to persist lists of any persistable element type:
>
>   implicit functor CSP_list(C: CSP) : CSP with type t = C.t list =
>   struct
>     type t = C.t list
>
>     let rec lift : C.t list -> C.t list code = function
>         [] -> .< [] >.
>       | x :: xs -> .< .~(csp x) :: .~(lift xs) >.
>   end
>
> These two instances make it possible to use the CSP function to
> persist stx values, or lists of stx values, or lists of lists of stx
> values (etc.):
>
>    # let ba = B A in .< .~(csp ba) >.;;
>    - : stx code = .<Stx.B Stx.A>.
>
>   # let l = [A; B A] in .< .~(csp l) >.;;
>   - : stx list code = .<[Stx.A; Stx.B Stx.A]>.
>
>   # let ll = [[A; B A]] and ba = B A in .< .~(csp ll), .~(csp ba) >.;;
>   - : (stx list list * stx) code = .<([[Stx.A; Stx.B Stx.A]], (Stx.B
> Stx.A))>.
>
> It's easy to imagine having the csp function built in to MetaOCaml, so
> that we could write .< x >.  (or some similarly convenient syntax) to
> mean .< .~(csp x) >...
>
> You can try out the switch with OPAM in the usual way:
>
>    opam update
>    opam switch 4.02.1+modular-implicits-ber
>    eval `opam config env`
>
> Jeremy.
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 4713 bytes --]