[Caml-list] camlp4 for js_of_ocaml toplevels

[Caml-list] camlp4 for js_of_ocaml toplevels

[Andy Ray]

Hi,

Is anyone working on making camlp4 extensions available in a
js_of_ocaml toplevel?

I have seen the work the OCamlPro guys have done to get pa_js going by
hacking the compiler itself - serious kudos for that!

Is there a reason why that's easier than js_of_ocaml compiling a
(static) version of camlp4?

Will a ppx future help in this regard?  I am particularly interested
in the pa_js, lwt, tyxml and cow extensions or something similar.

Cheers,
Andy

(Apologies if this message turns up twice...I've got far too many
email accounts these days)

Re: [Caml-list] camlp4 for js_of_ocaml toplevels

[Anil Madhavapeddy]

On 22 Feb 2014, at 03:07, Andy Ray <andy.ray@ujamjar.com> wrote:

> Hi,
> 
> Is anyone working on making camlp4 extensions available in a
> js_of_ocaml toplevel?
> 
> I have seen the work the OCamlPro guys have done to get pa_js going by
> hacking the compiler itself - serious kudos for that!
> 
> Is there a reason why that's easier than js_of_ocaml compiling a
> (static) version of camlp4?

Leo White put together a custom compiler frontend for just this use
case a while back:

https://github.com/lpw25/ocaml-with-pp

The idea is that you build this with a static set of camlp4 extensions,
and it runs the input through camlp4 and then passes the AST directly
through to the compiler (via compiler-libs). 

It should be reasonably easy to adapt this to a toplevel model
as well -- let us know if you need a hand.

-anil

Re: [Caml-list] camlp4 for js_of_ocaml toplevels

[Andy Ray]

>> Is anyone working on making camlp4 extensions available in a
>> js_of_ocaml toplevel?
>>
>> I have seen the work the OCamlPro guys have done to get pa_js going by
>> hacking the compiler itself - serious kudos for that!
>>
>> Is there a reason why that's easier than js_of_ocaml compiling a
>> (static) version of camlp4?
>
> Leo White put together a custom compiler frontend for just this use
> case a while back:
>
> https://github.com/lpw25/ocaml-with-pp
>
> The idea is that you build this with a static set of camlp4 extensions,
> and it runs the input through camlp4 and then passes the AST directly
> through to the compiler (via compiler-libs).
>
> It should be reasonably easy to adapt this to a toplevel model
> as well -- let us know if you need a hand.


A good amount of hacking and I have some camlp4 code running via js_of_ocaml.

I basically took the compilation process of camlp4orf, added pa_js.cmo
and slightly modified the driver program so it took a static command
line of the form `-str "let a = b##c"`.  Also needed a bit of hacking
around temp file generation.

My next step is to try and figure out a way to get camlp4 to transform
"string -> string" rather than "hacked js_of_ocaml psuedo file ->
hacked console log in browser".  Can't say I ever really wanted to
know much about the insides of camlp4, but there you go....

One thing I noticed was when I (accidentally) had unix_isatty
returning false, and presumably camlp4 attempting to marshal a
compiler ast the js_of_ocaml runtime would die with a JSON error.  I
presume this is an issue with js_of_ocaml's marshalling code?

Cheers,
Andy

Re: [Caml-list] camlp4 for js_of_ocaml toplevels

[Anil Madhavapeddy]

On Mon, Feb 24, 2014 at 05:55:03PM +0000, Andy Ray wrote:
> >> Is anyone working on making camlp4 extensions available in a
> >> js_of_ocaml toplevel?
> >>
> >> I have seen the work the OCamlPro guys have done to get pa_js going by
> >> hacking the compiler itself - serious kudos for that!
> >>
> >> Is there a reason why that's easier than js_of_ocaml compiling a
> >> (static) version of camlp4?
> >
> > Leo White put together a custom compiler frontend for just this use
> > case a while back:
> >
> > https://github.com/lpw25/ocaml-with-pp
> >
> > The idea is that you build this with a static set of camlp4 extensions,
> > and it runs the input through camlp4 and then passes the AST directly
> > through to the compiler (via compiler-libs).
> >
> > It should be reasonably easy to adapt this to a toplevel model
> > as well -- let us know if you need a hand.
> 
> 
> A good amount of hacking and I have some camlp4 code running via js_of_ocaml.

Fantastic!

> I basically took the compilation process of camlp4orf, added pa_js.cmo
> and slightly modified the driver program so it took a static command
> line of the form `-str "let a = b##c"`.  Also needed a bit of hacking
> around temp file generation.
> 
> My next step is to try and figure out a way to get camlp4 to transform
> "string -> string" rather than "hacked js_of_ocaml psuedo file ->
> hacked console log in browser".  Can't say I ever really wanted to
> know much about the insides of camlp4, but there you go....
> 
> One thing I noticed was when I (accidentally) had unix_isatty
> returning false, and presumably camlp4 attempting to marshal a
> compiler ast the js_of_ocaml runtime would die with a JSON error.  I
> presume this is an issue with js_of_ocaml's marshalling code?

Sounds like it -- camlp4 in non-tty mode would just be using Marshal to
send the data.  It looks like js_of_ocaml has a hand-written Marshall
module that mimics the standard OCaml wire format:
https://github.com/ocsigen/js_of_ocaml/blob/master/runtime/marshall.js

I'm not really sure why a bug here would result in a JSON error without
looking into it more deeply though...

-anil

Re: [Caml-list] camlp4 for js_of_ocaml toplevels

[Olivier Levillain]

Hi,

> One thing I noticed was when I (accidentally) had unix_isatty
> returning false, and presumably camlp4 attempting to marshal a
> compiler ast the js_of_ocaml runtime would die with a JSON error.  I
> presume this is an issue with js_of_ocaml's marshalling code?

I think I recently stumbled on this one: the default camlp4 printer is
"auto", which prints text when stdout is a tty and binary AST when it is
not. To force the text or AST setting, you should use the -printer
option of camlp4, with either "Camlp4OCamlPrinter" (text) or
"Camlp4OCamlAstDumper" parameter.

The name of the available printers correspond to files in the
/usr/lib/ocaml/camlp4/Camlp4Printers (on Debian).

Sorry for the noise if I'm wrong.

Regards,
Olivier Levillain