[Caml-list] js_of_ocaml with node

[Caml-list] js_of_ocaml with node

[Helmut Brandl]

Hello all,

I am currently writing a software package in ocaml and I want to be able 
to compile it from ocaml to javascript to be able to run it under 
nodejs. I have found js_of_ocaml which seems to do a rather good job in 
compiling from bytecode to javascript.

I achieved the compilation to javascript which can run under nodejs as 
long as I don't use anything from the unix library. Unfortunately I need 
two functions from the unix library: 1. Get the last modification time 
of a file. 2. Create a directory. I cannot find anything equivalent in 
the standard library.

I tried to find some hints in the js_of_ocaml documentation but I 
haven't succeeded because to documentation is hard to read.

Can anybody give me some hints on how to get these two functions in a 
way that js_of_ocaml does the correct compilation?

Thanks in advance

Regards
Helmut

Re: [Caml-list] js_of_ocaml with node

[Sébastien Dailly]

Le 27/04/2015 22:32, Helmut Brandl a écrit :

> I achieved the compilation to javascript which can run under nodejs as
> long as I don't use anything from the unix library. Unfortunately I need
> two functions from the unix library: 1. Get the last modification time
> of a file. 2. Create a directory. I cannot find anything equivalent in
> the standard library.

This a javascript limitation, not a js_of_ocaml one. I think (and hope) 
there is no such possibilities with standard javascript to create 
directories on the client fs. (and this would be a great security flaw 
if browsers would allow that).

Regards,

-- 
Sébastien Dailly

Re: [Caml-list] js_of_ocaml with node

[Daniel Bünzli]

Le lundi, 27 avril 2015 à 22:32, Helmut Brandl a écrit :
> Can anybody give me some hints on how to get these two functions in a
> way that js_of_ocaml does the correct compilation?

If you are getting unknown primitive errors at link time then you can simply try to implement them yourself using node's filesystem APIs see:

http://ocsigen.org/js_of_ocaml/2.5/manual/linker

If you are not concerned about using Unix directly (because node.js is your only target) and simply need these functions you can bind the corresponding node API functions directly:

http://ocsigen.org/js_of_ocaml/2.5/manual/bindings

Best,

Daniel

Re: [Caml-list] js_of_ocaml with node

[Helmut Brandl]

On 04/27/2015 04:02 PM, Sébastien Dailly wrote:
> Le 27/04/2015 22:32, Helmut Brandl a écrit :
>
>> I achieved the compilation to javascript which can run under nodejs as
>> long as I don't use anything from the unix library. Unfortunately I need
>> two functions from the unix library: 1. Get the last modification time
>> of a file. 2. Create a directory. I cannot find anything equivalent in
>> the standard library.
>
> This a javascript limitation, not a js_of_ocaml one. I think (and 
> hope) there is no such possibilities with standard javascript to 
> create directories on the client fs. (and this would be a great 
> security flaw if browsers would allow that).
Maybe I have been misunderstood. I am not trying to run my code in a 
browser and access the client fs from the browers. My software package 
is compiled to bytecode of native using the ocam compiler. All I want to 
do is to create a javascript version and run it under nodejs (which runs 
on the client machine) to provide user which don't have an installed 
ocaml compiler suite on their machine to run my software package.

Note that nodejs has full access to the filesystem like any other 
program written in any programming language like C, ocaml, java, scala,...


>
> Regards,
>

Re: [Caml-list] js_of_ocaml with node

[Alain Frisch]

On 27/04/2015 22:32, Helmut Brandl wrote:
> Hello all,
>
> I am currently writing a software package in ocaml and I want to be able
> to compile it from ocaml to javascript to be able to run it under
> nodejs. I have found js_of_ocaml which seems to do a rather good job in
> compiling from bytecode to javascript.
>
> I achieved the compilation to javascript which can run under nodejs as
> long as I don't use anything from the unix library. Unfortunately I need
> two functions from the unix library: 1. Get the last modification time
> of a file. 2. Create a directory. I cannot find anything equivalent in
> the standard library.

You need to define bindings to node.js' "fs" API:

https://nodejs.org/api/fs.html

It should be possible to implement a subset of the Unix module through 
this API, or just to expose the few functions you would need.

For instance, with the gen_js_api I announced earlier today, binding the 
synchronous mkdir function would look like:

  val mkdir: string -> int -> unit
    [@@js.global "fs.mkdirAsync"]


Alain

Re: [Caml-list] js_of_ocaml with node

[Daniel Bünzli]

Le mardi, 28 avril 2015 à 00:03, Alain Frisch a écrit :
> val mkdir: string -> int -> unit
> [@@js.global "fs.mkdirAsync"]

That should be "fs.mkdirSync". Just curious, how would the binding to the async fs.mkdir look like ?  

Daniel

Re: [Caml-list] js_of_ocaml with node

[Sébastien Dailly]

Le 2015-04-27 23:16, Helmut Brandl a écrit :

>> This a javascript limitation, not a js_of_ocaml one. I think (and 
>> hope) there is no such possibilities with standard javascript to 
>> create directories on the client fs. (and this would be a great 
>> security flaw if browsers would allow that).
> Maybe I have been misunderstood. I am not trying to run my code in a
> browser and access the client fs from the browers. My software package
> is compiled to bytecode of native using the ocam compiler. All I want
> to do is to create a javascript version and run it under nodejs (which
> runs on the client machine) to provide user which don't have an
> installed ocaml compiler suite on their machine to run my software
> package.
> 
> Note that nodejs has full access to the filesystem like any other
> program written in any programming language like C, ocaml, java,
> scala,...
> 

You'r right. I was only thinking of client side javascript.

Sorry for the noise.

-- 
Sébastien Dailly

Re: [Caml-list] js_of_ocaml with node

[Alain Frisch]

On 04/28/2015 12:41 AM, Daniel Bünzli wrote:
> Le mardi, 28 avril 2015 à 00:03, Alain Frisch a écrit :
>> val mkdir: string -> int -> unit
>> [@@js.global "fs.mkdirAsync"]
>
> That should be "fs.mkdirSync".

Indeed!

> Just curious, how would the binding to the async fs.mkdir look like ?

Something like:

val mkdir_async: string -> int -> (js_exn option -> unit) -> unit
   [@@js.global "fs.mkdir"]

assuming some predefined bindings for js_exn.  (One could also decide to 
bind JS exception to OCaml's exn type, but the gain is not clear.)

There are other possible variations, such as making the mode optional:

val mkdir_async:
     string ->
     ?mode:(int[@js.default 0o777]) ->
     (js_exn option -> unit) ->
     unit
   [@@js.global "fs.mkdir"]


Here js.default forces the default value for the optional mode argument. 
  Without it, a missing value will send an `undefined` as the second 
argument (I don't know if fs.mkdir would be happy with it).  This is 
currently not supported, but one could also extend the tool to allow:

val mkdir_async:
     string ->
     ?mode:(int[@js.drop_if_missing]) ->
     (js_exn option -> unit) -> unit
   [@@js.global "fs.mkdir"]

so that the second argument is dropped in the JS call if not provided 
(but I don't see how to support that for OCaml callbacks in full 
generality, since the function needs to check the type of arguments to 
decide how to interpret them).


Alain

Re: [Caml-list] js_of_ocaml with node

[Alain Frisch]

On 04/28/2015 09:53 AM, Alain Frisch wrote:
> val mkdir_async:
>      string ->
>      ?mode:(int[@js.default 0o777]) ->
>      (js_exn option -> unit) ->
>      unit
>    [@@js.global "fs.mkdir"]
>
>
> Here js.default forces the default value for the optional mode argument.
>   Without it, a missing value will send an `undefined` as the second
> argument (I don't know if fs.mkdir would be happy with it).

Just to clarify:  the issue here is that node.js documents the function as:

  fs.mkdir(path[, mode], callback)


and its implementation is more liberal:

=================
function modeNum(m, def) {
   if (util.isNumber(m))
     return m;
   if (util.isString(m))
     return parseInt(m, 8);
   if (def)
     return modeNum(def);
   return undefined;
}

fs.mkdir = function(path, mode, callback) {
   if (util.isFunction(mode)) callback = mode;
   ...
       modeNum(mode, 511 /*=0777*/)
   ...
};
=================

(welcome to Javascript wonderful calling conventions!)


So both conventions are supported to omit the mode argument:

   fs.mkdir(path, undefined, callback)
   fs.mkdir(path, callback)


(one could actually pass anything as the second argument as long as it 
is not a number, not a string and not a function).

So it is ok to bind it simply as:

val mkdir_async: string -> ?mode:int -> (js_exn option -> unit) ->  unit
   [@@js.global "fs.mkdir"]



Alain

Re: [Caml-list] js_of_ocaml with node

[Drup]

You can go one step further here and expose lwt threads instead. It 
could be done automatically too, since transforming callbacks into lwt 
thread is always done the same way.

Le 28/04/2015 09:53, Alain Frisch a écrit :
> On 04/28/2015 12:41 AM, Daniel Bünzli wrote:
>> Le mardi, 28 avril 2015 à 00:03, Alain Frisch a écrit :
>>> val mkdir: string -> int -> unit
>>> [@@js.global "fs.mkdirAsync"]
>>
>> That should be "fs.mkdirSync".
>
> Indeed!
>
>> Just curious, how would the binding to the async fs.mkdir look like ?
>
> Something like:
>
> val mkdir_async: string -> int -> (js_exn option -> unit) -> unit
>   [@@js.global "fs.mkdir"]
>
> assuming some predefined bindings for js_exn.  (One could also decide 
> to bind JS exception to OCaml's exn type, but the gain is not clear.)
>
> There are other possible variations, such as making the mode optional:
>
> val mkdir_async:
>     string ->
>     ?mode:(int[@js.default 0o777]) ->
>     (js_exn option -> unit) ->
>     unit
>   [@@js.global "fs.mkdir"]
>
>
> Here js.default forces the default value for the optional mode 
> argument.  Without it, a missing value will send an `undefined` as the 
> second argument (I don't know if fs.mkdir would be happy with it).  
> This is currently not supported, but one could also extend the tool to 
> allow:
>
> val mkdir_async:
>     string ->
>     ?mode:(int[@js.drop_if_missing]) ->
>     (js_exn option -> unit) -> unit
>   [@@js.global "fs.mkdir"]
>
> so that the second argument is dropped in the JS call if not provided 
> (but I don't see how to support that for OCaml callbacks in full 
> generality, since the function needs to check the type of arguments to 
> decide how to interpret them).
>
>
> Alain
>

Re: [Caml-list] js_of_ocaml with node

[Helmut Brandl]

Thanks to all for the hints.

I think I have to dig into the documentation of js_of_ocaml to resolve 
the problem.

Regards
Helmut

On 04/28/2015 06:17 AM, Drup wrote:
> You can go one step further here and expose lwt threads instead. It 
> could be done automatically too, since transforming callbacks into lwt 
> thread is always done the same way.
>
> Le 28/04/2015 09:53, Alain Frisch a écrit :
>> On 04/28/2015 12:41 AM, Daniel Bünzli wrote:
>>> Le mardi, 28 avril 2015 à 00:03, Alain Frisch a écrit :
>>>> val mkdir: string -> int -> unit
>>>> [@@js.global "fs.mkdirAsync"]
>>>
>>> That should be "fs.mkdirSync".
>>
>> Indeed!
>>
>>> Just curious, how would the binding to the async fs.mkdir look like ?
>>
>> Something like:
>>
>> val mkdir_async: string -> int -> (js_exn option -> unit) -> unit
>>   [@@js.global "fs.mkdir"]
>>
>> assuming some predefined bindings for js_exn.  (One could also decide 
>> to bind JS exception to OCaml's exn type, but the gain is not clear.)
>>
>> There are other possible variations, such as making the mode optional:
>>
>> val mkdir_async:
>>     string ->
>>     ?mode:(int[@js.default 0o777]) ->
>>     (js_exn option -> unit) ->
>>     unit
>>   [@@js.global "fs.mkdir"]
>>
>>
>> Here js.default forces the default value for the optional mode 
>> argument.  Without it, a missing value will send an `undefined` as 
>> the second argument (I don't know if fs.mkdir would be happy with 
>> it).  This is currently not supported, but one could also extend the 
>> tool to allow:
>>
>> val mkdir_async:
>>     string ->
>>     ?mode:(int[@js.drop_if_missing]) ->
>>     (js_exn option -> unit) -> unit
>>   [@@js.global "fs.mkdir"]
>>
>> so that the second argument is dropped in the JS call if not provided 
>> (but I don't see how to support that for OCaml callbacks in full 
>> generality, since the function needs to check the type of arguments 
>> to decide how to interpret them).
>>
>>
>> Alain
>>
>
>
>

Re: [Caml-list] js_of_ocaml with node

[Mauricio Fernández]

On Wed, Apr 29, 2015 at 05:03:21PM -0500, Helmut Brandl wrote:
> Thanks to all for the hints.
> 
> I think I have to dig into the documentation of js_of_ocaml to
> resolve the problem.

I've got some bindings for nodejs stuff including FS operations, sockets,
events, processes, etc.; will try to publish it tomorrow. Note that some
nodejs functions require variadic callbacks as supported by the patches
available at https://github.com/ocsigen/js_of_ocaml/issues/273

The hand-written bindings look like this (just a sneak peek):

node_require.eliom:

    {client{
    let require x =
      let require = Js.Unsafe.eval_string "require" in
        require##call(require, Js.string x)
    }}

    (* vim: set ft=ocaml: *)


node_fs.eliom:


    {client{
    open Printf
    open Lwt

    class type error =
    object
      method message : Js.js_string Js.t Js.prop
    end

    type maybe_error = error Js.t Js.opt

    class type stats =
    object
      method dev     : int Js.readonly_prop
      method ino     : int Js.readonly_prop
      method mode    : int Js.readonly_prop
      method nlink   : int Js.readonly_prop
      method uid     : int Js.readonly_prop
      method gid     : int Js.readonly_prop
      method rdev    : int Js.readonly_prop
      method size    : float Js.t Js.readonly_prop
      method blksize : int Js.readonly_prop
      method blocks  : int Js.readonly_prop
      method atime   : Js.date Js.t Js.readonly_prop
      method mtime   : Js.date Js.t Js.readonly_prop
      method ctime   : Js.date Js.t Js.readonly_prop
    end

    and watch_options =
    object
      method persistent : bool Js.t Js.prop
    end

    class type fs =
    object
      method existsSync : Js.js_string Js.t -> bool Js.t Js.meth
      method exists : Js.js_string Js.t -> (bool Js.t -> unit) Js.callback -> unit Js.meth

      method readFile :
        Js.js_string Js.t ->
        (maybe_error -> Node_net.buffer Js.t -> unit) Js.callback -> unit Js.meth

      method readFileSync : Js.js_string Js.t -> Node_net.buffer Js.t Js.meth

      method readdirSync : Js.js_string Js.t -> Js.js_string Js.t Js.js_array Js.t Js.meth

      method statSync : Js.js_string Js.t -> stats Js.t Js.opt Js.meth
      method lstatSync : Js.js_string Js.t -> stats Js.t Js.opt Js.meth

      method mkdir :
        Js.js_string Js.t -> int -> (maybe_error -> unit) Js.callback -> unit Js.meth
      method mkdirSync : Js.js_string Js.t -> int -> unit Js.meth

      method unlink : Js.js_string Js.t -> (maybe_error -> unit) Js.callback -> unit Js.meth
      method unlinkSync : Js.js_string Js.t -> unit Js.meth

      method watch : 'a. Js.js_string Js.t -> (#watch_options as 'a) Js.t ->
                     (Js.js_string -> Js.js_string -> unit) Js.callback -> unit Js.meth
    end

    let fs : fs Js.t = Node_require.require "fs"


    let err_callback1 u err =
      match Js.Opt.to_option err with
          None -> Lwt.wakeup_later u ()
        | Some err -> Lwt.wakeup_later_exn u (Failure (Js.to_string err##message))

    let bool_callback1 u b = Lwt.wakeup_later u (Js.to_bool b)

    let wrap_async1 callback f =
      let t, u = Lwt.wait () in
        f (Js.wrap_callback (callback u));
        t

    let exists s = wrap_async1 bool_callback1 (fun cb -> fs##exists(Js.string s, cb))
    let unlink s = wrap_async1 err_callback1 (fun cb -> fs##unlink(Js.string s, cb))

    let read_file fname =
      let t, u = Lwt.wait () in

      let on_read err data =
        if Js.Opt.test err then
          Lwt.wakeup_later_exn u (Failure (sprintf "could not read %S" fname))
        else Lwt.wakeup_later u (Js.to_string data##toString())
      in
        fs##readFile(Js.string fname, Js.wrap_callback on_read);
        t
    }}
    (* vim: set ft=ocaml: *)

-- 
Mauricio Fernández