[Caml-list] cppo, ocamlbuild, and dependencies

[Caml-list] cppo, ocamlbuild, and dependencies

[François Pottier]

Dear OCaml users,

I am trying to use cppo and ocamlbuild together.

Suppose the file a.cppo.ml contains the line:
   #include "b.cppo.ml".

Then, the build fails. As far as I understand, ocamlbuild is not aware 
of the
dependency a.cppo.ml -> b.cppo.ml, so the source file b.cppo.ml is not even
copied to the _build directory.

I am using the build rules bundled with cppo (in the module 
Ocamlbuild_cppo).

Shouldn't these build rules be improved so as to perform a genuine 
dependency
computation? Has anyone run into this issue?

Thanks,

--
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] cppo, ocamlbuild, and dependencies

[Gabriel Scherer]

Yes, the cppo ocamlbuild plugin
  https://github.com/mjambon/cppo/blob/master/ocamlbuild_plugin/ocamlbuild_cppo.ml
does not currently perform any kind of dynamic dependency computation.

I think the best way for this to be implemented would be at the level
of cppo, have a kind of -list-includes flag that would run on a
.cppo.* file and return the list of includes. Then you could easily
change the cppo rule to call this first, dynamically build all these
dependencies, and proceed with normal cppo processing. (I believe this
is roughly how `gcc -M` works.)

In the meantime, I implemented an amusing hack at the ocamlbuild level
which is to simply parse the failure message and, if it complains
about a missing file, recursively build that file as a dynamic
dependency. It seems to work.

(The code below is just a change over the main function of cppo's
ocamlbuild plugin. To use it in your program, copy the cppo's
ocamlbuild source in your myocamlbuild.ml, replace the relevant
function, and at the end add "let () = dispatch dispatcher" or
whatever dispatch logic you wish.)

let cppo_rules ext =
  let dep   = "%(name).cppo"-.-ext
  and prod1 = "%(name: <*> and not <*.cppo>)"-.-ext
  and prod2 = "%(name: <**/*> and not <**/*.cppo>)"-.-ext in
  let cppo_rule prod env build =
    let dep = env dep in
    let prod = env prod in
    let tags = tags_of_pathname prod ++ "cppo" in
    let cppo = S[A "cppo"; T tags; S [A "-o"; Px prod]; P dep] in
    let rec retry tmp =
      let cppo_to_tmp = S[cppo; Sh "2>"; P tmp ] in
      let cppo_cmd = Command.to_string (Cmd cppo_to_tmp) in
      match Sys.command cppo_cmd with
      | 0 -> Nop
      | _ ->
        let cppo_result = read_file tmp in
        let error = Str.regexp
            {|Error: Cannot find included file "\([^"]*\)"|} in
        (* If we cannot find the dependency,
           or if building the dependency fails,
           we return the original cppo command.

           This will do a bit of useless work but give a better error
           message than having the rule raise an exception here, as it
           will present the failure in terms of the cppo error message
           the user knows about, instead of backtracking in other
           rules. *)
        match Str.search_forward error cppo_result 0 with
        | exception Not_found -> Cmd cppo
        | _pos ->
          let dep = Str.matched_group 1 cppo_result in
          match build [[dep]] with
          | [Outcome.Good _dep] -> retry tmp
          | _ -> Cmd cppo
    in
    with_temp_file "ocamlbuild_cppo" "stderr" retry
  in
  rule ("cppo: *.cppo."-.-ext^" -> *."-.-ext)
    ~dep ~prod:prod1 (cppo_rule prod1);
  rule ("cppo: **/*.cppo."-.-ext^" -> **/*."-.-ext)
    ~dep ~prod:prod2 (cppo_rule prod2)


P.S.: Upon sending this message, GMail says:
> It seems like you forgot to attach a file.
> You wrote "find included" in your message, but there are no files attached.
> Send anyway?

On Tue, Jan 31, 2017 at 10:50 PM, François Pottier
<francois.pottier@inria.fr> wrote:
>
> Dear OCaml users,
>
> I am trying to use cppo and ocamlbuild together.
>
> Suppose the file a.cppo.ml contains the line:
>   #include "b.cppo.ml".
>
> Then, the build fails. As far as I understand, ocamlbuild is not aware of
> the
> dependency a.cppo.ml -> b.cppo.ml, so the source file b.cppo.ml is not even
> copied to the _build directory.
>
> I am using the build rules bundled with cppo (in the module
> Ocamlbuild_cppo).
>
> Shouldn't these build rules be improved so as to perform a genuine
> dependency
> computation? Has anyone run into this issue?
>
> Thanks,
>
> --
> François Pottier
> francois.pottier@inria.fr
> http://gallium.inria.fr/~fpottier/

Re: [Caml-list] cppo, ocamlbuild, and dependencies

[François Pottier]

On 01/02/2017 11:19, Gabriel Scherer wrote:
> In the meantime, I implemented an amusing hack at the ocamlbuild level

Thanks Gabriel for your quick reply.

This seems amusing indeed, but does not quite work for me. Running 
ocamlbuild
once fails, complaining that A.ml does not exist (whereas A.cppo.ml exists).
If I immediately run ocamlbuild again, the build succeeds.

I think I would prefer to use a slightly less amusing approach and just find
out, ahead of time, what the dependencies are. In the rule that transforms
%.cppo.ml to %.ml, could I just read the source file, looking #include
patterns, and ask that the included files be built first?

--
François Pottier
francois.pottier@inria.fr
http://gallium.inria.fr/~fpottier/

Re: [Caml-list] cppo, ocamlbuild, and dependencies

[Gabriel Scherer]

Yes, of course, your approach would also work
(`Ocamlbuild_plugin.read_file` is you friend, followed by the parsing
technique of your choice). The only issue is that you may
over-approximate dependencies depending on how closely you approximate
cppo's grammar (and I don't know, maybe there is a way to have
computed inclusion instead of just file path literals?), so this logic
would be easier to implement correctly in cppo itself.

You may find inspiration in the part of the manual that describes how
to write rules with dynamic dependencies (the explained example is the
code of the (.itarget -> .otarget) rule):

  https://github.com/ocaml/ocamlbuild/blob/master/manual/manual.adoc#Subsec_Rules__Dynamic_dependencies

On Wed, Feb 1, 2017 at 2:40 PM, François Pottier
<francois.pottier@inria.fr> wrote:
>
> On 01/02/2017 11:19, Gabriel Scherer wrote:
>>
>> In the meantime, I implemented an amusing hack at the ocamlbuild level
>
>
> Thanks Gabriel for your quick reply.
>
> This seems amusing indeed, but does not quite work for me. Running
> ocamlbuild
> once fails, complaining that A.ml does not exist (whereas A.cppo.ml exists).
> If I immediately run ocamlbuild again, the build succeeds.
>
> I think I would prefer to use a slightly less amusing approach and just find
> out, ahead of time, what the dependencies are. In the rule that transforms
> %.cppo.ml to %.ml, could I just read the source file, looking #include
> patterns, and ask that the included files be built first?
>
>
> --
> François Pottier
> francois.pottier@inria.fr
> http://gallium.inria.fr/~fpottier/