Ocamlbuild plugins

Ocamlbuild plugins

[Dario Teixeira]

Hi,

(I sent a similar message to this ocaml-beginner's list, but I have the
feeling this will require the assistance of some heavy-weights...)

First, -- and sorry if the answer is obvious -- I still haven't been able
to find Ocamlbuild's documentation about the construction of plugins.  All
I discovered about Ocamlbuild is a brief presentation, the (incomplete)
user manual, and the wiki, which are found at the following addresses:

http://gallium.inria.fr/~pouillar/ocamlbuild/ocamlbuild-presentation.html
http://gallium.inria.fr/~pouillar/ocamlbuild/ocamlbuild-user-guide.html
http://brion.inria.fr/gallium/index.php/Ocamlbuild

Now, the wiki has a few example plugins, and I've tried to dissect
them, but I still have this nagging feeling there must be somewhere
some documentation that introduces how these plugins work.  I've been
cursing at google for a while now, so I am asking directly on this
list -- where are those missing pieces?  (It's okay if they're not
in English)

Anyway, what I intend to do is actually very simple.  In "classic"
makefile notation it is expressed as follows:  (note the use of findlib)

database.cmo: database.ml
        PGDATABASE=lambdium ocamlfind ocamlc -package
threads,pgocaml.statements -syntax camlp4o -thread -c $<

So basically I declare an environment variable PGDATABASE, and
invoke the compiler (via findlib) with a Camlp4 preprocessing stage.
(The pgocaml.statements package instructs findlib about the module to
use for Camlp4; the contents of findlib's META file are listed at the
end of this message).

I've been looking at how to encode this in Ocamlbuild but I've tripped
over some basic problems.  First, how are plugins invoked?  Second,
I cannot find the "Ocamlbuild_plugin" module that is opened by all the
examples (I am using GODI).  Third, how do I even start to implement
the Ocamlbuild code to compile the code above?

Thanks in advance for your help!
Cheers,
Dario Teixeira

________________________________________________________________________________
name="pgocaml"
version="1.0"
description="PG'OCaml is a set of OCaml bindings for the PostgreSQL database."
requires="unix,extlib,csv,pcre,calendar"
archive(byte)="pgocaml.cma"
archive(native)="pgocaml.cmxa"

package "statements" (
  requires = "pgocaml,camlp4"
  version = "1.0"
  description = "Syntax extension: PostgreSQL statements checked at
compile-time"
  archive(syntax,preprocessor) = "pa_pgsql.cmo"
  archive(syntax,toploop) = "pa_pgsql.cmo"
)






      ___________________________________________________________
Yahoo! Answers - Got a question? Someone out there knows the answer. Try it
now.
http://uk.answers.yahoo.com/ 

Re: [Caml-list] Ocamlbuild plugins

[Nicolas Pouillard]

Excerpts from Dario Teixeira's message of Mon Nov 05 21:34:09 +0100 2007:
> Hi,
> 
> (I sent a similar message to this ocaml-beginner's list, but I have the
> feeling this will require the assistance of some heavy-weights...)
> 
> First, -- and sorry if the answer is obvious -- I still haven't been able
> to find Ocamlbuild's documentation about the construction of plugins.  All
> I discovered about Ocamlbuild is a brief presentation, the (incomplete)
> user manual, and the wiki, which are found at the following addresses:
> 
> http://gallium.inria.fr/~pouillar/ocamlbuild/ocamlbuild-presentation.html
> http://gallium.inria.fr/~pouillar/ocamlbuild/ocamlbuild-user-guide.html
> http://brion.inria.fr/gallium/index.php/Ocamlbuild

On  my  page [1] there is also a plugin example [2] and the API [3], where the
module  signature [4] can be a good starting point about what tools we have in
an ocamlbuild plugin.

[1]: http://gallium.inria.fr/~pouillar
[2]: http://gallium.inria.fr/~pouillar/ocamlbuild/plugin_example.html 
[3]: http://gallium.inria.fr/~pouillar/ocamlbuild/html
[4]: http://gallium.inria.fr/~pouillar/ocamlbuild/html/Signatures.html

> Now, the wiki has a few example plugins, and I've tried to dissect
> them, but I still have this nagging feeling there must be somewhere
> some documentation that introduces how these plugins work.  I've been
> cursing at google for a while now, so I am asking directly on this
> list -- where are those missing pieces?  (It's okay if they're not
> in English)

There  is  certainly  some  missing pieces. Roughly an ocamlbuild plugin is an
OCaml  module  called  Myocamlbuild  that should take place a the root of your
project  (myocamlbuild.ml).  Automatically  ocamlbuild will take care of it by
compiling  it  and  by  making  a  myocamlbuild  binary  that is the classical
ocamlbuild plus your module.

With  this  module  you can make effects on the ocamlbuild engine by extending
rules,  flag  injectors, dependencies injectors, libraries declarations... You
can  also  change  default  options  of  ocamlbuild  to  avoid giving flags to
ocamlbuild itself.

In  order  to  properly sequence your effects ocamlbuild tells you to register
your  effects  using  the  `dispatch'  function  that  you will found in every
ocamlbuild  plugin.  This  function receive an argument that is an event, like
After_rules that means that rules of the standard library have been setup.

> Anyway, what I intend to do is actually very simple.  In "classic"
> makefile notation it is expressed as follows:  (note the use of findlib)
> 
> database.cmo: database.ml
>         PGDATABASE=lambdium ocamlfind ocamlc -package
> threads,pgocaml.statements -syntax camlp4o -thread -c $<

Here you follow the make convention that says :

"when you have an exception to a rule, provide another rule more precise"

In  ocamlbuild  that's  still feasible but we tend to avoid it in order to not
rely  on  rules  directly. This facilitate having rules in a library. To do so
ocamlbuild provide another way to specify exceptions to a rule.

This  mechanism  allow  to inject flags (-syntax ... -thread) precisely where
they are needed.

Firstly since you want to use ocamlfind you should start with this plugin [5].

Then  since  -thread  is  a  standard  thing you can use the tag "thread" (and
remove threads from the list of package).

For -syntax you should add your own declaration like in [6] but using -package:

let flags in = S[A"-package"; A"pgocaml.statements"; A"-syntax"; A"camlp4o"] in
flag ["ocaml"; "compile"; "use_pgocaml_statements"];
flag ["ocaml"; "ocamldep"; "use_pgocaml_statements"];

You can then tag your files in the _tags file:

"database.ml": use_pgocaml_statements, thread

[5]: http://brion.inria.fr/gallium/index.php/Using_ocamlfind_with_ocamlbuild
[6]: http://brion.inria.fr/gallium/index.php/A_plugin_for_camlp4_syntax_extension_%28pa_openin%29

> So basically I declare an environment variable PGDATABASE, and
> invoke the compiler (via findlib) with a Camlp4 preprocessing stage.
> (The pgocaml.statements package instructs findlib about the module to
> use for Camlp4; the contents of findlib's META file are listed at the
> end of this message).

The  non-handled part is the PGDATABASE env var. To do so you can fall back to
writing  your  specific  rule  with  ocamlbuild (the rule function), however I
don't  recommend  that.  The  second  solution should be give this argument to
ocamlbuild itself.

> I've been looking at how to encode this in Ocamlbuild but I've tripped
> over some basic problems.  First, how are plugins invoked?  Second,
> I cannot find the "Ocamlbuild_plugin" module that is opened by all the
> examples (I am using GODI).  Third, how do I even start to implement
> the Ocamlbuild code to compile the code above?

I don't know if GODI properly install as many things as ocamlbuild needs.

-- 
Nicolas Pouillard aka Ertai

Re: [Caml-list] Ocamlbuild plugins

[Dario Teixeira]

Hi,

And thanks a lot for your reply!

> On  my  page [1] there is also a plugin example [2] and the API [3],
> where the module  signature [4] can be a good starting point about what
> tools we have in an ocamlbuild plugin.

I had seen those; what was missing was an overview of what exactly was
a plugin, and how it was supposed to be called -- the big picture stuff.


> There  is  certainly  some  missing pieces. Roughly an ocamlbuild plugin
> is an OCaml  module  called  Myocamlbuild  that should take place a
> the root of your project  (myocamlbuild.ml).  Automatically  ocamlbuild
> will take care of it by compiling  it  and  by  making  a  myocamlbuild
> binary  that is the classical ocamlbuild plus your module.

Yeap, I kind of assumed something like that was going on, though of course
I wasn't sure about the details.  This little piece of information would
have made a huge difference if it was included in the documentation!

Also, when introducing a new library, it is always good practice to fully
qualify values.  I know that for people used to a library this might
seem like a waste of time, but for those learning it makes everything
a lot clearer.


> Firstly since you want to use ocamlfind you should start with this plugin

I have looked into it, but it has one huge problem: it assumes that all
files will use the same packages.  Though in practice it doesn't hurt
to pass unnecessary packages to the compiler, it's still killing a fly*
with a bazooka.  Moreover, while the comment at the bottom of that page
addresses this issue ("A more fine grained version can provide specific
-package options depending on the compiled file. This can be easily
done using the tag system"), I find the use of "this can easily be done"
a bit ironic given the present meagerness of Ocamlbuild's documentation...


>  let flags in = S[A"-package"; A"pgocaml.statements"; A"-syntax"; 
A"camlp4o"] in
>  flag ["ocaml"; "compile"; "use_pgocaml_statements"];
>  flag ["ocaml"; "ocamldep"; "use_pgocaml_statements"];
>
>  You can then tag your files in the _tags file:
>
>  "database.ml": use_pgocaml_statements, thread

Yap, that was precisely what I wanted to express.  Though I am having
trouble parsing that example you just gave.  Is "flag" the function
"flag" from module Flags, or just a misspelling of the "flags" value
you just declared?  Also, "let flags in" doesn't seem grammatical.
(This just illustrates the need of always using fully qualified values
when giving examples!).


>  I don't know if GODI properly install as many things as ocamlbuild needs.

I would assume it does, but perhaps the GODI folks can enlighten us on that?

Anyway, thanks again for your help, and I hope you'll find the time
to expand a little on the user guide...  It seems the biggest obstacle
to a wider adoption of Ocamlbuild is the lack of proper documentation
(a common complaint in the OCaml world, unfortunately).

Kind regards,
Dario

*Note: no flies were actually hurt during the writing of this post.



      ________________________________________________________
Nervous about who has your email address? Yahoo! Mail can help you win the war against spam.
http://uk.docs.yahoo.com/mail/addressguard2.html

Re: [Caml-list] Ocamlbuild plugins

[Nicolas Pouillard]

Excerpts from Dario Teixeira's message of Tue Nov 06 17:23:01 +0100 2007:
> Hi,
> 
> And thanks a lot for your reply!
> 
> > On  my  page [1] there is also a plugin example [2] and the API [3],
> > where the module  signature [4] can be a good starting point about what
> > tools we have in an ocamlbuild plugin.
> 
> I had seen those; what was missing was an overview of what exactly was
> a plugin, and how it was supposed to be called -- the big picture stuff.
> 
> 
> > There  is  certainly  some  missing pieces. Roughly an ocamlbuild plugin
> > is an OCaml  module  called  Myocamlbuild  that should take place a
> > the root of your project  (myocamlbuild.ml).  Automatically  ocamlbuild
> > will take care of it by compiling  it  and  by  making  a  myocamlbuild
> > binary  that is the classical ocamlbuild plus your module.
> 
> Yeap, I kind of assumed something like that was going on, though of course
> I wasn't sure about the details.  This little piece of information would
> have made a huge difference if it was included in the documentation!
> 
> Also, when introducing a new library, it is always good practice to fully
> qualify values.  I know that for people used to a library this might
> seem like a waste of time, but for those learning it makes everything
> a lot clearer.
> 
> 
> > Firstly since you want to use ocamlfind you should start with this plugin
> 
> I have looked into it, but it has one huge problem: it assumes that all
> files will use the same packages.  Though in practice it doesn't hurt
> to pass unnecessary packages to the compiler, it's still killing a fly*
> with a bazooka.  Moreover, while the comment at the bottom of that page
> addresses this issue ("A more fine grained version can provide specific
> -package options depending on the compiled file. This can be easily
> done using the tag system"), I find the use of "this can easily be done"
> a bit ironic given the present meagerness of Ocamlbuild's documentation...

At this time, you're right.

> >  let flags in = S[A"-package"; A"pgocaml.statements"; A"-syntax"; 
> A"camlp4o"] in
> >  flag ["ocaml"; "compile"; "use_pgocaml_statements"];
> >  flag ["ocaml"; "ocamldep"; "use_pgocaml_statements"];
> >
> >  You can then tag your files in the _tags file:
> >
> >  "database.ml": use_pgocaml_statements, thread
> 
> Yap, that was precisely what I wanted to express.  Though I am having
> trouble parsing that example you just gave.  Is "flag" the function
> "flag" from module Flags, or just a misspelling of the "flags" value
> you just declared?  Also, "let flags in" doesn't seem grammatical.
> (This just illustrates the need of always using fully qualified values
> when giving examples!).

Sorry this example is totally badly wrote (due to the lack of email type checking).

Here is a correct one:

open Ocamlbuild_plugin;;
open Command;;
dispatch begin function
| After_rules ->
    let some_flags =
      S[A"-package"; A"pgocaml.statements"; A"-syntax"; A"camlp4o"]
    in
    flag ["ocaml"; "compile"; "use_pgocaml_statements"]  some_flags;
    flag ["ocaml"; "ocamldep"; "use_pgocaml_statements"] some_flags;
| _ -> ()
end;;

flag  [1]  refer  to  Ocamlbuild_plugin.flag  that  is  an  exported alias for
Ocamlbuild_pack.Flags.flag.

[1]: http://gallium.inria.fr/~pouillar/ocamlbuild/html/Signatures.PLUGIN.html

> >  I don't know if GODI properly install as many things as ocamlbuild needs.
> 
> I would assume it does, but perhaps the GODI folks can enlighten us on that?

Does anyone use ocamlbuild with plugins with a GODI setup?

> Anyway, thanks again for your help, and I hope you'll find the time
> to expand a little on the user guide...

I hope too.

>  It seems the biggest obstacle
> to a wider adoption of Ocamlbuild is the lack of proper documentation
> (a common complaint in the OCaml world, unfortunately).

Sadly you're right. As always any help/contribution will be appreciated :)

-- 
Nicolas Pouillard aka Ertai

Re: [Caml-list] Ocamlbuild plugins

[Dario Teixeira]

Hi,

> (...)
> Sadly you're right. As always any help/contribution will be appreciated :)

Well, I'll do my part: once I grok Ocamlbuild I'll profusely document
the myocamlbuild.ml for my project.

Anyway, thanks again for your help.  I have at the moment a myocamlbuild.ml
that can handle almost everything I need; the only bit that isn't working
is precisely the part that handles camlp4.  So, here's the myocamlbuild.ml:
______________________________________________________________

open Ocamlbuild_plugin
open Command

let ocamlfind x =
        let packages = "calendar, ocsigen" in
        S[A "ocamlfind"; x; A "-package"; A packages];;

dispatch begin function
        | Before_options ->
             Options.ocamlc := ocamlfind & A"ocamlc";
             Options.ocamlopt := ocamlfind & A"ocamlopt";
             Options.ocamldep := ocamlfind & A"ocamldep"
        | After_rules ->
                let some_flags = S[A"-package"; A"pgocaml.statements";
A"-syntax"; A"camlp4o"] in
                flag ["ocaml"; "compile"; "use_pgocaml_statements"] 
some_flags;
                flag ["ocaml"; "ocamldep"; "use_pgocaml_statements"]
some_flags;
                flag ["ocaml"; "link"] (A"-linkpkg")
        | _ -> ()
end;;
______________________________________________________________

I also have a _tags file with just one line:
"database.ml": use_pgocaml_statements, thread

And this is the error I get when I run "ocamlbuild database.cmo":
______________________________________________________________

+ ocamlfind ocamldep -package 'calendar, ocsigen' -package pgocaml.statements
-syntax camlp4o -pp '-package pgocaml.statements -syntax camlp4o' -modules
database.ml > database.ml.depends
Warning: -pp overrides the effect of -syntax partly
sh: Illegal option -p
Preprocessing error on file database.ml
Command exited with code 2.
______________________________________________________________

My guess is that some builtin rule is trying to invoke camlp4 with -pp,
which is altogether unnecessary because ocamlfind takes care of that
automatically with the -syntax option.  Any thoughts?

Thanks for your time,
Dario



      _____________________________________________________
Do not compromise. Get unlimited storage and first rate spam protection with Yahoo! Mail.
http://uk.mail.yahoo.com

Re: [Caml-list] Ocamlbuild plugins

[Nicolas Pouillard]

Excerpts from Dario Teixeira's message of Wed Nov 07 14:39:19 +0100 2007:
> Hi,
> 
> > (...)
> > Sadly you're right. As always any help/contribution will be appreciated :)
> 
> Well, I'll do my part: once I grok Ocamlbuild I'll profusely document
> the myocamlbuild.ml for my project.
> 
> Anyway, thanks again for your help.  I have at the moment a myocamlbuild.ml
> that can handle almost everything I need; the only bit that isn't working
> is precisely the part that handles camlp4.  So, here's the myocamlbuild.ml:
> ______________________________________________________________
> 
> open Ocamlbuild_plugin
> open Command
> 
> let ocamlfind x =
>         let packages = "calendar, ocsigen" in
>         S[A "ocamlfind"; x; A "-package"; A packages];;
> 
> dispatch begin function
>         | Before_options ->
>              Options.ocamlc := ocamlfind & A"ocamlc";
>              Options.ocamlopt := ocamlfind & A"ocamlopt";
>              Options.ocamldep := ocamlfind & A"ocamldep"
>         | After_rules ->
>                 let some_flags = S[A"-package"; A"pgocaml.statements";
> A"-syntax"; A"camlp4o"] in
>                 flag ["ocaml"; "compile"; "use_pgocaml_statements"] 
> some_flags;
>                 flag ["ocaml"; "ocamldep"; "use_pgocaml_statements"]
> some_flags;
>                 flag ["ocaml"; "link"] (A"-linkpkg")
>         | _ -> ()
> end;;
> ______________________________________________________________
> 
> I also have a _tags file with just one line:
> "database.ml": use_pgocaml_statements, thread
> 
> And this is the error I get when I run "ocamlbuild database.cmo":
> ______________________________________________________________
> 
> + ocamlfind ocamldep -package 'calendar, ocsigen' -package pgocaml.statements
> -syntax camlp4o -pp '-package pgocaml.statements -syntax camlp4o' -modules
> database.ml > database.ml.depends
> Warning: -pp overrides the effect of -syntax partly
> sh: Illegal option -p
> Preprocessing error on file database.ml
> Command exited with code 2.

Arg  too bad, that's an ocamlbuild bug. I didn't see it since I always use -pp
and not -syntax of ocamlfind.

You should try to use the .cmo of your syntax extension.
Use these declarations and replace pa_openin.cmo by the cmo of pgocaml.statements.

(* One add pa_openin.cmo to the ocaml pre-processor when use_opening is set *)
flag ["ocaml"; "pp"; "use_openin"] (A"pa_openin.cmo");

(* Running ocamldep on ocaml code that is tagged use_openin will require the cmo *)
dep ["ocaml"; "ocamldep"; "use_openin"] ["pa_openin.cmo"];

Let me know if it works...
-- 
Nicolas Pouillard aka Ertai

Re: Ocamlbuild plugins

[Alan Falloon]

Nicolas Pouillard wrote:
> Does anyone use ocamlbuild with plugins with a GODI setup?

I use a GODI install and ocamlbuild (with a plugin) without any issues. 
Of course, you do need to ensure you install the 3.10 section from GODI, 
but other than that everything worked out of the box (as far as 
ocamlbuild is concerned).