OSR: META files for packages containing syntax extensions

OSR: META files for packages containing syntax extensions

[Dario Teixeira]

================================================================
   OSR: META files for packages containing syntax extensions
================================================================

1. OBJECTIVE:
=============

This recommendation aims to ensure that the META files accompanying
Ocaml packages implementing some Camlp4-based syntax extension are
Findlib-compliant.


2. RATIONALE:
=============

Whether by invoking ocamlfind directly or by using the generic Ocamlbuild
plugin [1], the use of syntax extensions in Ocaml can be significantly
simplified if the package defining the extension includes a META file
that is Findlib+Camlp4 aware.  Though the Findlib manual describes
the Camlp4 support as being experimental [2], it works well enough in
practice to warrant a more widespread use.  Despite this fact, not all
packages available via GODI are presently compliant.


3. WHOM IT CONCERNS:
====================

On the one hand, package maintainers, who should make sure the packages
under their wing follow the guidelines in this OSR.  On the other hand,
the users themselves: either by using the Ocamlbuild plugin [1], or
by manually using the findlib facilities for syntax extensions in their
makefiles, they can make life easier for others who wish to compile/modify
their code.


4. DESCRIPTION:
===============

The Findlib manual already contains a comprehensive description [2].
The material contained here merely illustrates how the META file
should be written for a number of common scenarios.


4.1. Package that IS A syntax extension:
========================================

Suppose we have a syntax extension "pa_listcomp" for list comprehensions.
This extension does not rely on any other libraries, and is meant to be
packaged individually.  Here's how its META file should look like:

-----------------------------------------------
name "pa_listcomp"
requires = "camlp4"
version = "1.0"
description = "Syntax extension for list comprehensions"
archive(syntax,preprocessor) = "pa_listcomp.cmo"
archive(syntax,toploop) = "pa_listcomp.cmo"
-----------------------------------------------

Another common occurrence is for the syntax extension to rely on a runtime
module or library included with the package.  Suppose pa_listcomp relied
on a "runtime_listcomp" module; here's how its META file should look
like instead:

-----------------------------------------------
name "pa_listcomp"
requires = "camlp4"
version = "1.0"
description = "Syntax extension for list comprehensions"
archive(syntax,preprocessor) = "pa_listcomp.cmo"
archive(syntax,toploop) = "pa_listcomp.cmo runtime_listcomp.cmo"
archive(syntax,byte) = "runtime_listcomp.cmo"
archive(syntax,native) = "runtime_listcomp.cmx"
-----------------------------------------------

Topic for discussion: for the sake of standardisation, it would be
nice if all packages of this type were to share a common prefix.
The most obvious candidate is "pa_".


4.2. Package with optional syntax extension:
============================================

A good example is PG'OCaml.  Users of this library can link against the
"pgocaml" package if they only intend to use the low-level functions.
However, there is also an optional syntax extension that enables the
embedding of SQL statements.  This syntax extension is enabled by using
the "pgocaml.statements" package (note that "statements" is defined as
a subpackage of "pgocaml").  Here's the current META file:

-----------------------------------------------
name="pgocaml"
version="1.1"
description="PG'OCaml is a set of OCaml bindings for the PostgreSQL database."
requires="unix,extlib,csv,pcre,calendar"
archive(byte)="pgocaml.cma"
archive(native)="pgocaml.cmxa"

package "statements" (
  requires = "pgocaml,camlp4"
  version = "1.1"
  description = "Syntax extension: PostgreSQL statements checked at
compile-time"
  archive(syntax,preprocessor) = "pa_pgsql.cmo"
  archive(syntax,toploop) = "pa_pgsql.cmo"
  )
-----------------------------------------------

Topic for discussion: without prejudice for those cases where an alternative
name may be far more suitable, it would be nice if packages of this type
containing a single syntax extension were to agree on a common name for the
subpackage defining that extension.  Some possibilities are "extension",
"pa", "pp", etc.


4.3. Package that is a container for several syntax extensions:
===============================================================

I would place P4ck in this category, though it is presently not available
via GODI, and I don't know if its author would like to see it added as
a single godi-p4ck package or if split into individual packages.

Suppose we have a dummy package "foobar", which is simply a container for two
independent syntax extensions, "pa_openin" and "pa_memoization".  Users can
refer to each extension as "foobar.pa_openin" and "foobar.pa_memoization".
Here's how the META file for foobar should look like:

-----------------------------------------------
package "pa_openin" (
  requires = "camlp4"
  version = "1.0"
  description = "Open_in syntax extension"
  archive(syntax,preprocessor) = "pa_openin.cmo"
  archive(syntax,toploop) = "pa_openin.cmo"
  )

package "pa_memoization" (
  requires = "camlp4"
  version = "1.0"
  description = "Memoization syntax extension"
  archive(syntax,preprocessor) = "pa_memoization.cmo"
  archive(syntax,toploop) = "pa_memoization.cmo"
  )
-----------------------------------------------

Topic for discussion: since each syntax extension is fairly autonomous
and has the potential to become an independent package,  I reckon its
name should follow the same conventions that are eventually agreed upon
for section 4.1.


REFERENCES:
===========

[1] Ocamlbuild Wiki - "Using ocamlfind with ocamlbuild":
http://brion.inria.fr/gallium/index.php/Using_ocamlfind_with_ocamlbuild

[2] The findlib User's Guide - "Does Findlib support camlp4?":
http://www.ocaml-programming.de/packages/documentation/findlib/guide-html/x412.html






      ___________________________________________________________ 
Rise to the challenge for Sport Relief with Yahoo! For Good  

http://uk.promotions.yahoo.com/forgood/

Re: OSR: META files for packages containing syntax extensions

[Sylvain Le Gall]

Hello,

On 13-03-2008, Dario Teixeira <darioteixeira@yahoo.com> wrote:
>================================================================
>    OSR: META files for packages containing syntax extensions
>================================================================
>
> 1. OBJECTIVE:
>=============
>
> This recommendation aims to ensure that the META files accompanying
> Ocaml packages implementing some Camlp4-based syntax extension are
> Findlib-compliant.
>
>

Very good initiative: simple, efficient and first good step for a
standardization process ;-)


[...]

>
> 4.2. Package with optional syntax extension:
>============================================
>
> A good example is PG'OCaml.  Users of this library can link against the
> "pgocaml" package if they only intend to use the low-level functions.
> However, there is also an optional syntax extension that enables the
> embedding of SQL statements.  This syntax extension is enabled by using
> the "pgocaml.statements" package (note that "statements" is defined as
> a subpackage of "pgocaml").  Here's the current META file:
>
> -----------------------------------------------
> name="pgocaml"
> version="1.1"
> description="PG'OCaml is a set of OCaml bindings for the PostgreSQL database."
> requires="unix,extlib,csv,pcre,calendar"
> archive(byte)="pgocaml.cma"
> archive(native)="pgocaml.cmxa"
>
> package "statements" (
>   requires = "pgocaml,camlp4"
>   version = "1.1"
>   description = "Syntax extension: PostgreSQL statements checked at
> compile-time"
>   archive(syntax,preprocessor) = "pa_pgsql.cmo"
>   archive(syntax,toploop) = "pa_pgsql.cmo"
>   )
> -----------------------------------------------
>
> Topic for discussion: without prejudice for those cases where an alternative
> name may be far more suitable, it would be nice if packages of this type
> containing a single syntax extension were to agree on a common name for the
> subpackage defining that extension.  Some possibilities are "extension",
> "pa", "pp", etc.
>
>
> 4.3. Package that is a container for several syntax extensions:
>===============================================================
>
> I would place P4ck in this category, though it is presently not available
> via GODI, and I don't know if its author would like to see it added as
> a single godi-p4ck package or if split into individual packages.
>
> Suppose we have a dummy package "foobar", which is simply a container for two
> independent syntax extensions, "pa_openin" and "pa_memoization".  Users can
> refer to each extension as "foobar.pa_openin" and "foobar.pa_memoization".
> Here's how the META file for foobar should look like:
>
> -----------------------------------------------
> package "pa_openin" (
>   requires = "camlp4"
>   version = "1.0"
>   description = "Open_in syntax extension"
>   archive(syntax,preprocessor) = "pa_openin.cmo"
>   archive(syntax,toploop) = "pa_openin.cmo"
>   )
>
> package "pa_memoization" (
>   requires = "camlp4"
>   version = "1.0"
>   description = "Memoization syntax extension"
>   archive(syntax,preprocessor) = "pa_memoization.cmo"
>   archive(syntax,toploop) = "pa_memoization.cmo"
>   )
> -----------------------------------------------
>
> Topic for discussion: since each syntax extension is fairly autonomous
> and has the potential to become an independent package,  I reckon its
> name should follow the same conventions that are eventually agreed upon
> for section 4.1.

4.3 and 4.2 should be merged. 

All syntax extension should be contained in a package "statements" (or
whatever other name is chosen). If there is only 1 syntax extension, the
package "statements" is directly the syntax extension. If there is
several extension, each one get its own package with a name related to
its function.

For p4ck, here is the META is propose:
name = "p4ck"
version="0.0.0"
description="Various syntax extension"

package "statements"
(
  package "openin" (
    requires = "camlp4"
    version = "1.0"
    description = "Open_in syntax extension"
    archive(syntax,preprocessor) = "pa_openin.cmo"
    archive(syntax,toploop) = "pa_openin.cmo"
  )

  package "memoization" (
    requires = "camlp4"
    version = "1.0"
    description = "Memoization syntax extension"
    archive(syntax,preprocessor) = "pa_memoization.cmo"
    archive(syntax,toploop) = "pa_memoization.cmo"
  )

  requires = "p4ck.statements.openin,p4ck.statements.memoization"
)

Which gives "ocamlfind -list":
p4ck                (version: 0.0.0)
p4ck.statements     (version: n/a)
p4ck.statements.memoization (version: 1.0)
p4ck.statements.openin (version: 1.0)

The "statements" package should represent all the extension contained in
it.

In this case:
$ ocamlfind -query -a-format p4ck.statements -r -predicates "syntax,toploop"
pa_openin.cmo
pa_memoization.cmo

$ ocamlfind -query -a-format p4ck.statements.openin -r -predicates "syntax,toploop"
pa_openin.cmo

This also simplifies migration from 1 to many syntax extension in the same
package, because "statements" package will still contains all the former
syntax extension. However, this is not a good solution for many to 1
syntax extension (you will still have to define all sub packages just
requiring the "statements" package).

Regards,
Sylvain Le Gall

Re: OSR: META files for packages containing syntax extensions

[Eric Cooper]

> Topic for discussion: without prejudice for those cases where an
> alternative name may be far more suitable, it would be nice if
> packages of this type containing a single syntax extension were to
> agree on a common name for the subpackage defining that extension.
> Some possibilities are "extension", "pa", "pp", etc.

If it doesn't break anything, I suggest "syntax".

-- 
Eric Cooper             e c c @ c m u . e d u

Re: OSR: META files for packages containing syntax extensions

[Sylvain Le Gall]

On 13-03-2008, Eric Cooper <ecc@cmu.edu> wrote:
>> Topic for discussion: without prejudice for those cases where an
>> alternative name may be far more suitable, it would be nice if
>> packages of this type containing a single syntax extension were to
>> agree on a common name for the subpackage defining that extension.
>> Some possibilities are "extension", "pa", "pp", etc.
>
> If it doesn't break anything, I suggest "syntax".
>

What about "camlp4"/"camlp5" ? At least it will allow to make the
difference between the two.

Regards,
Sylvain Le Gall

Re: [Caml-list] Re: OSR: META files for packages containing syntax extensions

[Dario Teixeira]

> 
> What about "camlp4"/"camlp5" ? At least it will allow to make the
> difference between the two.
> 

Hi,

I think that distinction should be private to the META file, without being
visible and exported into userland (I don't see a good reason why the user
of a syntax extension should care if it is implemented with camlp4 or camlp5).
The "requires" attribute of the META file could be used to inform findlib or
Ocamlbuild of which preprocessor to use.

Cheers,
Dario



      ___________________________________________________________ 
Rise to the challenge for Sport Relief with Yahoo! For Good  

http://uk.promotions.yahoo.com/forgood/

Re: [Caml-list] Re: OSR: META files for packages containing syntax extensions

[Dario Teixeira]

> 
> If it doesn't break anything, I suggest "syntax".
> 

Well noted.  I'm now using "syntax" as the candidate denomination.

Cheers,
Dario



      ___________________________________________________________ 
Rise to the challenge for Sport Relief with Yahoo! For Good  

http://uk.promotions.yahoo.com/forgood/

Re: [Caml-list] Re: OSR: META files for packages containing syntax extensions

[Dario Teixeira]

Hi,

> 4.3 and 4.2 should be merged. 
> 
> All syntax extension should be contained in a package "statements" (or
> whatever other name is chosen). If there is only 1 syntax extension, the
> package "statements" is directly the syntax extension. If there is
> several extension, each one get its own package with a name related to
> its function.

Thanks for your comments, Sylvain.  Below is a modification to the original
proposal that incorporates your comments (with a few tweaks); under this new
proposal, *all* syntax extensions have a ".syntax" suffix for findlib users:

4.1. Package is a syntax extension:

"openin"        - the runtime lib required by the extension (if applicable)
"openin.syntax" - The syntax extension itself

4.2. Package with optional syntax extension:

"pgocaml"        - refers to the library without extension;
"pgocaml.syntax" - refers to all syntax extensions in package;

4.3. Package is a collection of syntax extensions:

"p4ck"               - all runtime libs for all extensions in collection?
"p4ck.openin"        - runtime support for the openin extension (if applicable)
"p4ck.openin.syntax" - the openin syntax extension
"p4ck.syntax"        - all syntax extensions in the collection


Though I'd prefer to avoid the Java syndrome of being.namespaced.into.oblivion,
this new proposal does have the advantage of unifying all special cases.

Any thoughts?

Cheers,
Dario



      ___________________________________________________________ 
Rise to the challenge for Sport Relief with Yahoo! For Good  

http://uk.promotions.yahoo.com/forgood/

Re: OSR: META files for packages containing syntax extensions

[Sylvain Le Gall]

On 14-03-2008, Dario Teixeira <darioteixeira@yahoo.com> wrote:
> Hi,
>
>> 4.3 and 4.2 should be merged. 
>> 
>> All syntax extension should be contained in a package "statements" (or
>> whatever other name is chosen). If there is only 1 syntax extension, the
>> package "statements" is directly the syntax extension. If there is
>> several extension, each one get its own package with a name related to
>> its function.
>
> Thanks for your comments, Sylvain.  Below is a modification to the original
> proposal that incorporates your comments (with a few tweaks); under this new
> proposal, *all* syntax extensions have a ".syntax" suffix for findlib users:
>
> 4.1. Package is a syntax extension:
>
> "openin"        - the runtime lib required by the extension (if applicable)
> "openin.syntax" - The syntax extension itself
>
> 4.2. Package with optional syntax extension:
>
> "pgocaml"        - refers to the library without extension;
> "pgocaml.syntax" - refers to all syntax extensions in package;
>
> 4.3. Package is a collection of syntax extensions:
>
> "p4ck"               - all runtime libs for all extensions in collection?
> "p4ck.openin"        - runtime support for the openin extension (if applicable)
> "p4ck.openin.syntax" - the openin syntax extension
> "p4ck.syntax"        - all syntax extensions in the collection
>
>
> Though I'd prefer to avoid the Java syndrome of being.namespaced.into.oblivion,
> this new proposal does have the advantage of unifying all special cases.
>
> Any thoughts?
>

Nope, this is great.

Regards,
Sylvain Le Gall

Re: [Caml-list] Re: OSR: META files for packages containing syntax extensions

[Dario Teixeira]

> 
> Nope, this is great.
> 

Excellent!  I'll wait for further comments before resubmitting the OSR.
If it's accepted, hopefully all GODI packages will quickly make the
migration.  It would also be nice if the extensions contained in
P4ck were to be packaged for GODI -- either independently or as
a collection.  Volunteers?

Cheers,
Dario



      ___________________________________________________________ 
Rise to the challenge for Sport Relief with Yahoo! For Good  

http://uk.promotions.yahoo.com/forgood/

Re: [Caml-list] OSR: META files for packages containing syntax extensions

[Stefano Zacchiroli]

On Thu, Mar 13, 2008 at 12:40:30PM +0000, Dario Teixeira wrote:
> Though the Findlib manual describes the Camlp4 support as being
> experimental [2], it works well enough in practice to warrant a more
> widespread use.

I've nothing in contrary this wonderful proposal, even because in Debian
we have been fans of META files since the very beginning, requiring them
as per our internal policy.

Nevertheless, a paragraph like the above one IMO needs to be backed by
Gerd, as the author of findlib. I won't base a proposal like yours on
something which is still declared to be experimental.

Cheers.

-- 
Stefano Zacchiroli -*- PhD in Computer Science ............... now what?
zack@{upsilon.cc,cs.unibo.it,debian.org}  -<%>-  http://upsilon.cc/zack/
(15:56:48)  Zack: e la demo dema ?    /\    All one has to do is hit the
(15:57:15)  Bac: no, la demo scema    \/    right keys at the right time

Re: [Caml-list] OSR: META files for packages containing syntax extensions

[Gerd Stolpmann]

Am Donnerstag, den 13.03.2008, 16:05 +0100 schrieb Stefano Zacchiroli:
> On Thu, Mar 13, 2008 at 12:40:30PM +0000, Dario Teixeira wrote:
> > Though the Findlib manual describes the Camlp4 support as being
> > experimental [2], it works well enough in practice to warrant a more
> > widespread use.
> 
> I've nothing in contrary this wonderful proposal, even because in Debian
> we have been fans of META files since the very beginning, requiring them
> as per our internal policy.
> 
> Nevertheless, a paragraph like the above one IMO needs to be backed by
> Gerd, as the author of findlib. I won't base a proposal like yours on
> something which is still declared to be experimental.

This is outdated. It's not experimental anymore. I should describe the
syntax extension feature at a better place in the manual than an FAQ.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------

Re: [Caml-list] OSR: META files for packages containing syntax extensions

[blue storm]

> Topic for discussion: for the sake of standardisation, it would be
> nice if all packages of this type were to share a common prefix.
> The most obvious candidate is "pa_".

Though this can be refined later, i think we could do something a bit
more elaborate there.
I have not used camlp4 pre-3.10 (camlp5 these days), but if i
understood correctly, prefixes were used to describe the internal
machinery of the extension : pa_ denotes a parser extension, while pr_
denotes a printer extension. I do not know of any other prefix.

With the new camlp4, those suffixes are not required, but they were
kept in order to ease the transition (i'd assume pa_r.cmo actually
refers to Camlp4Parser/Camlp4OCamlRevisedParser.cmo). This is a good
idea imho, but i think that if we stick to that convention (describing
the actual effect of an extension in his name) we should adopt a
naming convention a bit more powerful, to reflect the actual abilities
of camlp4 : for example, we can have AST filters, wich are neither
parsers nor printers.

Here would be a draft suggestion concerning this naming scheme :
- pa_ for parsers
- pr_ for printers
- pf_ for filters
- p_ in uncertain cases

Some extensions use eg. both a parser and a filter; i guess they'd
need to chose the most meaningful prefix (are they primarily intended
to modify the existing syntax, or change a behavior ?).
There are some code that can be described as a "syntax extension", but
that does not use any of the camlp4 parser/printer/filter tools. The
'extensions' using the library type-conv (
http://www.ocaml.info/home/ocaml_sources.html#type-conv ), for
example, use a specific registration mechanism. I guess the p_ prefix
would be appropriate for those corner cases.

I think it is important to keep a camlp5 compatibility (even if i'm
not sure ocamlfind support camlp5 yet); the re-use of the standard
prefixes provide this.



> Another common occurrence is for the syntax extension to rely on a runtime
> module or library included with the package.

Another common case is when the syntax extension rely on a
preprocessor-time module or library (eg. type-conv). It could be
helpful to add a description of that possibility.

Is there a way to specify a dependency on a specific preprocessor
(between camlp4o and camlp4r, for example ?). Most extension actually
won't work with both syntaxes (classical and revised), and it could be
helpful to make it apparent in the package description (although most
of the time an horrible preprocessor-runtime error will happen).
More generally, i'm concerned with the possible incompatibilities
between syntax extensions (however, this may be a bit off-topic).


Thanks for that very useful proposal.

Re: [Caml-list] OSR: META files for packages containing syntax extensions

[Dario Teixeira]

Hi,

> Some extensions use eg. both a parser and a filter; i guess they'd
> need to chose the most meaningful prefix (are they primarily intended
> to modify the existing syntax, or change a behavior ?).
> There are some code that can be described as a "syntax extension", but
> that does not use any of the camlp4 parser/printer/filter tools. The
> 'extensions' using the library type-conv (
> http://www.ocaml.info/home/ocaml_sources.html#type-conv ), for
> example, use a specific registration mechanism. I guess the p_ prefix
> would be appropriate for those corner cases.

Thanks for ellucidation.  Alternatively, since the preprocessor is
oblivious to the name of the package (as you say, even Camlp5 may
only care about the name of the .cmo file), we can simply declare
this to be a documentation problem and to not enforce any convention
regarding the package name (except for the previously discussed .syntax
suffix).

Therefore, the package implementing the "openin" extension could be
called "pa_openin", "openin", or whatever the author prefers due to
historical reasons.  The only requirement is that the syntax extension
itself should reside in an "openin.syntax" subpackage.


> Another common case is when the syntax extension rely on a
> preprocessor-time module or library (eg. type-conv). It could be
> helpful to add a description of that possibility.

I think that is also a documentation issue.  Ideally, the META files
should be constructed in such a way that the user should not have to
care about extension dependencies.  As an example, consider the Sexplib
syntax extension, which itself depends on the type-conv extension.
On my system, I have patched the META files so that this is transparent
to the Sexplib users.

Here's my META file for Sexplib:

--------------------------------------------------------
name="sexplib"
version="3.0.0"
description="Sexplib - automated S-expression conversions"
requires="bigarray"
archive(byte)="sexplib.cma"
archive(native)="sexplib.cmxa"

package "syntax" (
  requires = "sexplib,type-conv.syntax,camlp4"
  version = "3.0.0"
  description = "Syntax extension for Sexplib"
  archive(syntax,preprocessor) = "pa_sexp_conv.cmo"
  archive(syntax,toploop) = "pa_sexp_conv.cmo"
  )
--------------------------------------------------------

And the META file for Type-conv:

--------------------------------------------------------
name="type-conv"
version="1.0.0"
description="Type-conv - support library for preprocessor type conversions"

package "syntax" (
  requires = "camlp4"
  version = "1.0.0"
  description = "Syntax extension for type-conv"
  archive(syntax,preprocessor) = "pa_type_conv.cmo"
  archive(syntax,toploop) = "pa_type_conv.cmo"
  )
--------------------------------------------------------


> Is there a way to specify a dependency on a specific preprocessor
> (between camlp4o and camlp4r, for example ?). Most extension actually
> won't work with both syntaxes (classical and revised), and it could be
> helpful to make it apparent in the package description (although most
> of the time an horrible preprocessor-runtime error will happen).
> More generally, i'm concerned with the possible incompatibilities
> between syntax extensions (however, this may be a bit off-topic).

I don't know if there's support for those cases (are there any in the wild?).
Perhaps some of Findlib's gurus can enlighten us...

Cheers,
Dario



      ___________________________________________________________ 
Rise to the challenge for Sport Relief with Yahoo! For Good  

http://uk.promotions.yahoo.com/forgood/

Re: [Caml-list] OSR: META files for packages containing syntax extensions

[Gerd Stolpmann]

Am Freitag, den 14.03.2008, 15:33 +0000 schrieb Dario Teixeira:
> > Is there a way to specify a dependency on a specific preprocessor
> > (between camlp4o and camlp4r, for example ?). Most extension actually
> > won't work with both syntaxes (classical and revised), and it could be
> > helpful to make it apparent in the package description (although most
> > of the time an horrible preprocessor-runtime error will happen).
> > More generally, i'm concerned with the possible incompatibilities
> > between syntax extensions (however, this may be a bit off-topic).
> 
> I don't know if there's support for those cases (are there any in the wild?).
> Perhaps some of Findlib's gurus can enlighten us...

Currently this is not possible. We could do something like

error(camlp4r) = "This extension is not compatible with revised syntax"

if there is really a demand for it.

Gerd

> 
> Cheers,
> Dario
> 
> 
> 
>       ___________________________________________________________ 
> Rise to the challenge for Sport Relief with Yahoo! For Good  
> 
> http://uk.promotions.yahoo.com/forgood/
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Phone: +49-6151-153855                  Fax: +49-6151-997714
------------------------------------------------------------

Re: [Caml-list] OSR: META files for packages containing syntax extensions (take 2)

[Dario Teixeira]

Hi,

As promised, here's version 2.0 of the OSR.  Please make your voice
be heard concerning bugs/suggestions/criticisms/etc.  If everybody
is happy with it, it will be placed in the Wiki.

Best regards,
Dario Teixeira


================================================================
   OSR: META files for packages containing syntax extensions
================================================================

1. OBJECTIVE:
=============

This recommendation aims to ensure that the META files accompanying
Ocaml packages implementing some Camlp4-based syntax extension are
Findlib-compliant.


2. RATIONALE:
=============

Whether by invoking ocamlfind directly or by using the generic Ocamlbuild
plugin [1], the use of syntax extensions in Ocaml can be significantly
simplified if the package defining the extension includes a META file
that is Findlib+Camlp4 aware.  Despite this fact, not all packages
available via GODI are presently compliant.


3. WHOM IT CONCERNS:
====================

On the one hand, package maintainers, who should make sure the packages
under their wing follow the guidelines in this OSR.  On the other hand,
the users themselves: either by using the Ocamlbuild plugin [1], or
by manually using the findlib facilities for syntax extensions in their
makefiles, they can make life easier for others who wish to compile/modify
their code.


4. DESCRIPTION:
===============

The Findlib manual already contains a comprehensive reference concerning
Findlib's support for Camlp4 syntax extensions [2].  The material
contained here merely illustrates how the META file should be written for
a number of common scenarios.  The reader will note that the goal has been
to unify as much as possible the conventions used, even if at the expense
of always requiring the use of subpackages.  In particular, we always
mandate a subpackage called "syntax" to hold the syntax extension itself.


4.1. Package that IS A syntax extension:
========================================

Suppose we have a syntax extension "listcomp" for list comprehensions.
This extension does not rely on any other libraries, and is meant to be
packaged individually.  However, due to the requirement that all syntax
extension packages are named "syntax", it must be defined in a subpackage
and placed inside a container package.  Here's how its META file should
look like:

-----------------------------------------------
name "listcomp"
version = "1.0"
description = "Container for list comprehensions"

package "syntax"
   (
   version = "1.0"
   description = "Syntax extension for list comprehensions"
   requires = "camlp4"
   archive(syntax,preprocessor) = "pa_listcomp.cmo"
   archive(syntax,toploop) = "pa_listcomp.cmo"
   )
-----------------------------------------------


Another common occurrence is for the syntax extension to rely on a
runtime module or library included with the package.  Suppose listcomp
relied on a "runtime_listcomp" module; here's how its META file should
look like instead:

-----------------------------------------------
name "listcomp"
version = "1.0"
description = "Runtime support for list comprehensions"
archive(byte)="runtime_listcomp.cma"
archive(native)="runtime_listcomp.cmxa"

package "syntax"
   (
   version = "1.0"
   description = "Syntax extension for list comprehensions"
   requires = "listcomp,camlp4"
   archive(syntax,preprocessor) = "pa_listcomp.cmo"
   archive(syntax,toploop) = "pa_listcomp.cmo"
   )
-----------------------------------------------


4.2. Package with optional syntax extension:
============================================

A good example is PG'OCaml.  Users of this library can link against the
"pgocaml" package if they only intend to use the low-level functions.
However, there is also an optional syntax extension that enables the
embedding of SQL statements within Ocaml code.  This syntax extension
is enabled by using the "pgocaml.syntax" subpackage.  Note that this
case is essentially identical to the previously shown syntax extension
relying on a runtime library.  Here's the proposed META file:

-----------------------------------------------
name="pgocaml"
version="1.1"
description="PG'OCaml is a set of OCaml bindings for the PostgreSQL database."
requires="unix,extlib,csv,pcre,calendar"
archive(byte)="pgocaml.cma"
archive(native)="pgocaml.cmxa"

package "syntax"
   (
   version = "1.1"
   description = "Syntax extension: PostgreSQL statements checked at
compile-time"
   requires = "pgocaml,camlp4"
   archive(syntax,preprocessor) = "pa_pgsql.cmo"
   archive(syntax,toploop) = "pa_pgsql.cmo"
   )
-----------------------------------------------


4.3. Package that is a container for several syntax extensions:
===============================================================

I would place P4ck in this category, though it is presently not available
via GODI, and I don't know if its author would like to see it added as
a single godi-p4ck package or if split into individual packages.

Suppose we have a dummy package "foobar", which is simply a container
for two independent syntax extensions, "openin" and "memoization".
Users can refer to each extension as "foobar.openin.syntax" and
"foobar.memoization.syntax".  Moreover, and if applicable, the user may
wish to use all syntax extensions defined in package foobar.  In this
case, "foobar.syntax" should load them all.  Here's how the META file
for foobar should look like:

-----------------------------------------------
name "foobar"
version = "1.0"
description = "Collection of useful syntax extensions"

package "openin"
   (
   version = "1.0"
   description = "Container for openin syntax extension"
   package "syntax"
      (
      version = "1.0"
      requires = "camlp4"
      description = "Syntax extension for openin"
      archive(syntax,preprocessor) = "pa_openin.cmo"
      archive(syntax,toploop) = "pa_openin.cmo"
      )
   )

package "memoization"
   (
   version = "1.0"
   description = "Container for memoization syntax extension"
   package "syntax"
      (
      version = "1.0"
      requires = "camlp4"
      description = "Syntax extension for memoization"
      archive(syntax,preprocessor) = "pa_memoization.cmo"
      archive(syntax,toploop) = "pa_memoization.cmo"
      )
   )

package "syntax"
   (
   version = "1.0"
   requires = "foobar.openin.syntax,foobar.memoization.syntax"
   description = "All syntax extensions contained by foobar"
   )
-----------------------------------------------


REFERENCES:
===========

[1] Ocamlbuild Wiki - "Using ocamlfind with ocamlbuild":
http://brion.inria.fr/gallium/index.php/Using_ocamlfind_with_ocamlbuild

[2] The findlib User's Guide - "Does Findlib support camlp4?":
http://www.ocaml-programming.de/packages/documentation/findlib/guide-html/x412.html




      ___________________________________________________________ 
Rise to the challenge for Sport Relief with Yahoo! For Good  

http://uk.promotions.yahoo.com/forgood/