ocaml support in autotools

ocaml support in autotools

[Guillaume Rousse]

Hello.

I'd like to have ocaml support in autotools (autoconf + automake), to
avoid heavy code duplication between several projects (at least
ocmalimages and activedvi).

Autoconf support is quite simple to do, and even already done (I easily
found  reference to
http://www.lri.fr/~filliatr/ftp/ocaml/misc/configure.in. But instead of
loosely distributing those macros in a custom package, as currently done
in Debian, I'd like to submit them directly to autoconf maintainers
(with some modification, and with author's agreement of course). They
are OK for this, provided there is some prior consensus in ocaml
community first, to avoid continual macro change request of course.

Automake support is more complex, but apparently someone already tried
it, as discussed at
http://caml.inria.fr/pub/ml-archives/caml-list/2003/12/c9bf9f5d4423a9259f12ccc805fe4c1
d.fr.html
http://sourceware.org/ml/automake/2003-12/msg00153.html
I'd be interested in current status of this effort before further
investigation.

Comments ?
-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: [Caml-list] ocaml support in autotools

[Stefano Zacchiroli]

On Mon, Jul 31, 2006 at 06:14:44PM +0200, Guillaume Rousse wrote:
> I'd like to have ocaml support in autotools (autoconf + automake), to
> avoid heavy code duplication between several projects (at least
> ocmalimages and activedvi).

I would totally welcome such support, but I never followed the path of
looking for the documentation on how to integrate official support in
autotools. If you are willing to it would be great!

> Autoconf support is quite simple to do, and even already done (I easily
> found  reference to
> http://www.lri.fr/~filliatr/ftp/ocaml/misc/configure.in. But instead of

After a quick look at it, I found disappointing there is no support at
all for findlib. I definitely want to be able to:
1) test whether or not findlib is available
2) test whether or not a given findlib-compatible library is available
   or not (assuming findlib is available of course)

I have some autoconf code that already do that, it is available at [1]
(look for "$OCAMLFIND query"). Feel free to reuse it.

Cheers.

[1] http://helm.cs.unibo.it/websvn/filedetails.php?repname=helm&path=%2Ftrunk%2Fhelm%2Fsoftware%2Fconfigure.ac&rev=0&sc=0

-- 
Stefano Zacchiroli -*- Computer Science PhD student @ Uny Bologna, Italy
zack@{cs.unibo.it,debian.org,bononia.it} -%- http://www.bononia.it/zack/
If there's any real truth it's that the entire multidimensional infinity
of the Universe is almost certainly being run by a bunch of maniacs. -!-

Re: [Caml-list] ocaml support in autotools

[Erik de Castro Lopo]

Guillaume Rousse wrote:

> Hello.
> 
> I'd like to have ocaml support in autotools (autoconf + automake),

I'd love to see autotol support as well.

> Autoconf support is quite simple to do, and even already done (I easily
> found  reference to
> http://www.lri.fr/~filliatr/ftp/ocaml/misc/configure.in. But instead of
> loosely distributing those macros in a custom package, as currently done
> in Debian, I'd like to submit them directly to autoconf maintainers
> (with some modification, and with author's agreement of course). They
> are OK for this, provided there is some prior consensus in ocaml
> community first, to avoid continual macro change request of course.

I've never used autotools for Ocaml stuff, but thats because I
have not yet distributed any Ocaml code. That should change
in the next couple of months and I'd be happy to start looking
at this now.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
"I ran it on my DeathStation 9000 and demons flew out of my nose." --Kaz

Re: [Caml-list] ocaml support in autotools

[Christian Lindig]

On Aug 1, 2006, at 10:15 AM, Erik de Castro Lopo wrote:
> I'd like to have ocaml support in autotools (autoconf + automake),

Having a ./configure is very convenient for developers and users alike. 
Hoever, it does not have to be generated from Autoconf. The OCaml 
distribution uses manually written shell scripts, I am using a Perl 
script (since Perl is basically always available). You can take a look 
here:

	https://trac.vistabella.de/ocamlburg/browser/trunk/configure

I found it much easier to implement the checks that I need in my own 
little Perl script rather than relying on Autotools.

-- Christian

Re: [Caml-list] ocaml support in autotools

[Erik de Castro Lopo]

Christian Lindig wrote:

> Having a ./configure is very convenient for developers and users alike. 

Agreed.

> Hoever, it does not have to be generated from Autoconf. The OCaml 
> distribution uses manually written shell scripts,

Yes, I am aware than I can write my own configure script but I 
would *much* rather user autoconf and automake if they both
support Ocaml. I already use autoconf, automake and libtool
for two C libraries.

> I found it much easier to implement the checks that I need in my own 
> little Perl script rather than relying on Autotools.

I have and will continue to write M4 macros for autoconf as
required.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
"Crap can work. Given enough thrust pigs will fly, but it's not necessary a
good idea."  -- Alexander Viro on linux-kernel mailing list

Re: [Caml-list] ocaml support in autotools

[Hendrik Tews]

Christian Lindig <lindig@vistabella.de> writes:

   I found it much easier to implement the checks that I need in my own
   little Perl script rather than relying on Autotools.
   
I have the same experience. However, this is only the case
because proper ocaml support is missing from autoconf.

Bye,

Hendrik

Re: [Caml-list] ocaml support in autotools

[skaller]

On Tue, 2006-08-01 at 13:30 +0200, Hendrik Tews wrote:
> Christian Lindig <lindig@vistabella.de> writes:
> 
>    I found it much easier to implement the checks that I need in my own
>    little Perl script rather than relying on Autotools.
>    
> I have the same experience. However, this is only the case
> because proper ocaml support is missing from autoconf.

I use python myself. Either way, Autotools are exceptionally
evil and best avoided at all costs. They rarely even work
on Linux, especially libtool which is utter garbage.

Not much hope of it being useful on Windows. Really,
we don't need this junk when we have real programming 
languages like Python or Perl to do the work -- and they're
a lot more portable than autocrud ever will be.

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] ocaml support in autotools

[Guillaume Rousse]

skaller wrote:
> Not much hope of it being useful on Windows. Really,
> we don't need this junk when we have real programming 
> languages like Python or Perl to do the work -- and they're
> a lot more portable than autocrud ever will be.
Nice troll, but completly OT.
-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: [Caml-list] ocaml support in autotools

[skaller]

On Tue, 2006-08-01 at 14:50 +0200, Guillaume Rousse wrote:
> skaller wrote:
> > Not much hope of it being useful on Windows. Really,
> > we don't need this junk when we have real programming 
> > languages like Python or Perl to do the work -- and they're
> > a lot more portable than autocrud ever will be.
> Nice troll, but completly OT.

Is 'ocaml support in autotools' and my comment is 
that autotools are crud, there are better ways to do things, 
and I would HATE to see Ocaml packages released with autocrud
build system instead of the quality build tools it
deserves.

Seems like YOU are the troll .. since your comment is 
in fact unrelated to the topic.

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] ocaml support in autotools

[Guillaume Rousse]

Christian Lindig wrote:
> 
> On Aug 1, 2006, at 10:15 AM, Erik de Castro Lopo wrote:
>> I'd like to have ocaml support in autotools (autoconf + automake),
> 
> Having a ./configure is very convenient for developers and users alike.
> Hoever, it does not have to be generated from Autoconf. The OCaml
> distribution uses manually written shell scripts, I am using a Perl
> script (since Perl is basically always available). You can take a look
> here:
> 
>     https://trac.vistabella.de/ocamlburg/browser/trunk/configure
> 
> I found it much easier to implement the checks that I need in my own
> little Perl script rather than relying on Autotools.
You're perfectly right, after all only the result really matters from a
user point of view.

However, from my own experience, people attempting to implement by
themselves a build system, instead of using/contributing to an existing
one (autotools is the not the only one) generally only consider their
own constraints, or the one they are aware of, and generaly fails in
specific scenarios, most notably packaging (properly tagged dynamic
librairies, parallel build, cross-compilation, specific optimisation
flags, fake root installation, etc...). Your own private wheel may be
more convenient for you, but it is generally less robust than one
routinely used by hundreds of people.

Of course, this is recurrente debate :)
-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: [Caml-list] ocaml support in autotools

[Christian Lindig]

On Aug 2, 2006, at 2:46 PM, Guillaume Rousse wrote:
> However, from my own experience, people attempting to implement by
> themselves a build system, instead of using/contributing to an existing
> one (autotools is the not the only one) generally only consider their
> own constraints, or the one they are aware of, and generaly fails in
> specific scenarios [..]

I totally agree that building on the knowledge of Autotools would be 
better. But in my experience I have to write a lot of shell code in 
configure.in because the tests I require are not provided (or I can't 
find them). There is no guarantee that this code does not break in 
specific scenarios (but the chance of breaking is slimmer - I admit).

-- Christian

Re: [Caml-list] ocaml support in autotools

[Hendrik Tews]

Guillaume Rousse <Guillaume.Rousse@inria.fr> writes:

   I'd like to have ocaml support in autotools (autoconf + automake), to
   avoid heavy code duplication between several projects (at least
   ocmalimages and activedvi).
   
I would very much like to see this. Please go ahead. I did some
autoconf hacking for the CCSL compiler, see the files aclocal.ml
and configure.ac in
http://wwwtcs.inf.tu-dresden.de/~tews/ccsl/viewcvs.cgi/Ccslc/
Feel free to reuse that code.

The main problem with autoconf from my point of view (at least in
2.57, the latest version I looked at) is that it provides a lot
of stuff, which is only relevant for C/C++ projects and which you
cannot easily disable. For instance the configure options
--includedir. Further, there is no way to add an additional
configure option. I have therefore overridden the internal macros
_AC_INIT_PARSE_ARGS and _AC_INIT_HELP in a private aclocal.m4.
However, now I have to adopt my changes to every new autoconf
version. Idially, there should be a simple way to disable each of
the standard configure options and to add new ones.

   http://www.lri.fr/~filliatr/ftp/ocaml/misc/configure.in

This version searches in the PATH, but sometimes the right ocaml
version is not in the path.

Bye,

Hendrik Tews

Re: [Caml-list] ocaml support in autotools

[Erik de Castro Lopo]

Hendrik Tews wrote:

> Further, there is no way to add an additional
> configure option.

Does AC_ARG_ENABLE not do the trick?

I use something like this:

    AC_ARG_ENABLE(gcc-werror,
	AC_HELP_STRING([--enable-gcc-werror], [enable -Werror in all Makefiles]))

which sets the variable $enable_gcc_werror.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
A Microsoft Certified System Engineer is to computing what a
MacDonalds Certified Food Specialist is to fine cuisine.

Re: [Caml-list] ocaml support in autotools

[Guillaume Rousse]

Hendrik Tews wrote:
> Guillaume Rousse <Guillaume.Rousse@inria.fr> writes:
> 
>    I'd like to have ocaml support in autotools (autoconf + automake), to
>    avoid heavy code duplication between several projects (at least
>    ocmalimages and activedvi).
>    
> I would very much like to see this. Please go ahead. I did some
> autoconf hacking for the CCSL compiler, see the files aclocal.ml
> and configure.ac in
> http://wwwtcs.inf.tu-dresden.de/~tews/ccsl/viewcvs.cgi/Ccslc/
> Feel free to reuse that code.
I had a quick look, but I'm not sure of the interest of a specific
ocamlbin parameter, whereas you may as well just use PATH:

AC_PATH_PROG(CAMLC, ocamlc)
if test -z "$CAMLC"; then
    AC_MSG_ERROR([objective-caml compiler not found, check your PATH])
fi

Standard macros, such as AC_PROG_CC, also relies on default PATH, and
don't provide additional options for supplying program specific one.

> The main problem with autoconf from my point of view (at least in
> 2.57, the latest version I looked at) is that it provides a lot
> of stuff, which is only relevant for C/C++ projects and which you
> cannot easily disable. For instance the configure options
> --includedir. Further, there is no way to add an additional
> configure option. I have therefore overridden the internal macros
> _AC_INIT_PARSE_ARGS and _AC_INIT_HELP in a private aclocal.m4.
> However, now I have to adopt my changes to every new autoconf
> version. Idially, there should be a simple way to disable each of
> the standard configure options and to add new ones.
You can't disable existing ones. However, it don't see anything C/C++
specific apart environment variable among them, and they don't really
hurt anyway.

For new ones, you have AC_ARG_WITH and AC_ARG_ENABLE, fur subtle
semantic differences.
-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: ocaml support in autotools

[Grigory Batalov]

On Mon, 31 Jul 2006 18:14:44 +0200
Guillaume Rousse <Guillaume.Rousse@inria.fr> wrote:

> I'd like to have ocaml support in autotools (autoconf + automake), to
> avoid heavy code duplication between several projects (at least
> ocmalimages and activedvi).
> 
> Autoconf support is quite simple to do, and even already done (I easily
> found  reference to
> http://www.lri.fr/~filliatr/ftp/ocaml/misc/configure.in.

  We have some modified scripts here too:
  http://oops.tepkom.ru/projects/template/
  There is also a short usage description.

> But instead of
> loosely distributing those macros in a custom package, as currently done
> in Debian,

  Which Debian package? (autoconf ?)

> I'd like to submit them directly to autoconf maintainers
> (with some modification, and with author's agreement of course). They
> are OK for this, provided there is some prior consensus in ocaml
> community first, to avoid continual macro change request of course.
> 
> Automake support is more complex, but apparently someone already tried
> it, as discussed at
> http://caml.inria.fr/pub/ml-archives/caml-list/2003/12/c9bf9f5d4423a9259f12ccc805fe4c1d.fr.html
> http://sourceware.org/ml/automake/2003-12/msg00153.html
> I'd be interested in current status of this effort before further
> investigation.

Re: [Caml-list] Re: ocaml support in autotools

[Guillaume Rousse]

Grigory Batalov wrote:
>> But instead of
>> loosely distributing those macros in a custom package, as currently done
>> in Debian,
> 
>   Which Debian package? (autoconf ?)
ocaml-tools, according to
http://lists.debian.org/debian-ocaml-maint/2003/10/msg00369.html
-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: [Caml-list] ocaml support in autotools

[Guillaume Rousse]

Dmitri Boulytchev wrote:
>     We are currently using a set of autoconf+automake customized scripts to
> configure and build all of our OCaml projects.  The scripts are available
> through http://oops.tepkom.ru/projects/template.
It seems the most advanced work from all I've been suggested to look at,
so I used it as a work base. I'd have some questions however about
desirable behaviour for those macros.

First, I'd like to keep interfaces and behaviour as close as possible
from standard autoconf macros, in particular avoid taking any decision
in the macro themselves. Very few of them, for example, throw an error
if test fails, they either set variable or takes an action as argument,
so as to let the programmer decides what to do.

So, I'd like to avoid any use of AC_MSG_ERROR or AC_MSG_WARN in those
macros, only AC_MSG_RESULT for printing information. The use of
action-if-not-found argument in macro checking for several programs is
also unpractical, as its exact semantic is quite fuzzy (what if one of
them but not the others are found ?). So, those macros should only set
useful variables, without side effect.

They are exceptions, tough, for compiler macros-related macros, such as
AC_PROG_CC and AC_PROG_CPP. I guess this is because they are unlikely to
get used in conditional context. So I guess it would be quite reasonable
to make AC_PROG_OCAML fails if no working ocaml is found. And probably
also AC_PROG_CAMLP4. But definitevely not for others, and particulary
for macros macros looking for ocamllex or ocamlyacc: in C world, lex and
yacc-generated sources are generally distributed, to make lex and yacc
use non mandatory on installation platforms, and as a consequence,
AC_PROG_LEX and AC_PROG_YACC are not fatal macros.

So my current opinion is:
- have AC_PROG_OCAML fails if ocamlc is not found
- have AC_PROG_CAMLP4 fails if ocamlp4 is not found
- have all other macros never fail

Second, what to do with optimised versions ? It is desirable for the
user or the developer to be able to select between optimised and
non-optimised version of a given tool ?

If not, having a single variable for each tool, silently defined to the
optimised version if available and usable, would be the best option.
OCAMLC would then correspond to ocamlc.opt or ocamlc

If yes, a combination of user options (through AC_ARG_ENABLE),
developper options (conditional macros arguments), or distinct variables
(OCAMLC and OCAMLC_OPT) would do the trick.

Third, I'd like some standard macro to allow the user to select if he
wants bytcode compilation, native compilation, or both, the same as you
have when using libtool for producing either static or dynamic
libraries. Does it makes senses to add those switches to AC_PROG_OCAML,
and to define additional variables, such as OCAML_WANT_BYTECODE and
OCAML_WANT_NATIVE ?

Fourth, in the AC_PROG_FINDLIB, there is a switch allowing the user to
disallow the use of ocamlfind, even if present on the system. Why is
thie behaviour desirable ?

BTW, my global ocaml knwoledge is quite slim, so I may miss details :)
-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: [Caml-list] ocaml support in autotools

[Erik de Castro Lopo]

Guillaume Rousse wrote:

> So my current opinion is:
> - have AC_PROG_OCAML fails if ocamlc is not found
> - have AC_PROG_CAMLP4 fails if ocamlp4 is not found
> - have all other macros never fail
> 
> Second, what to do with optimised versions ? It is desirable for the
> user or the developer to be able to select between optimised and
> non-optimised version of a given tool ?
> 
> If not, having a single variable for each tool, silently defined to the
> optimised version if available and usable, would be the best option.
> OCAMLC would then correspond to ocamlc.opt or ocamlc

I thing you should have separate AC_PROG_OCAMLC and AC_PROG_OCAMLOPT
with separate variables.
 
> Third, I'd like some standard macro to allow the user to select if he
> wants bytcode compilation, native compilation, or both, the same as you
> have when using libtool for producing either static or dynamic
> libraries. Does it makes senses to add those switches to AC_PROG_OCAML,
> and to define additional variables, such as OCAML_WANT_BYTECODE and
> OCAML_WANT_NATIVE ?

Works for me.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
I have found that good programmers either do not make the kind
of mistakes that Ada can prevent, or insert enough checks that
they catch those mistakes about as efficiently as an Ada
environment can. At that point, the use of Ada gives no further
productivity advantage.

Re: ocaml support in autotools

[Grigory Batalov]

On Thu, 03 Aug 2006 14:56:05 +0200
Guillaume Rousse <Guillaume.Rousse@inria.fr> wrote:

> First, I'd like to keep interfaces and behaviour as close as possible
> from standard autoconf macros, in particular avoid taking any decision
> in the macro themselves. Very few of them, for example, throw an error
> if test fails, they either set variable or takes an action as argument,
> so as to let the programmer decides what to do.
> 
> So, I'd like to avoid any use of AC_MSG_ERROR or AC_MSG_WARN in those
> macros, only AC_MSG_RESULT for printing information. The use of
> action-if-not-found argument in macro checking for several programs is
> also unpractical, as its exact semantic is quite fuzzy (what if one of
> them but not the others are found ?). So, those macros should only set
> useful variables, without side effect.

  Any examples? =)

> They are exceptions, tough, for compiler macros-related macros, such as
> AC_PROG_CC and AC_PROG_CPP. I guess this is because they are unlikely to
> get used in conditional context. So I guess it would be quite reasonable
> to make AC_PROG_OCAML fails if no working ocaml is found. And probably
> also AC_PROG_CAMLP4. But definitevely not for others, and particulary
> for macros macros looking for ocamllex or ocamlyacc: in C world, lex and
> yacc-generated sources are generally distributed, to make lex and yacc
> use non mandatory on installation platforms, and as a consequence,
> AC_PROG_LEX and AC_PROG_YACC are not fatal macros.
> 
> So my current opinion is:
> - have AC_PROG_OCAML fails if ocamlc is not found
> - have AC_PROG_CAMLP4 fails if ocamlp4 is not found
> - have all other macros never fail
> 
> Second, what to do with optimised versions ? It is desirable for the
> user or the developer to be able to select between optimised and
> non-optimised version of a given tool ?
> 
> If not, having a single variable for each tool, silently defined to the
> optimised version if available and usable, would be the best option.
> OCAMLC would then correspond to ocamlc.opt or ocamlc

  As we do right now.

> If yes, a combination of user options (through AC_ARG_ENABLE),
> developper options (conditional macros arguments), or distinct variables
> (OCAMLC and OCAMLC_OPT) would do the trick.

  We use OCAMLCDOTOPT for ocamlc.opt and OCAMLOPTDOTOPT for ocamlopt.opt
  inside AC_PROG_OCAML macro, but they are not available to user yet.

  It seems like optimized tools work faster (?). What if we will prefer
  optimized tools by default, but switch to bytecoded if user gives
  --without-opttools configure argument ?

  BTW, in my distribution ocamlc is a soft link to ocamlc.opt =).

> Third, I'd like some standard macro to allow the user to select if he
> wants bytcode compilation, native compilation, or both, the same as you
> have when using libtool for producing either static or dynamic
> libraries. Does it makes senses to add those switches to AC_PROG_OCAML,
> and to define additional variables, such as OCAML_WANT_BYTECODE and
> OCAML_WANT_NATIVE ?

  Configure options
  --enable-native sets OCAML_BUILD_NATIVE
  --enable-bytecode sets OCAML_BUILD_BYTECODE
  or similar ?

  Optimized objects have different file extension (.cmx instead of .cmo),
  so it is quite easy to define another build rule in Makefile.

> Fourth, in the AC_PROG_FINDLIB, there is a switch allowing the user to
> disallow the use of ocamlfind, even if present on the system. Why is
> thie behaviour desirable ?

  I guess, we have found a broken findlib on some platform and desided
  to optionally disable it.

> BTW, my global ocaml knwoledge is quite slim, so I may miss details :)

Re: [Caml-list] Re: ocaml support in autotools

[skaller]

On Fri, 2006-08-04 at 04:40 +0400, Grigory Batalov wrote:

BTW: anyone working on this should examine the Debian Ocaml Policy.
Sorry no link off hand, ask on

debian-ocaml-maint@lists-debian.org

At least any macros should work well with Debian packagers
requirements, people there have high expertise packaging Ocaml
stuff.

In particular .. you should note that 'detecting' ocaml libraries
is VERY HARD because they're locked to a fixed version of Ocaml:
the Ocaml ABI changes with every release (including patches).

AFAIK, you simply cannot 'autoconf' check a required library
is available: the check will not reveal the version of Ocaml
used to build that library .. and it must be built with
the version of Ocaml being used for this tarball build
or the library is useless.

I think what you actually need to do is:

(a) find the appropriate Ocaml program
(b) find the candidate library
(c) check the time stamps on both, and reject the library
unless its date is newer than the ocaml program.

[My own personal solution to this is (a) never use third party
libraries and (b) if I have to, bundle the source code:
always build everything from source. For C this would be
a nightmare .. for Ocaml it is no problem, the compiler
is so fast, and it gives you control of compilation options..
not that there are many]

>   We use OCAMLCDOTOPT for ocamlc.opt and OCAMLOPTDOTOPT for ocamlopt.opt
>   inside AC_PROG_OCAML macro, but they are not available to user yet.
> 
>   It seems like optimized tools work faster (?). What if we will prefer
>   optimized tools by default, but switch to bytecoded if user gives
>   --without-opttools configure argument ?

I think you have this procedure:

(a) you detect and set variable for support for

native compiler: one of: ocamlopt.opt, ocamlopt
bytecode compiler: one of: ocamlc.opt, ocamlc

interface compiler: one of: ocamlc.opt, ocamlc
 (this is the same program as bytecode compiler, but it 
  should be managed separately)

plus other tools (ocamldep, ocamldoc, findlinb, etc).

(b) There at least are 3 models of compilation:

native code, bytecode, standalone bytecode
(the latter is a single executable file containing the 
bytecode interpreter and bytecode in one package).

The user has several choices:

1. Chose fastest model: 
  native code if possible, otherwise bytecode

2. Chose standalone executable (rare):
  native code if possible, otherwise standalone bytecode

3. Chose bytecode

4. Chose native code

5. BOTH bytecode and nativecode 

Developers often want (5) both, but it plays interesting
havoc with naive build scripts: the generated programs
must have distinct names (libraries and object files
already do).

My own build system can't handle this :)

> > Fourth, in the AC_PROG_FINDLIB, there is a switch allowing the user to
> > disallow the use of ocamlfind, even if present on the system. Why is
> > thie behaviour desirable ?
> 
>   I guess, we have found a broken findlib on some platform and desided
>   to optionally disable it.

Also because a developer may want to check a system builds
without requiring findlib.

but the real reason is simpler: some people write Makefiles
which require findlib, and will not work without it.

Some people write Makefiles that will not work with it.

Some people try to support both.


-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] Re: ocaml support in autotools

[skaller]

On Fri, 2006-08-04 at 15:32 +1000, skaller wrote:
> On Fri, 2006-08-04 at 04:40 +0400, Grigory Batalov wrote:
> 
> BTW: anyone working on this should examine the Debian Ocaml Policy.
> Sorry no link off hand, ask on
> 
> debian-ocaml-maint@lists-debian.org

Oops .. should be lists.debian.org, sorry!

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] Re: ocaml support in autotools

[Guillaume Rousse]

skaller wrote:
> On Fri, 2006-08-04 at 15:32 +1000, skaller wrote:
>> On Fri, 2006-08-04 at 04:40 +0400, Grigory Batalov wrote:
>>
>> BTW: anyone working on this should examine the Debian Ocaml Policy.
>> Sorry no link off hand, ask on
>>
>> debian-ocaml-maint@lists-debian.org
> 
> Oops .. should be lists.debian.org, sorry!
I guess debian people (Hi Sven!) follow this list too ? I'd prefer to
keep the discussion in one place only.

-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: [Caml-list] Re: ocaml support in autotools

[Anil Madhavapeddy]

On Fri, Aug 04, 2006 at 03:32:26PM +1000, skaller wrote:
> 
> AFAIK, you simply cannot 'autoconf' check a required library
> is available: the check will not reveal the version of Ocaml
> used to build that library .. and it must be built with
> the version of Ocaml being used for this tarball build
> or the library is useless.
> 
> I think what you actually need to do is:
> 
> (a) find the appropriate Ocaml program
> (b) find the candidate library
> (c) check the time stamps on both, and reject the library
> unless its date is newer than the ocaml program.
> 

You can simply run a test link against the OCaml library as part
of the AC_OCAML_CHECK_LIB (or whatever its called), just as is done
for C library testing.  This will error out almost always if the
versions are out of sync.

-- 
Anil Madhavapeddy                                 http://anil.recoil.org
University of Cambridge                          http://www.cl.cam.ac.uk

Re: [Caml-list] Re: ocaml support in autotools

[Guillaume Rousse]

Anil Madhavapeddy wrote:
> On Fri, Aug 04, 2006 at 03:32:26PM +1000, skaller wrote:
>> AFAIK, you simply cannot 'autoconf' check a required library
>> is available: the check will not reveal the version of Ocaml
>> used to build that library .. and it must be built with
>> the version of Ocaml being used for this tarball build
>> or the library is useless.
>>
>> I think what you actually need to do is:
>>
>> (a) find the appropriate Ocaml program
>> (b) find the candidate library
>> (c) check the time stamps on both, and reject the library
>> unless its date is newer than the ocaml program.
>>
> 
> You can simply run a test link against the OCaml library as part
> of the AC_OCAML_CHECK_LIB (or whatever its called), just as is done
> for C library testing.  This will error out almost always if the
> versions are out of sync.
I also favor this option.

-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: [Caml-list] Re: ocaml support in autotools

[Stefano Zacchiroli]

[-- Attachment #1: Type: text/plain, Size: 1815 bytes --]

On Fri, Aug 04, 2006 at 03:32:26PM +1000, skaller wrote:
> BTW: anyone working on this should examine the Debian Ocaml Policy.
> Sorry no link off hand, ask on

The policy is available on line at this URL:

  http://pkg-ocaml-maint.alioth.debian.org/ocaml_packaging_policy.html/index.html

> At least any macros should work well with Debian packagers
> requirements, people there have high expertise packaging Ocaml
> stuff.
> 
> In particular .. you should note that 'detecting' ocaml libraries
> is VERY HARD because they're locked to a fixed version of Ocaml:
> the Ocaml ABI changes with every release (including patches).

Still, I don't see the relationship of the policy with ocaml autotools
support. The faced issues are different. We have the need of ensuring no
two binaries packages could be installed at the same time if they rely
on different .cmi assumptions. In other words we want to ensure that
libraries that cannot be linked together could be installed at the same
time.

You may want to perform such a check at configure time, to ensure that
linking will succeed, but IMO it would be overkilling and not really
needed. After all the target user of autotools is a developer, not the
final user. When the final user has to compile libraries by himself,
then the infrastructure it uses (e.g. source based distributions or GODI
for we ocamlers) for compiling is responsible to ensure that the
versions of the library he is building are linkable together.

Cheers.

-- 
Stefano Zacchiroli -*- Computer Science PhD student @ Uny Bologna, Italy
zack@{cs.unibo.it,debian.org,bononia.it} -%- http://www.bononia.it/zack/
If there's any real truth it's that the entire multidimensional infinity
of the Universe is almost certainly being run by a bunch of maniacs. -!-

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [Caml-list] Re: ocaml support in autotools

[skaller]

On Fri, 2006-08-04 at 14:48 +0200, Stefano Zacchiroli wrote:
> On Fri, Aug 04, 2006 at 03:32:26PM +1000, skaller wrote:
> > BTW: anyone working on this should examine the Debian Ocaml Policy.
> > Sorry no link off hand, ask on
> 
> The policy is available on line at this URL:
> 
>   http://pkg-ocaml-maint.alioth.debian.org/ocaml_packaging_policy.html/index.html

Thanks..

> > At least any macros should work well with Debian packagers
> > requirements, people there have high expertise packaging Ocaml
> > stuff.

[]

> Still, I don't see the relationship of the policy with ocaml autotools
> support. 

I mention the policy document because it provides context which
may help factor autotools macros/variables etc.

There is a relationship: Debian packagers have to deal with
people's build systems.

> You may want to perform such a check at configure time, to ensure that
> linking will succeed, but IMO it would be overkilling and not really
> needed. After all the target user of autotools is a developer, not the
> final user. 

That's fine when everything works.. the problem comes when
something goes wrong. Then the person with the error is going
to try all sorts of things to solve the problem .. using
the latest cvs/svn repository version for example .. which
typically only have the autotools original sources, not
the generated scripts.

Perhaps not on Debian or GODI .. but it is common for Ocaml 
programmers to get inconsistent libraries simply because there's
no way to keep track of what libraries you have -- happens
to me all the time when I upgrade (in fact, its a bug in 
the Ocaml install procedure itself .. it doesn't properly
clean out the installation target)

So maybe a trial link is overkill .. when it succeeds:
it might be useful is it fails though.

In any case, this kind of thing is precisely what configuration
scripts actually do. Autoconf generated script regularly
compiles, links, and even runs C programs .. and so does
my Python hosted config script. For example we had some
code failing and it turned out our test of Linux epoll
was faulty. We just tried to compile a file using
create_epoll() but that compiles and links on Linux 2.4 kernels..
it doesn't work though. You actually have to executed the function
and check it doesn't return -1 to know if you have epoll support.

Exactly how one would do this in a cross compilation environment
I don't know (Felix is a cross-cross compiler). 

-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net

Re: [Caml-list] Re: ocaml support in autotools

[Stefano Zacchiroli]

On Sat, Aug 05, 2006 at 10:36:16AM +1000, skaller wrote:
> Exactly how one would do this in a cross compilation environment
> I don't know (Felix is a cross-cross compiler). 

An idea can be provide a couple of autoconf macros, something like
(sorry, I'm not really fluent in autoconf, I may have got an unusal
name, wrong syntax, ....):

- AC_CHECK_OCAML_MD5(Foo, 13c5192c85629bcaad3e2d74c22f08d4)

  This will ensure that all ocaml libraries checked for later using
  other macros (e.g. one that look for a given findlib library) trust
  the Foo interface to have the given md5sum

  Implement what's needed, but it will be probably hard to maintain for
  developers

- AC_CHECK_OCAML_CMI(foo.cmi)

  As above, but ensure that all libraries have the same md5sum for all
  the interfaces referenced by foo.cmi. You can easily extend this idea
  and provide an .mli which will be compiled and then used as a
  reference.

Cheers.

-- 
Stefano Zacchiroli -*- Computer Science PhD student @ Uny Bologna, Italy
zack@{cs.unibo.it,debian.org,bononia.it} -%- http://www.bononia.it/zack/
If there's any real truth it's that the entire multidimensional infinity
of the Universe is almost certainly being run by a bunch of maniacs. -!-

Re: [Caml-list] Re: ocaml support in autotools

[Guillaume Rousse]

I'm trying to factorize answers here.

skaller wrote:
> At least any macros should work well with Debian packagers
> requirements, people there have high expertise packaging Ocaml
> stuff.
Indeed.

> In particular .. you should note that 'detecting' ocaml libraries
> is VERY HARD because they're locked to a fixed version of Ocaml:
> the Ocaml ABI changes with every release (including patches).
Right, I introduced an automatic versioned dependency in mandriva rpm
for this reason.

> AFAIK, you simply cannot 'autoconf' check a required library
> is available: the check will not reveal the version of Ocaml
> used to build that library .. and it must be built with
> the version of Ocaml being used for this tarball build
> or the library is useless.
> 
> I think what you actually need to do is:
> 
> (a) find the appropriate Ocaml program
> (b) find the candidate library
> (c) check the time stamps on both, and reject the library
> unless its date is newer than the ocaml program.
Right, and it also support my point on distinguishing between retrieving
information, and testing it is accurate.

By why not try to compile and link for (c) instead of trusting
timestamps ? This is an higher-level check, and closer from the actual
problem moreover.

> [My own personal solution to this is (a) never use third party
> libraries and (b) if I have to, bundle the source code:
> always build everything from source.
Are you trying to compete with Java and PHP developper here ? Using
private code copies may make your (programmer) life easier, but just
make distribution package maintainers works more complex, as they have
to remove them and make sure your code build with system version.

That's a wrong solution to a real problem.

> For C this would be
> a nightmare .. for Ocaml it is no problem, the compiler
> is so fast, and it gives you control of compilation options..
> not that there are many]
> 
>>   We use OCAMLCDOTOPT for ocamlc.opt and OCAMLOPTDOTOPT for ocamlopt.opt
>>   inside AC_PROG_OCAML macro, but they are not available to user yet.
>>
>>   It seems like optimized tools work faster (?). What if we will prefer
>>   optimized tools by default, but switch to bytecoded if user gives
>>   --without-opttools configure argument ?
It is precisely my point: is so much flexibility desirable ?

> I think you have this procedure:
> 
> (a) you detect and set variable for support for
> 
> native compiler: one of: ocamlopt.opt, ocamlopt
> bytecode compiler: one of: ocamlc.opt, ocamlc
> 
> interface compiler: one of: ocamlc.opt, ocamlc
>  (this is the same program as bytecode compiler, but it 
>   should be managed separately)
I don't understand the interest of that specific point.

> plus other tools (ocamldep, ocamldoc, findlinb, etc).
This is the simplest model, which is also the more robust.

I have some proposal that seems quite simple to achieve:
- for each of those alternative, have AC_PROG_OCAML export two
variables, FOO and FOO_OPT
- by default, FOO would point to the same value as FOO_OPT if available
- a user switch would prevent this behaviour

So, the developper could enforce the use of only optimised versions, or
let the choice to the user with a correct default. Does it sounds good ?

> (b) There at least are 3 models of compilation:
> 
> native code, bytecode, standalone bytecode
> (the latter is a single executable file containing the 
> bytecode interpreter and bytecode in one package).
> 
> The user has several choices:
> 
> 1. Chose fastest model: 
>   native code if possible, otherwise bytecode
> 
> 2. Chose standalone executable (rare):
>   native code if possible, otherwise standalone bytecode
> 
> 3. Chose bytecode
> 
> 4. Chose native code
> 
> 5. BOTH bytecode and nativecode 
> 
> Developers often want (5) both, but it plays interesting
> havoc with naive build scripts: the generated programs
> must have distinct names (libraries and object files
> already do).
> 
> My own build system can't handle this :)
Ouch, it hurts...

We're just dealing with autoconf macros here (no automake support yet),
meaning developer still has to write the corresponding Makefile.
Providing users switches in those macros force the developper to handle
them. So, I'd rather keep uneeded complexity out, at least for the moment.

Allowing 3, 4 & 5 is possible with the following scheme, based on
AM_PROG_LIBTOOL:
- by default, AC_PROG_OCAML provides --enable-native and
--enable-bytecode switches, defaulting to true
- AC_DISABLE_OCMAL_NATIVE turns off the default value for --enable-native
- AC_DISABLE_OCMAL_BYTECODE turns off the default value for
--enable-bytecode
Those switches just set OCAML_BUILD_NATIVE and OCAML_BUILD_BYTECODE
variables.

>>> Fourth, in the AC_PROG_FINDLIB, there is a switch allowing the user to
>>> disallow the use of ocamlfind, even if present on the system. Why is
>>> thie behaviour desirable ?
>>   I guess, we have found a broken findlib on some platform and desided
>>   to optionally disable it.
If there is an explicit list of platform/findlib versions known to be
broken, or a typical way to diagnose such problems, this ought to be
managed directly in the macro itself, with some kind of error message,
not through a user switch.

> Also because a developer may want to check a system builds
> without requiring findlib.
> 
> but the real reason is simpler: some people write Makefiles
> which require findlib, and will not work without it.
> 
> Some people write Makefiles that will not work with it.
> 
> Some people try to support both.
But you don't need an explicit option for this: if you really want to be
findlib-independant, test with findlib installed and without findlib
installed.

My point is: don't put too much personal preferences in those shared
macros, especially as someone else will have to maintain them. They
should only bring a minimal number of consensual primitives, upon wich
developers may design their own build script that fit their own taste.

If you look at pkg-config, for instance, there is no AC_PROG_PKCONFIG
macro that automatically bring you a switch to disable its use. However,
PKG_CHECK_MODULE allows you to override pkg-config results automatically
each time you use it for a given library.

-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: [Caml-list] ocaml support in autotools

[Guillaume Rousse]

Dmitri Boulytchev wrote:
>     We are currently using a set of autoconf+automake customized scripts to
> configure and build all of our OCaml projects.  The scripts are available
> through http://oops.tepkom.ru/projects/template.
And yet another interface question, this time more related to
AC_CHECK_OCAML_MODULE

Basically, your implementation tries to discover minimal include flags
needed to make a module available, using various strategies:
- no additional flags
- ocamlfind-based additional flags
- developper defined (through additional macro option), user-defined
(through configure option) and confidure-defined (through running
`ocamlc -where`) flags

This is not really similar to default AC_CHECK_LIB macro, which just
determine if using all currently available flags, searched library is
available or not. Nor to PKG_CHECK_MODULE (closest ocamlfind equivalent
in C world) that just returns flags computed by pkg-config.

Knowing if an ocmal module is part of core (thus no need for additional
-I flag), or has ocmalfind support can be statically determined, not at
configure run-time. So, I'd better gives AC_CHECK_OCAML_MODULE a
behaviour more similar to AC_CHECK_LIB, and split ocmalfind logic into a
distinct AC_CHECK_OCAMLFIND_MODULE (or any other better name) that would
just set proper variables.

BTW, the macro seems to be misnamed: what you check for actually is a
set of modules (the second argument), not just a single one. I guess you
want to check for something larger, such as lablgtk, which has several
modules, right ? In that case, I'd rather call it
AC_CHECK_OCAML_MODULES, or something else if there is a dedicated word
in OCaml world to call a set of modules distributed together (library in
C word, distribution in Perl world, package, ...).
-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: [Caml-list] ocaml support in autotools

[Erik de Castro Lopo]

Guillaume Rousse wrote:

> This is not really similar to default AC_CHECK_LIB macro, which just
> determine if using all currently available flags, searched library is
> available or not.

AC_CHECK_LIB is rather primitive.

> Nor to PKG_CHECK_MODULE (closest ocamlfind equivalent
> in C world) that just returns flags computed by pkg-config.

PKG_CHECK_MODULE would be a better model.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
"The lusers I know are so clueless, that if they were dipped in
 clue musk and dropped in the middle of pack of horny clues, on
 clue prom night during clue happy hour, they still couldn't get
 a clue."   --Michael Girdwood, in the monastery

Re: [Caml-list] ocaml support in autotools

[Guillaume Rousse]

Erik de Castro Lopo wrote:
> Guillaume Rousse wrote:
> 
>> This is not really similar to default AC_CHECK_LIB macro, which just
>> determine if using all currently available flags, searched library is
>> available or not.
> 
> AC_CHECK_LIB is rather primitive.
> 
>> Nor to PKG_CHECK_MODULE (closest ocamlfind equivalent
>> in C world) that just returns flags computed by pkg-config.
> 
> PKG_CHECK_MODULE would be a better model.
But it does only retrieve information, it does no actual testing.

My point is separating issues (retrieving information and testing
compilation) allow more versatility for the programmer.

-- 
Guillaume Rousse
Projet Estime, INRIA
Domaine de Voluceau
Rocquencourt - B.P. 105
78153 Le Chesnay Cedex - France

Re: ocaml support in autotools

[Grigory Batalov]

On Thu, 03 Aug 2006 17:10:53 +0200
Guillaume Rousse <Guillaume.Rousse@inria.fr> wrote:

> >     We are currently using a set of autoconf+automake customized scripts to
> > configure and build all of our OCaml projects.  The scripts are available
> > through http://oops.tepkom.ru/projects/template.
> And yet another interface question, this time more related to
> AC_CHECK_OCAML_MODULE
> 
> Basically, your implementation tries to discover minimal include flags
> needed to make a module available, using various strategies:
> - no additional flags
> - ocamlfind-based additional flags
> - developper defined (through additional macro option), user-defined
> (through configure option) and confidure-defined (through running
> `ocamlc -where`) flags
> 
> This is not really similar to default AC_CHECK_LIB macro, which just
> determine if using all currently available flags, searched library is
> available or not. Nor to PKG_CHECK_MODULE (closest ocamlfind equivalent
> in C world) that just returns flags computed by pkg-config.

  Well, we often develop several libraries "in place", without installing,
  so ../project1 ../project2 cross-referrences are widely used. This
  is why custom paths in AC_CHECK_OCAML_MODULE appeared.

> Knowing if an ocmal module is part of core (thus no need for additional
> -I flag), or has ocmalfind support can be statically determined, not at
> configure run-time. So, I'd better gives AC_CHECK_OCAML_MODULE a
> behaviour more similar to AC_CHECK_LIB, and split ocmalfind logic into a
> distinct AC_CHECK_OCAMLFIND_MODULE (or any other better name) that would
> just set proper variables.

  Ok, this looks good.

> BTW, the macro seems to be misnamed: what you check for actually is a
> set of modules (the second argument), not just a single one. I guess you
> want to check for something larger, such as lablgtk, which has several
> modules, right ? In that case, I'd rather call it
> AC_CHECK_OCAML_MODULES, or something else if there is a dedicated word
> in OCaml world to call a set of modules distributed together (library in
> C word, distribution in Perl world, package, ...).

  Sure. At the beginning we had just one module to check =), but later
  expanded macro to several. AC_CHECK_OCAML_PKG (ocamlfind calls them
  "packages") should be better. What for camplp4 files, there is several
  types ("parsers", "printers"), but they all probably should be named
  "extensions", so AC_CHECK_CAMLP4_EXT would be nice.
  (Correct me, if I'm wrong.)

Re: [Caml-list] ocaml support in autotools

[Sylvain Le Gall]

Hello,

On Mon, Jul 31, 2006 at 06:14:44PM +0200, Guillaume Rousse wrote:
> Hello.
> 
> I'd like to have ocaml support in autotools (autoconf + automake), to
> avoid heavy code duplication between several projects (at least
> ocmalimages and activedvi).
> 
> Autoconf support is quite simple to do, and even already done (I easily
> found  reference to
> http://www.lri.fr/~filliatr/ftp/ocaml/misc/configure.in. But instead of
> loosely distributing those macros in a custom package, as currently done
> in Debian, I'd like to submit them directly to autoconf maintainers
> (with some modification, and with author's agreement of course). They
> are OK for this, provided there is some prior consensus in ocaml
> community first, to avoid continual macro change request of course.
> 
> Automake support is more complex, but apparently someone already tried
> it, as discussed at
> http://caml.inria.fr/pub/ml-archives/caml-list/2003/12/c9bf9f5d4423a9259f12ccc805fe4c1
> d.fr.html
> http://sourceware.org/ml/automake/2003-12/msg00153.html
> I'd be interested in current status of this effort before further
> investigation.
> 
> Comments ?

I spend several hours (in fact weeks), trying to hack automake to make
it work with OCaml target... I don't think it could work. As said in
this thread, automake is mainly made for C/C++ like langages... In
particular if you want to compile something, you need to have a ".o"
somewhere in your compilation path (don't ask me why, that's thing i
have discovered trying to do this). Automake is actually a good tool for
doing what it is made for : provide portable build system to C/C++
project. I don't think it is a general purpose build system.

But anyway, if you think you could perform to make automake works for
ocaml, let me know !

Maybe you can have a look at the "carbon ml"/"mlmm" which  is an ocaml 
project and use automake.

Concerning autoconf, it is a great tool, the configure.in you have
already discovered is the best available source for doing ocaml stuff
with autoconf...

Kind regard
Sylvain Le Gall