Re: [Caml-list] One build system to rule them all?

[Francois Berenger <francois.berenger@inria.fr>]

On 09/12/2014 04:08 PM, Yotam Barnoy wrote:
> OK so here's my summary of what I've heard here so far:
>
> - Ocaml does not yet have a (public) build system that the majority of
> users can be happy with for all of their project needs.
> - It seems difficult to have a build system that's simple enough to use
> for small projects, flexible enough to handle the more complicated
> projects, and that can scale.
> - While declarative, dsl-based tools are appealing, they get bogged down
> as you add features to them, making them become unwieldy.
> - A better solution seems to be using a real programming language to
> describe packages. In our case that should clearly be ocaml itself. The
> trick is making the API simple.

There is maybe an API in Haskell already:
https://hackage.haskell.org/package/shake

> Looking at some of the the available options:
> - ocamlbuild (dsl-based) is a decent basic choice, but lacks advanced
> features and has become weighed down by multiple file types (_tags,
> myocamlbuild etc).
> - OASIS (dsl-based) seems to be a good description layer for projects,
> currently farming out the difficult work of building stuff to
> ocamlbuild, but not really a contender in this race except as an
> abstraction layer.
> - omake (dsl-based) has some fans, but becomes overly-complex for more
> complicated projects.
> - ocp-build (dsl-based) has had nothing but criticism on this thread. Is
> there any ongoing work to address these criticisms?
> - Jenga (uses ocaml) seems promising but has heavy dependencies, and is
> currently not geared towards light projects.
> - assemblage (uses ocaml) seems promising as well, and seems focused on
> simplicity. Any comment from assemblage contributors on its status?
>
> Also, while I understand some of the windows hatred, the fact is, most
> of the world uses windows. Even using cygwin as a translation layer
> hurts ocaml's ability to reach people. Ideally, a windows programmer
> could download opam, ocaml and perhaps a mingw compiler, and begin to
> build projects without any need for a shell or anything POSIX-like. This
> means that the build system needs to obviate the need for any shell
> commands or makefiles.
>
> On Thu, Sep 11, 2014 at 6:37 AM, Yaron Minsky <yminsky@janestreet.com
> <mailto:yminsky@janestreet.com>> wrote:
>
>     On Thu, Sep 11, 2014 at 3:53 AM, Francois Berenger
>     <francois.berenger@inria.fr <mailto:francois.berenger@inria.fr>> wrote:
>      > On 09/10/2014 08:59 PM, Yotam Barnoy wrote:
>      >>
>      >> Here's part of my motivation for starting this discussion now: I
>      >> recently experienced a hard-drive failure and had to switch to
>     my backup
>      >> windows laptop. I was dismayed that opam is still not available for
>      >> Windows, and while wodi is available, it's not nearly as well
>     developed,
>      >> and apparently requires many hacks for different packages.
>      >>
>      >> It seems to me that these hacks are some of the things that make
>      >> platform compatibility hard for opam. Each build system has
>     different
>      >> requirements and methods of compilation, and making this approach
>      >> cross-platform compatible is difficult. Ideally, a build system will
>      >> abstract away the things that are not inter-platform compatible.
>     This is
>      >> why I don't like people using makefiles. Makefiles are not available
>      >> natively on Windows, and they often contain other bits of shell code
>      >> that isn't available on all POSIX platforms, let alone on
>     Windows. Not
>      >> to mention the fact that makefiles have a very tricky and sensitive
>      >> syntax.
>      >>
>      >> I don't expect one build system to match everyone's
>     requirements, but I
>      >> think if we get together, make a list of requirements, and try
>     to get
>      >> over our personal biases, we can find something that works for most
>      >> people, and with some group effort, can work for even more people in
>      >> more use-cases. I'm not suggesting that we invent something new, but
>      >> that we take something good and make it better, as well as make an
>      >> effort to learn that tool and convert packages to use that tool.
>      >>
>      >> So here are some requirements I can think of (using some of the
>      >> suggestions that have been brought up):
>      >> - Easy to use, especially for small projects (large projects can
>     afford
>      >> to put more time into their build systems)
>      >> - Abstract away platform considerations as much as possible. No
>      >> dependence on specific shells and POSIX utilities.
>      >> - Allows compilation of C files, which is quite common in ocaml
>     packages.
>      >> - Scalable to many directories and files
>      >> - Uses ocamlfind to locate packages
>      >> - Handles camlp4 and ppx
>      >> - Parallel & incremental compilation
>      >>
>      >> About Jenga: I took a quick look at Jenga, and even though I'm
>     impressed
>      >> by its capabilities, the amount of code needed to be written
>     even for
>      >> simple projects is overwhelming. It's clearly a very flexible and
>      >> powerful tool, but I'd say it's too flexible for the mainstream.
>     There's
>      >> always room for a build system that's integrated into ocaml
>     itself (like
>      >> Shake), but I think declarative build systems tend to be easier to
>      >> comprehend for the average user.
>      >>
>      >> ocp-build actually looks very interesting. The manual (which is
>     here:
>      >>
>      >>
>     http://github.com/OCamlPro/ocp-build/blob/master/docs/user-manual/user-manual.pdf?raw=true)
>      >> is incomplete, but contains a nice survey of the existing build
>     tools,
>      >> and motivation for making ocp-build. Has anyone had experience with
>      >> ocp-build? Opam seems to be using it, but they also use a makefile
>      >> (why?) with a bunch of shell commands inside (which is precisely the
>      >> problem from my perspective). ocp-build is supposedly compatible
>     with
>      >> Windows, too.
>      >
>      >
>      > Here is one thing I dislike in ocp-build: you have to explicitely
>     list all
>      > the source files (and maybe same thing for Jenga).
>
>     Jenga is quite flexible, so you could write your jengaroot either way
>     (to demand or not demand explicit source file lists.)  Our internal
>     jengaroot does not require explicit source file lists.
>
>     Not to confuse people, though, Jenga isn't really suitable for people
>     just picking up and using yet.  We haven't yet written a suitable
>     jengaroot for general use yet (though there is a version of our
>     internal jengaroot that we've released, but it's not yet ready for
>     prime time.)  So Jenga is interesting as an option to discuss, but not
>     yet a practical option we want to encourage casual users to try.
>
>      > I am way too lazy to do that, especially since other tools are able
>      > to infer this list automatically (a library is just a list of
>     modules,
>      > an executable has a single entry point, the main function in a
>     single file).
>      >
>      >> Yotam
>      >>
>      >>
>      >> On Wed, Sep 10, 2014 at 11:17 AM, Leonardo Laguna Ruiz
>     <modlfo@gmail.com <mailto:modlfo@gmail.com>
>      >> <mailto:modlfo@gmail.com <mailto:modlfo@gmail.com>>> wrote:
>      >>
>      >>     My pick is ocamlbuild because:
>      >>
>      >>     - It works the same way in all platforms that I work (linux,osx,
>      >>     windows msvc port, cygwin)
>      >>     - If I can compile ocaml, then I have ocamlbuild
>      >>     - It does not require external libraries
>      >>
>      >>     I have to say that I feel a little bit annoyed by the fact
>     that some
>      >>     other build systems or tools, do not work correctly in all the
>      >>     platforms that I want to support.
>      >>
>      >>     Leonardo
>      >>
>      >>
>      >>
>      >>     On 9/10/2014 4:23 PM, Gerd Stolpmann wrote:
>      >>
>      >>         Am Mittwoch, den 10.09.2014, 15:29 +0200 schrieb Francois
>      >> Berenger:
>      >>
>      >>             More seriously, concerning build systems, we clearly
>     have
>      >>             quite some
>      >>             choice on the OCaml shelf:
>      >>             - obuild
>      >>             - ocamlbuild
>      >>             - omake
>      >>             - oasis (which in fact uses ocamlbuild, don't forget
>     that)
>      >>
>      >>         oasis is not a build system. It is a package description
>     format
>      >> that
>      >>         describes how to invoke the build (and more). So far
>     there is only
>      >>         built-in knowledge for ocamlbuild, but this may change,
>     and you
>      >> can
>      >>         already call any external tool, so you can wrap any
>     build system
>      >> you
>      >>         want. The intention is here more to create a uniform API for
>      >>         starting
>      >>         the build, which may help packagers and other people who
>      >>         routinely build
>      >>         software.
>      >>
>      >>         Gerd
>      >>
>      >>             - jenga
>      >>             - [...]
>      >>
>      >>             My preffered is obuild
>      >>             (https://github.com/ocaml-__obuild/obuild
>      >>             <https://github.com/ocaml-obuild/obuild>),
>      >>             for the terseness, readability and centralization of
>     its build
>      >>             descriptions. I would love to see the user community of
>      >>             obuild grow,
>      >>             so that we can get rid of more bugs, be able to
>     compile _any_
>      >>             OCaml project with it and implement even more cool
>     features
>      >>             (contributors are very welcome).
>      >>
>      >>             I don't want a ring to rule them all, jut a ring
>     that fits
>      >>             _my_ finger. ;)
>      >>
>      >>             --
>      >>             Regards,
>      >>             Francois.
>      >>
>      >>
>      >>
>      >>     --
>      >>     Caml-list mailing list.  Subscription management and archives:
>      >> https://sympa.inria.fr/sympa/__arc/caml-list
>      >>     <https://sympa.inria.fr/sympa/arc/caml-list>
>      >>     Beginner's list: http://groups.yahoo.com/group/__ocaml_beginners
>      >>     <http://groups.yahoo.com/group/ocaml_beginners>
>      >>     Bug reports: http://caml.inria.fr/bin/caml-__bugs
>      >>     <http://caml.inria.fr/bin/caml-bugs>
>      >>
>      >>
>      >
>      > --
>      > Regards,
>      > Francois.
>      >
>      > --
>      > Caml-list mailing list.  Subscription management and archives:
>      > https://sympa.inria.fr/sympa/arc/caml-list
>      > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>      > Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>     --
>     Caml-list mailing list.  Subscription management and archives:
>     https://sympa.inria.fr/sympa/arc/caml-list
>     Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>     Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>

-- 
Regards,
Francois.