Re: [Caml-list] One build system to rule them all?

[Yotam Barnoy <yotambarnoy@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 9923 bytes --]

OK so here's my summary of what I've heard here so far:

- Ocaml does not yet have a (public) build system that the majority of
users can be happy with for all of their project needs.
- It seems difficult to have a build system that's simple enough to use for
small projects, flexible enough to handle the more complicated projects,
and that can scale.
- While declarative, dsl-based tools are appealing, they get bogged down as
you add features to them, making them become unwieldy.
- A better solution seems to be using a real programming language to
describe packages. In our case that should clearly be ocaml itself. The
trick is making the API simple.

Looking at some of the the available options:
- ocamlbuild (dsl-based) is a decent basic choice, but lacks advanced
features and has become weighed down by multiple file types (_tags,
myocamlbuild etc).
- OASIS (dsl-based) seems to be a good description layer for projects,
currently farming out the difficult work of building stuff to ocamlbuild,
but not really a contender in this race except as an abstraction layer.
- omake (dsl-based) has some fans, but becomes overly-complex for more
complicated projects.
- ocp-build (dsl-based) has had nothing but criticism on this thread. Is
there any ongoing work to address these criticisms?
- Jenga (uses ocaml) seems promising but has heavy dependencies, and is
currently not geared towards light projects.
- assemblage (uses ocaml) seems promising as well, and seems focused on
simplicity. Any comment from assemblage contributors on its status?

Also, while I understand some of the windows hatred, the fact is, most of
the world uses windows. Even using cygwin as a translation layer hurts
ocaml's ability to reach people. Ideally, a windows programmer could
download opam, ocaml and perhaps a mingw compiler, and begin to build
projects without any need for a shell or anything POSIX-like. This means
that the build system needs to obviate the need for any shell commands or
makefiles.

On Thu, Sep 11, 2014 at 6:37 AM, Yaron Minsky <yminsky@janestreet.com>
wrote:

> On Thu, Sep 11, 2014 at 3:53 AM, Francois Berenger
> <francois.berenger@inria.fr> wrote:
> > On 09/10/2014 08:59 PM, Yotam Barnoy wrote:
> >>
> >> Here's part of my motivation for starting this discussion now: I
> >> recently experienced a hard-drive failure and had to switch to my backup
> >> windows laptop. I was dismayed that opam is still not available for
> >> Windows, and while wodi is available, it's not nearly as well developed,
> >> and apparently requires many hacks for different packages.
> >>
> >> It seems to me that these hacks are some of the things that make
> >> platform compatibility hard for opam. Each build system has different
> >> requirements and methods of compilation, and making this approach
> >> cross-platform compatible is difficult. Ideally, a build system will
> >> abstract away the things that are not inter-platform compatible. This is
> >> why I don't like people using makefiles. Makefiles are not available
> >> natively on Windows, and they often contain other bits of shell code
> >> that isn't available on all POSIX platforms, let alone on Windows. Not
> >> to mention the fact that makefiles have a very tricky and sensitive
> >> syntax.
> >>
> >> I don't expect one build system to match everyone's requirements, but I
> >> think if we get together, make a list of requirements, and try to get
> >> over our personal biases, we can find something that works for most
> >> people, and with some group effort, can work for even more people in
> >> more use-cases. I'm not suggesting that we invent something new, but
> >> that we take something good and make it better, as well as make an
> >> effort to learn that tool and convert packages to use that tool.
> >>
> >> So here are some requirements I can think of (using some of the
> >> suggestions that have been brought up):
> >> - Easy to use, especially for small projects (large projects can afford
> >> to put more time into their build systems)
> >> - Abstract away platform considerations as much as possible. No
> >> dependence on specific shells and POSIX utilities.
> >> - Allows compilation of C files, which is quite common in ocaml
> packages.
> >> - Scalable to many directories and files
> >> - Uses ocamlfind to locate packages
> >> - Handles camlp4 and ppx
> >> - Parallel & incremental compilation
> >>
> >> About Jenga: I took a quick look at Jenga, and even though I'm impressed
> >> by its capabilities, the amount of code needed to be written even for
> >> simple projects is overwhelming. It's clearly a very flexible and
> >> powerful tool, but I'd say it's too flexible for the mainstream. There's
> >> always room for a build system that's integrated into ocaml itself (like
> >> Shake), but I think declarative build systems tend to be easier to
> >> comprehend for the average user.
> >>
> >> ocp-build actually looks very interesting. The manual (which is here:
> >>
> >>
> http://github.com/OCamlPro/ocp-build/blob/master/docs/user-manual/user-manual.pdf?raw=true
> )
> >> is incomplete, but contains a nice survey of the existing build tools,
> >> and motivation for making ocp-build. Has anyone had experience with
> >> ocp-build? Opam seems to be using it, but they also use a makefile
> >> (why?) with a bunch of shell commands inside (which is precisely the
> >> problem from my perspective). ocp-build is supposedly compatible with
> >> Windows, too.
> >
> >
> > Here is one thing I dislike in ocp-build: you have to explicitely list
> all
> > the source files (and maybe same thing for Jenga).
>
> Jenga is quite flexible, so you could write your jengaroot either way
> (to demand or not demand explicit source file lists.)  Our internal
> jengaroot does not require explicit source file lists.
>
> Not to confuse people, though, Jenga isn't really suitable for people
> just picking up and using yet.  We haven't yet written a suitable
> jengaroot for general use yet (though there is a version of our
> internal jengaroot that we've released, but it's not yet ready for
> prime time.)  So Jenga is interesting as an option to discuss, but not
> yet a practical option we want to encourage casual users to try.
>
> > I am way too lazy to do that, especially since other tools are able
> > to infer this list automatically (a library is just a list of modules,
> > an executable has a single entry point, the main function in a single
> file).
> >
> >> Yotam
> >>
> >>
> >> On Wed, Sep 10, 2014 at 11:17 AM, Leonardo Laguna Ruiz <
> modlfo@gmail.com
> >> <mailto:modlfo@gmail.com>> wrote:
> >>
> >>     My pick is ocamlbuild because:
> >>
> >>     - It works the same way in all platforms that I work (linux,osx,
> >>     windows msvc port, cygwin)
> >>     - If I can compile ocaml, then I have ocamlbuild
> >>     - It does not require external libraries
> >>
> >>     I have to say that I feel a little bit annoyed by the fact that some
> >>     other build systems or tools, do not work correctly in all the
> >>     platforms that I want to support.
> >>
> >>     Leonardo
> >>
> >>
> >>
> >>     On 9/10/2014 4:23 PM, Gerd Stolpmann wrote:
> >>
> >>         Am Mittwoch, den 10.09.2014, 15:29 +0200 schrieb Francois
> >> Berenger:
> >>
> >>             More seriously, concerning build systems, we clearly have
> >>             quite some
> >>             choice on the OCaml shelf:
> >>             - obuild
> >>             - ocamlbuild
> >>             - omake
> >>             - oasis (which in fact uses ocamlbuild, don't forget that)
> >>
> >>         oasis is not a build system. It is a package description format
> >> that
> >>         describes how to invoke the build (and more). So far there is
> only
> >>         built-in knowledge for ocamlbuild, but this may change, and you
> >> can
> >>         already call any external tool, so you can wrap any build system
> >> you
> >>         want. The intention is here more to create a uniform API for
> >>         starting
> >>         the build, which may help packagers and other people who
> >>         routinely build
> >>         software.
> >>
> >>         Gerd
> >>
> >>             - jenga
> >>             - [...]
> >>
> >>             My preffered is obuild
> >>             (https://github.com/ocaml-__obuild/obuild
> >>             <https://github.com/ocaml-obuild/obuild>),
> >>             for the terseness, readability and centralization of its
> build
> >>             descriptions. I would love to see the user community of
> >>             obuild grow,
> >>             so that we can get rid of more bugs, be able to compile
> _any_
> >>             OCaml project with it and implement even more cool features
> >>             (contributors are very welcome).
> >>
> >>             I don't want a ring to rule them all, jut a ring that fits
> >>             _my_ finger. ;)
> >>
> >>             --
> >>             Regards,
> >>             Francois.
> >>
> >>
> >>
> >>     --
> >>     Caml-list mailing list.  Subscription management and archives:
> >>     https://sympa.inria.fr/sympa/__arc/caml-list
> >>     <https://sympa.inria.fr/sympa/arc/caml-list>
> >>     Beginner's list: http://groups.yahoo.com/group/__ocaml_beginners
> >>     <http://groups.yahoo.com/group/ocaml_beginners>
> >>     Bug reports: http://caml.inria.fr/bin/caml-__bugs
> >>     <http://caml.inria.fr/bin/caml-bugs>
> >>
> >>
> >
> > --
> > Regards,
> > Francois.
> >
> > --
> > Caml-list mailing list.  Subscription management and archives:
> > https://sympa.inria.fr/sympa/arc/caml-list
> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 13533 bytes --]