Re: [Caml-list] GODI is shutting down

[Gabriel Scherer <gabriel.scherer@gmail.com>]

> I think that's fair, but the devil is in the details. The core of OASIS
> is sound, but is hampered by its need to build on top of ocamlfind
> and ocamlbuild, and the ensuing impedance mismatches that follow (e.g.
> manually editing _tags files, and trying to debug which tool a failure
> happens in is an advanced operation). OPAM packages represent an output
> collection of (several optionally built) ocamlfind packages, and
> OASIS sits awkwardly in between these two right now, with ocamlbuild
> crashing the party and getting drunk in the kitchen.

While we're at it (and hoping a diversion might restores some
constructiveness in this discussion), I have also been thinking about
this aspect of OASIS that I don't really like. I use ocamlbuild
-use-ocamlfind, but I'd rather build my _tags and myocamlbuild.ml
files completely separately from OASIS, and avoid that intermixing of
several tools. OASIS could decide which files to build (by calling
whichever build system I use with those as targets) and include in the
user installation, but I'd rather have full control of the build
system independently.

In fact, that's more or less what there is in Batteries; we use
ocamlbuild, with a Makefile on top of it (to easily integrate some
hacks such as inline tests using
https://github.com/vincent-hugot/iTeML ), and use the XCustomBuild
option of OASIS to tell it to use the makefile. This avoids having
autogenerated stuff in the ocamlbuild setup, but currently OASIS has
no say on which files to build (it gets what "make all" produces).

In theory, integrating OASIS and the build system would allow OASIS to
diffuse best practices at the build system level as well. In practice
as a package developer I'm not fond on this combination, and I would
rather let a QA pass do the kind of checks to avoid what a tight OASIS
+ build system integration would have made impossible in the first
place (missing .cmt or what not).

PS: ocamlbuild has been getting bugfixes and improvements relatively
fast these last few months, in particular thanks to the completely
benevolous efforts of Wojchiech Meyer. I'm confident it will continue
to improve on the known-problematic fronts (terse documentation,
unfriendly failure behaviours, disappointing parallelization, etc) in
the future, which may help for some of your "fast Mirage" goals.

On Wed, Jul 24, 2013 at 6:39 PM, Anil Madhavapeddy <anil@recoil.org> wrote:
> On 24 Jul 2013, at 08:58, Markus Mottl <markus.mottl@gmail.com> wrote:
>
>> My guess is that you are not totally happy with Oasis at this point so
>> this may be a topic to address.  Being an Oasis user, I would say that
>> the Oasis tools may not be as mature yet as one would hope, but the
>> Cabal-like package specification syntax seems decent enough to be
>> supported by OPAM.  It really shouldn't be that hard to automate the
>> process of updating OPAM metadata (including version constraints)
>> based on changes in Oasis package specifications.
>
> I think that's fair, but the devil is in the details. The core of OASIS
> is sound, but is hampered by its need to build on top of ocamlfind
> and ocamlbuild, and the ensuing impedance mismatches that follow (e.g.
> manually editing _tags files, and trying to debug which tool a failure
> happens in is an advanced operation). OPAM packages represent an output
> collection of (several optionally built) ocamlfind packages, and
> OASIS sits awkwardly in between these two right now, with ocamlbuild
> crashing the party and getting drunk in the kitchen.
>
> OASIS is structured very well to help get around this situation though.
> It has a library that takes care of parsing the _oasis file, and then a
> plugin generator that currently uses ocamlbuild to drive this specification
> into concrete outputs.  Meanwhile, the OCaml compiler has been exposing
> more of its innards via compiler-libs. Leo and I are taking advantage of
> this by trying to folding all of these layers into a new "packaging-friendly"
> OCaml driver that statically links:
>
> - all the camlp4 libraries required for a project, and directly calls it
>   instead of going via -pp (avoiding a fork/marshal).
> - links to OPAM/findlib to find dependencies directly, avoiding a shell
>   out to ocamlfind.
> - can publish a static schedule of its activities to help Makefiles
>   figure out what actually got built (similar to the recent ocamlbuild
>   request).
>
> The idea is to have a "ocamlmkcompiler" that builds a binary for-purpose,
> as "ocamlmktop" does today for toplevels.  It shouldn't even need any
> patches to the official compiler, thanks to compiler-libs.  More to the
> point, it should make it far easier to build a camlp4-aware js_of_ocaml
> toplevel, to fit into the experiments that OCamlPro is doing with web
> IDE interfaces.
>
> Anyway, these are still experiments and quite likely to run into unexpected
> issues as we build them, and we'll have to consider all the alternatives
> before folding them all into OPAM.
>
> Right now, OPAM dictates the *minimal* possible set of requirements for
> a package since it just uses shell commands.  It should be possible to
> specialise this for common cases such as a OASIS library, but it does
> require some careful thought first.  Luckily, we can use OPAM itself to
> conduct such experiments [1].
>
> All such discussions and experiments are very welcome on the mailing
> list (http://lists.ocaml.org/listinfo/opam-devel) to help establish
> everyone's use cases.  Daniel Buenzli's input has been particularly
> interesting to date, as his libraries adopt a very minimal shell-script
> driven approach that doesn't use OASIS, but still distributes META files.
> That's just one example of someone that doesn't use OASIS directly.
> The Core OASIS files are also a bit special due to their configuration
> requirements.  Xen/Mirage is also quite demanding in terms of the
> compilation environment.
>
> Here's my goal: I want to have near instant-rebuilds of Mirage packages.
> I know it's possible to pull this off with the existing compiler tools,
> since the monolithic Mirage tree (with lovingly crafted ocamlbuild rules
> and judicious use of native code compiler binaries) could rebuild the
> entire webserver in ~10s in parallel.  Nowadays, with OPAM, ocamlbuild,
> ocamlfind and camlp4, it takes around 2-3 minutes, which just isn't
> good enough.
>
> -anil
>
> [1] http://opam.ocamlpro.com/pkg/debug-camlp4-log.base.html: I hacked
> up a debug camlp4 to trace all the uses of camlp4 in the OPAM repository,
> during the wg-camlp4 discussion on the topic of where to go next with it.
> This just required doing a mass OPAM install, drinking some coffee, and
> reading the resulting trace logs.
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs