Re: [Caml-list] why is building ocaml hard?

[Tom Ridge <tom.j.ridge+caml@googlemail.com>]

[-- Attachment #1: Type: text/plain, Size: 4646 bytes --]

Gerd raises interesting technical points.

A possibly-too-simple point of view which is not really technical at all,
but usability: for a src/ directory, with (possibly nested) subdirectories
(used for organization of files not naming of modules), and with no
external dependencies, I think most users would like to write `ocamlfind
ocamlc src/` and have it just work. If there are external dependencies,
then `ocamlfind -pkgs x,y,z ocamlc src/` (or similar) should just work.

Now, ocaml's build process is very flexible, so we should not expect
everything to be so simple. But in the simple case, things should ideally
be simple.


On 10 July 2016 at 12:03, Gerd Stolpmann <info@gerd-stolpmann.de> wrote:

> Am Samstag, den 09.07.2016, 21:16 -0700 schrieb Martin DeMello:
> > My consistent experience with OCaml has been that the build systems
> > are fiddly and hard to work with, but I've never seen a discussion of
> > why this is so (as opposed to problems with specific build tools).
> > Supposing you were to start from scratch and develop a new build
> > system in a bottom up manner, starting with a set of libraries and
> > utilities and working your way up to a framework or dsl, what would
> > the difficult steps be?
>
> Well, I don't think you can improve it that much (except... see below).
> Since I've taken over omake and studied it carefully, I think I have
> good insights into the problems.
>
> In short, the big difficulty of OCaml is the strict build topology. You
> need to build a module before the caller of the module. Most build-tool
> related failures come from that. Note that many other languages have
> more relaxed build topologies, or work around the problem by doing
> 2-pass builds (i.e. you first pre-compile and extract interfaces for the
> whole project, and do the actual code generation in the second pass).
>
> Let's have a closer look why it is relatively error-prone to extract the
> dependencies. The tool in question is ocamldep. It is fairly dumb in so
> far it is only parsing the source code, and then looks at all
> module-related constructs (open, include, module, etc.). Because it
> never looks into already compiled interfaces and also proceeds file by
> file, it may sometimes emit wrong dependency information. For example,
> when there is
>
> open M1
> open M2
>
> at the beginning of a file, ocamldep doesn't know whether M2 is another
> top-level module, or whether it is a submodule of M1. ocamldep normally
> errs on the side of generating too many dependencies, which is then
> tried to be corrected by only accepting those deps corresponding to
> existing files. In this example, this would mean that a dependency to M2
> is emitted when there is a file M2.ml. Note that this is wrong when M2
> is actually a submodule of M1 AND the file M2.ml exists.
>
> So how to fix this? In my opinion there are two solutions. You can
> either have a more intelligent ocamldep (e.g. one that reads in
> non-local cmi files and uses that information and also tries to
> interpret all project ml files at once and not file by file - btw, did
> anybody check whether there is an algorithm that precisely solves the
> problem?). The other solution path is to mark toplevel modules in the
> syntax of OCaml (e.g. you'd have to do "open ^M2" is M2 is a toplevel
> module).
>
> Besides ocamldep, there are also other aspects that affect the
> dependency analysis. E.g. with omake there is a distinction of
> project-local and other dependencies, and you need to set the
> OCAMLINCLUDES variable to add other directories to the local part of the
> analysis, whereas the non-local deps are nowadays handled with
> ocamlfind. First of all, this distinction is not really clear to every
> user, and second, there are some difficulties in processing that when
> both concepts overlap (e.g. you want to also get project-local
> dependency expansion with ocamlfind).
>
> Note that recently the dependency analysis even became harder because of
> flambda. Now cmx files play a bigger role. In particular a cmx file can
> refer to another cmx file that isn't a direct dependency.  In other
> words, there is a second kind of dependency that is added by the code
> generator. Current build tools cannot record these dependencies yet.
>
> Gerd
>
> --
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
> My OCaml site:          http://www.camlcity.org
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
> ------------------------------------------------------------
>
>

[-- Attachment #2: Type: text/html, Size: 5673 bytes --]