Re: [Caml-list] About ocamlbuild

[Pierre-Etienne Meunier <pierreetienne.meunier@gmail.com>]

Hi,

My original idea was not at all to start a new ocaml-tools war, I'm sorry if my first message was poorly formulated. Let me make myself clear, I wrote a typesetting system in ocaml, in order to write my PhD thesis. The way it works is a little strange, it compiles your source to ocaml source code (our language is quite cool, because based on a dypgen grammar, but you can use another one if you have time to write a parser and code generator). Then it compiles this code with a library called "Typography", that does all the document processing stuff, optimization of the page breaks, and so on. Finally, it calls another library to output to different formats, such as pdf, plain text, svg, opengl, or anything else.

These are clearly "separable but not independent" projects (drivers, document formats, font library, typesetting library, parser/code generator). So compiling everything without duplication is not that trivial, and I really would like to write an ocaml program to compile it, maybe using a more complex build system than just one file with the ocamlbuild api. I understand that ocamlbuild is still in an early stage of development, this is why I'm sending these mails. The code is split among different directories, and using a library interface in the compilation of another library still seems difficult in ocamlbuild today. The documentation explains how to use an internal library in program, but it forces you to use a particular layout of your files. This is why I feel that a library would be more versatile: we can imagine a makefile compiling the build system first, possibly split between several modules, then calling it to compile the rest, like xmonad does, for instance.

Also, if you are writing a document with our system, separated between different files, you want the system to compile it automatically, without too much recompilation. The problem is extensibility: for instance, if you are writing a bibliography library (I wrote one, but there may be others in the future), you certainly do not want to touch our code in order to compile it. Instead, you want to write a dynlinkable module explaining to our system how the documents depend on bibliography files, and how to compile everything in what order.
To do this, I had to rewrite a (small) ocamlbuild-like library. My point is that ocamlbuild exposes (or documents) a too small part of its internals to be re-used inside another project.


Now, I completely understand Xavier's points:

- about _tags, and this is also my philosophy when I write ocaml code myself. I asked the question because I needed magic comments in the build system of my typesetting stuff, mostly because the ocaml source code that I compile is produced automatically, and meant to be executed only once. Writing the correct _tags files without conflicts between the different files, with the correct directory inclusions for instance, would be much more complicated. This is the only case where I see the point of a mixed system.

- about the command line options of ghc: clearly, this also has to do with the fact that ocamlopt manages to compile files of equivalent sizes about twenty or thirty times faster than ghc in the cases I tried. The theoretical problems solved are not the same, either.

Finally, Gabriel, I would be happy to contribute code to ocaml, only my time is a scarce resource right now. Maybe some day…

Cheers,
Pierre


Em 16/11/2012, às 11:53, Gabriel Scherer escreveu:

> Regardless of the overly heated tone of the discussion, I find the
> idea of having "ocamlbuild as a library" interesting. Let me detail
> what I believe was the idea of the proposal.
> 
> Currently we have a shared ocamlbuild executable that would contain
> the core logic, and be passed plugins that are invoked by the core
> logic at some point of the program. We could consider an inversion of
> control where the executable logic is exposed as a library, and users
> write scripts use this library (to invoke the ocamlbuild processing),
> the script being compiled as whole programs rather than plugins. In
> effect, you give the tool to let any project define it's own "tailored
> ocamlbuild" easily. There could be little change of usage and
> behavior, it's just a different point of view, that has the advantages
> already evoked in this thread (facility to link with other stuff,
> etc.).
> 
> This idea has already been used in other projects where "configuration
> is compilation", such as the XMonad window manager (
> http://xmonad.org/ ); the xmonad.hs users write is not compiled to a
> dynamically loaded plugin, it itself invokes the "xmonad program" as a
> library function, passing it (from the language) configuration
> information.
> 
> I find this idea interesting, but it will require some amount of work.
> My guess as to why Xavier Clerc's reply is a bit grumpy is that he is
> often the one tasked with implementing those changes. On the other
> hand, everyone is free to contribute to ocamlbuild, and I've been
> trying to publicly encourage people to contribute, with little success
> so far. So Pierre, if you want to have a try at implementing this
> idea, by all means, do! We'll see when someone has some working patch
> if it can be integrated upstream.
> 
> Regarding in-file tags/pragmas, I find such changes should be
> considered much more cautiously, as those could mean changes to the
> OCaml tooling (or worse, language) as a whole, not simply ocamlbuild
> itself. Again, nobody's forbidden to come up with a good,
> well-specified, simple proposal that would please everyone, but that
> is a quite difficult task and any such proposal should expect to get
> rejected.
> 
> On Fri, Nov 16, 2012 at 11:33 AM, Arnaud Spiwack
> <Arnaud.Spiwack@lix.polytechnique.fr> wrote:
>> Well,
>> 
>> I must say the sort of mafioso-like reasoning - "you really do not want to
>> do that" - doesn't really make sense to me.
>> 
>> Here are valid reasons not to include a feature X in a tool:
>> 
>> I don't have time to do X
>> I don't know how to do X
>> Having X would prevent me from having Y which I'd rather
>> 
>> Any other reason is bad.
>> 
>> People want to have tags and compilation pragmas? Well, let them, they
>> probably have valid reasons to (some options may be better seen global to
>> the project and other local to files). People want to interleave compiling
>> the build system and compiling the project. They have a good reason for that
>> too. Also: ocamlbuild doesn't only compile ocaml files.
>> 
>> Related to that last point, and to the initial question: as far as I'm
>> aware, an ocamlbuild plugin cannot depend on a custom library (to go around
>> that we can use a myocamlbuild.ml which compiles a mymyocamlbuild.ml (or
>> something) with the appropriate linking options, then copy it in a
>> _build_stage2 directory, then Unix.excv an ocamlbuild -no-plugin -build-dir
>> _build_stage2. It would be nice if there were a standard and simple way to
>> do this sort of thing). In particular there is no principled way to
>> distribute a tool together with an ocamlbuild library against which to
>> compile our myocamlbuild.ml plugins (hence extending what ocamlbuild can
>> "natively" compile).
>> 
>> 
>> 
>> 
>> On 15 November 2012 23:36, Xavier Clerc <xavier.clerc@inria.fr> wrote:
>>> 
>>> ----- Mail original -----
>>>> On 11/15/2012 4:48 PM, Xavier Clerc wrote:
>>>>>> 3 - Finally, I find the idea of tags good, for backward
>>>>>> compatibility
>>>>>> reasons (you do not have to change your code), but not enough. For
>>>>>> instance, in haskell (and some compilers written in ocaml), you
>>>>>> can
>>>>>> add "tags" at the beginning of your files. You would start your
>>>>>> ocaml files with comments such as:
>>>>>> 
>>>>>> (* #OPTIONS -rectypes *)
>>>>> Well, I find tags pretty convenient, and do not dislike the comment
>>>>> approach. However, I am pretty sure I do not want the related
>>>>> information
>>>>> to be scattered over multiple locations.
>>>> It's exactly this reason that I'm strongly in favor of magic comments
>>>> or
>>>> pragmas or something in .ml files to do do this kind of thing.
>>>> Splitting the data needed to compile a module into two parts: the
>>>> module
>>>> and a makefile (or _tags or OMakefile or whatever) seems a violation
>>>> of
>>>> your "multiple locations" policy.
>>> 
>>> I do not follow your line of thought: every information about how to
>>> compile is clearly in build files, and not in source code files.
>>> If I wonder how the command-line for a given compilation is determined,
>>> there is only one place to look.
>>> 
>>> Again, I do not claim that the current state of affairs (everything
>>> in build files) is better than a pure pragma-based solution (everything
>>> in source files). My objection is about a mixed system where one should
>>> have to consult both build and source file in order to know which
>>> options are passed to compile a given module.
>>> 
>>> 
>>>> While I grant that some
>>>> compilation
>>>> options (whether to compile in debug/profiling/native mode, location
>>>> of
>>>> external libraries) should not be included in a source file, I'm
>>>> strongly in favor of having the build system work out the details of
>>>> dependencies (internal to project and external, installed in global
>>>> location) and even what camlp4 to run based on the contents of the
>>>> file
>>>> itself and not elsewhere.
>>> 
>>> To be fair, I should point out that Coq is extracting camlp4 information
>>> from source files in order to determine how to treat them.
>>> 
>>> 
>>> Just a final remark: compared to Haskell, OCaml compilers feature a
>>> limited
>>> number of command-line switches having an influence on the semantics of a
>>> source file(*). I regard this as a pleasant thing, and think this makes
>>> the
>>> issue at hand quite minor.
>>> 
>>> 
>>> Kind regards,
>>> 
>>> Xavier Clerc
>>> 
>>> 
>>> 
>>> 
>>> (*) we are only talking about labels, applicative functors, and recursive
>>> types
>>> 
>>> --
>>> Caml-list mailing list.  Subscription management and archives:
>>> https://sympa.inria.fr/sympa/arc/caml-list
>>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>> 
>> 
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs