Re: [Caml-list] Accelerating compilation

[Romain Bardou <romain.bardou@inria.fr>]

Le 06/09/2013 22:51, Fabrice Le Fessant a écrit :
> On Fri, Sep 6, 2013 at 5:20 PM, Romain Bardou <romain.bardou@inria.fr> wrote:
>>>> 3) Parallel compilation in Ocamlbuild
>>>>
>>>> Of course it would help but it is not easy to implement so I'm just
>>>> putting it there to be exhaustive.
>>>
>>> I'm not sure what you are referring to, OCamlBuild does already
>>> support parallel builds.
>>
>> Does it? I actually thought the -j option was ignored.
>>
>> I just did a quick test and I gain about 5 seconds with -j on a 1min15
>> build (I had cleaned, recompiled and recleaned before so that caching by
>> the file system would not impact the result too much), so it does seem
>> to be a *little* faster :)
> 
> FWIW, I recently compiled Merlin with both ocamlbuild and ocp-build,
> on a quad-core with "-j 10" for both (the link to the ocp-build
> description file is in the latest OCamlPro's report), ocamlbuild needs
> 13s where ocp-build only needs 4s to compile everything.
> 
> As Xavier suggested, I think the main problem is that ocamlbuild must
> discover the files to compile dynamically, starting from the last
> module, so it cannot parallelize much. On the contrary, ocp-build has
> the list of files in the description of the project, so for example,
> it can call ocamldep on all of them almost at the same time.
> 
> ocp-build has an option to choose between digest and (timestamp +
> inode), the default. When I benchmarked the two modes, I could notice
> a small difference, but not more then 5%, so the digests are unlikely
> to be responsible for the problem.
> 
> Finally, another optimization that could be considered for ocamlbuild
> is to call camlp4 only once per file, and then call ocamldep, ocamlc
> and ocamlopt on the preprocessed file. I could notice a huge
> difference on Core, for example, when this optimization was
> implemented first in ocp-build (about a 50% speed-up, if I remember
> well).

I was hoping to hear from ocp-build :) Interesting observations here. I
would have thought the digests would make more of a difference. Good to
know!

Cheers,

-- 
Romain Bardou