Concurrent ocamlbuild

Concurrent ocamlbuild

[Jon Harrop]

When I'm not cuddling my dual core, I enjoy using tools such as POVRay, 
pbzip2, quake4-smp and ocamlbuild.

Unfortunately, ocamlbuild only seems to spawn a single compile at a time, 
leaving my other core twiddling its thumbs.

Call me crazy, but I was thinking of forking ocamlbuild and doing a complete 
rewrite in a high-level language like JoCaml. However, I fear my boss will 
object on the grounds that finding me a successor versed in fringe languages 
like JoCaml would be prohibitively difficult. I'll ask her when she's 
finished making lunch but, in the mean time, what do you guys think?

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e

Re: [Caml-list] Concurrent ocamlbuild

[Erick Tryzelaar]

Did you see the -j N option?

 > ocamlbuild -h
...
  -j <N>                      Allow N jobs at once (0 for unlimited)
...


Jon Harrop wrote:
> When I'm not cuddling my dual core, I enjoy using tools such as POVRay, 
> pbzip2, quake4-smp and ocamlbuild.
>
> Unfortunately, ocamlbuild only seems to spawn a single compile at a time, 
> leaving my other core twiddling its thumbs.
>
> Call me crazy, but I was thinking of forking ocamlbuild and doing a complete 
> rewrite in a high-level language like JoCaml. However, I fear my boss will 
> object on the grounds that finding me a successor versed in fringe languages 
> like JoCaml would be prohibitively difficult. I'll ask her when she's 
> finished making lunch but, in the mean time, what do you guys think?

Re: [Caml-list] Concurrent ocamlbuild

[Jon Harrop]

On Sunday 04 November 2007 18:06, Erick wrote:
> Did you see the -j N option?
>
>  > ocamlbuild -h
>
> ...
>   -j <N>                      Allow N jobs at once (0 for unlimited)
> ...

I hadn't noticed that, thanks. However, it doesn't seem to work (at least for 
this project). My old Makefile takes 21s and ocamlbuild with -j anything 
still takes at least 45s. Moreover, the Makefile isn't as parallel as it 
could be...

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e

Re: [Caml-list] Concurrent ocamlbuild

[Nicolas Pouillard]

Excerpts from jon's message of Mon Nov 05 11:11:42 +0100 2007:
> On Sunday 04 November 2007 18:06, Erick wrote:
> > Did you see the -j N option?
> >
> >  > ocamlbuild -h
> >
> > ...
> >   -j <N>                      Allow N jobs at once (0 for unlimited)
> > ...
> 
> I hadn't noticed that, thanks. However, it doesn't seem to work (at least for 
> this project). My old Makefile takes 21s and ocamlbuild with -j anything 
> still takes at least 45s. Moreover, the Makefile isn't as parallel as it 
> could be...

Is it native compilation, byte-code compilation?

The  byte-code  one  offer  more opportunities for parallelism than the native
one  if  we  respect  dependencies  to  allow cross module inlining. Since the
compiler  does not complain about not found ".cmx" you can do more parallelism
in  your  Makefile.  However  that's risky and can leads to worst performances
(less inlining).

Sometimes  using  libraries  (.mllib)  can  give  more  parallelism  than  the
traditional ocamlbuild discovery of sources.

So you can try...

$ cat mylib.mllib
# Put here all modules needed by your main program
# You can gather them by looking at the link command produced by ocamlbuild
A B C D E

$ cat myocamlbuild.ml
open Ocamlbuild_plugin;;
open Ocamlbuild_pack;;
dispatch begin function
| After_rules ->
    ocaml_lib "mylib"
| _ -> ()
end;;

$ cat _tags
<myprog.{byte,native}>: use_mylib

$ ocamlbuild mylib.cma mylib.cmxa myprog.byte myprog.native

HTH,
-- 
Nicolas Pouillard aka Ertai