Re: [Caml-list] Benchmarks against imperative languages

[Jon Harrop <jon@ffconsultancy.com>]

On Saturday 04 March 2006 14:36, Basile STARYNKEVITCH wrote:
> We all know (by experience) that Ocaml performs quite well. We also
> know that for most (but not all) of the software we are coding, the
> cost and time of development does significantly matter, and a 10%
> decrease in performance is not that important, hence Ocaml brings a
> real win.

Yes and no. I think the relative performance of OCaml is quite variable. Here 
is a range of examples:

1. Straightforwardly-implemented algorithms can be vastly faster when written 
in a functional style and optimising imperative equivalents can be tricky, 
e.g. set theoretic operations and the "n"th-nearest neighbour example from my 
ocaml book:

  http://www.ffconsultancy.com/products/ocaml_for_scientists/complete/

2. Any problems that are near the limit of practical feasibility are likely to 
benefit enormously from improved development time. This includes the limits 
of computational complexity, memory usage and algorithm-implementation 
complexity.

3. Floating-point intensive algorithms can be miraculously (IMHO) fast for an 
FPL, e.g. my ray tracer benchmark on AMD (particularly AMD64):

  http://www.ffconsultancy.com/free/ray_tracer/languages.html

4. Certain operations can be surprisingly slow, e.g. my Sudoku solver is 
several times slower in OCaml because ocamlopt does not optimise integer div 
or mod by a constant:

  http://www.ffconsultancy.com/free/sudoku/

The relative development speed in OCaml and C++ is also quite variable. Some 
applications, like compilers and interpreters, clearly benefit enormously 
from languages like OCaml. Other applications, like small and simple 
numerical programs (e.g. the ray tracer) benefit from the availability of 
libraries like Blitz++ and Boost for C++. In the former case, development can 
be an order of magnitude faster in OCaml. In the latter case, the time taken 
to develop similar-performance programs can be roughly the same.

> A real answer would be to have a team of programmers fluent in Ocaml
> write a code (an real-sized application of hundreds of KLOC of source
> code, representing several man-years of effort) which has exactly the
> same precise specification than an existing code written in C. But
> this will never happen (it is too costly but quite useless an
> experiment). For example, nobody will recode in Ocaml an exact clone
> of gcc-4.1, firefox-1.5, or mysql-5.0!

I have cited my most relevant example before - a vector graphics library which 
was 4x longer in C++, the worst-case performance was 5x faster in OCaml and 
the development time was an order of magnitude faster in OCaml. It was a 
rewrite from C++ to OCaml, so it is biased. However, I ditched C++ because it 
was infeasible to do the rewrite in C++ and I have not looked back since. 
Also, I can't say how OCaml compares to other FPLs. The code size was 
~10kLOC.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
Objective CAML for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists