Re: [Caml-list] Closing the performance gap to C

[Oleg <oleg@okmij.org>]

> @Ivan: Your point about the optimization is well taken, but please
> consider that the runge-kutta integrator is a sophisticated, general
> numerical method. Usually, the user of such a method has neither the
> knowledge nor the time to manually adapt it, but rather implements a
> standard interface. Therefore, the unused arguments and
> algebraic/trigonometric identities have to be resolved by the compiler.
> Otherwise, we could as well compute the exact solution, anyways ;).

I would like to concur with Ivan and to strongly emphasize his
point. This discussion about compiler optimizations has been done
before -- decades before -- in the HPC community. We know the
conclusion: the era of compiler optimizations is over. All further
gains in HPC computing can only be expected from *domain-specific*
optimizations, specific to a particular domain. Because the most
profitable optimizations are domain-specific, one cannot, should not
and must not count on *general-purpose* compiler to do them. Compiler
writers should think of optimizations that universally profitable,
rather than applicable only in specific (and not entirely clear-cut)
circumstances. The compiler cannot do algebraic/trigonometric
identities! First of all, there are too many of them, and it is not
clear in which direction to apply them. Second, most of the identities
are not generally valid, in the presence of NaNs etc (for example, 0*x
= 0 is not valid for FP computations).

A lot has been written about it. This list is not the best place to
discuss all this work. Let me give three references:

-- The recent talk by Paul Kelly
  http://www.doc.ic.ac.uk/~phjk/Presentations/2015-09-LCPC-Keynote-PaulKelly-V03-ForDistribution.pdf
plus many more publications from his group
        http://www.doc.ic.ac.uk/~phjk/phjk-Publications.html
He talks about finite element methods, far more sophisticated than
Runge-Kutta

-- A very old paper with lots of references to the debate about
compiler optimizations
        http://www-rocq.inria.fr/~acohen/publications/CDGHKP06.ps.gz

-- Stream Fusion, to completeness paper mentioned on this list a
couple of months ago. The paper shows yet another example that
domain-specific optimizations really work, generating code as if it
were written by hand. The paper takes great pains to emphasize why
the optimizations cannot be carried out by a general-purpose
compiler. Please be sure to read the discussion section near the end,
with some notable quotes about GHC optimizer.