Re: [Caml-list] ANN: Brand-new BER MetaOCaml for OCaml 4.00.1

[oleg@okmij.org]

Francois Berenger wrote:
> Could it be (ab)used to translate easily a whole program's OCaml source
> into C code?

In fact at one point MetaOCaml did so -- translated the generated OCaml
code to C or Fortran. This feature -- called offshoring -- was used to
generate FFT kernels. The generated C code was good enough to plug as
it was into the FFTW benchmarking and testing framework. During the
overhaul offshoring has been separated and left for clean-up and
re-writing. Now there is a place for it in the overall architecture.


> I understand that this is very interesting from a research
> perspective, but practically meta-programming is only useful as long
> as performances are the concern, for these problems when some critical
> information is not known until runtime; I can't think of another usage
> for runtime code specialisation anyway.
>
> So the question that immediately arises is then: why is metaocaml
> supporting byte code only? Is a metaocamlopt planned, envisaged,
> doable, in a galaxy not too far away?

Before discussing the plans for the native run, let me clear a
misconception. In two largish MetaOCaml projects I recall offhand,
MetaOCaml was used to generate the optimal code for FFT kernels and
for Gaussian Elimination. Gaussian Elimination is a large family of
the algorithms, parameterized by domain (integer, float, polynomial),
by the choice of pivoting, by the need to compute determinant, rank,
permutation matrix, by the layout of matrices and by a half a dozen
more aspects. Because Gaussian Elimination is often used in inner
loops, the performance matters a great deal. Therefore, all this
parameterization has to be done statically. Jacques Carette and I used
MetaOCaml to generate GE procedures for particular parameter choices.
The generated code was printed out, to be compiled in a
library -- by the native OCaml compiler. Let me emphasize: the code
generator was byte-code OCaml, but the generated code could be stored
and compiled by the native OCaml. We used the byte-code run, but only
for testing. The FFT project was similar: the generator was byte-code,
the byte-code run was used for testing, the generated code was C 
and compiled with the Intel C compiler.

The point is that the byte-code generator produces the code that can
be compiled by any suitable compiler. The mode (byte-code or native)
of the generator and of the generated code are not related. In my
experience, the existing MetaOCaml is already useful in practice --
but more could and should be done.

Regarding native MetaOCaml. As far as generating code is concerned, it
can be done already. (I should try to make optmetaocamlc -- it should
just work). The generated code can be stored on file, compiled any way
we wish -- and linked back using dynlink. What is needed currently is
to make this process automatic and convenient. The refactoring of 
MetaOCaml should help. The functions to run the code are now `user
level'. That is, to change them or to add new ones, one does not have
to re-compile MetaOCaml. Adding a new run function is as simple as
adding a new library function -- just making sure the compiler
can find your *.cmi and *.cmo/*.cmx files.

I too want the end result quicker. Yet basics have to be done
first. At the very least we need to get the codebase that one can work
with and understand, and to be able to add features without breaking
anything.  I think basics is mostly done.