Re: [Caml-list] [OSR] Caml Community Code

[Jon Harrop <jon@ffconsultancy.com>]

On Friday 01 February 2008 13:56:36 Christopher L Conway wrote:
> Jon,
>
> So far as I can tell, there is exactly one person on this mailing list
> who is interested in forking OCaml.

We should clarify exactly what we mean by "forking OCaml".

I get the impression that you are asking if I want to make a replica of 
INRIA's OCaml, diverge its evolution and steal as many existing OCaml users 
as possible.

> So the relevant questions is: are *you* going to be forking OCaml?

I want to improve OCaml and I am evaluating how this might be done. Forking is 
one possibility.

One thing I would like to do is try to reconcile existing "OCaml-derived" 
distributions, taking the best from each of them. I am happy to call 
these "forks" but perhaps that has bad connurtations.

For example, Alain Frisch recently said:

  "In particular, we have our locally-patched version of OCaml and all the 
third-party libraries (either in source or binary form) in the repository."

To me, that means LexiFi forked OCaml for their own purposes. Many other 
industrial users have also forked OCaml. More importantly, these forks are 
often degenerate: they reimplement the same missing functionality, often in 
slightly different and incompatible ways.

I would like to encourage existing users to go public with their own forks so 
that we can reconcile them in order to build a single, newer, better, 
enhanced OCaml that everyone is free to share.

Lots of other people are thinking along similar lines but I, for one, am not 
at all clear on who is doing what and, in particular, what exactly their 
goals are.

For example, perhaps many (most?) of OCaml's issues can be addressed without 
altering the language by retrofitting an enhanced standard library onto 
OCaml. Perhaps this can be done transparently (i.e. without changing compile 
lines or code) in such a way that we can all share it compatibly (i.e. 
without breaking the brittle bindings) and we are all free to contribute to 
it. This would be fantastic! Perhaps that is what "Community OCaml" is all 
about but, from what I have read, heard and watched, I believe their effort 
will not implement this transparently, i.e. it will be invasive because they 
do not wish to supercede the stdlib (either by patching it or by automating 
the inclusion of augmentations). I would rather have something transparent 
because it would let users solve their problems more easily.

> I would strongly urge you not to. 

May I ask you to explain what exactly you mean and why you want to actively 
discourage such work? I have been assuming that some people are worried about 
having the relatively-small OCaml community fragmented across competing OCaml 
distributions but perhaps I am wrong.

Perhaps it will also help if I try to enumerate some of the issues that I 
would like to address. Many of OCaml's issues are trivial and have already 
been fixed:

. OCaml does not have a "try..finally" or "unwind-protect" contruct even 
though many other languages have shown that this is extremely useful and 
OCaml's "immigrants" will sorely miss it. Implementing this missing feature 
as a syntactic construct is so trivial that it is everyone's first tutorial 
on using camlp4. But working around this using a non-standard camlp4 macro is 
not a production-quality solution. It requires all compile commands to be 
altered, changes the compiler's responses to all errors and breaks developer 
tools (e.g. the Emacs mode). The production-quality solution is to alter 
either the language itself and its tools (i.e. hack the compiler) or to at 
least add a try_finally combinator to the standard library.

. OCaml's standard library could be improved a great deal. Easy-to-use 
functions for reading files would be enormously beneficial. Many of the 
current modules (e.g. String) are "missing" many functions. The efficiency of 
some of the implementations (e.g. Set) can be greatly improved.

. OCaml's top-level runs interpreted bytecode and, consequently, is many times 
slower than the interactive sessions of "competing" language implementations 
like SBCL (Lisp) and F#. Alain Frisch has already implemented a native-code 
top-level for OCaml called "ocamlnat" in his "natdynlink" fork of OCaml. I 
found this extremely useful and would like it to be easier for other people 
to benefit from this work.

Ultimately, I would also like to address some of OCaml's more fundamental 
problems but these are much harder to solve:

. OCaml's approach to bindings are a great way to avoid the angst of broken 
ABIs but the approach does not scale. Perhaps a better solution would be for 
the compiler to automate the generation of "used" interfaces and 
transparently go via functors when interfacing to libraries. This would 
loosen the bindings without completely breaking them as C does, making it 
possible to use ordinary package systems with lots of packages and making it 
much easier for people like me to distribute closed-source libraries to OCaml 
users.

. OCaml's FFI is much harder to use reliably compared to many other languages 
(C, C++, C#, F#, Lisp). It should be possible to load a C DLL and use it 
directly from OCaml without having to write and compile C stubs. Same goes 
for C++ and other languages but those are harder. LLVM may well be the holy 
grail for this.

. OCaml's lack of overloading is an impediment when writing numerical code 
that uses many different types.

. OCaml lacks some obvious types like 32-bit floats as a general storage 
format.

. OCaml lacks arbitrary unboxing of types which makes it harder to optimize 
certain kinds of code (e.g. FFTs) and impossible to write efficient 
interfaces to some libraries (e.g. OpenGL).

This raises two important questions for me:

1. Is anyone trying to transparently augment the stdlib?

2. What does LLVM facilitate for us?

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e