Re: [Caml-list] The closing gap (warning: long, inflammatory rant)

["Berke Durak" <berke.durak@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 5276 bytes --]

On Mon, Apr 21, 2008 at 1:17 PM, Jon Harrop <jon@ffconsultancy.com> wrote:

> Quad cores are already the norm.
>
> An *eight* core Dell Precision T7400 now costs only £1,171. Our desktop
> machines will be replaced with these eight core machines before the end of
> this year.


Well it's worse than what I thought then.


> > I'd hate to see Ocaml become one of the slower languages.
>
> For many applications, that already happened.
>

By slower, I meant something like "not within a factor of 1.5/2 of the top
five fastest languages".
I assume you are referring to parallelizable, number-crunching tasks?

> That way one could embed multiple Ocaml libraries in programs written in
> > other languages (which could allow easier monetization of Ocaml software
>
> There are many other issues that prevent the commercialization of OCaml
> libraries.


That could take care of at least one of those issues.  What are the others?
.NET
interoperability I guess.


> > Also, we could explore alternate paths to paralellism, such as running
> > multiple Ocaml interpreters/runtimes in different threads with disjoint
> > heaps. Those threads would run truly in parallel on different CPUs.
>
> Communication is too slow for that to be useful outside a few niche
> applications.


The problem to be solved here is not throughput, but paralellizability.
Note that bus and cache coherency protocols already do lots of
communication,
but in hardware.  So here you'd have software communication, which would be
quite slower, but your individual processes could run at full speed.

As for the applications, I'm thinking of Ocaml as a web programming language
as
a "niche" were such an approach might be useful.


> > I personally don't believe much in fine-grained parallelism using
> threads
> > and locks because it makes the GC difficult,
>
> This is certainly a difficult problem but you cannot compete without a
> concurrent GC now.


A concurrent GC should be developed.  But I think you can compete in some
"niches"
without a concurrent GC.

Web programming should not be dismissed as a lowly art anymore - and even if
it is lowly,
it is quite lucrative.

Look at Ruby, basically a disguised functional language with a
monkey-patchable object system.
Web developers embraced it extremely quickly.  And while web developers, on
average, are
not bright programmers (or programmers at all), their huge population means
that there is a sizable
portion of the brighter and brightest programmers that do "web developing" -
be it front-end or
back-end.  Do I need to remind everyone that the web is the most important
technological development
in the last twenty years?

Now data center owners love Ruby-based apps, since you need 60 servers to do
300 requests/second.
So Ruby is not the fastest language on Earth - it was second-to-last on the
latest Alioth shootout.
And people are slightly starting to get disillusioned with Ruby, mostly for
performance reasons,
and migrating to Python.  Note that neither of those languages have good
parallelism support nor are
especially fast.

So Ruby and Python, which are slow and do not have concurrent GCs, let alone
a half-decent type system,
enjoyed widespread adoption.  Why?  And can Ocaml do the same?

I think dynamic typing and the metareflexive capabilities are not the reason
behind Python and Ruby's success.
Nor is embeddability.  The main reason, I think, is absent/automatic/easy
compilation.

Compilation and linking are extremely painful things, especially when you
want to start to learn a new language
in good faith.  Java has a relatively good packaging/loading model which is
part of its success.  Ocaml is
terrible at this.

Yet, unlike Javascript or PHP, Python and Ruby are system programming
languages, in which you can write
command-line tools or a small HTTP server without needing a Java runtime
with a RSS of 300MB .  As is Ocaml.

Also, Ruby and Python smell neither Sun nor Microsoft Enterpriseiness.  They
are 100% FOSS-compatible.  As is Ocaml,
unlike F#.  Java is going a long way with OpenJDK, but it's still not
FOSS-kosher.

This means that there is a gap to fill - people already understood that
closures are useful,
they are starting to realize that higher-order functions are useful too and
that without static typing,
their applications become unmaintainable and they can't cooperate
effectively.

Scala or Groovy won't fill it because they are not natives in JVM-land and
they have the unremovable
Java-enterprise stench.

Erlang may be scalable but is not general-purpose and people hate to learn
extra languages.

Haskell won't fill that niche either, because if *I*'m not ready to wrestle
with monads to get some type-theoretical
effect safety (only to start peppering my code with unsafePerformIO and
strictness annotations), then the average
developer certainly won't be.

So there is a gap to be filled, and Ocaml could be the next fashionable web
programming language if we fix
a few things or two:
- Compilation and package headache,
- Missing batteries.

If someone manages to write a good concurrent GC it'll be quite nice.
Meanwhile, I see heap-disjoint
parallelism as a low hanging fruit.

I think the market is ripening for such an offer.  But the gap is closing.
We must hurry!
-- 
Berke

[-- Attachment #2: Type: text/html, Size: 6771 bytes --]