RE: [Caml-list] Benchmark suggestion

["Brandon J. Van Every" <vanevery@indiegamedesign.com>]

Wolfgang Müller wrote:
>
> 2. Graduate students with the usual experience in functional and
> imperative/OO programming solve the problem given ample
> documentation of
> the language and three days of time (like the ICFP contest?).
>
> This would give an idea if one reaches reasonably quickly
> useful results
> when switching to a given language. In my experience OCaml is
> also very good at that.
>
> What do you think about these ideas?

I think one has to define the motivation for the benchmark.  Is it
language evangelism, i.e. making people aware of OCaml?  or of FP
languages generally?  If language evangelism is the point, then one
should look at *all* the factors that go into picking new languages.
The devil's in the details.

For instance, the Shootout led me to OCaml, but further inquiry is
leading me towards Standard ML because it has better int / 32-bit float
/ C interfacing.  These efficiency concerns are rather important to me,
they affect the most basic ways that I personally program.  It's way
easier for me to learn to blow off OO than to try to add such things to
OCaml.  I'm still going to organize a Seattle SIG, but I'm going to
re-Christen it as an "ML" SIG.  I'm reasonably convinced that ML is the
right language family for my problems, it's just down to which ML is
best for my specific needs.

Actually I think comparative benchmarking for some goal other than
evangelism is kinda pointless.  If you want your own app to go faster at
something, you write a benchmark for your own app.  You only look at
app-app or snippet-snippet comparisons when you're trying to decide
which option to pick.

Otherwise why are you bothering?  You are wasting your time.  I do mean
*time*, with my OpenGL device driver background I'm very conscious of
how much production time benchmarking consumes.  Even when you have all
the tests already written and you're just running them.  It takes time
to run and analyze the results of a benchmark.  I've learned not to sit
around running benchmarks all day, as that's not coding.  It's better to
develop instincts about what optimizations are worth it, and sanity
check those instincts with benchmarks a few times in a week.

I'd say the same for picking languages.  I have a lot of instincts about
language evangelism now, having looked at so many of them in the past
year.


Cheers,                         www.indiegamedesign.com
Brandon Van Every               Seattle, WA

20% of the world is real.
80% is gobbledygook we make up inside our own heads.

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.708 / Virus Database: 464 - Release Date: 6/18/2004

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners