Hello,

I think that this benchmark is lacking in the algorithms department. Where
is a dynamic programming problem? A graph algorithm? Anything with
non-trivial time/space complexity? Anything a little more complex than
matrix product?

Also, it's not uncommon to disallow low-level optimizations such as writing
memory allocators and async file access when comparing implementations of an
algorithm, but such restrictions should be carried out uniformly.  In such a
benchmark I would expect each entry to stick to their guns, i.e. use only
the standard libraries and programming styles for instance. Linking in
foreign libraries must most definitely be disallowed. So, if in Java, it's
necessary to call the garbage collector explicitly from time to time, and we
had to do that for a long time, so be it. Or again, if in Java, performance
will suffer unless you only use arrays of integral types, the implementer
may wish to implement as much as is possible with arrays, though I wonder if
it is not better to choose the most natural implementation style for the
particular language. In the case of Java, the claim was that object-oriented
was some kind of a programming-aid that can replace talented programmers :)
It's unfortunate of course that some kinds of optimizations always have to
be made by hand, for instance in functional languages many compilers do not
have deforestation.

Otherwise, of course, any implementation may include a compiler for the
fastest language and present a program in that language, which is not the
objective.

An alternative objective could be to compare the shortest and most
comprehensible, if possible line-to-line compatible implementation of a
given pseudocode in different languages. That would be extremely informative
for serious algorithm researchers! If a computer scientist isn't sure of the
performance of the primitives, he cannot make sure his implementation will
comply with the time-complexity of the given algorithm.

Best Regards,

On Wed, Nov 24, 2010 at 2:23 AM, Isaac Gouy <igouy2@yahoo.com> wrote:

>  <oliver <at> first.in-berlin.de> writes:
>
> > > Even back in 2001, Doug Bagley had noted all the things that were
> > > wrong with the tasks on his "The Great Computer Language Shootout".
> >
> > And what was wrong in his eyes?
>
> Find out for yourself:
>
> http://web.archive.org/web/20010617014807/www.bagley.org/~doug/shootout/<http://web.archive.org/web/20010617014807/www.bagley.org/%7Edoug/shootout/>
>
>
> > So, now the comparisions are perfect?
>
> Has anyone said so?
>
>
> > What problems were removed?
>
> All of them.
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>



-- 
Eray Ozkural, PhD candidate.  Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct