Maybe the solution is to get one of our numbers among the maintainers of the shootout. This would guarantee, if not objectivity, then at least informed choices wrt OCaml.
On Nov 24, 2010, at 2:36 AM, Eray Ozkural wrote:
Hello,
I think that this benchmark is lacking in the algorithms department. Where is a dynamic programming problem? A graph algorithm? Anything with non-trivial time/space complexity? Anything a little more complex than matrix product?
Also, it's not uncommon to disallow low-level optimizations such as writing memory allocators and async file access when comparing implementations of an algorithm, but such restrictions should be carried out uniformly. In such a benchmark I would expect each entry to stick to their guns, i.e. use only the standard libraries and programming styles for instance. Linking in foreign libraries must most definitely be disallowed. So, if in Java, it's necessary to call the garbage collector explicitly from time to time, and we had to do that for a long time, so be it. Or again, if in Java, performance will suffer unless you only use arrays of integral types, the implementer may wish to implement as much as is possible with arrays, though I wonder if it is not better to choose the most natural implementation style for the particular language. In the case of Java, the claim was that object-oriented was some kind of a programming-aid that can replace talented programmers :) It's unfortunate of course that some kinds of optimizations always have to be made by hand, for instance in functional languages many compilers do not have deforestation.
Otherwise, of course, any implementation may include a compiler for the fastest language and present a program in that language, which is not the objective.
An alternative objective could be to compare the shortest and most comprehensible, if possible line-to-line compatible implementation of a given pseudocode in different languages. That would be extremely informative for serious algorithm researchers! If a computer scientist isn't sure of the performance of the primitives, he cannot make sure his implementation will comply with the time-complexity of the given algorithm.
Best Regards,
--
Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs