From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from majordomo@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id LAA02001; Sat, 19 Jun 2004 11:04:26 +0200 (MET DST) X-Authentication-Warning: pauillac.inria.fr: majordomo set sender to owner-caml-list@pauillac.inria.fr using -f Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id LAA01977 for ; Sat, 19 Jun 2004 11:04:25 +0200 (MET DST) Received: from btr0x1.rz.uni-bayreuth.de (btr0x1.rz.uni-bayreuth.de [132.180.8.29]) by nez-perce.inria.fr (8.12.10/8.12.10) with ESMTP id i5J94OEV025821 for ; Sat, 19 Jun 2004 11:04:24 +0200 Received: from btn1x1.inf.uni-bayreuth.de (btn1x1.inf.uni-bayreuth.de [132.180.192.6]) by btr0x1.rz.uni-bayreuth.de (8.12.10/8.12.1) with ESMTP id i5J94NbK015068; Sat, 19 Jun 2004 11:04:23 +0200 (MEST) Received: from inf.uni-bayreuth.de (btn1x1.inf.uni-bayreuth.de [132.180.192.6]) by btn1x1.inf.uni-bayreuth.de (8.12.3/8.12.3/SuSE Linux 0.6) with SMTP id i5J94M73000308; Sat, 19 Jun 2004 11:04:22 +0200 Received: from pd95fc928.dip.t-dialin.net ([217.95.201.40]) (SquirrelMail authenticated user wmueller) by btn1x1.inf.uni-bayreuth.de with HTTP; Sat, 19 Jun 2004 11:04:22 +0200 (CEST) Message-ID: <34996.217.95.201.40.1087635862.squirrel@btn1x1.inf.uni-bayreuth.de> Date: Sat, 19 Jun 2004 11:04:22 +0200 (CEST) Subject: Re: [Caml-list] Benchmark suggestion (Was: Programming Language Shootout) From: "=?iso-8859-1?Q?Wolfgang_M=FCller?=" To: In-Reply-To: <20040619022648.7864665d.nicolas.francois@free.fr> References: <06D9583C-C0FD-11D8-8AC7-000393863F70@exomi.com> <20040619022648.7864665d.nicolas.francois@free.fr> X-Priority: 3 Importance: Normal X-MSMail-Priority: Normal Cc: X-Mailer: SquirrelMail (version 1.2.7) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Miltered: at nez-perce with ID 40D40198.000 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! X-Loop: caml-list@inria.fr X-Spam: no; 0.00; caml-list:01 shootout:01 mueller:99 shootout:01 python:01 inverted:01 ocaml:01 ocaml:01 benchmark:02 benchmark:02 benchmarking:02 compile:02 bytes:02 wolfgang:02 wolfgang:02 Sender: owner-caml-list@pauillac.inria.fr Precedence: bulk Well, I knew OCaml before the shootout :-) , but I loved the shootout results a lot. OCaml helped me once in a project I had to code on a small slow portable computer, where c++ was too slow to compile, JAVA needed too much RAM, and Python/Perl Scripting was out of the question for execution speed reasons. We then recently did some benchmarking at our place comparing JAVA against c++ for an indexing application (inverted files). I also did some initial tests with OCaml, however, as it was out of the question to convince my peers to use OCaml I abandoned :-( What was emphasized by the results of that benchmark is that you can always do fancy optimisations, but what matters in projects that have to be finished in time is the relation between programmer time and (execution time || functionality). JAVA took in our precise scenario about twice the time of c++, which we deemed in our context fast enough, as we can save developer time when using JAVA (e.g. faster debug-build-execute). In other contexts this might be different. In general you might approximate programmer time via number of lines or the suggested compressed number of bytes (in absence of a better measure), and the execution time you can measure, as well as the functionality. What I would dream of, would be two classes of shootout-like benchmarks: 1. Gurus in every language solve given standard problems, like e.g. parsing of web pages, indexing text, etc.etc. (suitable problems would need to be found). Here we would learn how fast and/or short someone really good can make it, some upper bound of the performance reachable. 2. Graduate students with the usual experience in functional and imperative/OO programming solve the problem given ample documentation of the language and three days of time (like the ICFP contest?). This would give an idea if one reaches reasonably quickly useful results when switching to a given language. In my experience OCaml is also very good at that. What do you think about these ideas? Cheers, Wolfgang ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners