From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.1 required=5.0 tests=AWL,MAILTO_TO_SPAM_ADDR autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from concorde.inria.fr (concorde.inria.fr [192.93.2.39]) by yquem.inria.fr (Postfix) with ESMTP id 268B5BC0B for ; Wed, 31 Jan 2007 18:03:43 +0100 (CET) Received: from hedwig1.umh.ac.be (hedwig2.umh.ac.be [193.190.193.73]) by concorde.inria.fr (8.13.6/8.13.6) with ESMTP id l0VH3gRX025083 for ; Wed, 31 Jan 2007 18:03:42 +0100 Received: from poincare.swapping.umh.ac.be (mathwifi.swapping.umh.ac.be [10.102.100.203]) by hedwig1.umh.ac.be (8.13.6/8.13.6) with ESMTP id l0VH5BP41605746; Wed, 31 Jan 2007 18:05:18 +0100 Received: from localhost ([127.0.0.1]) by poincare.swapping.umh.ac.be with esmtp (Exim 4.63) (envelope-from ) id 1HCIrp-0005c1-Ss; Wed, 31 Jan 2007 18:03:29 +0100 Date: Wed, 31 Jan 2007 18:03:29 +0100 (CET) Message-Id: <20070131.180329.224137902.Christophe.Troestler+ocaml@umh.ac.be> To: "O'Caml Mailing List" Subject: Re: [Caml-list] Benchmarking different dispatch types From: Christophe TROESTLER In-Reply-To: <45AED8C8.3080808@gmail.com> References: X-Face: #2fb%mPx>rRL@4ff~TVgZ"<[:,oL"`TUEGK/[8/qb58~C>jR(x4A+v/n)7BgpEtIph_neoLKJBq0JBY9:}8v|j Organization: University of Mons-Hainaut X-Mailer: Mew version 5.1.52 on Emacs 22.0.93 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Multipart/Mixed; boundary="--Next_Part(Wed_Jan_31_18_03_29_2007_504)--" Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.1 (www dot roaringpenguin dot com slash mimedefang) X-Miltered: at concorde with ID 45C0CBEE.000 by Joe's j-chkmail (http://j-chkmail . ensmp . fr)! X-Spam: no; 0.00; christophe:01 troestler:01 christophe:01 troestler:01 ocaml:01 umh:01 foc:01 foc:01 ocamlopt:01 -inline:01 cmxa:01 cmxa:01 ocamlc:01 -inline:01 2007,:98 X-Attachments: name="gray.ml" ----Next_Part(Wed_Jan_31_18_03_29_2007_504)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit > (warning: too few iterations for a reliable count) > > Interesting, but are they meaningful? The warnings from Benchmark are > troubling, but I didn't have any immediate ideas on how to get rid of > them. The warnings mean what they say. The running time of the calls (as opposed to the whole process: calls + repetition loop), is estimated to be less than 0.1 second and is therefore too imprecise to produce meaningful results. On Wed, 17 Jan 2007, "Nathaniel Gray" wrote: > > let results = latencyN 40000 > [("function", call_f, ()); > ("method", call_o, ()); > ("closure", call_fc, ()); > ("obj. closure", call_foc, ())] Knowing the right number of calls to have a meaningful measurment is not always easy, espcially for fast running code like yours. Actually, as Edgar Friendly pointed out, in your case, even max_int is not enough... In order to get rid of this you should: - Use the throughputN function which allows you to set the running time of the function instead of the number of calls; - Use the new version Benchmark module which now uses Int64.t to store the number of calls. Moreover, with this new version,... On Wed, 17 Jan 2007, Edgar Friendly wrote: > > Rate > function -5.36871e+10/s ^ ...this cannot happen anymore. You are also advised to set the ~repeat flag to a value of 5-10 in order to trigger some statistical tests that will put the relative rates of the tested functions in between brackets if they are not significantly different. For your problem, your code (attached) gives the following results (on a Intel 2Ghz dual core): Rate method obj. closure closure function method 156624689+- 738397/s -- -60% -68% -84% obj. closure 387882394+- 4665352/s 148% -- -20% -61% closure 486459254+- 1330609/s 211% 25% -- -51% function 984016184+-24771072/s 528% 154% 102% -- Since the (i,j) entry is rij = (ri / rj - 1) where ri is the rate of line i and rj the rate of col j, if we want to compare the times ti = 1/ri, one gets tj = (1 + rij) ti. Thus one has to look at the last line to have the times relative to the function one: t(closure) = (1 + 102%) t(function) ~ 2x t(obj.closure) = (1 + 154%) t(function) ~ 2.5x t(method) = (1 + 528%) t(function) ~ 6.3x This is quite coherent with Jacques Garrigue results. Hope this helps, ChriS ----Next_Part(Wed_Jan_31_18_03_29_2007_504)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="gray.ml" (* Test method dispatch vs. function dispatch vs. closure dispatch *) let f x = x + 100 let call_f () = f 1 let o = object method f_o x = x + 100 end let call_o () = o#f_o 1 let f_c () x = x + 100 let f_c' = f_c () let call_fc () = f_c' 1 let o_c = object method f_oc () x = x + 100 end let f_oc' = o_c#f_oc () let call_foc () = f_oc' 1 open Benchmark let () = let results = latencyN ~repeat:5 1100000000L (* let results = throughputN ~repeat:5 1 *) [("function", call_f, ()); ("method", call_o, ()); ("closure", call_fc, ()); ("obj. closure", call_foc, ())] in tabulate results (* ocamlopt -o gray.com -inline 0 -I benchmark unix.cmxa benchmark.cmxa gray.ml ocamlc -o gray.exe -inline 0 -I benchmark unix.cma benchmark.cma gray.ml *) ----Next_Part(Wed_Jan_31_18_03_29_2007_504)----