From: Brian Hurt <brian.hurt@qlogic.com>
To: Xavier Leroy <xavier.leroy@inria.fr>
Cc: onlyclimb <onlyclimb@163.com>, <caml-list@inria.fr>
Subject: Re: [Caml-list] speed
Date: Sat, 4 Jan 2003 19:13:11 -0600 (CST) [thread overview]
Message-ID: <Pine.LNX.4.33.0301041808020.2036-100000@eagle.ancor.com> (raw)
In-Reply-To: <20030103143221.B29601@pauillac.inria.fr>
Woo hoo. Language advocacy with benchmarks again.
Feel free to replace this whole post with a comment about "lies, damned
lies, and cross language benchmarks". It amounts to the same thing.
On Fri, 3 Jan 2003, Xavier Leroy wrote:
> > Is it normal that my ocaml program is only 2 times faster than the java
> > counterpart ?(using the same method and complied into native. jdk is 1.4.1
>
> You know, many compiler researchers would kill their whole families to
> get speedups by a factor of 2 :-)
>
> James Gosling gave a talk at INRIA recently where he repeated the
> party line that JDK 1.4 runs as fast, or even faster, than C++.
Quibble #1: *what* C++? Most of the time, when I see C++ benchmarked,
what's really being benchmarked is C compiled with a C++ compiler, or at
most C with classes. My experience with C++ tells me that if you actually
use the features of C++- RTTI, templates, STL, exceptions, operator
overloading, etc- the code you produce is often much *slower* than Java.
With a language as feature rich/bloated as C++, which subset of the
language you use makes a huge difference in your performance. Ocaml has
the same problem in a lot of ways.
Quibble #2: define "equivelent program".
> So, by transitivity, you're implying that OCaml is twice as fast as C++.
> Yippee!
>
> More seriously: Java is nowhere as fast as a good C++ compiler (see
> e.g. http://www.coyotegulch.com/reviews/almabench.html for an
> independent, cross-language benchmark in numerical computing),
I note the coyote gulch benchmark shows IBM's Java to be more-or-less on
par with GCC 3.2. I note, btw, that GCC 3.2 is signifigantly better at
optimization than GCC 2.9x, producing code about 10% faster on average
IIRC according to the GCC maintainers themselves. Which tells me that
IBM's Java *is* better than GCC 2.9x. Which is still the most commonly
used compiler on Linux systems. Ditto for Windows. My own experience and
tests show me that MS VC++ 6.0 is no better than, and in many cases worse
than, GCC 2.9x for optimization.
> but it's not that slow either. A factor of 2 slower than ocamlopt
> sounds broadly reasonable, especially if the program doesn't stress
> the GC too much. Bagley's shootout (http://www.bagley.org/~doug/shootout/)
> seems to suggest a larger factor (JDK 1.3 slightly slower than OCaml
> bytecode), but his figures may be lowered by Java's slow start-up times.
Startup costs dominate in Bagley's shootout. Look at matrix
multiplication- the fastest tests (C, C++, and Ocaml) are running in
70-110 milliseconds. Most timers are accurate only to ~10 milliseconds,
which means the time for the C program to run could be anything from
600 millisecond to 800 milliseconds, for an error of +/-14.3%.
Java has huge start up costs. First off, you have the JIT. Then, there
is a time delay before hotspot kicks in an actually starts optimizing the
code to any signifigant extent. Notice that the pro-Java benchmarks run
the code to be benchmarked a few thousands or tens of thousands of times
before starting the timer, so that the hotspot optimizer has already been
over the code a couple of times. Or at least once, to bypass JIT time.
Is this a legitimate tactic? Lies, damned lies, and cross-language
benchmarks.
Note that I can also claim, with a straight face, that Ocaml is 5x
*slower* than Java. Take a look at Bagley's shootout on matrix
multiplication, comparing byte-code interpreted Java with byte-code
intepreted Ocaml. Which is a much more apples to apples comparison.
Then there is the question of *future* performance of the languages. In
the pro-Java camp, I direct your attention to HP's Dynamo project:
http://www.arstechnica.com/reviews/1q00/dynamo/dynamo-1.html
http://www.hpl.hp.com/cambridge/projects/Dynamo/
which showed that a virtual PA-RISC emulator could run the code up to 20%
faster than running the same code native. In the pro-Ocaml camp, Caml's
innate ease of reasoning about code open up, I think, a much larger array
of potiental optimizations for the compiler.
Of course, Java, Ocaml, and C++ all pale in comparison to the performance
of hand-tuned assembly language. Ergo, anyone who is using performance of
the generated code as the primary reason for picking a language should, by
all logic, be coding in assembly language.
Note that I, personally, think that performance should be the last reason
used to pick a language. Things like correctness of the code, available
libraries and environments, and existing talents and skills of the
workforce, should instead take precedence.
Brian
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
next prev parent reply other threads:[~2003-01-05 1:05 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-01-03 16:00 onlyclimb
2003-01-03 11:38 ` [Caml-list] speed Clemens Hintze
2003-01-03 11:47 ` [Caml-list] speed Noel Welsh
2003-01-02 16:45 ` Chet Murthy
2003-01-03 13:32 ` Xavier Leroy
2003-01-02 17:52 ` Chet Murthy
2003-01-03 14:53 ` Sven Luther
2003-01-03 15:28 ` Erol Akarsu
2003-01-02 17:53 ` Coyote Gulch test in Caml (was Re: [Caml-list] speed ) Chet Murthy
2003-01-03 15:10 ` Shawn Wagner
2003-01-03 15:56 ` Oleg
2003-01-04 18:31 ` Xavier Leroy
2003-01-18 22:49 ` Oleg
2003-01-18 23:50 ` Shawn Wagner
2003-01-20 21:23 ` David Chase
2003-01-20 21:39 ` Nickolay Semyonov-Kolchin
2003-01-21 0:54 ` Brian Hurt
2003-01-21 13:09 ` David Chase
2003-01-21 13:15 ` Daniel Andor
2003-01-21 20:26 ` Nickolay Semyonov-Kolchin
2003-01-19 10:33 ` Siegfried Gonzi
2003-01-19 10:34 ` Siegfried Gonzi
2003-01-21 9:56 ` [Caml-list] Re: Coyote Gulch test in Caml Xavier Leroy
2003-01-21 15:57 ` Brian Hurt
2003-01-27 16:58 ` Daniel Andor
2003-01-28 8:27 ` Christian Lindig
2003-01-05 1:13 ` Brian Hurt [this message]
2003-01-05 1:48 ` [Caml-list] speed Michael Vanier
2003-01-07 16:03 isaac gouy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Pine.LNX.4.33.0301041808020.2036-100000@eagle.ancor.com \
--to=brian.hurt@qlogic.com \
--cc=caml-list@inria.fr \
--cc=onlyclimb@163.com \
--cc=xavier.leroy@inria.fr \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).