[Caml-list] comparison with C performance

[Lex Stein <stein@eecs.harvard.edu>]

Hi,

A while ago I built an NFS server in OCaml (BDBFS) and the performance
stunk. It was 10x slower than the BSD in-kernel NFS server for metadata
operations. There was some speculation about what was causing this
slowness. It could have been a number of things. So in order for my
Advisor to let me continue programming in OCaml, I set out to show that it
wasn't due to the choice of OCaml.

The experiment consisted of 10,000 repeated RPC cycles across a 100Mbps
link. An RPC cycle consists of a NULL RPC followed by an RPC with a 20 and
24 byte string that is written to a Berkeley-DB database (via DB->put)
with the 20 bytes as key and 24 bytes as value. The C test never leaves C
code and calls directly into the Berkeley-DB C code. The OCaml test leaves
C above the RPC layer and enters the OCaml world, using the OCaml
Berkeley-DB interface (the one I wrote, I know Yaron Minsky has one too)
to write to the database. The following column shows the time taken by a
client (the same client across all 3 test configurations) to execute
100,000 RPC cycles. I ran the experiment 15 times. The square brackets
contain the standard deviation. The units are seconds.

		Test run at 5:00am 04-27-2003
		100,000 RPC cycs
C shunt: 	22.87s [1.20s]
OCaml shunts:
  bytecode: 	23.87s [0.96s]
  native:       22.20s [0.98s]

The result is that the C and OCaml native and C and OCaml bytecode are
not differentiable, due to the relative standard deviations. The OCaml
bytecode and native are differentiable, being more than one standard
deviation away from each other.

To get back to the original story: this has pointed me in the direction of
improving BDBFS' performance by improving the efficiency of the directory
listing and lookup algorithms rather than changing languages. OCaml seems
to fare just fine against C.

Lex

On Thu, 1 May 2003, Brian Hurt wrote:

>
> Given the number of performance-related discussions in this maillist of
> late, I thought I'd forward this article:
> http://www-106.ibm.com/developerworks/java/library/j-jtp04223.html
>
> It's about Java, but I think it's still worthwhile reading for Ocaml
> programmers.  The lesson to learn here is that performance is tricky- what
> you think will obviously be a problem often isn't, and what you think
> won't be a problem can be.  Make it work correctly first, then measure
> performance, then enhance for performance if necessary.
>
> Another comment that applies to both languages is that if it's a glaringly
> obvious problem, the compiler people are probably already working on it
> (or possibly already solved it).
>
> Brian
>
>
> -------------------
> To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
> Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners