fundata1 -- Karmic Social Capital Benchmark and Shootout

fundata1 -- Karmic Social Capital Benchmark and Shootout

[Alexy Khrabrov]

I am happy to announce fundata1 -- the largest-ever program per RAM allocation in Haskell, originally implemented in Clojure and then OCaml and Haskell for social network modeling.  

http://github.com/alexy/fundata1

It has now become the first large-scale social networking benchmark with a real dynamic social graph built from the actual Twitter gardenhose, with the data OK'd by Twitter and supplied along with the benchmark.

I wrote three reference implementations, all on github as well.  Clojure and OCaml are quite basic, while Haskell community had a chance to optimize its data structures and in fact fix a GC integer overflow while working on it.  You're welcome to fork and improve all of these implementations, and supply others!

There's a Google Group, 

http://groups.google.com/group/fundata/

to discuss the shootout.  There's also a blog about it and other functional things at

http://functional.tv/

Let the fun begin!

-- Alexy Khrabrov
firstname.lastnameATgmaildotcom

Re: [Caml-list] fundata1 -- Karmic Social Capital Benchmark and Shootout

[bluestorm]

[-- Attachment #1: Type: text/plain, Size: 621 bytes --]

I was mildly curious and directly went for performance results.
A few bits of information for those that dont want to explore the website
themselves :

- it's a really big data set; on the website you're advised to
    export OCAMLRUNPARAM='h=5G;s=1G'

- the Haskell implementation was developped first, tuned, and now takes 17
minutes to run. The OCaml implementation is a simple port of the Haskell
implementation (with the data structures adapted), and it takes 15 minute to
run. A younger Clojure implementation is at 30 minutes for now.

My hasty conclusion : the OCaml GC and the Hasthbl implementation scale
well.

[-- Attachment #2: Type: text/html, Size: 738 bytes --]