Hi Anders! 

How're you doing ? 

Thanks for the suggestion. We actually have a solution in production right now (www.besport.com) that relies on netshm. Unfortunately, the serialization / deserialization is killing the performance : about 200ms for a full request against 7ms for a simple Hashtbl lookup on our hardware. 

ocaml-everlasting did solve this performance bottleneck but we get unacceptable segmentation faults, and before debugging this library I'd rather learn about other approaches :)

Best regards,

William


On Thu, Dec 8, 2011 at 12:49 AM, Anders Fugmann <anders@fugmann.net> wrote:
Hi William,

Have you tried netshm from ocamlnet library? It seems that it would fit your requirements by placing memory outside the reach of the garbage collector.

Using netshm would require a lot of serialization / deserilization and memory copying, which might not be that efficient if you access the cache a lot.

Regards
Anders Fugmann




On 12/08/2011 05:35 AM, William Le Ferrand wrote:
Dear list,

We are building a cache in ocaml and we're wondering if it would make
sense to store ocaml values outside the reach of the gc. (gc on a 20GB
cache hangs the process for a second or so).

To run some experiments, we wrote a small library
(https://github.com/besport/ocaml-everlasting) that exposes two
functions, get and set.

When inserting a value, we copy recursively the blocs outside of the
reach of the gc (and put the resulting value in some C array). When
getting the value, we simply pass the pointer to the copied value to the
ocaml code (the structure is still coherent and the value is directly
usable). We also wrote an "update" function that compare a new value
with the existing value in cache, to avoid unnecessary memory
allocation/deallocation.

It does not seems very stable though, but I don't know if it is a bug in
the update function or simply because this approach is not reasonable.
Do you have any thoughts? Is there any clever way to build a large cache
in an ocaml app ?

Thanks in advance for any tips!

Best

William


--
William Le Ferrand

Mobile : (+1) (415) 683-1484
Web : http://williamleferrand.github.com/
<http://www.linkedin.com/in/williamleferrand>



--
Caml-list mailing list.  Subscription management and archives:
https://sympa-roc.inria.fr/wws/info/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs




--
William Le Ferrand

Mobile : (+1) (415) 683-1484
Web : http://williamleferrand.github.com/