[Caml-list] Storing ocaml values outside ocaml's heap

[Caml-list] Storing ocaml values outside ocaml's heap

[William Le Ferrand]

[-- Attachment #1: Type: text/plain, Size: 1202 bytes --]

Dear list,

We are building a cache in ocaml and we're wondering if it would make sense
to store ocaml values outside the reach of the gc. (gc on a 20GB cache
hangs the process for a second or so).

To run some experiments, we wrote a small library (
https://github.com/besport/ocaml-everlasting) that exposes two functions,
get and set.

When inserting a value, we copy recursively the blocs outside of the reach
of the gc (and put the resulting value in some C array). When getting the
value, we simply pass the pointer to the copied value to the ocaml code
(the structure is still coherent and the value is directly usable). We also
wrote an "update" function that compare a new value with the existing value
in cache, to avoid unnecessary memory allocation/deallocation.

It does not seems very stable though, but I don't know if it is a bug in
the update function or simply because this approach is not reasonable. Do
you have any thoughts? Is there any clever way to build a large cache in an
ocaml app ?

Thanks in advance for any tips!

Best

William


-- 
William Le Ferrand

Mobile : (+1) (415) 683-1484
Web : http://williamleferrand.github.com/
<http://www.linkedin.com/in/williamleferrand>

[-- Attachment #2: Type: text/html, Size: 1844 bytes --]

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[Stéphane Glondu]

Le 08/12/2011 05:35, William Le Ferrand a écrit :
> We are building a cache in ocaml and we're wondering if it would make
> sense to store ocaml values outside the reach of the gc. (gc on a 20GB
> cache hangs the process for a second or so).

Have you heard of the ancient [1] library?

[1] http://git.annexia.org/?p=ocaml-ancient.git;a=summary


Cheers,

-- 
Stéphane

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[William Le Ferrand]

[-- Attachment #1: Type: text/plain, Size: 825 bytes --]

hi Stephane

Yes, but it's not exactly what we're looking for : we plan to access the
data at a (very) high rate, so swapping/unswapping would probably kill the
performance ..


On Wed, Dec 7, 2011 at 10:04 PM, Stéphane Glondu <steph@glondu.net> wrote:

> Le 08/12/2011 05:35, William Le Ferrand a écrit :
> > We are building a cache in ocaml and we're wondering if it would make
> > sense to store ocaml values outside the reach of the gc. (gc on a 20GB
> > cache hangs the process for a second or so).
>
> Have you heard of the ancient [1] library?
>
> [1] http://git.annexia.org/?p=ocaml-ancient.git;a=summary
>
>
> Cheers,
>
> --
> Stéphane
>
>


-- 
William Le Ferrand

Mobile : (+1) (415) 683-1484
Web : http://williamleferrand.github.com/
<http://www.linkedin.com/in/williamleferrand>

[-- Attachment #2: Type: text/html, Size: 1721 bytes --]

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[Anders Fugmann]

Hi William,

Have you tried netshm from ocamlnet library? It seems that it would fit 
your requirements by placing memory outside the reach of the garbage 
collector.

Using netshm would require a lot of serialization / deserilization and 
memory copying, which might not be that efficient if you access the 
cache a lot.

Regards
Anders Fugmann



On 12/08/2011 05:35 AM, William Le Ferrand wrote:
> Dear list,
>
> We are building a cache in ocaml and we're wondering if it would make
> sense to store ocaml values outside the reach of the gc. (gc on a 20GB
> cache hangs the process for a second or so).
>
> To run some experiments, we wrote a small library
> (https://github.com/besport/ocaml-everlasting) that exposes two
> functions, get and set.
>
> When inserting a value, we copy recursively the blocs outside of the
> reach of the gc (and put the resulting value in some C array). When
> getting the value, we simply pass the pointer to the copied value to the
> ocaml code (the structure is still coherent and the value is directly
> usable). We also wrote an "update" function that compare a new value
> with the existing value in cache, to avoid unnecessary memory
> allocation/deallocation.
>
> It does not seems very stable though, but I don't know if it is a bug in
> the update function or simply because this approach is not reasonable.
> Do you have any thoughts? Is there any clever way to build a large cache
> in an ocaml app ?
>
> Thanks in advance for any tips!
>
> Best
>
> William
>
>
> --
> William Le Ferrand
>
> Mobile : (+1) (415) 683-1484
> Web : http://williamleferrand.github.com/
> <http://www.linkedin.com/in/williamleferrand>
>

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[William Le Ferrand]

[-- Attachment #1: Type: text/plain, Size: 2990 bytes --]

Hi Anders!

How're you doing ?

Thanks for the suggestion. We actually have a solution in production right
now (www.besport.com) that relies on netshm. Unfortunately, the
serialization / deserialization is killing the performance : about 200ms
for a full request against 7ms for a simple Hashtbl lookup on our hardware.

ocaml-everlasting did solve this performance bottleneck but we get
unacceptable segmentation faults, and before debugging this library I'd
rather learn about other approaches :)

Best regards,

William


On Thu, Dec 8, 2011 at 12:49 AM, Anders Fugmann <anders@fugmann.net> wrote:

> Hi William,
>
> Have you tried netshm from ocamlnet library? It seems that it would fit
> your requirements by placing memory outside the reach of the garbage
> collector.
>
> Using netshm would require a lot of serialization / deserilization and
> memory copying, which might not be that efficient if you access the cache a
> lot.
>
> Regards
> Anders Fugmann
>
>
>
>
> On 12/08/2011 05:35 AM, William Le Ferrand wrote:
>
>> Dear list,
>>
>> We are building a cache in ocaml and we're wondering if it would make
>> sense to store ocaml values outside the reach of the gc. (gc on a 20GB
>> cache hangs the process for a second or so).
>>
>> To run some experiments, we wrote a small library
>> (https://github.com/besport/**ocaml-everlasting<https://github.com/besport/ocaml-everlasting>)
>> that exposes two
>> functions, get and set.
>>
>> When inserting a value, we copy recursively the blocs outside of the
>> reach of the gc (and put the resulting value in some C array). When
>> getting the value, we simply pass the pointer to the copied value to the
>> ocaml code (the structure is still coherent and the value is directly
>> usable). We also wrote an "update" function that compare a new value
>> with the existing value in cache, to avoid unnecessary memory
>> allocation/deallocation.
>>
>> It does not seems very stable though, but I don't know if it is a bug in
>> the update function or simply because this approach is not reasonable.
>> Do you have any thoughts? Is there any clever way to build a large cache
>> in an ocaml app ?
>>
>> Thanks in advance for any tips!
>>
>> Best
>>
>> William
>>
>>
>> --
>> William Le Ferrand
>>
>> Mobile : (+1) (415) 683-1484
>> Web : http://williamleferrand.**github.com/<http://williamleferrand.github.com/>
>> <http://www.linkedin.com/in/**williamleferrand<http://www.linkedin.com/in/williamleferrand>
>> >
>>
>>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/**wws/info/caml-list<https://sympa-roc.inria.fr/wws/info/caml-list>
> Beginner's list: http://groups.yahoo.com/group/**ocaml_beginners<http://groups.yahoo.com/group/ocaml_beginners>
> Bug reports: http://caml.inria.fr/bin/caml-**bugs<http://caml.inria.fr/bin/caml-bugs>
>
>


-- 
William Le Ferrand

Mobile : (+1) (415) 683-1484
Web : http://williamleferrand.github.com/
<http://www.linkedin.com/in/williamleferrand>

[-- Attachment #2: Type: text/html, Size: 4468 bytes --]

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[Anil Madhavapeddy]

[-- Attachment #1: Type: text/plain, Size: 1100 bytes --]

I think Ancient is exactly what you're looking for.

Just disable swap (which you should do anyway on a big modern server), and it will let you keep in-memory, out-of-heap OCaml values that don't get scanned by the GC.

Anil

On 8 Dec 2011, at 08:19, William Le Ferrand wrote:

> hi Stephane
> 
> Yes, but it's not exactly what we're looking for : we plan to access the data at a (very) high rate, so swapping/unswapping would probably kill the performance .. 
> 
> 
> On Wed, Dec 7, 2011 at 10:04 PM, Stéphane Glondu <steph@glondu.net> wrote:
> Le 08/12/2011 05:35, William Le Ferrand a écrit :
> > We are building a cache in ocaml and we're wondering if it would make
> > sense to store ocaml values outside the reach of the gc. (gc on a 20GB
> > cache hangs the process for a second or so).
> 
> Have you heard of the ancient [1] library?
> 
> [1] http://git.annexia.org/?p=ocaml-ancient.git;a=summary
> 
> 
> Cheers,
> 
> --
> Stéphane
> 
> 
> 
> 
> -- 
> William Le Ferrand
> 
> Mobile : (+1) (415) 683-1484
> Web : http://williamleferrand.github.com/
> 


[-- Attachment #2: Type: text/html, Size: 2303 bytes --]

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[William Le Ferrand]

[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]

Ok thanks. I'll have a very close look at it very soon and hopefully
integrate it on www.besport.com backend then :)

Many thanks to all of you

Best

William

On Thu, Dec 8, 2011 at 1:04 AM, Anil Madhavapeddy <anil@recoil.org> wrote:

> I think Ancient is exactly what you're looking for.
>
> Just disable swap (which you should do anyway on a big modern server), and
> it will let you keep in-memory, out-of-heap OCaml values that don't get
> scanned by the GC.
>
> Anil
>
> On 8 Dec 2011, at 08:19, William Le Ferrand wrote:
>
> hi Stephane
>
> Yes, but it's not exactly what we're looking for : we plan to access the
> data at a (very) high rate, so swapping/unswapping would probably kill the
> performance ..
>
>
> On Wed, Dec 7, 2011 at 10:04 PM, Stéphane Glondu <steph@glondu.net> wrote:
>
>> Le 08/12/2011 05:35, William Le Ferrand a écrit :
>> > We are building a cache in ocaml and we're wondering if it would make
>> > sense to store ocaml values outside the reach of the gc. (gc on a 20GB
>> > cache hangs the process for a second or so).
>>
>> Have you heard of the ancient [1] library?
>>
>> [1] http://git.annexia.org/?p=ocaml-ancient.git;a=summary
>>
>>
>> Cheers,
>>
>> --
>> Stéphane
>>
>>
>
>
> --
> William Le Ferrand
>
> Mobile : (+1) (415) 683-1484
> Web : http://williamleferrand.github.com/
> <http://www.linkedin.com/in/williamleferrand>
>
>
>


-- 
William Le Ferrand

Mobile : (+1) (415) 683-1484
Web : http://williamleferrand.github.com/
<http://www.linkedin.com/in/williamleferrand>

[-- Attachment #2: Type: text/html, Size: 3357 bytes --]

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[Gerd Stolpmann]

Hi William,

this can be made working, and there is already a lot of support in
Ocamlnet (module Netsys_mem, espeically init_value). I'm now using this
all the time for populating shared memory blocks (which need to be outside
the ocaml heap) with normal ocaml values.

In Ocamlnet I'm using simply a bigarray for keeping the reference to the
off-heap memory area (remember, bigarrays are just pointers to C arrays).
This allows it to program the whole algorithm accessing such values in
Ocaml.

A warning though: Experience shows also that this type of programming is
very error-prone, as you always have to take the memory representation of
ocaml values into account.

Gerd


> Dear list,
>
> We are building a cache in ocaml and we're wondering if it would make
> sense
> to store ocaml values outside the reach of the gc. (gc on a 20GB cache
> hangs the process for a second or so).
>
> To run some experiments, we wrote a small library (
> https://github.com/besport/ocaml-everlasting) that exposes two functions,
> get and set.
>
> When inserting a value, we copy recursively the blocs outside of the reach
> of the gc (and put the resulting value in some C array). When getting the
> value, we simply pass the pointer to the copied value to the ocaml code
> (the structure is still coherent and the value is directly usable). We
> also
> wrote an "update" function that compare a new value with the existing
> value
> in cache, to avoid unnecessary memory allocation/deallocation.
>
> It does not seems very stable though, but I don't know if it is a bug in
> the update function or simply because this approach is not reasonable. Do
> you have any thoughts? Is there any clever way to build a large cache in
> an
> ocaml app ?
>
> Thanks in advance for any tips!
>
> Best
>
> William
>
>
> --
> William Le Ferrand
>
> Mobile : (+1) (415) 683-1484
> Web : http://williamleferrand.github.com/
> <http://www.linkedin.com/in/williamleferrand>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>


-- 
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
*** Searching for new projects! Need consulting for system
*** programming in Ocaml? Gerd Stolpmann can help you.

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[Anders Fugmann]

On 12/08/2011 09:56 AM, William Le Ferrand wrote:
> Hi Anders!
>
> How're you doing ?
>
> Thanks for the suggestion. We actually have a solution in production
> right now (www.besport.com <http://www.besport.com>) that relies on
> netshm. Unfortunately, the serialization / deserialization is killing
> the performance : about 200ms for a full request against 7ms for a
> simple Hashtbl lookup on our hardware.
200 ms seems too much - Nothing takes 200ms on modern hardware. not even 
on AWS :-)

Maybe the system was swapping or some other noise polluted your 
statistics. Which serialization format did you use? ocaml's standard 
marshaling, or did your write your own?


  > ocaml-everlasting did solve this performance bottleneck but we get
> unacceptable segmentation faults, and before debugging this library I'd
> rather learn about other approaches :)
What type of performance do you get with ocaml-everlasting?

Regards
Anders

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[Gaius Hammond]

Alternatively (depending on the datatypes), try in in-memory[1] SQLite[2] database:


[1] http://www.sqlite.org/inmemorydb.html

[2] http://www.ocaml.info/home/ocaml_sources.html


Gaius



------------------

-----Original Message-----
From: Stéphane Glondu <steph@glondu.net>
Date: Thu, 08 Dec 2011 07:04:19 
To: William Le Ferrand<William.Le-Ferrand@polytechnique.edu>
Cc: caml users<caml-list@inria.fr>
Subject: Re: [Caml-list] Storing ocaml values outside ocaml's heap

Le 08/12/2011 05:35, William Le Ferrand a écrit :
> We are building a cache in ocaml and we're wondering if it would make
> sense to store ocaml values outside the reach of the gc. (gc on a 20GB
> cache hangs the process for a second or so).

Have you heard of the ancient [1] library?

[1] http://git.annexia.org/?p=ocaml-ancient.git;a=summary


Cheers,

-- 
Stéphane



-- 
Caml-list mailing list.  Subscription management and archives:
https://sympa-roc.inria.fr/wws/info/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[Goswin von Brederlow]

William Le Ferrand <William.Le-Ferrand@polytechnique.edu> writes:

> Dear list, 
>
> We are building a cache in ocaml and we're wondering if it would make sense to
> store ocaml values outside the reach of the gc. (gc on a 20GB cache hangs the
> process for a second or so).
>
> To run some experiments, we wrote a small library (https://github.com/besport/
> ocaml-everlasting) that exposes two functions, get and set. 
>
> When inserting a value, we copy recursively the blocs outside of the reach of
> the gc (and put the resulting value in some C array). When getting the value,
> we simply pass the pointer to the copied value to the ocaml code (the structure
> is still coherent and the value is directly usable). We also wrote an "update"
> function that compare a new value with the existing value in cache, to avoid
> unnecessary memory allocation/deallocation.
>
> It does not seems very stable though, but I don't know if it is a bug in the
> update function or simply because this approach is not reasonable. Do you have
> any thoughts? Is there any clever way to build a large cache in an ocaml app ? 
>
> Thanks in advance for any tips!
>
> Best
>
> William

For a generic case you will have to inspect the values, tell the GC
about all the other ocaml values it points to and track any heap values
that have a reference to your external value so you don't delete it
before it is dead. In short you need to write a GC.

For special cases you can store completly self contained values, e.g. a
array of floats or a record of non pointer types. The simplest of types.


For a cache you often have millions of objects of the same type. And
often those contain pointers but do not share them. Then it becomes
feasable to copy everything they point to into the cache as well. Or
write a little C glue to store the data in an abstract or custom block
that does not contain pointers to outside the block. Often you can also
store the data much more compact than ocaml allows and thereby reduce
the memory footprint and increase cache efficiency.

MfG
        Goswin

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[oliver]

On Thu, Dec 08, 2011 at 09:04:50AM +0000, Anil Madhavapeddy wrote:
> I think Ancient is exactly what you're looking for.
> 
> Just disable swap (which you should do anyway on a big modern server), and it
> will let you keep in-memory, out-of-heap OCaml values that don't get scanned by
> the GC.
[...]


Completely disabling swap can crash the system.
Better just fine tune it, so that it uses swapping less often.

=> on Linux swappiness can be set via /proc:   /proc/sys/vm/swappiness


Ciao,
  Oliver

Re: [Caml-list] Storing ocaml values outside ocaml's heap

[Goswin von Brederlow]

oliver <oliver@first.in-berlin.de> writes:

> On Thu, Dec 08, 2011 at 09:04:50AM +0000, Anil Madhavapeddy wrote:
>> I think Ancient is exactly what you're looking for.
>> 
>> Just disable swap (which you should do anyway on a big modern server), and it
>> will let you keep in-memory, out-of-heap OCaml values that don't get scanned by
>> the GC.
> [...]
>
>
> Completely disabling swap can crash the system.

No it can't. Or at least that would be a serious bug.

With or without swap if you run out of memory the OOM killer activates
and your system goes down the drain. But it should never crash. Swap
just delays the inevitable.

> Better just fine tune it, so that it uses swapping less often.
>
> => on Linux swappiness can be set via /proc:   /proc/sys/vm/swappiness

What I saw is that often on a 16GB node people would start programs that
use up 15.99GB. As in try to maximize the memory they use. Now when some
other things run like a cron job the system would reach an OOM
situation. And that is where a little swap comes in handly.

So give the system 1GB swap so it can swap out enough of a 16GB batch
job to run cron and friends when they occasionally wake up. But nowhere
the amount the ancient swap = 2x ram rule would dictate.

MfG
        Goswin