RE: understanding weak

RE: understanding weak

[CUOQ Pascal]

[-- Attachment #1: Type: text/plain, Size: 1809 bytes --]


Warren Harris <warren@metaweb.com> wrote:
> I'd like to understand better how ocaml's weak pointers operate. 

You will be interested in the following important article:
http://portal.acm.org/citation.cfm?id=1411308
:)
 
>First, although it doesn't seem to be specified in the documentation,  
>I assume that weak pointers will *not* be reclaimed (e.g. from a weak  
>hash table) if the program retains some other reference to the object.  

Exactly.

>My second question relates specifically to my application. I would  
>like to have a primary cache of objects, and a secondary index into  
>sub-objects referenced from the primary cache. I.e. CacheA references  
>objects of type A; objects of type A reference objects of type B;  
>CacheB references objects of type B. I would like to guarantee that  
>weak references in CacheB are not flushed unless the corresponding  
>reference from CacheA is first flushed. I assume will be the case if a  
>non-weak reference from A to B is maintained.

The non-weak reference from A to B prevents B being unreachable
without A being unreachable, so yes, a reference from CacheB can
not disappear without the reference from CacheA disappearing
earlier or simultaneously.
This said, if what you want is really a cache, you would probably be
better off fixing its size and renewal strategy yourself than letting
the GC do it for you (by using weak pointers). What it will do has almost
no chance of being the best compromise between memory use and
speed for your application, and it may change without notice from
one version to the other. In short: don't use weak pointers to make caches.

>Can anyone verify?

If you want to experiment to confirm your impressions, the function
Gc.full_major is your friend.

Pascal





[-- Attachment #2: winmail.dat --]
[-- Type: application/ms-tnef, Size: 3459 bytes --]

Re: [Caml-list] RE: understanding weak

[Daniel Bünzli]

Le 30 oct. 08 à 19:48, CUOQ Pascal a écrit :

>> First, although it doesn't seem to be specified in the documentation,
>> I assume that weak pointers will *not* be reclaimed (e.g. from a weak
>> hash table) if the program retains some other reference to the  
>> object.
>
> Exactly.

The documentation should be fixed, as it really doesn't make that  
point clear. It says "A weak pointer is a value that the garbage  
collector may erase at any time" and this isn't what you want if you  
are looking for weak references.

Best,

Daniel

Re: [Caml-list] RE: understanding weak

[Warren Harris]

On Oct 30, 2008, at 11:48 AM, CUOQ Pascal - Pascal.CUOQ@cea.fr wrote:

>
> Warren Harris <warren@metaweb.com> wrote:
>> I'd like to understand better how ocaml's weak pointers operate.
>
> You will be interested in the following important article:
> http://portal.acm.org/citation.cfm?id=1411308
> :)

Thank you for the reference. Looks like the paper's timing couldn't be  
better.

>
>
>> First, although it doesn't seem to be specified in the documentation,
>> I assume that weak pointers will *not* be reclaimed (e.g. from a weak
>> hash table) if the program retains some other reference to the  
>> object.
>
> Exactly.
>
>> My second question relates specifically to my application. I would
>> like to have a primary cache of objects, and a secondary index into
>> sub-objects referenced from the primary cache. I.e. CacheA references
>> objects of type A; objects of type A reference objects of type B;
>> CacheB references objects of type B. I would like to guarantee that
>> weak references in CacheB are not flushed unless the corresponding
>> reference from CacheA is first flushed. I assume will be the case  
>> if a
>> non-weak reference from A to B is maintained.
>
> The non-weak reference from A to B prevents B being unreachable
> without A being unreachable, so yes, a reference from CacheB can
> not disappear without the reference from CacheA disappearing
> earlier or simultaneously.
> This said, if what you want is really a cache, you would probably be
> better off fixing its size and renewal strategy yourself than letting
> the GC do it for you (by using weak pointers). What it will do has  
> almost
> no chance of being the best compromise between memory use and
> speed for your application, and it may change without notice from
> one version to the other. In short: don't use weak pointers to make  
> caches.

Thanks for the advice -- but I thought this was exactly what weak hash  
tables were intended for. (The paper seems to indicate that too.) I  
realize that an explicit replacement policy can be more application- 
friendly, but if what you really want is just "unless  anyone is using  
this" then using weak hash tables would seem appropriate.

>
>
>> Can anyone verify?
>
> If you want to experiment to confirm your impressions, the function
> Gc.full_major is your friend.

I'd hate to do an empirical analysis of this only to find I hadn't  
uncovered the full truth once the app was in production... so asking  
seemed like the way to go. Seems like the manual should say more here.

Warren

Re: [Caml-list] RE: understanding weak

[Alain Frisch]

Warren Harris wrote:
> On Oct 30, 2008, at 11:48 AM, CUOQ Pascal - Pascal.CUOQ@cea.fr wrote:
>> In short: don't use weak pointers to make 
>> caches.
> 
> Thanks for the advice -- but I thought this was exactly what weak hash 
> tables were intended for.

Although there is some similarity between a weak table and a cache 
("store values, but feel free to get rid of them"), they are really 
different beasts: a good implementation of a weak table should release 
memory as soon as possible to avoid memory leak; a good implementation 
of a cache should instead keep data as long as possible (until there is 
a good reason to release it, like resource exhaustion). In particular, 
data stored in a OCaml weak table will be released during the next 
collection cycle when it is no longer referenced by any strong 
reference; this is probably not what you expect from a cache!

-- Alain

Re: understanding weak

[Rémi Vanicat]

Alain Frisch <alain@frisch.fr> writes:

> Warren Harris wrote:
>> On Oct 30, 2008, at 11:48 AM, CUOQ Pascal - Pascal.CUOQ@cea.fr wrote:
>>> In short: don't use weak pointers to make caches.
>>
>> Thanks for the advice -- but I thought this was exactly what weak
>> hash tables were intended for.
>
> Although there is some similarity between a weak table and a cache
> ("store values, but feel free to get rid of them"), they are really
> different beasts: a good implementation of a weak table should release
> memory as soon as possible to avoid memory leak; a good implementation
> of a cache should instead keep data as long as possible (until there
> is a good reason to release it, like resource exhaustion).

It's true, but it could be interesting to have a cache pointer/cache
table provided with the garbage collector: it's the GC that know when
there is not enough memory left, and that it should free those cache
pointer to have some.

-- 
Rémi Vanicat

Re: [Caml-list] RE: understanding weak

[Jon Harrop]

On Thursday 30 October 2008 18:48:26 CUOQ Pascal wrote:
> Warren Harris <warren@metaweb.com> wrote:
> > I'd like to understand better how ocaml's weak pointers operate.
>
> You will be interested in the following important article:
> http://portal.acm.org/citation.cfm?id=1411308

Is this freely available anywhere?

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: [Caml-list] RE: understanding weak

[Stefano Zacchiroli]

On Sat, Nov 01, 2008 at 01:08:17AM +0000, Jon Harrop wrote:
> > You will be interested in the following important article:
> > http://portal.acm.org/citation.cfm?id=1411308
> Is this freely available anywhere?

<rant>

Seriously, you SPAM everybody here with periodical announces of
non-free issues of the OCaml journal and don't even have access to a
random lab with an ACM subscription?

</rant>

SCNR.

Do as we all do: look on authors' homepage, mail them, try citeseer
cache, ...

-- 
Stefano Zacchiroli -*- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è sempre /oo\ All one has to do is hit the right
uno zaino        -- A.Bergonzoni \__/ keys at the right time -- J.S.Bach

Re: [Caml-list] RE: understanding weak

[kirillkh]

[-- Attachment #1: Type: text/plain, Size: 844 bytes --]

On Sat, Nov 1, 2008 at 12:37 PM, Stefano Zacchiroli <zack@upsilon.cc> wrote:

> On Sat, Nov 01, 2008 at 01:08:17AM +0000, Jon Harrop wrote:
> > > You will be interested in the following important article:
> > > http://portal.acm.org/citation.cfm?id=1411308
> > Is this freely available anywhere?
>
> <rant>
>
> Seriously, you SPAM everybody here with periodical announces of
> non-free issues of the OCaml journal and don't even have access to a
> random lab with an ACM subscription?


I think the difference is that ACM is about science and Jon's journal is not
(AFAIK). And since I think science should be free, I'm on Jon's side here,
though I agree that spamming is not nice.

Do as we all do: look on authors' homepage, mail them, try citeseer
> cache, ...


Would be easier, if everyone just published citeseer or direct links.

-Kirill

[-- Attachment #2: Type: text/html, Size: 1420 bytes --]

understanding weak

[Warren Harris]

[-- Attachment #1: Type: text/plain, Size: 986 bytes --]

I'd like to understand better how ocaml's weak pointers operate.  
First, although it doesn't seem to be specified in the documentation,  
I assume that weak pointers will *not* be reclaimed (e.g. from a weak  
hash table) if the program retains some other reference to the object.  
I.e. the weak pointer must be the last remaining pointer to the object  
for reclamation to occur.

My second question relates specifically to my application. I would  
like to have a primary cache of objects, and a secondary index into  
sub-objects referenced from the primary cache. I.e. CacheA references  
objects of type A; objects of type A reference objects of type B;  
CacheB references objects of type B. I would like to guarantee that  
weak references in CacheB are not flushed unless the corresponding  
reference from CacheA is first flushed. I assume will be the case if a  
non-weak reference from A to B is maintained. Can anyone verify?

Thanks,

Warren Harris
Metaweb Technologies


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3739 bytes --]