[Caml-list] meta-gc (Re: concurrent gc?)

[Caml-list] meta-gc (Re: concurrent gc?)

[Raoul Duke]

> I agree with Malcolm's experience, and my situation might be similar to
> yours: games -- a lot of allocations at high framerate. I'm guessing this
> from how evil you consider pauses to be. ;)

yes, interactive apps.

> allocators -- you still need to use dynamic allocations wisely. Here, I
> think the biggest problem is when you have a GC, you lose awareness of the
> allocations you're triggering.

yes, those are nice down to earth wise words that need to be said more
often in every class :-)

to kick the question up a meta-level: Does anybody have thoughts on
how to best orthogonalize actual memory allocation from where the
application says it wants it? I mean if I write a stupid loop that
makes a zillion particles in the middle of each frame render then yes
I'm an idiot, sure... but what things can be done to actually allow
such idiocy to squeak by?

Maybe that's a horrible thing to suggest as it just panders. But I
like to think of it more along the lines of trying to tease things
apart as much as possible, and then some more (to corrupt an infamous
meme).

[In some sense that is what GC purported to be when it was being sold
like snake oil :-) and for a long time "regular" programmers knew
better and would make fun of e.g. lisp programmers.]

[Caml-list] meta-gc (Re: concurrent gc?)

[John F. Carr]

 > to kick the question up a meta-level: Does anybody have thoughts on
 > how to best orthogonalize actual memory allocation from where the
 > application says it wants it? I mean if I write a stupid loop that
 > makes a zillion particles in the middle of each frame render then yes
 > I'm an idiot, sure... but what things can be done to actually allow
 > such idiocy to squeak by?

If you know in advance when you will need to run uninterrupted by
garbage collection, set the minor heap size large before that period.

If the minor heap size is X words and a collection has just completed,
the next collection will occur when any of the following is true:

1. You have allocated X words of small objects, i.e. those that
are 256 words or less exclusive of header.  These are allocated
from the minor heap.

2. You have allocated X words of large objects, i.e. those that are
257 words or more exclusive of header.  These are allocated from the
major heap and count as "old" even though they are new.

3. You have stored too many newly created small objects into objects
that are either large or existed before the last GC, i.e. created too
many old to young references.  Set OCAMLRUNPARAM=v=8 and look for
"ref_table threshold crossed" to see how often you hit this condition.
It should be rare.

Example:

	let a = Array.create 300 0
	let b = Array.create 300 {x=0}
	let c = [|0|]
	let d = 0
	let e = 0.0

and assume all of these values are live (used later).

The first line allocates 301 words in the major heap.

The second line allocates 301 words in the major heap, 2 words
in the minor heap, and creates 300 major to minor (old to young)
references.

The third line allocates 2 words in the minor heap.

The fourth line does not allocate memory from the heap.

The fifth line may allocate 0, 2, or 3 words from the minor heap.

Re: [Caml-list] meta-gc (Re: concurrent gc?)

[Raoul Duke]

super cool, thank you.

Re: [Caml-list] meta-gc (Re: concurrent gc?)

[Nicolas Boulay]

[-- Attachment #1: Type: text/plain, Size: 925 bytes --]

2014-07-24 19:37 GMT+02:00 Raoul Duke <raould@gmail.com>:

> ...
>
> to kick the question up a meta-level: Does anybody have thoughts on
> how to best orthogonalize actual memory allocation from where the
> application says it wants it? I mean if I write a stupid loop that
> makes a zillion particles in the middle of each frame render then yes
> I'm an idiot, sure... but what things can be done to actually allow
> such idiocy to squeak by?
> ...
>

On the game world, they have invented entity-component-system (
https://en.wikipedia.org/wiki/Entity_component_system ) data-driven
software architecture. The idea is to be very flexible with a "generic kind
of object" (entity), with typed component. An entity is only an integer.
The component are found inside a collection with the entity identifier.

So each component are allocated in it's own contener, so data are more
compact, and memory access appear less random.

[-- Attachment #2: Type: text/html, Size: 1482 bytes --]