[Caml-list] Implementation plan for freeing dynamically loaded code

[Caml-list] Implementation plan for freeing dynamically loaded code

[Nuutti Kotivuori]

This is an initial outline of an implementation plan for making
dynamically loaded code garbage collectable.

1. Custom block for code parts

Implement a small module providing custom blocks, which give pointers
to malloc()'d blocks and a finalization function which free()'s the
blocks at that time.

Allocation would call stat_alloc() and store the address in the custom
block, finalization would call stat_free() on the address - and some
function is provided to give the address as a string type, like
Meta.static_alloc does.

2. Two new bytecode instructions, CLOSUREDYN and CLOSURERECDYN

These two instructions would behave mostly like their original
counterparts, CLOSURE and CLOSUREREC, but would copy the *last*
element of the current environment to be an extra last element in the
generated closure. Environment should always be a closure pointer, so
CLOSURE and CLOSUREREC would work similarily in this respect I
believe.

3. Bytecode patcher for CLOSURE,CLOSUREREC -> CLOSUREDYN,CLOSURERECDYN

A simple bytecode traversing code that would change the CLOSURE
opcodes to CLOSUREDYN opcodes and CLOSUREREC opcodes to
CLOSURERECDYN. The 'size' of each instruction and almost ready code
for this can be taken from tools/dumpobj.ml.

4. Modification of dynlink.ml to use these

The dynlink library needs to take advantage of these new functions. It
needs to allocate the code block with the custom block code, it needs
to run the bytecode patcher on the bytecode before starting it - and
Meta.reify_bytecode needs to be altered to place a given value into
the closure it creates for the bytecode. That way when the CLOSUREDYN
calls are invoked when running the code block initially, they will
have the reference to the allocated block as the last element of the
environment. And since every code pointer that references this code
block has to have been created from a CLOSUREDYN (or CLOSURERECDYN)
instruction, we can safely use that last element of environment
everywhere.

And that's it.

Strictly speaking, the new bytecode instructions aren't at all needed
- but mangling the original bytecode to push the correct pointer on
the stack so that it becomes the last parameter seems like a too
difficult exercise. It becomes more viable if special compilation can
be expected from dynamically loaded code.

So, if anyone has improvement suggestions, or reasons why this
wouldn't work at all, please tell me. Otherwise, I shall probably get
to work in the near future and see what comes up. I wish to do a bunch
of tests on custom blocks in any case, to learn how they really
behave.

-- Naked

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Implementation plan for freeing dynamically loaded code

[Alex Baretta]

Nuutti Kotivuori wrote:
> This is an initial outline of an implementation plan for making
> dynamically loaded code garbage collectable.
...

I've been following this thread with great interest. I hope this plan 
will work out well and will be accepted by the Great Gurus of Inria.

I'm bugging the list to point out that such a solution is also terribly 
necessary to use ocaml as a scripting language within an application 
through the Toplevelib. Consider a long-living server application which 
receives and executes scripts in the form of ocaml source code. Such 
scripts would be bytecompiled by the toplevellib and linked into the 
application code. Since this code is not garbage collected, such an 
approach would lead to a memory leak and to the necessity of 
periodically "garbage-collecting" entire server processes and fork new 
ones. This is the approach I have taken in my projects up to now. Yet, I 
dream that such memory leaks be avoided altogether.

Alex

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners