Re: [Caml-list] CallCC using fork (and garbage collected processes)

[skaller <skaller@users.sourceforge.net>]

On Mon, 2007-07-16 at 06:30 +0100, Jon Harrop wrote:
> Just another crazy idea. So Oleg's callcc for OCaml works by copying the OCaml 
> bytecode stack and its rather nifty because you can trivially control invert 
> parsers and so forth.
> 
> You can replicate the native-code stack using fork. What if you wrap the 
> forked process in an object and set the finalizer to pass it a message 
> telling it to die. Then you could invoke the process (continuation) to 
> propagate computation. Maybe you can pass your child the pipe from your 
> parent so that it can respond directly and then die yourself (a tail call!).
> 
> I know this is really silly because you've got the OCaml GC collecting 
> processes (which is even worse than GCing OpenGL textures) but my machine can 
> fork processes pretty quickly and I'm just wondering if anyone's tried it?

A much reduced version of this idea has been tried
in C, swapping the C stack pointer and registers using
setjmp/longjmp and some assembler.

This is, of course, much faster than forking.

However the C method doesn't work, because Unix uses
linear bounded stacks, and there isn't enough address 
space to create enough stacks.

The forking idea has some interest because this problem
might go away. However it isn't necessary if you COPY
the stack, because the copy isn't executing so doesn't
need a bound. Of course, forking copies the stack
lazily (because it copies everything lazily) so it
actually might be quite efficient compared to copying.

Note the Fork idea will NOT work in ML anyhow, since
ML (including Ocaml) is a procedural programming language.
Fork won't support writing to shared memory (eg ref types).

Ultimately though, the only viable way to do this properly
is to use a spaghetti stack on the heap as Felix does.
If you're going to run with user space threads, you want
to be able to run millions of them and context switch
very quickly .. otherwise you can just use Threads
and the Event module for communications.

Felix has no trouble with a million threads on an AMD64 with
1G of RAM ..somehow I doubt your box has enough process
handles to spawn that many forked processes .. :)

MLton solves this problem by mmap()ing the stacks:
fast context switching AND solves the bound problem.
Stacks grow linearly and then get compacted .. its very cute!

The bottom line here is that programming language execution
models have to STOP using the stack except for very temporary
storage -- but even that isn't going to work, if you're calling
C code which calls back into your language.


-- 
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net