ocaml for embedding

ocaml for embedding

[Gaspard Bucher]

[-- Attachment #1: Type: text/plain, Size: 997 bytes --]

Hi List !

I'm writing an open source signal processing tool (mainly used for art
projects). Basically objects are connected together through slots and send
values to each other. Currently, objects are either C++ classes or Lua
scripts that can be edited live (live coding), but the API is scripting
language agnostic. This means that with the proper bindings (basically an
"eval" method and a way to "call" inside the language), anything is
possible.

I think it would be a shame to not support at least one functional language
in rubyk and it would be quite fun to have an application with different
languages talking together, each one being used for what it does best. So my
questions are:

1. Is it possible to embed an ocaml compiler ? (something that eats
std::string and produces a callable thing)
2. How would an interface between C++ and a compiled script work ? (passing
values, getting return values)

Thanks for any thoughts,

Gaspard

--
PS: signal processing tool: http://rubyk.org

[-- Attachment #2: Type: text/html, Size: 1223 bytes --]

Re: [Caml-list] ocaml for embedding

[David MENTRE]

Hello,

2010/7/8 Gaspard Bucher <gaspard@teti.ch>:
> 1. Is it possible to embed an ocaml compiler ? (something that eats
> std::string and produces a callable thing)

No, not the compiler. It is however possible to embed an interpreter
and to load byte code (and native code?) on demand:
  http://caml.inria.fr/pub/docs/manual-ocaml/manual032.html#toc134

> 2. How would an interface between C++ and a compiled script work ? (passing
> values, getting return values)

Through a C interface. See above link.

Sincerely yours,
david

Re: [Caml-list] ocaml for embedding

[Gaspard Bucher]

Hi David !

Thanks for the pointer. From my understanding of the use of
caml_startup (or caml_main), this means that the caml runtime is
global. In Rubyk, each "object" (processing unit) is separated from
all other objects (no globals) so that we can properly encapsulate
operations without side effects and provide fine grained locking
(concurrency is important).

Is there a way to avoid:

1. global locking (or locking only during script recompilation)
2. script level encapsulation

Point (2) would mean that if object "a" contains the script:

 let foo a, b = ... do something

and object "b" contains:

 let foo x, y = ... do something else

The recompilation of script "b" does not change the definition of
function "foo".

One possible solution would be to force the creation of singleton
classes for each script named after the script name (unique):

let script_a = object
  method foo a, b = ...
end;;

let script_b =
  method foo x, y = ... do something else (does not hide script_a::foo)
end;;

This would solve the encapsulation problem, but what about concurrent
calls to ocaml from C ? Is this possible ?

Gaspard

On Fri, Jul 9, 2010 at 9:06 AM, David MENTRE <dmentre@linux-france.org> wrote:
>
> Hello,
>
> 2010/7/8 Gaspard Bucher <gaspard@teti.ch>:
> > 1. Is it possible to embed an ocaml compiler ? (something that eats
> > std::string and produces a callable thing)
>
> No, not the compiler. It is however possible to embed an interpreter
> and to load byte code (and native code?) on demand:
>  http://caml.inria.fr/pub/docs/manual-ocaml/manual032.html#toc134
>
> > 2. How would an interface between C++ and a compiled script work ? (passing
> > values, getting return values)
>
> Through a C interface. See above link.
>
> Sincerely yours,
> david

Re: [Caml-list] ocaml for embedding

[David MENTRE]

Hello,

2010/7/9 Gaspard Bucher <gaspard@teti.ch>:
> From my understanding of the use of
> caml_startup (or caml_main), this means that the caml runtime is
> global.

Yes.

> Is there a way to avoid:
>
> 1. global locking (or locking only during script recompilation)

I don't think so.

> 2. script level encapsulation

I don't know.


You might play with the C symbols and a bit a C pre-processing to
generate several different OCaml runtimes that would be linked with
you application. But, as far as I know, the OCaml runtime has not be
designed to be included several times within the same application.

Sincerely yours,
david

Re: [Caml-list] ocaml for embedding

[Gaspard Bucher]

Thanks for the insight David.

I also found this email from Xavier Leroy
(http://caml.inria.fr/pub/ml-archives/caml-list/2002/11/64c14acb90cb14bedb2cacb73338fb15.en.html).
I will start with a single context (eventually built from several
scripts). If someone ever needs distributed computing, it's easy to
run several instances of Rubyk passing messages.

Just a final question on the topic: has JoCaml anything to do with
this concurrency (shared memory) question ?

Gaspard

On Mon, Jul 12, 2010 at 11:14 AM, David MENTRE <dmentre@linux-france.org> wrote:
> Hello,
>
> 2010/7/9 Gaspard Bucher <gaspard@teti.ch>:
>> From my understanding of the use of
>> caml_startup (or caml_main), this means that the caml runtime is
>> global.
>
> Yes.
>
>> Is there a way to avoid:
>>
>> 1. global locking (or locking only during script recompilation)
>
> I don't think so.
>
>> 2. script level encapsulation
>
> I don't know.
>
>
> You might play with the C symbols and a bit a C pre-processing to
> generate several different OCaml runtimes that would be linked with
> you application. But, as far as I know, the OCaml runtime has not be
> designed to be included several times within the same application.
>
> Sincerely yours,
> david
>

Re: [Caml-list] ocaml for embedding

[Eray Ozkural]

[-- Attachment #1: Type: text/plain, Size: 930 bytes --]

On Mon, Jul 12, 2010 at 4:19 PM, Gaspard Bucher <gaspard@teti.ch> wrote:

>  Just a final question on the topic: has JoCaml anything to do with
> this concurrency (shared memory) question ?


It seems to me that it can be implemented for a shared memory system. Why
not? But it's been designed with distributed memory in mind. So, I suppose
writing to shared memory could be slow. It'd be sort of like implementing
synchronization on top of a message passing system. Although I can see that
reading is OK. I would still like a nice and sound source-level support for
shared memory, if possible :) Do not all of us need such a feature? My cores
are idle. That's a pity. Yes, I did use ocamlmpi to get some speedup on
shared memory, but it's not the same thing. I shouldn't have to go back to
C++ just for implementing a shared memory algorithm.

Best,
-- 
Eray Ozkural, PhD candidate.  Comp. Sci. Dept., Bilkent University, Ankara

[-- Attachment #2: Type: text/html, Size: 1259 bytes --]

Re: [Caml-list] ocaml for embedding

[Eray Ozkural]

[-- Attachment #1: Type: text/plain, Size: 2110 bytes --]

On Tue, Jul 20, 2010 at 3:36 AM, Eray Ozkural <examachine@gmail.com> wrote:

> On Mon, Jul 12, 2010 at 4:19 PM, Gaspard Bucher <gaspard@teti.ch> wrote:
>
>>  Just a final question on the topic: has JoCaml anything to do with
>> this concurrency (shared memory) question ?
>
>
> It seems to me that it can be implemented for a shared memory system. Why
> not? But it's been designed with distributed memory in mind. So, I suppose
> writing to shared memory could be slow. It'd be sort of like implementing
> synchronization on top of a message passing system. Although I can see that
> reading is OK.
>

On second thought it's not so clear how reads could be efficient. I'd
assumed that somehow messages could be sent without making a separate copy
of the data, so in principle we could perform the reads just as fast.
Interesting question, though, thanks Gaspard. Now I realized it's not so
clear and I apologize for that mistake, because that would be only valid for
immutable data structures. And I suppose with immutable data structures you
get only part of the advantages of shared memory, right? What good is a
parallel RAM if it doesn't support concurrent writes?

So, I think, no matter what kind of abstract concurrency primitives you
provide, you'd still need proper shared memory + threads for the low-level
programmer. Or as I said, you can also get speedup using MPI, but usually
that's not the best speedup you are going to get because a) you can't avoid
copying huge messages across processes b) mpi implementation may have too
much penalty for finer-grain code, i.e. latency will become significant. I
think MPI or these high-level concurrent processes would be good on shared
memory if the problem is embarrasingly parallel. But as we all know only a
small minority of problems turn out to be that easy to solve in parallel!

At any rate, I'm sure we'll have it one day, I'm keeping my fingers crossed
:)

Best,

-- 
Eray Ozkural, PhD candidate.  Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct

[-- Attachment #2: Type: text/html, Size: 2943 bytes --]