embedding ocaml into a windows app: need gcc?

embedding ocaml into a windows app: need gcc?

[Joel Reymont]

Can I do without gcc if I want to embed the OCaml compiler into a  
commercial Windows app?

Do I need to become part of the OCaml consortium to do this?

Ideally, I would like to generate OCaml code at runtime and compile it  
into something that can be loaded by a runtime of some sort.

Compiling into a DLL would be ideal, is it possible?

Are there are other options?

	Thanks in advance, Joel

--
wagerlabs.com

Re: [Caml-list] embedding ocaml into a windows app: need gcc?

[Richard Jones]

On Sat, Sep 27, 2008 at 08:27:18PM +0100, Joel Reymont wrote:
> Can I do without gcc if I want to embed the OCaml compiler into a  
> commercial Windows app?

The OCaml compiler uses gcc in various stages (linking IIRC).  Also
needs an assembler.

> Do I need to become part of the OCaml consortium to do this?

You certainly need to read the license very carefully.

However is it necessary to embed the compiler?  Can you not just call
out to the compiler as a separate process?  It's surely easier to do
Sys.command, rather than working out the licensing and technical
issues involved in linking your program to the compiler.

> Ideally, I would like to generate OCaml code at runtime and compile it  
> into something that can be loaded by a runtime of some sort.
> 
> Compiling into a DLL would be ideal, is it possible?

ocamlopt -fPIC ... should be able to do it.

Or compile the code with ocamlc and use Dynlink.

Or wait for OCaml 3.11 which can use Dynlink on native code.

Rich.

-- 
Richard Jones
Red Hat

Re: [Caml-list] embedding ocaml into a windows app: need gcc?

[David Teller]

I don't know if this answers your question, but OCaml 3 now has Dynlink,
i.e. a manner of dynamically loading OCaml modules from OCaml. So if you
manage to get your code compiled at run-time, it shouldn't be too hard
to load it.

Cheers,
 David

On Sat, 2008-09-27 at 20:27 +0100, Joel Reymont wrote:
> Ideally, I would like to generate OCaml code at runtime and compile it  
> into something that can be loaded by a runtime of some sort.
> 
> Compiling into a DLL would be ideal, is it possible?
> 
> Are there are other options?

-- 
David Teller-Rajchenbach
 Security of Distributed Systems
  http://www.univ-orleans.fr/lifo/Members/David.Teller
 Angry researcher: French Universities need reforms, but the LRU act
brings liquidations. 

Re: [Caml-list] embedding ocaml into a windows app: need gcc?

[Mikkel Fahnøe Jørgensen]

2008/9/27 Joel Reymont <joelr1@gmail.com>:
> Can I do without gcc if I want to embed the OCaml compiler into a commercial
> Windows app?

I assume you mean native compilation.

As mentioned, you should consider using a standalone ocaml compiler to
avoid licensing issues, but either way:

The ocamlopt compiler generates assembly source code and doesn't need
a C compiler.
It needs a runtime library, a linker and an assembler.

On Windows you need to install masm - or find a compatible
alternative. (On Unix I think gcc works as assembler).

Masm can be downloaded without too much trouble - but if you start
deploying it to other users, you may have a problem. Likewise, you
must consider if your users have a linker installed and how you can
distribute it legally.

So assuming you have an assembler, ocaml compiler, linker and ocaml libraries:

If you want to attach the generated code to a live process, you need
to create a dll or a new process. If you want to attach to existing
code then start a process, you can use static linking.

So lets assume you want to create a dll.

On Windows you don't need to compile position independent code, last
time I looked, so -fPIC is not relevant.

You compile an ocaml library and link it with other code as a dll.
With the next ocaml version you might even be able to create a dll and
use it not from OCaml but from any other program. Then the problem is
solved - except for installing all the compile tools.

Without the above solution, I have created such dll's some years back.
More specifically to embed an ocamlyacc compiler in other Windows
programs.

You need to get the compile flags right, but it definitely is possible.

You can also create a COM dll if you want - and call it from .Net.
The benefit of a COM dll is that you get a clear calling interface and
can interface with .NET easily. The downside is that COM projects are
ugly.

If you create a simple dll, you need a way to manage your call backs.
I have an old ocamldll library for this purpose. It lets you register
multiple ocaml function names in a dll and call them from the
application by name.

The new windows dll solution scheduled for next OCaml version has a
more elegant solution where you don't have to register callbacks:

http://alain.frisch.fr/natdynlink.html

You can probably use this even with current OCaml version since it is
Windows specific, not OCaml specific.

Let me know if I should publish my ocamldll thing which will work, but
is not cutting edge.

All that being said:

I have considered the same thing, and it is messy because of the compiler tools.

I would like an OCaml support library that can compile and execute
similar to JaveScript engines, but we don't have that in any practical
form.

So I would recommend looking into embedding a javascript compiler
instead if performance is not absolutely critical - you could still
drive fast ocaml library routines.

In the past few months performance has exploded both in WebKit and
Mozilla. Google has also released a fast engine named V8. WebKit has
LGPL licensing issues, but Mozilla and V8 should be very embeddable.
All compilers have JIT compilation, Mozilla also 64-bit.

Or look at Haxe or Lua.

Regards,
Mikkel

'Compiler' module (was: embedding ocaml into a windows app: need gcc?)

[Dawid Toton]

> I would like an OCaml support library that can compile and execute
> similar to JaveScript engines, but we don't have that in any practical
> form.
> 

I also need this and I'm thinking about something like this:

module type Compiler =
sig
  val parse : Context.t -> string -> (Context.t * AST.t)
  val get_type : Context.t -> AST.t -> Type.t;
  val eval : Context.t -> AST.t -> Context.t * (Type.t * 
MarshalledValueOrSomething.t)
end

Is it really so hard to have it in OCaml? I'm envy of Python's Compiler 
module.

It could solve some of my problems, in particular for my 'parallel-like' 
evaluation ( 
http://www.nabble.com/'Nondeterministic'-evaluation-wrt-exceptions-td18653998.html 
). After some experimentation I know that I need type information during 
code transformation. I want to create a sort of interpreter.

That Compiler module would also enable me to have an editor with 
graphical representation of results intermixed with code (like 
Mathematica's notebook).

Where to look for suitable pieces of code? AFAIK camlp4 modules can't 
tell me type of anything. Should I start digging in OCaml compiler sources?

Dawid

Re: [Caml-list] 'Compiler' module (was: embedding ocaml into a windows app: need gcc?)

[Lukasz Stafiniak]

On Fri, Oct 3, 2008 at 11:34 PM, Dawid Toton <d0@wp.pl> wrote:
>
>> I would like an OCaml support library that can compile and execute
>> similar to JaveScript engines, but we don't have that in any practical
>> form.
>>
>
> I also need this and I'm thinking about something like this:
>
> module type Compiler =
> sig
>  val parse : Context.t -> string -> (Context.t * AST.t)
>  val get_type : Context.t -> AST.t -> Type.t;
>  val eval : Context.t -> AST.t -> Context.t * (Type.t *
> MarshalledValueOrSomething.t)
> end
>
> Is it really so hard to have it in OCaml? I'm envy of Python's Compiler
> module.
>
> It could solve some of my problems, in particular for my 'parallel-like'
> evaluation (
> http://www.nabble.com/'Nondeterministic'-evaluation-wrt-exceptions-td18653998.html
> ). After some experimentation I know that I need type information during
> code transformation. I want to create a sort of interpreter.
>
> That Compiler module would also enable me to have an editor with graphical
> representation of results intermixed with code (like Mathematica's
> notebook).
>
> Where to look for suitable pieces of code? AFAIK camlp4 modules can't tell
> me type of anything. Should I start digging in OCaml compiler sources?
>
You will be better off starting digging in MetaOCaml compiler sources :-)

Re: [Caml-list] 'Compiler' module (was: embedding ocaml into a windows app: need gcc?)

[David Teller]

Have you looked at the CDK's DynEval?

Other than that, it's the kind of thing we'd like to put in Batteries,
so if you write/find a better solution, please make it public.

Cheers,
 David

On Fri, 2008-10-03 at 22:34 +0100, Dawid Toton wrote:
> > I would like an OCaml support library that can compile and execute
> > similar to JaveScript engines, but we don't have that in any practical
> > form.
> > 
> 
> I also need this and I'm thinking about something like this:
> 
> module type Compiler =
> sig
>   val parse : Context.t -> string -> (Context.t * AST.t)
>   val get_type : Context.t -> AST.t -> Type.t;
>   val eval : Context.t -> AST.t -> Context.t * (Type.t * 
> MarshalledValueOrSomething.t)
> end
> 
> Is it really so hard to have it in OCaml? I'm envy of Python's Compiler 
> module.

-- 
David Teller-Rajchenbach
 Security of Distributed Systems
  http://www.univ-orleans.fr/lifo/Members/David.Teller
 Angry researcher: French Universities need reforms, but the LRU act
brings liquidations. 

Re: [Caml-list] 'Compiler' module (was: embedding ocaml into a windows app: need gcc?)

[Jon Harrop]

On Friday 03 October 2008 22:34:15 Dawid Toton wrote:
> > I would like an OCaml support library that can compile and execute
> > similar to JaveScript engines, but we don't have that in any practical
> > form.
>
> I also need this and I'm thinking about something like this:
>
> module type Compiler =
> sig
>   val parse : Context.t -> string -> (Context.t * AST.t)
>   val get_type : Context.t -> AST.t -> Type.t;
>   val eval : Context.t -> AST.t -> Context.t * (Type.t *
> MarshalledValueOrSomething.t)
> end
>
> Is it really so hard to have it in OCaml?

The OCaml compiler's current design makes it extremely difficult but it is 
certainly entirely feasible (F# already does this, for example).

> I'm envy of Python's Compiler module.

Given that the sole objective of run-time compilation is performance and 
Python is vastly slower than OCaml, why are you the envy of Python's Compiler 
(to interpreted bytecode) module?

> It could solve some of my problems, in particular for my 'parallel-like'
> evaluation (
> http://www.nabble.com/'Nondeterministic'-evaluation-wrt-exceptions-td186539
>98.html ). After some experimentation I know that I need type information
> during code transformation. I want to create a sort of interpreter.

I think (as Zheng implied) you want asynchronous workflows from F#. Note that 
these also handle cleanup of resources.

> That Compiler module would also enable me to have an editor with
> graphical representation of results intermixed with code (like
> Mathematica's notebook).

For a decent editor, you will need a lot more than that. I would certainly 
love to have such a thing but you also need access to the type checking phase 
to throwback type errors and having the data available in an efficient and 
accessible form would certainly be preferable.

I believe you can lex and parse OCaml using Camlp4 and then invoke the OCaml 
compiler with -dtypes to get the results of type checking. However, the 
latter is extremely inefficient. You really want the ability to restart the 
compiler from the end of the previous definition each time a file is edited.

> Where to look for suitable pieces of code? AFAIK camlp4 modules can't
> tell me type of anything. Should I start digging in OCaml compiler sources?

I'd start by reviewing asynchronous workflows.

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Re: [Caml-list] 'Compiler' module

[Dawid Toton]

> 
> Given that the sole objective of run-time compilation is performance and 
> Python is vastly slower than OCaml, why are you the envy of Python's Compiler 
> (to interpreted bytecode) module?
> 

 From my point of view this is more about manipulating code at runtime
and evaluating it (actually I'm not interested about performance).


> I think (as Zheng implied) you want asynchronous workflows from F#. Note that 
> these also handle cleanup of resources.

 From what is currently available on web I see that it's equivalent to
my current stop-gap solution using monads:
(async {blah blah}) in F# can be translated to (lazy (blah blah)) in
OCaml and Async.Run just uses some threads to do Lazy.force. Am I
missing some difference?

> 
> For a decent editor, you will need a lot more than that. I would certainly 
> love to have such a thing but you also need access to the type checking phase 
> to throwback type errors and having the data available in an efficient and 
> accessible form would certainly be preferable.
> 

I see no reason for hypothetical Compiler.eval function couldn't give me
full information about all errors.

As for availability of resulting data: this is not obvious at the
conceptual level.

One possibile solution is to decide that the host invoking Compiler.eval
has to treat resulting values as of abstract type. They might be subject
to marshalling and so on.

The other way is to have a module for manipulating data which type is
known at runtime. Then Compiler.eval would return value of type
TypedData.t = (Type.t, TypedData.abstract_type)

The second approach would make the whole Compiler-module-thing more useful.

Anyway, this is somewhat another story. RTTI would allow us to have
clean implementation of marshalling. We could have (at no performance
cost) Type.typeof operator that acts only on values of concrete type
(if only camlp4 knew types...).

> I believe you can lex and parse OCaml using Camlp4 and then invoke the OCaml 
> compiler with -dtypes to get the results of type checking. However, the 
> latter is extremely inefficient. You really want the ability to restart the 
> compiler from the end of the previous definition each time a file is edited.

I hope the first dirty solution would be to use compiled modules as the
context carrier. In case of a simple editor I'd have many tiny modules
that correspond to compiled chunks of evolvong code.
It would scale not so bad: compiler would have to load (log n) modules
for n chunks processed. But I have no idea what would be the practical
performance.

Dawid