OCaml on CLR/JVM? (RE: OCaml <--> ODBC/SQL Server)

OCaml on CLR/JVM? (RE: OCaml <--> ODBC/SQL Server)

[Don Syme]

Now I have to say the obvious: wouldn't it be wonderful if Caml interfaced
with either Java or the .NET Common Language Runtime seemlessly so we
wouldn't have to keep facing these kinds of questions and problems, and
could just leverage existing libraries?   

I'm very interested to know if there are people with some time to spare who
would be keen to work with me toward a .NET version of OCaml.  I've talked
this over from time to time with Xavier, and have done a lot of foundational
work for the core language when building a .NET compiler for Haskell.  If
you think would be interested, or would simply like to join a mailing list
devoted to talking about getting Caml running and interoperating on .NET,
then please let me know!

Cheers,
Don Syme


------------------------------------------------------------------------
At the lab:                                     
Microsoft Research Cambridge                  
St George House                             
Cambridge, CB2 3NH, UK
Ph: +44 (0) 1223 744797                        
http://research.microsoft.com/users/dsyme
email: dsyme@microsoft.com
------------------------------------------------------------------------




-----Original Message-----
From: Vincent Leleu [mailto:me@vleleu.com]
Sent: 05 February 2001 08:52
To: caml-list@inria.fr
Subject: OCaml <--> ODBC/SQL Server


*** Version Francaise a la fin ***

Hello,

I'd like to use OCaml as a CGI scripting language that process forms AND
especially query a SQL Server DB (7.0 or 2000).
I came across a library to easily manage the CGI forms & query strings in
OCaml. This is no probs.

What I cannot find around is a way to easily interrogate and interface in
OCaml with an ODBC data source OR a direct way to send/receive queries
from/to the DB.

If anyone has any idea or even a little clue on where to go, any comment is
welcome.

Thanks a lot

*******

Bonjour,

J'aimerais utiliser OCaml comme language acteur de script CGI qui traiterait
les 'forms' ET surtout interrogerait une base de donnee SQL Server 7.0 ou
2000.
J'ai pu trouver une bibliotheque afin de gerer facilement les forms CGI et
ligne de parametres URL. Pas de probleme avec ca.

Ce que je ne trouve pas est une solution pour facilement interroger et
communiquer d'OCaml avec une source de donnees ODBC OU directement une facon
d'envoyer/recevoir les requetes/donnees vers/de la base de donnees.

Si vous avez la plus petite idee ou piste a me communiquer, tout commentaire
sera le bien venu.

Merci enormement


Vincent Leleu
---
AB Productions
105-A Devonshire Sq.
Jackson, TN 38305
US
---

Re: OCaml on CLR/JVM?

[Xavier Leroy]

> Now I have to say the obvious: wouldn't it be wonderful if Caml interfaced
> with either Java or the .NET Common Language Runtime seemlessly so we
> wouldn't have to keep facing these kinds of questions and problems, and
> could just leverage existing libraries?   

I've been working on and off (mostly off, lately) on an OCaml/Java
interface that works by coupling the two systems at the C level via
their foreign-function interfaces (Java's JNI and OCaml's C
interface).  This was strongly inspired by the work of Erik Meijer et
al on a similar Haskell/Java interface.  (These Haskell guys sure are
at the bleeding edge of language interoperability.  This is the second
interop idea I steal from them, after the IDL/COM binding.)

The low-level coupling is surprisingly easy, including making the two
garbage collectors cooperate: both the JNI and OCaml's C interface
provide enough functionality to get the couping to work without *any*
modification on either of the implementations.  How nice!
The only limitation is that a cross-heap cycle (a Java object pointing
to a Caml block pointing back to the Java object) can never be
reclaimed... (Thanks to Martin Odersky for pointing this out.)

(Actually, the main problem is working around the bugs in Sun's JDK
1.2.2 for Linux.  These guys must be kidding.  Does anyone has a
recommendation for a solid, complete Java implementation (including
Java 2 and of course the JNI) for Linux?)

Of course, the low-level interface is type-unsafe, so the real fun is
to build a type-safe view of Java classes and objects as Caml classes
and objects, and conversely.  I'm still struggling with some of the
issues involved.  For instance, it turns out to be much simpler (for
the implementation, not for the final user!) to map Java objects to
values of abstract Caml types, and treat methods as functions over
these abstract types, than mapping Java objects to Caml objects.  That
was quite unexpected!

One thing I learnt is that the real problem with language
interoperability is not how to compile language X to virtual machine Y
(this can always be done, albeit more or less efficiently), but rather
how to map between X's data structures and objects and those of all
other languages Z1 ... Zn that also compile down to Y.  This is obvious
in retrospect, but I think many (myself included) often overlook this
point and believe that compiling to the same virtual machine is
necessary and sufficient for interoperability.  It is actually neither
necessary nor sufficient...

While this work started with the JVM, I'm pretty sure it can be made
to work with the .NET CLR, as soon as it will have a foreign-function
interface with features comparable to those of the JNI.  (And I'm sure
this will happen eventually, not only because it makes sense, but also
because Java has it, so .NET must too :-)

Stay tuned for further developments.

- Xavier Leroy

Re: OCaml on CLR/JVM?

[David Mentre]

Xavier Leroy <Xavier.Leroy@inria.fr> writes:

[ about Caml and Java GC cooperation ]
> The only limitation is that a cross-heap cycle (a Java object pointing
> to a Caml block pointing back to the Java object) can never be
> reclaimed... (Thanks to Martin Odersky for pointing this out.)

Regarding GC cooperation, some work has been done on the MALI memory
system [Bekkers86-0:confs]. As Olivier Ridoux explained to me and as far
as I remember, they consider 4 kinds of pointer, some of them to let
garbage reclamation be done latter by the other GC. However, I don't
know if their scheme would solve your inter-GC cycle reclamation. Maybe
you should ask Olivier.Ridoux@irisa.fr directly.

I would be interested to know the solution you applied to this issue. 

>  it turns out to be much simpler (for the implementation, not for the
> final user!) to map Java objects to values of abstract Caml types, and
> treat methods as functions over these abstract types, than mapping
> Java objects to Caml objects.  That was quite unexpected!

Of pure curiosity, why is it so difficult to map Java to Caml objects?
Is it the way control flow evolves between object methods that is
different? Is the typing of OCaml constraining too much the kind of
programs that can be written compared to Java?


Best regards,
d.


@InProceedings{Bekkers86-0:confs,
  author =       "Y. Bekkers and B. Canet and O. Ridoux and L. Ungaro",
  title =        "{MALI}: {A} Memory with a Real-time Garbage Collector
                 for Implementing Logic Programming Languages",
  booktitle =    "Proceedings of the International Symposium on Logic
                 Programming",
  organization = "IEEE Computer Society,",
  year =         "1986",
  month =        sep,
  publisher =    "The Computer Society Press",
  pages =        "258--265",
  ISBN =         "0-8186-0728-9",
}

There is also a "Publication interne" #611, IRISA, 1991, by Olivier
Ridoux : ftp://ftp.irisa.fr/local/lande/or-tr-irisa611-91.ps.Z

-- 
 David.Mentre@inria.fr -- http://www.irisa.fr/prive/dmentre/
 Opinions expressed here are only mine.

[Caml-list] Re: OCaml on CLR/JVM?

[Xavier Leroy]

> [The MALI memory system.]  I would be interested to know the solution
> you applied to this issue.

Currently, none: the cycle is never deallocated.  But thanks for the
interesting references.

> Of pure curiosity, why is it so difficult to map Java to Caml objects?

There are a zillion sources of mismatches; some are easy to solve,
some not so.  For instance:

- Static overloading: Java has it, Caml doesn't.  So what do you do
  when your Java class has several definitions of method m,
  distinguished only by their argument types?  My current solution is to 
  have distinct methods on the Caml side, named m, m', m'', m'3, m'4, etc.
  (Thanks God, the quote is not valid in Java identifiers!)  Of course
  this sucks because the programmer has to refer to an automatically-generated
  documentation to find the Caml name of a Java method used at a particular
  type.

- Type equivalence and subtyping policy: Java is by name, Caml is by
  structure.  Actually not a big deal, since abstract data types in
  Caml can be used in creative ways to simulate Java's by-name behavior.

- "null" objects: any Java object reference can be "null"; Caml
  objects are never "null".  One can map Java object references to
  a Caml "option" type, but this is quite heavy-handed on the Caml
  client side.  My current solution is to map a Java "null" to special
  Caml objects that raise exceptions whenever one of their methods is called.

- Publically-accessible fields: in Java, fields are accessible from
  the outside of an object; in Caml, only the object's methods can see them.
  Moreover, the Caml object is distinct from the Java object: it just
  delegates its methods to the Java object.  Solution: map Java fields
  to get/set methods on the Caml side.  In the reverse direction,
  it's harder: basically, a Caml object can only implement a Java interface
  (only methods, no fields).

- Recursion between classes: in Java, classes can be arbitrarily recursive,
  even across packages.  And they are: for instance, java.lang.Object
  (the root of the Java class hierarchy) is actually mutually
  recursive with about 20 classes spanning 4-5 packages!  In Caml,
  only classes defined simultaneously in the same module can refer
  recursively to each other.  This is my current show-stopper.
  The initial approach was to map a Java package to one Caml module,
  allowing inter-package recursion, but not cross-package recursion.
  But that is not enough!  One actually needs to do a "connected
  components" analysis on the Java classes, map each component to
  a Caml module, then re-assemble the modules per-package in order
  to give a nice user view...

In summary: it's hard.  But stay tuned for further news.

- Xavier Leroy
-------------------
To unsubscribe, mail caml-list-request@inria.fr.  Archives: http://caml.inria.fr

RE: OCaml on CLR/JVM?

[Fabrice Le Fessant]

Is the .NET VM open source ? Which part is Microsoft-independent ?

After loosing progressively some parts of the OS market, is Microsoft
 trying to conquer the VM market ? Why should all software developers
 always depend on some Microsoft software ? Before it was the OS. But
 now, many languages are OS independent ... And now, they want to
 capture the market again, through the .NET VM ?

Ocaml is not a _hack_, as I have read in some recent mails, but a
_good_ independent language. It should not change to follow a
commercial standart, which will itself change for commercial
reasons, as soon as the market is captured ...

If Microsoft wants its new product to be used, it is Microsoft problem
to port more languages to its VM, and not only say: "We have ported
our homemade languages to it (C#, C++, VB.NET) [because it was
designed for them], so, you see, we have proved it's the universal
VM. Now, do the same for your languages, or your language will not be
used anymore by our customers..."

So, why do we really need a .NET port of OCaml ? OCaml is working fine on
Windows, and on many other OS ...

- Fabrice

Re: OCaml on CLR/JVM?

[Toby Watson]

> > One thing I learnt is that the real problem with language
> > interoperability is not how to compile language X to virtual machine Y
> > (this can always be done, albeit more or less efficiently), but rather
> > how to map between X's data structures and objects and those of all
> > other languages Z1 ... Zn that also compile down to Y.
>
> To look at it another way, OCaml already shares a platform with C (at
least
> with the native-code compiler), so all the C libraries are already
> available.
> Yet it can still be a lot of effort to link with a C library.  Why should
> Java and .NET be any easier?  Also, look at the effort that went into
making
> an ML/Java system with MLj.


As I understand it part of the .NET effort is to specify *some* common
higher-level to decrease types the impedence mismatch between languages. I
think this is called the common language specification,  and there is some
further common type system (probably defines a protocol for collections and
such) CTS.

Whether or not they are successful - and I know the project7 is still
carrying out research to improve type system compatibility - the Idea is to
have practical interoperability between languages. Basically this is to
improve upon the situation where you can make method calls, or even marshall
objects between langauges but still need to generate pages of IDL or custom
headers to have a decent interface. This goal is aided by retaining the type
signatures and other information in binary files.

Some general but useful non-MS background:

http://www.cs.mu.oz.au/research/mercury/information/dotnet/mercury_and_dotne
t.html

not so non-MS

http://www.dnjonline.com/articles/essentials/iss21_essentials.html

cheers,
toby

RE: OCaml on CLR/JVM?

[Don Syme]

> > 
> > > Now I have to say the obvious: wouldn't it be wonderful if Caml
interfaced
> > > with either Java or the .NET Common Language Runtime seamlessly so we
> > > wouldn't have to keep facing these kinds of questions and problems,
and
> > > could just leverage existing libraries?   
> 
> Although this view is understandable, I think it is rather naive.  

Well, I didn't exactly propose a technical solution...  

Of course there's hard work to be done to realise this vision, but in
principle a clean interop story sure beats the endless rehashing of other
people's code in language X as a library in language Y.  Myself and others
involved in the Microsoft Project 7 are working on one approach to achieve
this interop, i.e. compiling languages directly to .NET MS-IL, in the style
of MLj, often adding extensions to the language in order to improve the
interop.  We are also working on improving the .NET infrastructure,
proposing support for features such as parametric polymorphism in MS-IL.  

Xavier is also working on a solution for OCaml, as he mentioned, though the
problem of how to reflect the constructs of an object model into ML, Haskell
or OCaml remains similar whichever approach you take to actually running the
stuff.

> To look at it another way, OCaml already shares a platform with C (at
least
> with the native-code compiler), so all the C libraries are already
> available.
> Yet it can still be a lot of effort to link with a C library.  Why should 
> Java and .NET be any easier?  Also, look at the effort that went into
making
> an ML/Java system with MLj.

There are several reasons why it is easier: exceptions, for example, can be
propogated across the interop boundary, without any effort at all if you
compile to MS-IL of Java bytecode.  If you're compiling to bytecode you can
also ensure more compatibilities of representations, e.g. make sure ML
int64's are exactly representationally equivalent to C's int64s.  Note if
you don't compile to a bytecode then you even have to marshal integers
across the interop boundary in Caml, though this could be automated.

You can also transfer objects more consistently, as the semantics of the
object models of Java and .NET are fairly simple in contrast to C, e.g. no
need to have an IDL to help interpret pointers as "in-out", "in", "out"
parameters.

While at a certain level I like Xavier's approach, i.e. maintaining two
runtimes, garbage collectors etc., I have troubles seeing it scaling to the
multi-language component programming envisioned as part of .NET approach
(and indeed currently in practice with C#, C++, VB.NET and other .NET
langauges).  Two GC's are already trouble enough (performance might suck as
they will both be tuned to fill up the cache), but if you have components
from 10 languages in one process?  10 GCs competing for attention?  Maybe it
can be made to work, but there's a certain conceptual clarity in just
accepting that a GC should form part of the computing infrastructure, and
share that service.  These are the aspects of the .NET approach that I find
quite compelling.

As an aside, I think it would be an interesting question to say "OK, let's
take it for granted that the end purpose of our language is to produce
components whose interface is expressed in terms of the Java or .NET type
systems, but which retains as many of the features and conceptual simplicity
of OCaml and ML as possible."  I'm not sure exactly what you'd end up with,
but whatever it was it could be the language to take over from C# and/or
Java (if that's what you're interested in...)  But without really taking
Java/.NET component building seriously right from the start I feel you're
always just going to end up with a bit of a hack - an interesting, usable
hack perhaps, but not a really _good_ language.

Probably the greatest recurring technical problem that I see in this kind of
work is that of type inference, and the way both the Java and .NET models
rely on both subtyping and overloading to help make APIs palatable.  Type
inference just doesn't work well with either subtyping or oveloading.  This
is a great, great shame, as it's obviously one of the main things ML has to
offer to improve productivity.  

Cheers,
Don

P.S. As for threads - I don't think the story is half as bad as you might
think.  After all, OCaml threads map down to Windows threads at some point,
and I just don't see that there are that many special logical properties of
typical ML and Caml threading libraries that make it semantically ridiculous
to share threads between languages (though it is true asynchronous
exceptions can make things hard when compiling to a bytecode).  But I'll
admit I'm not an expert on this.

RE: OCaml on CLR/JVM?

[Dave Berry]

> From: Xavier Leroy [mailto:Xavier.Leroy@inria.fr]
> 
> > Now I have to say the obvious: wouldn't it be wonderful if Caml
interfaced
> > with either Java or the .NET Common Language Runtime seamlessly so we
> > wouldn't have to keep facing these kinds of questions and problems, and
> > could just leverage existing libraries?   

Although this view is understandable, I think it is rather naive.  As Xavier
said:

> One thing I learnt is that the real problem with language
> interoperability is not how to compile language X to virtual machine Y
> (this can always be done, albeit more or less efficiently), but rather
> how to map between X's data structures and objects and those of all
> other languages Z1 ... Zn that also compile down to Y.  

To look at it another way, OCaml already shares a platform with C (at least
with the native-code compiler), so all the C libraries are already
available.
Yet it can still be a lot of effort to link with a C library.  Why should 
Java and .NET be any easier?  Also, look at the effort that went into making
an ML/Java system with MLj.

(To be fair, I've never tried linking OCaml to C myself; I'm only judging
the
difficulty by traffic on this mailing list.  It seems to be simpler than
many
other ML compilers, but still with potential pitfalls). 

Threads are another area of potential problems.  In fact they can be a total
minefield.  

Xavier also wrote:
> These Haskell guys sure are
> at the bleeding edge of language interoperability. 

I don't entirely agree with this -- they may be at the leading edge of
publicly
documented implementations, but I know of commercial Lisp and Dylan
implementations
that are notably more sophisticated.  Unfortunately the techniques that
those
implementations use are proprietary, so I guess this doesn't have much
practical
impact for anyone else.

That said, I was pleased (if a little envious) when the Haskell people
successfully
published some papers about foreign-language interfaces.  I had rather
assumed that the
academic community wouldn't be interested in such mundane engineering
issues.
I did notice that some of the papers still used greek letters and
denotational
semantic-style equations to describe a fairly straightforward translation -
perhaps
these softened the blow to the readers?  ;-)  (I hope this doesn't sound
harsh; I 
intend it mainly in jest).