Announcement: ocaml-based magnetism simulation package

Announcement: ocaml-based magnetism simulation package

[Thomas Fischbacher]

I thought I should mention this to keep people informed about
large projects done in OCaml.

Micromagnetism is a rather hot topic these days. There are
a number of packages around that allow one to do physical
simulations of sub-micron scale magnetic systems, but there
is one around (ours) which is written mostly in ocaml.
(The user interface is python, though.)

The rationale for using ocaml was that we needed both speed
and enough flexibility to do symbolic manipulations.
Actually, this micromagnetism package is just a library on
top of a much more generic multiphysics simulation system
that was designed to deal with a very broad class of
generalized reaction-diffusion systems, whose ultimate goal
is to automatically compile an abstract problem specification
(containing differential operators and equations of motion
in symbolic form) down to some numerical finite element code.

The first beta-release (still a bit wobbly) is at:

http://nmag.soton.ac.uk

-- 
best regards,
Thomas Fischbacher
t.fischbacher@soton.ac.uk

Re: [Caml-list] Announcement: ocaml-based magnetism simulation package

[Jon Harrop]

On Wednesday 21 November 2007 19:06, Thomas Fischbacher wrote:
> http://nmag.soton.ac.uk

Wow! That looks awesome. :-)

Are all of the graphics done from Python and none from OCaml? I'm interested 
in writing some visualization libraries for OCaml as it seem to be lacking in 
this area...

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e

Re: [Caml-list] Announcement: ocaml-based magnetism simulation package

[Thomas Fischbacher]

Jon Harrop wrote:

>>http://nmag.soton.ac.uk
> 
> Wow! That looks awesome. :-)
> 
> Are all of the graphics done from Python and none from OCaml? I'm interested 
> in writing some visualization libraries for OCaml as it seem to be lacking in 
> this area...

Graphics-related issues are user interface stuff and as such will always
be handled at the python level (= where all the interpretation of
physical data happens).

(I do, however, occasionally think about adding some debugging feature
that allows me to create an image from a sparse matrix showing where
the nonzero entries end up. But this would be only for debugging.)

-- 
best regards,
Thomas Fischbacher
t.fischbacher@soton.ac.uk

Re: [Caml-list] Announcement: ocaml-based magnetism simulation package

[Andrej Bauer]

Thomas Fischbacher wrote:
> (The user interface is python, though.)

I have suspected for a while that writing the computational part in
ocaml and the user interface in python is a good combination. What is
your experience? Are there any drawbacks? Would you do it again?

Andrej

Re: [Caml-list] caml and python (was: Announcement: ocaml-based magnetism simulation package)

[Thomas Fischbacher]

Andrej Bauer wrote:

>>(The user interface is python, though.) 
> 
> I have suspected for a while that writing the computational part in
> ocaml and the user interface in python is a good combination. What is
> your experience? Are there any drawbacks? Would you do it again?

We use a modified variant of Art (arty) Yerkes' old "pycaml" module.

Essentially, I had to re-write quite a bit of it, as both reference
counting on the Python side and Gc management on the OCaml side were
broken. One impotant issue is that the pycaml module (which also is
part of Debian) does not recognize properly that python library
primitives often differ in their behaviour concerning whether they
increase the reference count of a return value or not.

Also, we added a lot of extra stuff and goodies which allows us to
interface ML to python quite smoothly. When I call a ML-function from
python, dynamic type-checking of all the arguments is automatic and
generates appropriate python exceptions. Here is an example for a
python-wrapped ML function that uses our extended pycaml module:

let _py_raw_cvode_advance =
   python_pre_interfaced_function
     [|CamlpillType; (* the cvode *)
       CamlpillType; (* the result vector *)
       FloatType;    (* target time *)
       IntType;      (* max nr_steps, -1 for "no limit" *)
     |]
     (fun args ->
        let cvode = cvode_from_ocamlpill args.(0) in
        let FEM_field (_,_,v_result) = field_from_ocamlpill args.(1) in
        let target_time = pyfloat_asdouble args.(2) in
        let max_steps = pyint_asint args.(3) in
        let result_time =
	 Sundials.cvode_advance cvode target_time v_result max_steps
	 in
	 pyfloat_fromdouble result_time)
;;

It is nice to be able to implement other high-level python/caml
interface abstractions through functional tricks -- such as a caml
function constructing and accessing a higher-rank nested-list python
tensor.

How does it feel? If I had a need for some specific fast low-level
extension to python, I would prefer using python+ocaml a thousand
times over python+C. It is just so much nicer not having to worry
about reference counting issues. Even if you feel you can do it
right, there is always a bit of doubt. Doubly so if you are not
the sole developer. I would not let our PhD students write C
extensions to python, but I have no problems whatsoever letting
them extend python via ocaml functions.

One sort-of limitation is that we use our own python interpreter.
(I.e. the main program is written in OCaml and eventually provides
a python prompt or runs a python script.)

This is necessary in our situation as we also want to support running
in parallel under MPI control, and the MPI libraries do some nasty
arg-passing things that upset an unmodified python interpreter. So, we
do not load some object-code module containing all the ML runtime
environment system into a standard python interpreter; technically, that
should be possible as well on many platforms, but it is not relevant in
our particular case here.


On the other hand, of course, if one can completely avoid such
inter-language interfaces, the better. In principle, both Python
and OCaml are widely loved because of flexibility they both derive
from LISP. So, some situations, one may be better off using e.g. the
Gambit Scheme Compiler instead (which has, by the way, a terrific
C interface!)

It depends a lot on your constraints. If you want to build something
that can be easily integrated with lots of data processing libraries
that are readily available and accessible to casual programmers
(engineers, physicists, etc.), providing python support seems to be a
very good choice to me. It is rather popular among engineers these
days, which I consider mostly a good thing.

Also, if you want your PhD students to be productive at writing code
that has to run fast, and if they are spoilt enough from previous
exposure to imperative languages that you cannot really teach them
Lisp/Scheme within a short amount of time ;-), then ocaml certainly
is a very good choice. (In terms of what we teach our students about
caml, this is not much more than the core ML language plus a bit
about the C interface. They usually pick that up in less than 3 months.
With C++, they never could be productive after such a short amount
of time.)

So, I would say: in an ideal computing world, we would all be expert
Scheme hackers, but in the real world, python+ocaml can be an extremely
productive and powerful combination. I have used and combined many
systems in the past, but I would have difficulty naming anything
else I played with that appeared even remotely as powerful.

I know that in the longer run, we certainly should split nmag and
ship our new and improved pycaml module separately (after brushing
it up a bit) -- and a few other modules as well. But for now,
we are busy dealing with other issues. (We are, after all,
physicists studying real physical systems, so all this symbolic
mumbo-jumbo in nmag/nsim is just a tool to us.)

-- 
best regards,
Thomas Fischbacher
tf@functionality.de

Re: [Caml-list] caml and python

[Andrej Bauer]

Dear Thomas,

thank you for your reply. I did not realize you actually married python 
and ocaml so closely.

My inquiry was actually geared towards getting some sort of 
Mathematica-like front-end environment that could link to a 
computational "core" (or cores) running as separate processes. Are there 
any pooor man's frontends out there? I am aware of Texmacs and Sage. 
Texmacs strikes me a bit idiosyncratic, while Sage scares me in sheer 
size. I just want a nice little general-purpose frontend that can do an 
interactive loop and show pictures and math formulas as results.

Andrej

Re: [Caml-list] caml and python

[Thomas Fischbacher]

Andrej,

> thank you for your reply. I did not realize you actually married python 
> and ocaml so closely.

Well, we sort-of had to.

> My inquiry was actually geared towards getting some sort of 
> Mathematica-like front-end environment that could link to a 
> computational "core" (or cores) running as separate processes. Are there 
> any pooor man's frontends out there? I am aware of Texmacs and Sage. 
> Texmacs strikes me a bit idiosyncratic, while Sage scares me in sheer 
> size. I just want a nice little general-purpose frontend that can do an 
> interactive loop and show pictures and math formulas as results.

I have played around with Texmacs myself a few years ago, but considered
it way too weak for heavy-duty applications. I think that if visualizing
maths in a front-end is your main concern, you may be better off hooking
up the computational core to a webserver and use a web-browser as your
front-end. Then, you can think about MathML rendering and a JavaScript
or even Java-based user interface. (Concerning Java, it is rather
convenient to write Java applets in Per Bothner's Kawa Scheme rather
than in Java.) This can be a very convenient approach provided you have
full continuation support in your computational core.

Here is a prototype for a webbrowser-as-maths-user-interface I wrote
a few years ago (taking mzscheme as a basis, as this provides call/cc):

http://141.84.136.30:8000/

The preprint thaat describes the idea is here:

http://de.arxiv.org/abs/cs/0406002

The original motivation for a pattern-language based term manipulation
tool came from supergravity, where you have to do a lot of nasty
("fierzing") tensor and spinor algebra. However, I never found the
time to evolve this idea into a really useful tool for that particular
job. A more reasonable first goal for a demo application would be the
formalism of partial derivatives used in thermodynamics.

-- 
best regards,
Thomas Fischbacher
tf@functionality.de

Re: [Caml-list] caml and python

[Jon Harrop]

On Thursday 22 November 2007 15:21, Andrej Bauer wrote:
> My inquiry was actually geared towards getting some sort of
> Mathematica-like front-end environment that could link to a
> computational "core" (or cores) running as separate processes. Are there
> any pooor man's frontends out there? I am aware of Texmacs and Sage.
> Texmacs strikes me a bit idiosyncratic, while Sage scares me in sheer
> size. I just want a nice little general-purpose frontend that can do an
> interactive loop and show pictures and math formulas as results.

We have a complete typesetting engine using TeX fonts with Mathematica's 
layout and hardware-accelerated rendering using OpenGL via Smoke. This is not 
yet in the free bytecode edition of Smoke but if anyone is interested I'll 
gladly ship it.

Smoke includes picking (selection of objects with the mouse) so you can easily 
write a GUI front-end with typesetting using it (we already did this for 
Presenta). Smoke is also cross platform, so the result would run out of the 
box on Linux, Mac OS X and Windows. We also now have OCaml compilation 
environments on all of these platforms, so I'll get binaries for everything 
up on our site ASAP.

I am interested in the idea of writing a cross platform typeset IDE building 
upon OCaml using exactly this technology. We should collaborate!

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/products/?e

Re: [Caml-list] caml and python (was: Announcement: ocaml-based magnetism simulation package)

[Stefano Zacchiroli]

On Wed, Nov 21, 2007 at 11:33:54PM +0000, Thomas Fischbacher wrote:
> We use a modified variant of Art (arty) Yerkes' old "pycaml" module.
>
> Essentially, I had to re-write quite a bit of it, as both reference
> counting on the Python side and Gc management on the OCaml side were
> broken. One impotant issue is that the pycaml module (which also is
> part of Debian) does not recognize properly that python library
> primitives often differ in their behaviour concerning whether they
> increase the reference count of a return value or not.

Are you going to release a modified (and fixed!) version of PyCaml as a
standalone library? Given that you found several issues in PyCaml I bet
the whole community would be very happy to see something like that! If
you claim that the original maintainer of PyCaml is no longer active
maybe you can take over maintenance or something like that, how about
it?

Cheers.

-- 
Stefano Zacchiroli -*- PhD in Computer Science ............... now what?
zack@{cs.unibo.it,debian.org,bononia.it} -%- http://www.bononia.it/zack/
(15:56:48)  Zack: e la demo dema ?    /\    All one has to do is hit the
(15:57:15)  Bac: no, la demo scema    \/    right keys at the right time

Re: [Caml-list] caml and python

[Thomas Fischbacher]

Stefano Zacchiroli wrote:

>>We use a modified variant of Art (arty) Yerkes' old "pycaml" module.
> 
> Are you going to release a modified (and fixed!) version of PyCaml as a
> standalone library? Given that you found several issues in PyCaml I bet
> the whole community would be very happy to see something like that! If
> you claim that the original maintainer of PyCaml is no longer active
> maybe you can take over maintenance or something like that, how about
> it?

As I said, we strongly plan to do so, but at present, manpower is our
main limiting factor. Our modifications have to be streamlined and
documented properly. As it is, you can just download the nsim
sources, unpack it, go to the pycaml directory, type "make install"
(well, actually, you should run the configuration script first)
and should end up with our pycaml module installed under ocamlfind
control. It is sufficiently independent to be used as a separate module.

-- 
best regards,
Thomas Fischbacher
tf@functionality.de