Re: Portability of applications written in OCAML

[Gerd Stolpmann <Gerd.Stolpmann@darmstadt.netsurf.de>]

On Tue, 15 Feb 2000, Claude Marche wrote:

>One idea could be distributing bytecode: is it true the any bytecode
>can be executed on any architecture and OS as soon as the right
>ocamlrun is used? (Even under non unix OSs like Microsoft-Windows or
>Mac-OS?) 

As far as I know: Yes. I did it already, and it worked even for systems
with different endianess. But I did it only for Unixes.

>If yes, anyone interested in using the application would
>need only to install the suitable ocamlrun for its architecture/OS
>combination.  It would allow any potentially interested user to try
>the application, and in case he finds it suitable for its need, he
>would be motivated to install ocaml for compiling from the sources and
>getting a faster application.
>
>For this to work, a kind of archive of various versions of ocamlrun
>should exist somewhere on the web, allowing to choose the right
>architecture/OS, also the right ocaml version, the right libraries
>like libc 5 or 6 or glibc.

Good idea.

>Nevertheless, there is a problem for applications that use other
>libraries like unix, str, nums, threads, that need to be compiled in
>-custom mode. If I understand well, binaries compiled with -custom
>contain the native code of the interpreter itself, so are not portable
>anymore. Techically speaking, is it possible to build an ocamlrun
>interpreter that contains those libraries statically linked?

This is possible by using the -make-runtime option.

But: This is not the best solution, as the number of libraries is growing,
and every time a new library (even a new version) would be needed by an
application, also new bytecode interpreters for all platforms would be
necessary. With two consequences:

1) The interpreters become very large, and there will be certainly people
   saying that OCaml is a big fat all-memory-eating language

2) The interpreters for platforms which are not everybody's darling will
   rapidly become too old to execute many applications.

My suggestion: Every modern operating system can link libraries dynamically.
This is also possible for the parts of the OCaml libraries written in C
which need to be available in the runtime system. The runtime system consists
of three parts:

1) The bytecode interpreter
2) The additional C-libraries
3) A startup routine which initializes the interpreter and configures the
   additional primitives available by 2)

There are no problems with parts 1 and 2 and dynamic libraries. Unfortunately,
part 3 is a generated C function, i.e. a different version is needed for every
combination of C-libraries. If we want dynamic libraries part 3 must be
replaced by a configurable dynamic loader. This is not too difficult, I think
I can write one (at least for Unix); the task of the dynamic loader is 
reading the list of necessary primitives from the bytecode file, loading the
associated libraries, and setting up the table that maps the primitives to
object-code symbols.

If this is done, we can link part 1 and the new loader that replaces part 3 to
the new, generic main executable which is able to run EVERY application for
which the necessary dynamic libraries are available.

So:

- For every platform, we need only one generic bytecode interpreter and one
  set of shared libraries

- Only libraries needed for the application are loaded into memory; the memory
  footprints become much smaller

- If new libraries are available, it is possible to compile them independently
  of the other libraries

- Multiple versions of the same library can easily be supported.

- It is often not necessary to recompile the libraries if a new version
  of the OCaml interpreter is available (at least if symbol-level 
  compatibility is given; the C FFI did not change often in the past).

>If this is technically feasible, I would like to know people who are
>interested in using this for distribution of their own application in
>bytecode form. Moreover, for building this archive of ocamlrun
>binaries, many people are needed to cover a wide range of
>architecture/OS combinations. I have access myself to PCs under Linux
>(Redhat distribution) and SPARC/Solaris 2.7, that's all.

Interested. I have some Web applications, and some TK applications.

I can offer 
 - Linux with libc5
 - Linux with glibc6.0
 - Linux with glibc6.1
   (Yes, I had problems to run OCaml applications compiled for glibc6.0 under
    glibc6.1!)
 - FreeBSD 3.3
 - Solaris 2.6
 - AIX 4.2

Gerd
-- 
----------------------------------------------------------------------------
Gerd Stolpmann      Telefon: +49 6151 997705 (privat)
Viktoriastr. 100             
64293 Darmstadt     EMail:   Gerd.Stolpmann@darmstadt.netsurf.de (privat)
Germany                     
----------------------------------------------------------------------------