Re: [Caml-list] OCaml packaging problems

[Gerd Stolpmann <info@gerd-stolpmann.de>]

On 2002.05.14 10:54 Xavier Leroy wrote:
> Concerning this ld.conf issue, I disagree both with Sven Luther's solution
> (a tool that adds/removes lines from this file) and with Vitaly
> Lugovsky's suggestion (multiple configuration files in a directory).
> 
> The ld.conf mechanism was modeled after the /etc/ld.so.conf file used
> by the Unix dynamic loader.  It is intended to list a small number of
> standard directories that contain shared libraries, typically one
> directory for the "system" libraries (i.e. those provided by the OCaml
> core distribution), one for local extensions (e.g. /usr/local/lib),
> and perhaps one or two for especially large packages with many libraries
> (e.g. /usr/X11R6/lib).  

Traditionally, Unixes try to minimize the directories that contain
code, i.e. executables and (later) shared libraries are stored in only
a few directories. Today people tend to justify this way of managing 
code without realizing that it is the consequence of a design 
decision of the operating system, and not naturally the best choice.

The point is that important parts of the program loader run in userland,
and not inside the kernel. Especially searching the libraries is done for
every started program again, because it is not possible to have an index 
that could speed this process up. At least an automatic index is not
possible (you would need hooks on file system operations for it, and that
would require modifications in the kernel, e.g. a filesystem with
database capabilities). Some Unixes have such an index,  but it must be 
manually updated (ldconfig).

This means: The lack of a certain facility is the reason why Unix organizes
the directories by its operational needs, and not "by package", or in a
completely different way.
 
> When you install a package that provides a C shared library, you don't
> install it in a package-dependent directory and add a line to
> /etc/ld.so.conf with this directory; you install in /usr/lib or
> /usr/local/lib, perhaps via a symbolic link.  I urge everyone to use
> the same scheme for OCaml shared libraries.

In my opinion, there is no real reason why ocaml should follow the Unix
example, because the ocaml loader needs not to manage the whole system,
and has much lower scalability requirements. Currently I have 9 directories
in my ld.conf, and even if I had twenty directories I would not notice
any significant startup deferrals. (Yes, the search needs quadratic
time, but the file system cache helps very much to keep the costs
low.)

Furthermore, for users with much higher requirements it is possible to
implement the index like some of the Unix loaders do: by an ld.cache
that must be updated by starting an update program.

Finally, I want to point out that ocaml views the C libraries primarily as 
_dynamic_ libraries and not _shared_ libraries, so a shared directory
is conceptually misleading. The cma is shared, not the C library behind
it. Ideally, there would be a unique ID that joins the cma and the C
library, and that makes the association between the cma and the C library
exclusive. (So no other cma can load the C library, e.g. because I have
installed several versions of the same module with the danger of a
name clash.)

Well, again we would need database capabilities to do the job right...
Nobody can generate such a unique ID for us.

> (And, yes, I haven't followed this recommendation in some of the
> standard OCaml libraries (labltk) or some of my own extensions,
> but I've realized my error and intend to fix this in release 3.05.)

Which error? Too less directories!
 
> So, to summarize, I strongly suggest the following approach for RPMs
> or Debian packages:
> - The main Caml package can add one or two lines to ld.conf,
>   e.g. /var/lib/ocaml/shlibs (for libraries installed by other packages)
>   and /usr/local/lib/ocaml/shlibs (for local stuff)
> - Packages for additional Caml libraries install their shared libraries
>   in /var/lib/ocaml/shlibs, either directly or via a symlink.
> 
> This is simple, consistent with C shared libraries, and supports
> deinstallation of additional Caml libraries without any hassle.

Yes, for packaged ocaml installations this is the simplest solution. But
not everybody uses such installations, and for manually maintained systems
this way introduces another source for errors. 

My summary is different: The Unix way of handling shared library is driven
by operational requirements (fast library lookup), and is conceptually not
the best choice. In a better world, it would be possible to link programs
with libraries in a very controlled way, but that would require database
capabilities on the filesystem layer. However, for the needs of a small
subsystem like ocaml, it is not hopeless to maintain the required index
tables manually.

Gerd
-- 
----------------------------------------------------------------------------
Gerd Stolpmann      Telefon: +49 6151 997705 (privat)
Viktoriastr. 45             
64293 Darmstadt     EMail:   gerd@gerd-stolpmann.de
Germany                     
----------------------------------------------------------------------------
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners