RE: Module hierarchies

[Gerd Stolpmann <gerd@gerd-stolpmann.de>]

On Tue, 09 Jan 2001, Dave Berry wrote:
>I think you have to step back and ask what you're trying to achieve.  In
>particular, there are two related issues.  One is nested namespaces, which
>is obviously desirable. The second is units of delivery, by which I mean
>DLLs, Components, Libraries, Executables, object files, or groups of these.
>As a generic term, I'll call these "deliverables".  (I'd like to use
>"packages", but this would cause confusion with Java and Ada -- neither of
>which actually package the results of compiling a so-called "package").
>
>I'll use an example with which I'm more familiar: the SML Basis Library.
>This contains several modules, some of which are grouped in various ways.
>One such grouping is the IO modules; these are designed so that you can use
>the rest of the library without having to use the IO modules, e.g. in case
>you have your own IO subsystem.  So we want to be able to load a subset of
>the full library.

Your example shows that there are several aspects:

- How to statically refer to a module at compile time = namespace aspect
- How to separate the code base into separately linkable/loadable parts
  = library aspect
- How to use/initialize the code, i.e. there may be fragments of libraries
  which can be used independently of the rest of the library

In O'Caml, we have currently:

- A library (always in archive format) consists of several toplevel
  modules
- When linking the executable, only the actually referred modules of the
  libraries are included and initialized

It is very simple to use libraries, but it might be difficult to produce them,
because unfortunately the toplevel modules of the library are the
compilation units when the library is created. For example, if I want to
make a library with, say, IO.File, IO.Net, and IO.Base, I have only one
compilation unit IO, although it is desirable to have thee such units which are
linked together later.

>[...]
>So, how should this be packaged, and how should it be referenced?  You don't
>necessarily want to distribute the library as a single DLL, because you
>might want to use only some parts of it.  But you still want a nested
>namespace.  So it's a bad idea to require that a namespace refers to a
>single compiled object.   This is effectively what SML does -- a namespace
>is a structure, and if one part of a structure is available then all parts
>are (although an implementation might do something clever with delayed
>loading).

DLLs are more problematic than static libraries when only parts of the
libraries are referred to, because it is more difficult to initialize a DLL
only partly. (But I can still deliver the library as single DLL if the
operating system has a mechanism that loads only the needed parts.)

>You do want some way of distributing the library as a small number of files.
>Distributing large numbers of object files is a pain, especially if they
>have to be uninstalled in certain places.  Archive files (ar, jar) are one
>solution.  You might also have a library that you want to distribute as a
>DLL. 
>
>Then you need to express dependencies.  I think it's useful to be able to do
>this at the library level -- e.g. to say that application A depends on
>Basis.IO, and application B depends on Basis.* .  You don't want to have to
>list each file separately.  Possibly this can be done by command-line
>arguments, e.g. compile -I Basis\IO ?

This is what I implemented in findlib, my package manager for O'Caml. It
focuses on the delivery problems of libraries: Where is the library installed?
Which archives belong to a library? Which dependencies exist?

However, findlib is restricted because it only manages the details of the
command line arguments of the O'Caml compiler. So it can't find out namespace
conflicts, nor it can rearrange namespaces.

The findlib URL: 
http://www.ocaml-programming.de/packages/documentation/findlib/

>So there are lots of issues to consider.  At Harlequin we wrote several
>project management systems for MLWorks, and a couple more for Dylan.  They
>all involved difficult design decisions.   My current belief is that you
>need a separate notion of library, being a namespace that corresponds to a
>deliverable, with some notion of dependencies at the library level, as well
>as dependencies between the modules in a library.

I think that namespaces and libraries are simply independent notions. The
current O'Caml implementation does this right when libraries are used; i.e. you
can form a library from arbitrary modules. What is missing is the possibility
to reorganize namespaces when libraries are created.

Of course, DLLs are missing, too. 

Gerd
-- 
----------------------------------------------------------------------------
Gerd Stolpmann      Telefon: +49 6151 997705 (privat)
Viktoriastr. 100             
64293 Darmstadt     EMail:   gerd@gerd-stolpmann.de
Germany                     
----------------------------------------------------------------------------