Re: Module hierarchies

[Xavier Leroy <Xavier.Leroy@inria.fr>]

> An alternative is to adopt the Java convention, in which a module such a
>      engine/graphics/texture/manager.ml
> is automagically mapped to Engine.Graphics.Texture.Manager.

Yes, this has been suggested already on this list.  Problem number one
is, as you said:

> The difficulty
> now is what to do about the file/module
>      engine/graphics/texture.ml <=> Engine.Graphics.Texture
> It seems to me the easiest solution is to assume that a directory/file
> layout has the semantics of a single file in which the modules are
> catenated in depth-first order.

This is one solution, but this ordering for submodules is somehow
arbitrary.  More pragmatically, it seems very hard (in the current
implementation) to maintain the correspondence between a directory and
a structure with sub-modules corresponding to the directory elements.

An alternative solution (suggested by Judicaël Courant some time ago)
would be to have a new command that groups together several
separately-compiled modules into one module having the original
modules as sub-structures.  E.g.

        ocamlnewmagiccommand -o lib.cmo a.cmo b.cmo c.cmo

would generate lib.cmo and lib.cmi files equivalent to the following
source code for lib.ml:

        module A = struct (* contents of a.ml *) end
        module B = struct (* contents of b.ml *) end
        module C = struct (* contents of c.ml *) end

In other terms, while the current OCaml library archive files (.cma files)
generated by "ocamlc -a" are "flat" and introduce no additional
structuring, the new command would do both library archiving and
introducing of a layer of structuring.

Of course, the order of .cmo files on the command line would determine
the order of the sub-modules, thus relieving the compiler from
guessing this order.

(As an aside, it is interesting to note that the Linux kernel sources
-- a large source tree indeed -- uses "ld -r" in subdirectories to group
together the object files for each subdirectory in one easy to
manipulate .o file.  This is kind of the same idea, except that of
course C's namespace is flat, so no additional structuring is introduced.)

I still have no idea how hard it is to implement Judicaël's scheme,
though.

Coming back to the general problem of structuring a large OCaml
project, my experience with the OCaml compiler itself is that the
solution based on a flat module namespace + subdirectories to
partition the files + a big Makefile at the top works out quite well
for projects of about 100 KLOC, and could scale up some more, although
perhaps not to 1 MLOC.  In particular, one big Makefile is a lot
easier to maintain than a zillion tiny recursive Makefiles.

When comparing with Java, you have to keep in mind that Java source
files are smaller and more numerous than Caml source files, since the
latter can contain several classes as well as submodules.  (Not to
mention that a 10-line OCaml datatype declaration is roughly
equivalent to 11 Java classes, each in its own file...)  So, the need
to break up a Java project into several packages appears earlier than
the need to break up a Caml project into several directories.

Still, I'd be very interested to know how others "do it" with large
OCaml projects.

Happy new year,

- Xavier Leroy