Nesting Modules

Nesting Modules

[Tom Hawkins]

Is it possible to nest modules defined in separate files?

For example, assume I have...

   top.ml
   top.mli
   bottom.ml
   bottom.mli

Can I instruct the compiler to nest Bottom somewhere in Top, to obtain 
clean, hierarchical names (eg: Top.Bottom.some_function)?  Of course I 
can inline bottom.ml in top.ml, but then the file become rather large.

-Tom

Re: [Caml-list] Nesting Modules

[Pietro Abate]

On Tue, Nov 01, 2005 at 10:59:27AM -0600, Tom Hawkins wrote:
> Can I instruct the compiler to nest Bottom somewhere in Top, to obtain 
> clean, hierarchical names (eg: Top.Bottom.some_function)?  Of course I 
> can inline bottom.ml in top.ml, but then the file become rather large.

There should be some way in camlp4 to inline files with and
#include "file.ml" directive... Or at least I've a vague memory
of this... I remember of a thread on this ml a while ago but I never
use this trick. Does anybody can confirm this ?

hope this helps.

p

-- 
++ Blog: http://blog.rsise.anu.edu.au/?q=pietro
++ 
++ "All great truths begin as blasphemies." -George Bernard Shaw
++ Please avoid sending me Word or PowerPoint attachments.
   See http://www.fsf.org/philosophy/no-word-attachments.html

Re: [Caml-list] Nesting Modules

[Martin Jambon]

On Wed, 2 Nov 2005, Pietro Abate wrote:

> On Tue, Nov 01, 2005 at 10:59:27AM -0600, Tom Hawkins wrote:
>> Can I instruct the compiler to nest Bottom somewhere in Top, to obtain
>> clean, hierarchical names (eg: Top.Bottom.some_function)?  Of course I
>> can inline bottom.ml in top.ml, but then the file become rather large.
>
> There should be some way in camlp4 to inline files with and
> #include "file.ml" directive... Or at least I've a vague memory
> of this... I remember of a thread on this ml a while ago but I never
> use this trick. Does anybody can confirm this ?

Some time ago I wrote some code which works partially. You can 
include a file from within a submodule, i.e. you can do this:

   module A = struct USE "module_a.ml" end

The problem is line numbering. Setting Pcaml.input_file doesn't seem 
to have any effect. If anyone knows how to fix that please let me know:

(* camlp4o pa_extend.cmo q_MLast.cmo -loc loc pa_use.ml *)

let parse_stream s =
   let f = !Pcaml.parse_implem in
   let rec loop () =
     match f s with
         l, true -> List.map fst l @ loop ()
       | l, false -> List.map fst l in
   loop ()

let parse_file file =
   let ic = open_in file in
   let stream = Stream.of_channel ic in
   let current_file = !Pcaml.input_file in
   Pcaml.input_file := file; (* it doesn't work *)
   let l = parse_stream stream in
   close_in ic;
   Pcaml.input_file := current_file;
   l

EXTEND
   Pcaml.str_item: [
     [ "USE"; file = STRING ->
         let l = parse_file file in
         <:str_item< declare $list:l$ end >> ]
   ];
END

(********************************)

Martin

--
Martin Jambon, PhD         http://martin.jambon.free.fr
Freedom for the regexps!   http://martin.jambon.free.fr/micmatch-howto.html

Re: [Caml-list] Nesting Modules

[Robert Roessler]

Tom Hawkins wrote:
> Is it possible to nest modules defined in separate files?
> 
> For example, assume I have...
> 
>   top.ml
>   top.mli
>   bottom.ml
>   bottom.mli
> 
> Can I instruct the compiler to nest Bottom somewhere in Top, to obtain 
> clean, hierarchical names (eg: Top.Bottom.some_function)?  Of course I 
> can inline bottom.ml in top.ml, but then the file become rather large.

It seems like the "include" construct might be of help...

Robert Roessler
roessler@rftp.com
http://www.rftp.com

Re: [Caml-list] Nesting Modules

[Chris King]

On 11/1/05, Tom Hawkins <tom@confluent.org> wrote:
> Is it possible to nest modules defined in separate files?

One way to do what you're looking to do, is to write in top.ml:

module Bottom = Bottom

This will essentially re-export Bottom in Top.  Unfortunately, this
requires that you link bottom.cmo in with your code, which causes
Bottom to be visible at the top level.

A similar effect can be obtained with the -pack flag:

ocamlc -pack top.ml bottom.ml foo.cmo

Top and Bottom can now be referenced as Foo.Top and Foo.Bottom, and
only foo.cmo needs to be linked.  I know this isn't exactly what
you're looking for, but it might do the trick just the same.

- Chris King

Re: [Caml-list] Nesting Modules

[Daniel Bünzli]

e 2 nov. 05 à 03:03, Chris King a écrit :

> One way to do what you're looking to do, is to write in top.ml:
>
> module Bottom = Bottom
>
> This will essentially re-export Bottom in Top.  Unfortunately, this
> requires that you link bottom.cmo in with your code, which causes
> Bottom to be visible at the top level.

It will not be visible to the user if you don't provide bottom.cmi.  
The problem of linking can be alleviated by packing everything in a  
cma library. In other words what you can do is

 > echo 'let f x = x' > bottom.ml
 > echo 'module Bottom = Bottom' > top.ml
 > ocamlc -c bottom.ml
 > ocamlc -c top.ml
 > ocamlc -a -o toplib.cma bottom.cmo top.cmo
 > rm bottom.cmi
 > ocaml toplib.cma
         Objective Caml version 3.09.0

# Bottom.f;;
Unbound value Bottom.f
# Top.Bottom.f;;
- : 'a -> 'a = <fun>

When you want to link an executable using Top you now need to specify  
toplib.cma (instead of top.cmo).

Note that using 'include' poses the same problems of visibility and  
linking so you will have to adopt the same technique. In fact I think  
that 'module Bottom = Bottom' is exactly the same as 'module Bottom =  
struct include Bottom end', the former being more obvious. 'include'  
is only usefull if you need to extend the structure with new  
elements, e.g. 'module Bottom = struct include Bottom let g x = ...  
end'.

Daniel

Re: [Caml-list] Nesting Modules

[Tom Hawkins]

Daniel Bünzli wrote:
> e 2 nov. 05 à 03:03, Chris King a écrit :
> 
>> One way to do what you're looking to do, is to write in top.ml:
>>
>> module Bottom = Bottom
>>
>> This will essentially re-export Bottom in Top.  Unfortunately, this
>> requires that you link bottom.cmo in with your code, which causes
>> Bottom to be visible at the top level.
> 
> 
> It will not be visible to the user if you don't provide bottom.cmi.  The 
> problem of linking can be alleviated by packing everything in a  cma 
> library. In other words what you can do is
> 
>  > echo 'let f x = x' > bottom.ml
>  > echo 'module Bottom = Bottom' > top.ml
>  > ocamlc -c bottom.ml
>  > ocamlc -c top.ml
>  > ocamlc -a -o toplib.cma bottom.cmo top.cmo
>  > rm bottom.cmi
>  > ocaml toplib.cma
>         Objective Caml version 3.09.0
> 
> # Bottom.f;;
> Unbound value Bottom.f
> # Top.Bottom.f;;
> - : 'a -> 'a = <fun>
> 
> When you want to link an executable using Top you now need to specify  
> toplib.cma (instead of top.cmo).

Building a cma/cmxa is fine -- I am working on a library after all.  But 
for this to work, how are the mli files handled?  I tried several 
variations of the following, but again, I'm faced with "Unbound module 
type Bottom":

echo 'let hello = "hello"' > bottom.ml
echo 'val hello : string' > bottom.mli
echo 'module Bottom = Bottom' > top.ml
echo 'module type Bottom = Bottom' > top.mli
ocamlc -c bottom.mli
ocamlc -c bottom.ml
ocamlc -c top.mli
File "top.mli", line 1, characters 21-27:
Unbound module type Bottom

This must be a common problem.  Surely I'm not the first person 
constructing a hierarchical library.  OCaml claims to have a 
hierarchical module system, but I have yet to see an easy way to build a 
hierarchy of modules.  Am I really going to have to revert to a 
preprocessor?!?

-Tom

Re: [Caml-list] Nesting Modules

[Daniel Bünzli]

Le 2 nov. 05 à 13:00, Tom Hawkins a écrit :

> Building a cma/cmxa is fine -- I am working on a library after  
> all.  But for this to work, how are the mli files handled?  I tried  
> several variations of the following, but again, I'm faced with  
> "Unbound module type Bottom":

It is true that when I applied the scheme I described to my own code  
I inelegantly cut and pasted Bottom's type in Top's one.

It seems that although a compiled file a.ml defines a module named A,  
a compiled file a.mli doesn't define a module type A, strange  
asymmetry. Apparently an old feature wish about this is in the  
bugtracking system [1]. Maybe we should wish again ? The problem of  
the wish is that granting it could break existing code.

Daniel

[1] <http://pauillac.inria.fr/bin/caml-bugs/feature%20wish?id=1471>

Re: [Caml-list] Nesting Modules

[Tom Hawkins]

William D. Neumann wrote:
> On Tue, 1 Nov 2005, Tom Hawkins wrote:
> 
>> Is it possible to nest modules defined in separate files?
>>
>> For example, assume I have...
>>
>>  top.ml
>>  top.mli
>>  bottom.ml
>>  bottom.mli
>>
>> Can I instruct the compiler to nest Bottom somewhere in Top, to obtain 
>> clean, hierarchical names (eg: Top.Bottom.some_function)?  Of course I 
>> can inline bottom.ml in top.ml, but then the file become rather large.
> 
> 
> Sort of.  You can use include to minimize the file bloat.  A stupid 
> example follows:

Thanks William, this helps.  One question below...

> 
> ------
> (************************************************************************ *
> * bottom.ml
> *
> ************************************************************************)
> 
> let mname = "Bottom"
> 
> let id x = x -----
> 
> -----
> (************************************************************************
> *
> *  bottom.mli
> *
> ************************************************************************)
> 
> val mname : string
> val id : 'a -> 'a
> -----
> 
> -----
> (************************************************************************
> *
> *  top.ml
> *
> ************************************************************************)
> 
> let mname = "Top"
> 
> let id x = x
> 
> let double x = x + x
> 
> module Bottom =
>   struct
>     include Bottom
>   end
> -----
> 
> -----
> (************************************************************************
> *
> *  top.mli
> *
> ************************************************************************)
> 
> val mname : string
> 
> val id : 'a -> 'a
> 
> val double : int -> int
> 
> module Bottom : sig
>   val mname : string
>   val id : 'a -> 'a
> end

Why repeat Bottom's specification?  Why can't we do...

module Bottom : sig
   include Bottom
end

It appears to be valid syntax, but I get a "Unbound module type Bottom" 
when I try it.

-Tom

Re: [Caml-list] Nesting Modules

[Richard Jones]

On Tue, Nov 01, 2005 at 10:59:27AM -0600, Tom Hawkins wrote:
> Can I instruct the compiler to nest Bottom somewhere in Top, to obtain 
> clean, hierarchical names (eg: Top.Bottom.some_function)?  Of course I 
> can inline bottom.ml in top.ml, but then the file become rather large.

If you want clean, hierarchical names it's much better to use '_' to
separate the hierarchies (eg. Top_Bottom).  The reason for this is
that it allows others to extend your hierarchy by adding other modules
(eg. someone else can easily then create Top_MyExtension).

This means a bit of file renaming:

  top.mli
  top.ml
  top_Bottom.mli
  top_Button.ml

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com

Re: [Caml-list] Nesting Modules

[Daniel Bünzli]

Le 2 nov. 05 à 10:46, Richard Jones a écrit :

> If you want clean, hierarchical names it's much better to use '_' to
> separate the hierarchies (eg. Top_Bottom).  The reason for this is
> that it allows others to extend your hierarchy by adding other modules
> (eg. someone else can easily then create Top_MyExtension).

This doesn't scale well, suppose that I want to extend Top_Bottom,  
then I have a module called Top_Bottom_Myextension. I think 'include'  
is a better mechanism if you need to extend modules.

Another problem of this solution is that you cannot open modules  
hierarchically, e.g. open Top but use the dot notation for Bottom.

Daniel

Re: [Caml-list] Nesting Modules

[Richard Jones]

On Wed, Nov 02, 2005 at 10:57:37AM +0100, Daniel Bünzli wrote:
> This doesn't scale well, suppose that I want to extend Top_Bottom,  
> then I have a module called Top_Bottom_Myextension. I think 'include'  
> is a better mechanism if you need to extend modules.

Well that's if you want to add more functions to Top_Bottom; obviously
'include' would be the way forwards there (or the equivalent notion
where you mirror the definitions from the base module which is how
ExtLib works).  However consider a database layer:

  Database
  Database_Postgres
  Database_MySQL

Obviously '_' is the way to go here if you want multiple third parties
to provide database modules.  'include' wouldn't work at all here.

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com

Re: [Caml-list] Nesting Modules

[Daniel Bünzli]

Le 2 nov. 05 à 11:47, Richard Jones a écrit :

>  consider a database layer:
>
>   Database
>   Database_Postgres
>   Database_MySQL
>
> Obviously '_' is the way to go here if you want multiple third parties
> to provide database modules.  'include' wouldn't work at all here.

You are right. I think 'include' doesn't work here because there is a  
notion of choice. If I understand well your example, functors  
wouldn't help here because it is not that you want to get a unified  
interface (frontend) from a specific database implementation  
(backend), but you really want to access database specific features  
via the third party modules, right ?

Daniel

Re: [Caml-list] Nesting Modules

[Richard Jones]

On Wed, Nov 02, 2005 at 11:57:45AM +0100, Daniel Bünzli wrote:
> Le 2 nov. 05 à 11:47, Richard Jones a écrit :
> > consider a database layer:
> >
> >  Database
> >  Database_Postgres
> >  Database_MySQL
> >
> >Obviously '_' is the way to go here if you want multiple third parties
> >to provide database modules.  'include' wouldn't work at all here.
> 
> You are right. I think 'include' doesn't work here because there is a  
> notion of choice. If I understand well your example, functors  
> wouldn't help here because it is not that you want to get a unified  
> interface (frontend) from a specific database implementation  
> (backend), but you really want to access database specific features  
> via the third party modules, right ?

Yes; I'm really comparing it to Perl's modules.  In Perl's module
repository, CPAN, they have a zillion modules and so need to name them
sensibly and hierarchically.  For example under "Net"[1] you have
"Net::DHCP", "Net::Daemon", "Net::FTP", etc...  There is no way to
"open Net" (or its equivalent) in Perl - this is just a useful way to
organise modules.

May not be applicable to OCaml of course.

Rich.

[1] http://www.cpan.org/modules/by-module/Net/

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com

Re: [Caml-list] Nesting Modules

[Tom Hawkins]

Richard Jones wrote:
> On Tue, Nov 01, 2005 at 10:59:27AM -0600, Tom Hawkins wrote:
> 
>>Can I instruct the compiler to nest Bottom somewhere in Top, to obtain 
>>clean, hierarchical names (eg: Top.Bottom.some_function)?  Of course I 
>>can inline bottom.ml in top.ml, but then the file become rather large.
> 
> 
> If you want clean, hierarchical names it's much better to use '_' to
> separate the hierarchies (eg. Top_Bottom).  The reason for this is
> that it allows others to extend your hierarchy by adding other modules
> (eg. someone else can easily then create Top_MyExtension).
> 
> This means a bit of file renaming:
> 
>   top.mli
>   top.ml
>   top_Bottom.mli
>   top_Button.ml

But then you're forced to 
Drag_Along_Every_Module_To_Get_To_Something.useful.  ;-)  As a library 
user, I would much prefer:

   open Drag.Along.Every.Module.To.Get.To
   Something.useful ()


-Tom

Re: [Caml-list] Nesting Modules

[Richard Jones]

On Wed, Nov 02, 2005 at 05:59:04AM -0600, Tom Hawkins wrote:
> But then you're forced to 
> Drag_Along_Every_Module_To_Get_To_Something.useful.  ;-)  As a library 
> user, I would much prefer:
> 
>   open Drag.Along.Every.Module.To.Get.To
>   Something.useful ()

I'm sure you would, but the design of .cma/.cmxa files makes that
impossible if you also want to allow other people to add modules into
the same namespace.

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com

Re: [Caml-list] Nesting Modules

[Christophe TROESTLER]

On Wed, 02 Nov 2005, Tom Hawkins <tom@confluent.org> wrote:
> 
> But then you're forced to 
> Drag_Along_Every_Module_To_Get_To_Something.useful.  ;-)  As a library 
> user, I would much prefer:
> 
>    open Drag.Along.Every.Module.To.Get.To
>    Something.useful ()

You can alway do

  module Something = Drag_Along_Every_Module_To_Get_To_Something
  Something.useful()

My 2¢,
ChriS

Re: [Caml-list] Nesting Modules

[Tom Hawkins]

I found a convenient way to compose a set modules into a hierarchical 
library.  First define the set of modules (ml/mli file pairs) -- these 
become the leaves in the module tree.  Next, define the branch modules 
in the hierarchy using "module Leaf = Leaf" (ml only, no mli).  Then 
compile and link.  The only restriction is that all functional code must 
reside in leaf modules.  Here's an example...

# Define the leaf modules.
echo 'let name = "leaf1"' > leaf1.ml
echo 'val name : string'  > leaf1.mli
echo 'let name = "leaf2"' > leaf2.ml
echo 'val name : string'  > leaf2.mli
echo 'let name = "leaf3"' > leaf3.ml
echo 'val name : string'  > leaf3.mli

# Define the branch modules.
echo 'module Leaf2 = Leaf2;; module Leaf3 = Leaf3'     > branch2.ml
echo 'module Leaf1 = Leaf1;; module Branch2 = Branch2' > branch1.ml

# Compile all leaf modules.
ocamlc -c leaf1.mli
ocamlc -c leaf1.ml
ocamlc -c leaf2.mli
ocamlc -c leaf2.ml
ocamlc -c leaf3.mli
ocamlc -c leaf3.ml

# Compile all branch modules.
ocamlc -c branch2.ml
ocamlc -c branch1.ml

# Link the library.
ocamlc -a -o branch1.cma leaf1.cmo leaf2.cmo leaf3.cmo branch2.cmo 
branch1.cmo

# Delete all *.cmi files except top.
rm leaf*.cmi
rm branch2.cmi

# Run.
ocaml branch1.cma

         Objective Caml version 3.09.0

# Branch1.Branch2.Leaf3.name;;
- : string = "leaf3"




Thanks for everyone's help!

-Tom

Re: [Caml-list] Nesting Modules

[Gerd Stolpmann]

Sorry, but this is not a good solution. The leaves are only hidden from
the user, but can still cause nameclashes. (E.g. if one adds another
Leaf1 module later.)

The proper solution is to use -pack (and -for-pack in O'Caml 3.09):

ocamlc -o branch2.cmo -pack leaf2.cmo leaf3.cmo
ocamlc -o branch1.cmo -pack leaf1.cmo brachch2.cmo

Then throw away leaf*.{cmi,cmo} and branch2.{cmi,cmo}. Everything is now
in branch1.{cmi,cmo}.

For O'Caml 3.09 and ocamlopt, you need to add the right -for-pack
options, e.g.

ocamlopt -for-pack Branch1.Branch2 -c leaf2.ml
ocamlopt -for-pack Branch1.Branch2 -c leaf3.ml
ocamlopt -for-pack Branch1 -c leaf1.ml
ocamlopt -o branch2.cmx -pack -for-pack Branch1 leaf2.cmx leaf3.cmx
ocamlopt -o branch1.cmx -pack leaf1.cmx branch2.cmx

For O'Caml 3.08 and ocamlopt, you need GNU objcopy, but no extra
-for-pack options.

Gerd

Am Mittwoch, den 02.11.2005, 08:02 -0600 schrieb Tom Hawkins:
> I found a convenient way to compose a set modules into a hierarchical 
> library.  First define the set of modules (ml/mli file pairs) -- these 
> become the leaves in the module tree.  Next, define the branch modules 
> in the hierarchy using "module Leaf = Leaf" (ml only, no mli).  Then 
> compile and link.  The only restriction is that all functional code must 
> reside in leaf modules.  Here's an example...
> 
> # Define the leaf modules.
> echo 'let name = "leaf1"' > leaf1.ml
> echo 'val name : string'  > leaf1.mli
> echo 'let name = "leaf2"' > leaf2.ml
> echo 'val name : string'  > leaf2.mli
> echo 'let name = "leaf3"' > leaf3.ml
> echo 'val name : string'  > leaf3.mli
> 
> # Define the branch modules.
> echo 'module Leaf2 = Leaf2;; module Leaf3 = Leaf3'     > branch2.ml
> echo 'module Leaf1 = Leaf1;; module Branch2 = Branch2' > branch1.ml
> 
> # Compile all leaf modules.
> ocamlc -c leaf1.mli
> ocamlc -c leaf1.ml
> ocamlc -c leaf2.mli
> ocamlc -c leaf2.ml
> ocamlc -c leaf3.mli
> ocamlc -c leaf3.ml
> 
> # Compile all branch modules.
> ocamlc -c branch2.ml
> ocamlc -c branch1.ml
> 
> # Link the library.
> ocamlc -a -o branch1.cma leaf1.cmo leaf2.cmo leaf3.cmo branch2.cmo 
> branch1.cmo
> 
> # Delete all *.cmi files except top.
> rm leaf*.cmi
> rm branch2.cmi
> 
> # Run.
> ocaml branch1.cma
> 
>          Objective Caml version 3.09.0
> 
> # Branch1.Branch2.Leaf3.name;;
> - : string = "leaf3"
> 
> 
> 
> 
> Thanks for everyone's help!
> 
> -Tom
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Telefon: 06151/153855                  Telefax: 06151/997714
------------------------------------------------------------

Re: [Caml-list] Nesting Modules

[Oliver Bandel]

On Tue, Nov 01, 2005 at 10:59:27AM -0600, Tom Hawkins wrote:
> Is it possible to nest modules defined in separate files?
> 
> For example, assume I have...
> 
>   top.ml
>   top.mli
>   bottom.ml
>   bottom.mli
> 

Do you look for something like this:

##############################################################
first:~/Programmierung/includes-in-ocaml oliver$ cat top.ml
let a = "Haha, I'm in top!"

module Bottom = struct include Bottom let c = "oiuh" end

first:~/Programmierung/includes-in-ocaml oliver$ cat bottom.ml
let b = "hello, this is the bottom!"
first:~/Programmierung/includes-in-ocaml oliver$ ocamlc -c bottom.ml
first:~/Programmierung/includes-in-ocaml oliver$ ocamlc -c top.ml   
first:~/Programmierung/includes-in-ocaml oliver$ ocamlc -i top.ml
val a : string
module Bottom : sig val b : string val c : string end
first:~/Programmierung/includes-in-ocaml oliver$ 
##############################################################

???

Ciao,
   Oliver