Partially hiding modules in packages

Partially hiding modules in packages

[Alexey Rodriguez]

[-- Attachment #1: Type: text/plain, Size: 2997 bytes --]

Dear list,

My question is about how to hide modules (or parts thereof) in
an ocaml package from the outside world (users of the package).

I am trying to build an ocaml package with internal functionality
(types and functions)
that I do not want to expose. I have two modules in the package implemented by
foo.ml and bar.ml. The structure of each .mli looks like this:

> module Foo = sig
>   type t
>   val make : t
>   val change : t -> t
>
>   (* Internal, not to be exposed. *)
>   val unsafe_change : t -> t
> end

> module Bar = sig
>   type t
>   val combine : t -> Foo.t -> t (* uses unsafe_change *)
> end

As you see Foo exports unsafe_change because Bar needs it.
I compile both modules using "-for-pack Foobar" and build them using
the "-pack" option.
However, now I want to hide "Foo.unsafe_change" from the package
"export list" (if there
is such a thing) so that main.ml cannot call it. I see the following options:

* Merge foo.ml and bar.ml so that unsafe_change does not need to be exposed.
  I don't like this, the modules could be very large.
* Split Foo into Foo_internal and Foo, and somehow keep Foo_internal
  invisible. I don't know how to do this. I would use "exposed-modules" in
  Cabal which is a package manager for Haskell.
* Add the file foobar.mli which contains the signatures of Foo and Bar
but hiding
  Foo.unsafe_change. I think it could work, but I would have to repeat
much of the Foo
  and Bar interfaces. I tried this but it didn't work, ocaml complains
as follows:

> Error: The implementation (obtained by packing)
>        does not match the interface foobar.mli:
>        The field `Foo' is required but not provided

  I am attaching a tarball of this attempt in case someone knows what
went wrong.
  The example is a bit different from the example in the email.

So the question is:

  What solution do you follow when you want to hide (part of) a module
in a package?
  Keep in mind that the hidden parts should still be exposed to the
modules inside
  the package.

A related question is how to expose the representation Foo.t so that
Bar can manipulate it
directly, but at the same time keep it hidden from users of the
package. The following easy
hack works. Create foo_type.mli with the definition of the type t:

> type t = int

and redefine Foo as follows:

> module Foo = sig
>   type t = Foo_type.t
>   val make : t
>   val change : t -> t
>
>   (* Internal, not to be exposed. *)
>   val unsafe_change : t -> t
> end

Now "t" is visible for Bar to use but we still have to hide it from
the outside world. How do
we do it? Well, just use "-pack" on foo.cmx and bar.cmx. We can omit
Foo_types because it has
no .cmx. As a consequence Foo.t becomes abstract. This probably works
by accident and maybe it
is not a good idea to rely on this trick.

I would appreciate if you can share your experiences and perhaps point
to some ocaml library
that does module or partial module hiding so I can use it as an example.

Thank you!

Cheers,

Alexey

[-- Attachment #2: packages.tgz --]
[-- Type: application/x-gzip, Size: 613 bytes --]

Re: [Caml-list] Partially hiding modules in packages

[blue storm]

The problem with your packages.tgz example is that you use "module
type Foo = .." in the .mli. This gives the signature of a module type,
that is, it refers to a _module type_ defined in the implementation
file.

What you want to do here is to give the signature of a _module_, not a
module types, so here is the correct syntax :

  module Foo : sig
    type foo_t

    val initial : foo_t
    val show : foo_t -> string
  end


Regarding your original problem, I've had the same needs and came up
with a slightly different solution : in order to avoid the additional
indirection level related to -pack (Foobar.Foo), is used a flattened
representation by adding a "foobar.ml" file containing only :

  include Foo

(and possibly include of other modules in the package). Then the
foobarl.mli is :

  type foo_t

  val initial : foo_t
  val show : foo_t -> string

And values can be referred with Foobar.foo, instead of Foobar.Foo.foo.
Of course this is only useful if you don't want the user to see the
internal module hierarchy, wich may not be what you had in mind.

Re: [Caml-list] Partially hiding modules in packages

[Alain Frisch]

On 9/9/2009 9:00 PM, Alexey Rodriguez wrote:
> My question is about how to hide modules (or parts thereof) in
> an ocaml package from the outside world (users of the package).

It is not a well-known fact, but it is possible to provide an explicit 
interface for the packaged module (just create an ad hoc foobar.mli and 
compile it before doing the -pack'ing).

-- Alain

Re: [Caml-list] Partially hiding modules in packages

[Jean-Christophe Filliâtre]

Hi,

Alexey Rodriguez a écrit :
> My question is about how to hide modules (or parts thereof) in
> an ocaml package from the outside world (users of the package).
> 
> * Add the file foobar.mli which contains the signatures of Foo and Bar
> but hiding
>   Foo.unsafe_change. I think it could work, but I would have to repeat
> much of the Foo
>   and Bar interfaces. I tried this but it didn't work, ocaml complains
> as follows:

That's the solution we followed in Mlpost (http://mlpost.lri.fr/) and it
works fine (you may have a look at the Makefile.in in mlpost sources).

Indeed, you have to repeat (parts of) the modules interfaces, but since
it is where we put all the ocamldoc documentation, in a single file, it
appears to be a satisfying solution.

-- 
Jean-Christophe

Re: [Caml-list] Partially hiding modules in packages

[Alexey Rodriguez]

Thanks to all who replied. I have solved my problem, but I still have
a question regarding clashing type definitions.

On Wed, Sep 9, 2009 at 9:40 PM, blue storm <bluestorm.dylc@gmail.com> wrote:
> The problem with your packages.tgz example is that you use "module
> type Foo = .." in the .mli. This gives the signature of a module type,
> that is, it refers to a _module type_ defined in the implementation
> file.
>
> What you want to do here is to give the signature of a _module_, not a
> module types, so here is the correct syntax :
>
>  module Foo : sig
>    type foo_t
>
>    val initial : foo_t
>    val show : foo_t -> string
>  end

Well spotted! I already encountered this before and at the time I
understood it, but I guess my brain forgot about it yesterday. Thanks
for checking the files, now it works. Whether the modules are
flattened in Foobar or not, I think it is orthogonal to the issue of
hiding (we both use the same solution). In our case I prefer to keep
the hierarchy as the packages tend to have many big modules.

I also checked the source code of mlpost, thanks for the tip
Jean-Christophe. This is probably the most promising solution.

Now I am trying to see how much reuse of "signatures" can be achieved.
Currently I defined the FOO and BAR signatures in sigs.ml. I include
them in the local mli files, for example:

foo.mli:
> include Sigs.FOO with type foo_t = int
>
> val unsafe_modify : foo_t -> foo_t

bar.mli:
> include Sigs.BAR with type foo_t = Foo.foo_t

And now the package signature is:
> module Foo : Sigs.FOO
> module Bar : Sigs.BAR with type foo_t = Foo.foo_t

This works great: the representation of Foo.foo_t is hidden and the
unsafe function is not visible. This requires adding the type foo_t to
the signature of Bar:

> module type BAR = sig
>  type bar_t
>  type foo_t
>  val initial : bar_t
>  val combine : bar_t -> foo_t -> foo_t
> end

So there is no free lunch here, I can reuse the signatures to avoid
repetition but I have to introduce extra type aliases to relate
signatures at include time. This could add some complexity for modules
sharing many types, and now I cannot include both FOO and BAR in the
same module due to clashing definitions of foo_t.

The only way that comes to my mind to avoid such clashes is to invent
new type alias names even though they refer to the same type. Is there
a nice methodology to avoid clashing type definitions when I want to
include two module (interfaces) that share types?

Cheers,

Alexey

Re: [Caml-list] Partially hiding modules in packages

[Guillaume Yziquel]

blue storm a écrit :
> 
> Regarding your original problem, I've had the same needs and came up
> with a slightly different solution : in order to avoid the additional
> indirection level related to -pack (Foobar.Foo), is used a flattened
> representation by adding a "foobar.ml" file containing only :
> 
>   include Foo
> 
> (and possibly include of other modules in the package). Then the
> foobarl.mli is :
> 
>   type foo_t
> 
>   val initial : foo_t
>   val show : foo_t -> string
> 
> And values can be referred with Foobar.foo, instead of Foobar.Foo.foo.
> Of course this is only useful if you don't want the user to see the
> internal module hierarchy, wich may not be what you had in mind.

Where do you put the foobar.ml? I've been trying it all over, I do not 
see how you can flatten something that you pack.

Do you put foobar.ml at the same level of your directory foobar/, or in 
you directory foobar/ ?

Or am I understanding you wrong?

All the best,

-- 
      Guillaume Yziquel
http://yziquel.homelinux.org/