Questions concerning modules as first-class values

Questions concerning modules as first-class values

[Dario Teixeira]

Hi,

3.12 will bring modules as first-class values to the Ocaml language.  
This is an interesting development whose full range of applications will
surprise us for many years, I suspect.  It also raises a few questions,
two of which I'll throw to the audience.

The first is purely one of terminology.  The table below describes how up 
until now core "elements" of the Ocaml language could be described; note the
use of ad-hoc terms such as "value-ish", "module-ish", and even "elements".
Moreover, since modules are now also first-class "values", the terminology 
is bound to get confusing.  So, what should the proper names for all these
"elements" be?

-------------------------------------------------
|               | 0-arity       | n-arity (n>0) |
-------------------------------------------------
| Valueish      | value         | function      |
|               |               |               |
| Moduleish     | module        | functor       |
-------------------------------------------------


Second, does the promotion of modules to first-class values also extend to
higher-order modules, ie, functors?  For the sake of symmetry I reckon it 
ought to.  As an illustration, consider again the same table, but extended to
a third dimension indicating with a boolean (T or F) whether each "element"
is a first-class "value" (the third dimension is represented after a slash). 
It seems right that the question mark in the functor cell should also be a "T".
But is it?

-------------------------------------------------
|               | 0-arity       | n-arity (n>0) |
-------------------------------------------------
| Valueish      | value/T       | function/T    |
|               |               |               |
| Moduleish     | module/T      | functor/?     |
-------------------------------------------------


Thanks for your time!
Best regards,
Dario Teixeira

Re: [Caml-list] Questions concerning modules as first-class values

[Julien Signoles]

[-- Attachment #1: Type: text/plain, Size: 1248 bytes --]

Hello,

2010/6/2 Dario Teixeira <darioteixeira@yahoo.com>

> -------------------------------------------------
> |               | 0-arity       | n-arity (n>0) |
> -------------------------------------------------
> | Valueish      | value         | function      |
> |               |               |               |
> | Moduleish     | module        | functor       |
> -------------------------------------------------
>

I disagree with your terminology. Below is the one I use:

-------------------------------------------------
> |               | 0-arity       | n-arity (n>0) |
> -------------------------------------------------
> | Value     | constant    | function       |
> |               |                   |                    |
> | Module  | structure     | functor        |
> -------------------------------------------------
>


> Second, does the promotion of modules to first-class values also extend to
> higher-order modules, ie, functors?
>

If I well understood what Alain Frisch and Xavier Leroy explained, modules
(including both structures and functors) become first class value:
structures may be converted to records and functors to functions; and
conversely. But I let more informed person confirm this.

--
Julien

[-- Attachment #2: Type: text/html, Size: 1905 bytes --]

Re: [Caml-list] Questions concerning modules as first-class values

[Alain Frisch]

On 06/02/2010 03:43 PM, Julien Signoles wrote:
> If I well understood what Alain Frisch and Xavier Leroy explained,
> modules (including both structures and functors) become first class
> value: structures may be converted to records and functors to functions;
> and conversely. But I let more informed person confirm this.

Not really. What you describe is the internal compilation strategy for 
modules (structures -> records; functors -> functions), which did not 
change with the introduction of 1st class modules.

What's possible in OCaml 3.12 is to turn a module (be it a structure or 
a functor) into a "package", which is a first-class value. A package is 
just the wrapper around a module that makes it a first-class value. In 
particular, a package that wraps a structure (resp. a functor) is *not* 
converted to a record (resp. a function).

What you can with a package:
- whatever you can do with a first-class value (pass it to a function, 
store it in a data structure, etc);
- unwrap it back into the underlying module.


We've been using first-class modules for more than one year at LexiFi 
and we have never found a use for packing functors into first-class 
values, but I've no doubt people will find some. In particular, this 
allows the program to decide at runtime (given an environment variable 
or a command-line argument) to use some particular (functorized) 
implementation of a data structure over another.


Alain

Re: [Caml-list] Questions concerning modules as first-class values

[Eric Cooper]

On Wed, Jun 02, 2010 at 06:28:07PM +0200, Alain Frisch wrote:
> In particular, this allows the program to decide at runtime (given
> an environment variable or a command-line argument) to use some
> particular (functorized) implementation of a data structure over
> another.

Is it possible to marshal and unmarshal these packages?  That might be
a nice way to implement a plug-in architecture.

-- 
Eric Cooper             e c c @ c m u . e d u

Re: [Caml-list] Questions concerning modules as first-class values

[Alain Frisch]

On 02/06/2010 19:36, Eric Cooper wrote:
> Is it possible to marshal and unmarshal these packages?

Yes, this is possible, but:

1. Extremely dangerous: there is no runtime type-checking. If you 
marshal a module, you need to unmarshal it to the same package type, 
exactly (no coercion allowed).

2. Rather useless: the code for functions within modules is not 
marshaled (as for regular functions). If you marshal a module that 
define functions, you need to unmarshal it in the same program.

 > That might be a nice way to implement a plug-in architecture.

Given the points above, I don't think so!


Alain

Re: [Caml-list] Questions concerning modules as first-class values

[Jacques Garrigue]

From: Alain Frisch <alain@frisch.fr>
> On 02/06/2010 19:36, Eric Cooper wrote:
>> Is it possible to marshal and unmarshal these packages?
> 
> Yes, this is possible, but:
> 
> 1. Extremely dangerous: there is no runtime type-checking. If you
> marshal a module, you need to unmarshal it to the same package type,
> exactly (no coercion allowed).
> 
> 2. Rather useless: the code for functions within modules is not
> marshaled (as for regular functions). If you marshal a module that
> define functions, you need to unmarshal it in the same program.
> 
>> That might be a nice way to implement a plug-in architecture.
> 
> Given the points above, I don't think so!

While marshalling of first-class modules is indeed not a good idea, I
think that dynamic loading of first-class modules might provide a nice
plug-in architecture.
For instance, you can easily build a list of such plug-ins, with an
uniform interface, and add elements to this list through dynamic
loading. They can even expose internal data, with an abstract type.

There is however a difficulty if you want to save and load data used
by these plugins uniformly. Namely, even if all these plugins expose
some data with a type "t", the "t" is actually going to be different
in each, and the type system will enforce it. This is good, but it
will be seen as different for the same plugin in a different run of
the same application. So you need to do some standard hacking around,
having unique ids for each plugin, and some magic to recover the data.
Otherwise this seems a nice approach, easier than using objects, since
objects don't let you expose internal data of different types.
(Well, actually you can probably do this with objects too, using a
polymorphic method to encoding existential types in CPS style, but
this is going to be costly and not very natural.)

Jacques Garrigue

Re: [Caml-list] Questions concerning modules as first-class values

[Alain Frisch]

On 04/06/2010 03:34, Jacques Garrigue wrote:
> For instance, you can easily build a list of such plug-ins, with an
> uniform interface, and add elements to this list through dynamic
> loading. They can even expose internal data, with an abstract type.

Indeed!

That's basically our primary use of first-class modules at LexiFi. Our 
application maintains several sets of extensible notions (like 
"instruments" or "models") and it can load plugins which can register 
new such objects. Roughly speaking, each instrument is represented as a 
first-class module of type (module S) which looks like:

module type S = sig
   type t            (* the type of parameters for the instruments *)
   val t: t ttype    (* a dynamic representation of the type t *)
   val name: string  (* name to identify this instrument, presented
                        in a menu to the end-user *)
   val run: t -> contract (* build a fresh contract, which is a notion 
independent from the instrument *)

   type attrs   (* the type of attributes for this instrument *)
   val attrs: attrs ttype (* dynamic representation of attrs *)
   val compute: t -> contract -> attrs (* calculate attributes
                                    from initial parameters and
                                    a contract with history *)

   (* ... *)
end


The idea is that each instrument defines a number of abstract types 
(parameters, attributes) and operations on these types. It also exports 
values that gives the concrete structural representation of these 
abstract types (the 'a ttype components above) to enable generic 
operations such as automatic generation of GUI for showing/editing 
values of these types, or safe marshaling.  These "type" values can be 
derived automatically from the actual type definitions (either with a 
syntax extension, or with a compiler modification in our case).




Alain

Re: [Caml-list] Questions concerning modules as first-class values

[Dario Teixeira]

Hi,

> I disagree with your terminology. Below is the one I use:
>
> ------------------------------------------
> |          | 0-arity     | n-arity (n>0) |
> ------------------------------------------
> | Value    | constant    | function      |
> |          |             |               |
> | Module   | structure   | functor       |
> ------------------------------------------

This terminology is indeed better.  But note that since modules are now
first-class "values", the word "value" itself becomes ambiguous.  I'm not
trying to be picky; I think that having clear, unambiguous terminology is
essential to properly convey a message, particularly to beginners.


> If I well understood what Alain Frisch and Xavier Leroy explained,
> modules (including both structures and functors) become first class
> value: structures may be converted to records and functors to functions;
> and conversely. But I let more informed person confirm this.

I am not sure about the implementation details, but at least syntax-wise
I did not get the impression that structures would be converted to records.
What do you mean, exactly?

Cheers,
Dario Teixeira