modules local to functions.

modules local to functions.

[Andrew Conway]

Resume en tres pauvre Francais: je voudrais utiliser un "functor" au milleau
d'une fonctionne, en utilisant les parametres de la fonctionne dans le "sig".
Je ne peut pas le faire. Pourquoi pas?

---------------------------------

I have a functor such as follows:

  module type Specifyn = sig val n : int end;;

  module Makespecific(N:Specifyn) = struct
	type mytype = int
	let mirror () : mytype = N.n
  end;;

I can apply it at the main level like:

  module Fred = Makespecific(struct let n = 7 end);;

However, there are cases when I want to define a module in the middle
of a function (eg a functor of packed binary arrays where I want to 
determine the number of bits used at run time depending upon the 
arguement of the function). That is, I would like to be abe to do:

  let toto digits =
	module Fred = Makespecific(struct let n = digits end) in
	Fred.mirror ();;

which seems to be reasonably within the spirit and syntax of csl. 
Even if that is not possible, something else like:

   let toto digits =
	let u = Makespecific(struct let n = digits end).mirror () in
	u
   ;;

would also be useful although not an entirely satisfactory solution.

Is this plausible? I seem to remember a discussion something along this
line a while ago, but it may have been for sml.

Thanks for making csl available,

Andrew.

Re: modules local to functions.

[Christian Boos]

 > I have a functor such as follows:
 > 
 >   module type Specifyn = sig val n : int end;;
 > 
 >   module Makespecific(N:Specifyn) = struct
 > 	type mytype = int
 > 	let mirror () : mytype = N.n
 >   end;;
 > 
 > I can apply it at the main level like:
 > 
 >   module Fred = Makespecific(struct let n = 7 end);;
 > 
 > However, there are cases when I want to define a module in the middle
 > of a function (eg a functor of packed binary arrays where I want to (...)


Hello,

I have encountered a similar problem yesterday! 
My understanding is that YOU CAN'T do that in CSL at all: functor
application means code generation, and there's no runtime code
generation!

One way to circumvent this problem is to change slightly the semantic
of your argument module, for example:

   module type Specifyn = sig val n : unit -> int end
				      ^^^^^^^

This should be not too inconvenient, and almost as simple to use as
yours:

    module Makespecific(N:Specifyn) = struct
       type mytype = int
       let mirror () : mytype = N.n ()
				    ^^
    end
    
    let toto_fred_n = ref 10
    
    module Fred = Makespecific(struct let n () = !toto_fred_n end) 
    
    let toto digits =
      toto_fred_n := digits;
      Fred.mirror ()


(This particular solution may even be expressed simpler:

    module type Specifyn = sig val n : int ref end
					   ^^^
    
    module Makespecific(N:Specifyn) = struct
       type mytype = int
       let mirror () : mytype = !N.n
				^
    end
    
    module Fred_N = struct let n = ref 10 end
    
    module Fred = Makespecific(Fred_N) 
    
    let toto digits =
      Fred_N.n := digits;
      Fred.mirror ()
	
)

Hope it helps!


Christian 

Re: modules local to functions.

[Andrew Conway]

I wrote:

> > However, there are cases when I want to define a module in the middle
> > of a function (eg a functor of packed binary arrays where I want to (...)

Christian Boos commented:

>I have encountered a similar problem yesterday! 
>My understanding is that YOU CAN'T do that in CSL at all: functor
>application means code generation, and there's no runtime code
>generation!

Ah. I expected (though I hadn't looked into the code at all) that
a run time structure creation (eg struct let n = variable end) was
very similar to the creation of a tuple/normal structure, just requiring
sticking some values into a block of memory. I then expected that a 
module creation from a functor was very similar to a partial function
application, (with perhaps a bit of execution) and could thus be done 
at run time (as distinct from, say C++ templates which produce specialised 
code for each application).

But it may well be significantly more complex than that.

> One can get around the problem by defining n as a function or reference
> and changing it dynamically.

Ugh! I hadn't thought of that, and it is a good idea even if just
a little inelegant. Unfortunately it is not appropriate in my case as
I have several values in the structure which get defined when the module
is created and which depend on n...ie:


    module Makespecific(N:Specifyn) = struct
       type mytype = int
       let computednumber = complexfunction N.n 
       let mirror () : mytype = computednumber
    end

and I only want to call complexfunction once for each module instantiation.

Of course, I could make "computednumber" a reference, and I could have
a function "change_n_and_recompute_everything", but again it is an inelegant
solution that is hard to understand, and I end up with "!"s all over the
code for no algorithmically justifyable reason.

Yeah yeah, I know I am being picky...

Thanks,

Andrew.

Re: modules local to functions.

[Xavier Leroy]

Andrew Conway wrote:

> However, there are cases when I want to define a module in the middle
> of a function [...] That is, I would like to be abe to do:
> 
>   let toto digits =
> 	module Fred = Makespecific(struct let n = digits end) in
> 	Fred.mirror ();;

Christian Boos replied:

> My understanding is that YOU CAN'T do that in CSL at all: functor
> application means code generation, and there's no runtime code
> generation!

This is wrong. Functors are compiled once and for all, and functor
application proceeds exactly as function application.

Ada generics and C++ templates are generally implemented by
recompilation as Christian described, but that's just because these
languages don't implement lexical closures in their full generality.

So, there is no deep reason why modules could not be defined locally
to a (core-language) expression. Standard ML allows this (the "let
structure" and "local" constructs). That was left out of Caml Special
Light for simplicity. In particular, not having module expressions
inside core-language expressions avoids mutual recursion between the
module language and the core language; this makes both the theory and
the implementation simpler.

I still need to be convinced of the usefulness of local functor
applications (Andrew Conway's example was too sketchy to illuminate me),
but it can definitely be done.

- Xavier Leroy

Re: modules local to functions.

[Andrew Conway]

I wrote a while ago:

> However, there are cases when I want to define a module in the middle
> of a function [...] That is, I would like to be abe to do:

Xavier Leroy replied:

>I still need to be convinced of the usefulness of local functor
>applications (Andrew Conway's example was too sketchy to illuminate me),
>but it can definitely be done.

OK, here comes my attempt.

I want to write a function that uses a large number of arrays as a
essential part of its process. These arrays contain small positive
values, in a range specified in the function (eg 0 to 2). As these
arrays are copied around very frequently, and memory consumption
is significant, it is worth while having them as a packed array.

So, I implemented a packed array functor that takes as arguement
the number of bits needed, and returns a module implementing that
packed array.

I would like the code to be as memory efficient as reasonably possible, and 
so, for instance, when the range is 0 to 1, the packed array should be of
just one bit per location.

[ I am using it in a dynamic programming problem. It already wants to use more
  than the available memory, so I have to sacrifice time for memory already.
  Packing from 2 bits to 1 bits roughly divides the execution time by a 
  factor of 4. And no, I can't see a significantly better algorithm 
  unfortunately, and it is not from a lack of looking. ]

There are other parameters to the function, and I would like to make
a table of the value of the function for all parameters. 

At the moment what I do is declare (at file scope) a module PA with
a fixed number of bits. I then generate that section of the table
that can be done efficiently with that sized packed array. I then 
edit the source, recompile with a different sized packed array, 
and print out another section of the table, etc.

This multiple-recompilation is an annoyance. In this particular case
it is not a major problem as I only want to do it a few times (and
a few times for debugging, and a few times when I change the function
slightly). However, it is a real example of when declaring a module
inside a function would significantly increase the straightforward 
expressability of the language.

[ Note: I am aware that there are lots of work arounds, and in particular
  that one could regard functors as syntactic sugar, but they are very nice
  syntactic sugar for a really ugly alternative. ]

Andrew.

Error message format (was Re: modules local to functions.)

[Christian Boos]

Xavier Leroy writes:
 > Christian Boos replied:
 > 
 > > My understanding is that YOU CAN'T do that in CSL at all: functor
 > > application means code generation, and there's no runtime code
 > > generation!
 > 
 > This is wrong. Functors are compiled once and for all, and functor
 > application proceeds exactly as function application.
 > 

[poor english version]

Hello,

  C++ has twisted my mind...

  I would make a suggestion about the information given by the
compiler when there is a mismatch between an interface and an
implementation.
  Usually, it ends up with something like :

	Values do not match:
	  val f : t1 -> t4 -> t5
	is not included in
	  val f : t1 -> t2 -> t3

	(for example)

  My point is that it would be usefull to be more precise, that is, to
say if the error is with t1 not included in t1, or t4 not included in
t2, ... 

Bye.


--------------------------------------------
Bonjour,

  Autant pour moi ! Ca m'apprendra a raisonner par analogie avec des
langages d'outre-tombe (C++) plutot que de lire le code de CSL avant
de parler ...


  Au passage, je profite de ce mail pour faire une petite suggestion
concernant le message d'erreur genere lorsqu'il y a incompatibilite
entre l'implementation et l'interface d'un module.

  Le message se termine usuellement en donnant une incompatibilite de
type ou une incompatibilite de valeur. Dans le cas des fonctions, ne
pourrait-il pas ajouter la localisation precise de l'erreur ?

  Par exemple, on specifie "sig val f : t1 -> t2 -> t3 end"
et dans l'implementation on construit une fonction f ayant un type
"val f : t1 -> t4 -> t5", avec t4 non compatible avec t2.
  Au lieu de s'arreter dans son message d'erreur a "val f : ... is not
included in val f : ...", il pourrait dire en plus "t4 is not included
in t2".  

 Ceci me semble tres utile car j'ai passe une apres-midi a essayer de
localiser une erreur de ce type, tout simplement parce que dans mon
cas, t4 et t5 etaient des types complexes (generes par des foncteurs)
que j'avais de facon alambiquee contraint a etre egaux a t2 et t3. 
  Je pensais donc que l'erreur venait de la ... alors qu'il s'agissait
d'une erreur toute bete entre t1 et t1 (memes noms, types differents),
erreur qui aurait ete localisee tout de suite si j'avais eu un message
comme en caml-light (la fameuse erreur "This expression has type foo,
but is used with type foo").  


Amicalement,

Christian

modules local to functions. (again)

[Christian Boos]

	Hello, 

  I want to restart the discussion about what the subject says (module
local to functions), because Xavier Leroy's conclusion (which I recall
below) was not satisfactory for me.

Xavier Leroy writes:
 > ...
 > 
 > I still need to be convinced of the usefulness of local functor
 > applications (Andrew Conway's example was too sketchy to illuminate me),
 > but it can definitely be done.
 > 

  Perhaps the following example will be more convincing !

  Let suppose that you fully play the game of modular programming,
that is, designing your algorithms in terms of independant parts,
basic algorithms (direct modules instantiations) and parametrable
algorithms (functors).
  In several situations, it may be usefull to instantiate functors
in many different, ways, by using various parameter modules.
Those different ways will produces different results, each equally
usefull in a particular, end-user-defined, context.
  For instance, you've got 3 modules of type T (A, B, C), four functors
of type T -> T' (F1, F2, F3, F4) and another functor G of type T' ->
T'', which produce the module you're interested in.
  Even in this small example, you've got 12 different ways to produce
a T'' module. 
  How to write a stand-alone program which allows the user to select
the kind of T'' functionnality he wants ?

  You may write:

    (* prog1.ml *)

    let base = ref "a" and fonct = ref 1
  
    module G_A1 = G(F1(A))
    module G_A2 = G(F2(A))
    ...                         (* nine other module instantiations *)
    module G_C4 = G(F4(C))
  
    let main = 
      Arg.parse [ "-base", String ((:=) base); "-f", Int ((:=) fonct)]
	  usage;
      let f =
	match (!base, !fonct) with
	  ("a", 1) -> G_A1.f
	| ("a", 2) -> G_A2.f
	| ...                   (* nine other selections *)
	| ("c", 4) -> G_C4.f
	| _ -> failwith "no such base modules or functors"
      in
      (* hopefully you don't need anything else from G ... *)
      f "youpi" ();
      exit 0


  You may also run in things such like automaticaly generate csl code
wich instantiates the module you need, compile it, and executing it (or
even dynamically load it); but that's hacking, not modular functionnal
programming.
  Neither solution is satisfactory.

  Let's cite X.L. again:

Xavier Leroy writes:
 > ...
 > 
 > So, there is no deep reason why modules could not be defined locally
 > to a (core-language) expression. Standard ML allows this (the "let
 > structure" and "local" constructs). That was left out of Caml Special
 > Light for simplicity. In particular, not having module expressions
 > inside core-language expressions avoids mutual recursion between the
 > module language and the core language; this makes both the theory and
 > the implementation simpler.
 > ...

  ... and make my two cents proposal:

  * allow local module instantiation, along with local "open"s
 
  * extend the langage such that with each expression <expr> is
    associated a module type.
	** existing <expr> construct will be associated the void
           module type (that is, the most general one: sig end). 

	** add the following constructs:

             'module <ModName>'
           which is a module constant, having 'unit' for regular
           (value) type, and it's own type module.  

             'module <ModName> = <expr> in <expr>'
           which is a module instantiation, also having 'unit' for
           regular type.

  Now, you have to perform two kinds of type-checking for expressions:
the usual one, plus the module type checking, but the later will be
trivial in almost all cases (matching 'sig end' against 'sig end').
  I'm not sure whether this proposal is realistic or not, but at
least, it provides an elegant way to rewrite the previous example
program:

    (* prog2.ml *)
    let base = ref "a" and fonct = ref 1
    let main = 
      Arg.parse [ "-base", String ((:=) base); "-f", Int ((:=) fonct)]
	  usage;
      module Base =
        match !base with
          "a"|"A" -> module A
        | "b"|"B" -> module B
        | "c"|"C" -> module C
        | _ -> failwith "unknown base module"
      in
      module F = 
        match !fonct with
          1 -> module F1
        | 2 -> module F2
        | 3 -> module F3
        | 4 -> module F4
        | _ -> failwith "unimplemented method"
      in
      module MyG = G(F(Base) in
      MyG.f "youpi again" 
        (MyG.g "now we can use everything from G without dispatching again !");
      exit 0



[French translation:  (:-) ]
  Pour une fois que je fais l'effort de tout ecrire directement en
anglais, vous pouvez bien faire l'effort de me lire en anglais, non ?



-- Christian Boos