[Caml-list] [ANN] ocamlopen 1.0.2

[Caml-list] [ANN] ocamlopen 1.0.2

[Leo P White]

Hi,

If anyone is interested, I have written a new version of my patch to 
add open extensible types to OCaml. It is available at:

https://sites.google.com/site/ocamlopen/

The only new feature is allowing variant declarations to be made 
extensible. This allows declarations like:

open type foo = A | B of int | ..

Having implemented it, I think that the extension might well be better 
off without this feature, so I am also releasing another version of 
the patch without it.

I have also written a much better example of how open types and GADTs 
might be used. It basically shows how classes can be created that 
permit a kind of nominative down-casting. I have included it below.

Finally, I have also added a feature request to Mantis if anyone would 
like to comment.

http://caml.inria.fr/mantis/view.php?id=5584

Regards,

Leo

--------------------------------

open type 'a class_name

exception Bad_cast

class type castable =
object
  method cast: 'a.'a class_name -> 'a
end

(* Lets create a castable class with a name*)

class type foo_t =
object
  inherit castable
  method foo: string
end

extend 'a class_name with Foo: foo_t class_name

class foo: foo_t =
object(self)
  method cast: type a. a class_name -> a =
    function
       Foo -> (self : #foo_t :> foo_t)
      | _ -> ((raise Bad_cast) : a)
  method foo = "foo"
end

(* Now we can create a subclass of foo *)

class type bar_t =
object
  inherit foo
  method bar: string
end

extend 'a class_name with Bar: bar_t class_name

class bar: bar_t =
object(self)
  inherit foo as super
  method cast: type a. a class_name -> a =
    function
        Bar -> (self : #bar_t :> bar_t)
      | other -> super#cast other
  method bar = "bar"
end

(* Now lets create a mutable list of castable objects *)

let clist :castable list ref = ref []

let push_castable (c: #castable) =
  clist := (c :> castable) :: !clist

let pop_castable () =
  match !clist with
      c :: rest ->
        clist := rest;
        c
    | [] -> raise Not_found;;

(* We can add foos and bars to this list, and retrive them *)

push_castable (new foo);;
push_castable (new bar);;
push_castable (new foo);;

let c1: castable = pop_castable ()
let c2: castable = pop_castable ()
let c3: castable = pop_castable ()

(* We can also downcast these values to foos and bars *)

let f1: foo = c1#cast Foo
let f2: foo = c2#cast Foo
let f3: foo = c3#cast Foo

let b2: bar = c2#cast Bar

Re: [Caml-list] [ANN] ocamlopen 1.0.2

[Gabriel Scherer]

Is there a description somewhere of the semantics of your language change?


More specifically:

1. You link to a paper by Andres Löh and Ralf Hinze, how close are you
of their proposal?
They mention open datatypes and functions, you propose open datatypes,
but not open functions? That would be understandable because open
pattern matching is a bit fishy (best-fit matching etc...).

2. What is the difference between your "open" and "extensible"
datatypes? The visual difference is the present of initial
constructors in extensible cases, but you apparently make much finer
distinctions.

3. What is the semantics of making a *constructor* private? My
intuition of private types is that (type t = private u) generates a
new type t that is a strict subtype of u (values of type t can be
coerced into u, but not the other way around). This intuition does not
hold anymore if some constructors are marked private, but not the
other.


I have some further questions:

4. When would you say that one should use polymorphic variants rather
than your open datatypes? (I know how to argue in the other direction:
unique constructors make for better error messages.)

5. What are the implications of your patch at the runtime / data
representation level? Could you elaborate a bit more on "a new tag to
represent extensions"? Have you conducted performance measurements?


(I find your work interesting. I have not looked at the code yet,
because I think someone should be able to get answer to the questions
above without having to look at the patch.)

On Wed, Apr 11, 2012 at 11:13 PM, Leo P White <lpw25@cam.ac.uk> wrote:
> Hi,
>
> If anyone is interested, I have written a new version of my patch to add
> open extensible types to OCaml. It is available at:
>
> https://sites.google.com/site/ocamlopen/
>
> The only new feature is allowing variant declarations to be made extensible.
> This allows declarations like:
>
> open type foo = A | B of int | ..
>
> Having implemented it, I think that the extension might well be better off
> without this feature, so I am also releasing another version of the patch
> without it.
>
> I have also written a much better example of how open types and GADTs might
> be used. It basically shows how classes can be created that permit a kind of
> nominative down-casting. I have included it below.
>
> Finally, I have also added a feature request to Mantis if anyone would like
> to comment.
>
> http://caml.inria.fr/mantis/view.php?id=5584
>
> Regards,
>
> Leo
>
> --------------------------------
>
> open type 'a class_name
>
> exception Bad_cast
>
> class type castable =
> object
>  method cast: 'a.'a class_name -> 'a
> end
>
> (* Lets create a castable class with a name*)
>
> class type foo_t =
> object
>  inherit castable
>  method foo: string
> end
>
> extend 'a class_name with Foo: foo_t class_name
>
> class foo: foo_t =
> object(self)
>  method cast: type a. a class_name -> a =
>   function
>      Foo -> (self : #foo_t :> foo_t)
>     | _ -> ((raise Bad_cast) : a)
>  method foo = "foo"
> end
>
> (* Now we can create a subclass of foo *)
>
> class type bar_t =
> object
>  inherit foo
>  method bar: string
> end
>
> extend 'a class_name with Bar: bar_t class_name
>
> class bar: bar_t =
> object(self)
>  inherit foo as super
>  method cast: type a. a class_name -> a =
>   function
>       Bar -> (self : #bar_t :> bar_t)
>     | other -> super#cast other
>  method bar = "bar"
> end
>
> (* Now lets create a mutable list of castable objects *)
>
> let clist :castable list ref = ref []
>
> let push_castable (c: #castable) =
>  clist := (c :> castable) :: !clist
>
> let pop_castable () =
>  match !clist with
>     c :: rest ->
>       clist := rest;
>       c
>   | [] -> raise Not_found;;
>
> (* We can add foos and bars to this list, and retrive them *)
>
> push_castable (new foo);;
> push_castable (new bar);;
> push_castable (new foo);;
>
> let c1: castable = pop_castable ()
> let c2: castable = pop_castable ()
> let c3: castable = pop_castable ()
>
> (* We can also downcast these values to foos and bars *)
>
> let f1: foo = c1#cast Foo
> let f2: foo = c2#cast Foo
> let f3: foo = c3#cast Foo
>
> let b2: bar = c2#cast Bar
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

Re: [Caml-list] [ANN] ocamlopen 1.0.2

[Alain Frisch]

I'll let Leo answer all your questions.  Here are just my 2 cents:

On 04/12/2012 02:56 PM, Gabriel Scherer wrote:
> 3. What is the semantics of making a *constructor* private?

I guess the idea is to allow pattern matching on that constructor, but 
not using it in positive position (to construct a value).

> 4. When would you say that one should use polymorphic variants rather
> than your open datatypes? (I know how to argue in the other direction:
> unique constructors make for better error messages.)

I've wanted such open datatypes several times.  One example is a message 
bus across an application: some components can yield messages to be 
dispatched to all registered components.  Messages can hold data, and 
the set of possible messages (with the type of their associated data) is 
extensible (say, because components can be loaded dynamically, or just 
to make the application's architecture more modular).  It makes sense to 
use an extensible datatype to represent messages.  Components can react 
to messages with pattern-matching and two components can interact if 
their share a common constructor definition.  This is simpler than 
encoding open datatypes with a "universal" type (with 
injections/projections).  Of course, one can use the existing "exn" 
type, but then we don't distinguish between exceptions and messages at all.


Alain

Re: [Caml-list] [ANN] ocamlopen 1.0.2

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 1399 bytes --]

>
> 4. When would you say that one should use polymorphic variants rather
>> than your open datatypes? (I know how to argue in the other direction:
>> unique constructors make for better error messages.)
>>
>
> I've wanted such open datatypes several times.  One example is a message
> bus across an application: some components can yield messages to be
> dispatched to all registered components.  Messages can hold data, and the
> set of possible messages (with the type of their associated data) is
> extensible (say, because components can be loaded dynamically, or just to
> make the application's architecture more modular).  It makes sense to use
> an extensible datatype to represent messages.  Components can react to
> messages with pattern-matching and two components can interact if their
> share a common constructor definition.  This is simpler than encoding open
> datatypes with a "universal" type (with injections/projections).  Of
> course, one can use the existing "exn" type, but then we don't distinguish
> between exceptions and messages at all.
>

Isn't this a good use case for polymorphic variants too ? Compared to open
datatypes, you could suffer from the weaker type checks though: if two
components are supposed to communicate through a constructor, which is
mispelled in one of the component, the error would not be noticed by the
compiler.

my own 2 cent ...
philippe.

[-- Attachment #2: Type: text/html, Size: 1805 bytes --]

Re: [Caml-list] [ANN] ocamlopen 1.0.2

[Alain Frisch]

On 04/12/2012 04:16 PM, Philippe Veber wrote:
> Isn't this a good use case for polymorphic variants too ?

I don't see how to use polymorphic variants here.  The message bus 
itself need to provide functions like:

   val dispatch: message -> unit
   val register_listener: (message -> unit) -> unit


How do you define the message type without having access to all possible 
message constructors?


-- Alain

Re: [Caml-list] [ANN] ocamlopen 1.0.2

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 1682 bytes --]

2012/4/12 Alain Frisch <alain@frisch.fr>

> On 04/12/2012 04:16 PM, Philippe Veber wrote:
>
>> Isn't this a good use case for polymorphic variants too ?
>>
>
> I don't see how to use polymorphic variants here.  The message bus itself
> need to provide functions like:
>
>  val dispatch: message -> unit
>  val register_listener: (message -> unit) -> unit
>
>
> How do you define the message type without having access to all possible
> message constructors?

I reckon you can't, but I mistakenly thought you could at least define the
bus without writing the full message type:

module Bus : sig
  type 'a t
  val create : unit -> 'a t
  val dispatch : 'a t -> 'a -> unit
  val register : 'a t -> ('a -> unit) -> unit
end
=
struct
  type 'a t = ('a -> unit) list ref
  let create () = ref []
  let dispatch bus msg =
    List.iter (fun f -> f msg) !bus
  let register bus f =
    bus := f :: !bus
end


let bus = Bus.create ()

let () = Bus.register bus (function `a -> print_char 'a')
let () = Bus.register bus (function `b n -> print_int n)

However this is not a legal program:

  let () = Bus.register bus (function `b n -> print_int n);;
                                      ^^^^
Error: This pattern matches values of type [< `b of 'a ]
       but a pattern was expected which matches values of type [< `a ]
       These two variant types have no intersection

Well, I mentionned polymorphic variants because you can at least define
listeners without knowing the full message type and even reuse them for
buses with different message types. So you have a certain flexibility,
compared to monomorphic types.

But ok, maybe the remark wasn't even worth 2 cents :o).

Cheers,
ph.

[-- Attachment #2: Type: text/html, Size: 4493 bytes --]

Re: [Caml-list] [ANN] ocamlopen 1.0.2

[Leo P White]

>1. You link to a paper by Andres Löh and Ralf Hinze, how close are you
>of their proposal?
>They mention open datatypes and functions, you propose open datatypes,
>but not open functions? That would be understandable because open
>pattern matching is a bit fishy (best-fit matching etc...).

I think that my open datatypes are basically the same as they propose, 
which is pretty much the same as the behaviour of exn in OCaml. 

>2. What is the difference between your "open" and "extensible"
>datatypes? The visual difference is the present of initial
>constructors in extensible cases, but you apparently make much finer
>distinctions.

This complication arises from the existence "extensible variant 
declarations", and is one of the reasons that I think the extension 
is probably better without them.

Basically, by an "extensible" type I mean one for which not all the 
constructors are given in the type declaration. Obviously all abstract 
types in a signature are extensible, because they may be implemented 
using a variant type. My extension allows variant type declarations to 
also be made extensible using the syntax:

type Foo = A | B of int | ..

Note that this extensibility cannot be hidden by a signature, the 
compiler needs to know if a variant is extensible in order to properly 
implement pattern matching.

By an "open" type I mean one that is allowed to be extended using an 
extension definition. Only extensible types can be declared open, 
becuase an extension definition adds constructors to a type that are 
not mentioned in its declaration. Unlike extensibility, the openness 
of a type can be hidden using a signature.


>3. What is the semantics of making a *constructor* private? My
>intuition of private types is that (type t = private u) generates a
>new type t that is a strict subtype of u (values of type t can be
>coerced into u, but not the other way around). This intuition does not
>hold anymore if some constructors are marked private, but not the
>other.

As Alain said, making a constructor private allows it to be used for 
pattern matching but not to create a value. This is an alternative 
intuition for how private variant types already work. 

I think that allowing private extensions could definitly be useful. 
Borrowing Alain's example of a messsage bus, if a message producer 
used a private message constructor then it could enusre that it was 
the only component producing messages with that constructor.

Note that the extension does not allow private abstract types to be 
declared open, because this would break the standard intuition for 
them.

However, it does allow private variant declarations to be made open 
and treats this as meaning that each of its ordinary constructors 
is private, but that public extensions *can* still be created and 
used in positive positions. I think that this is a bit of a muddle, 
and possibly another reason that extensible variant declarations 
should be left out.

>4. When would you say that one should use polymorphic variants rather
>than your open datatypes? (I know how to argue in the other direction:
>unique constructors make for better error messages.)

A good reason for using polymorphic variants is to create multiple types 
that share constructors. For instance a compiler might want one type to 
represent expressions and another to represent constant expressions. 
Using polymorphic variants constant expression values could be used 
directly as expressions values. This wouldn't be possible using open 
datatypes.

Note that there are other reasons to use open datatypes over polymorphic 
variants: they can be used properly with references, they can be 
controlled using modules and they can be GADTs.

>5. What are the implications of your patch at the runtime / data
>representation level? Could you elaborate a bit more on "a new tag to
>represent extensions"? Have you conducted performance measurements?

Just like exceptions, they are represented as constructors whose 
first field points to an address that is allocated by the extension 
definition to represent that extension. They have a special tag value 
so that structural equality knows to compare the first fields by 
address. Note that this is exactly what is required to fix the bug 
with structural equality on exceptions (4765).

I haven't conducted any performance measurements, but using extensions 
should be similar in cost to using exceptions. The ordinary variants 
in an extensible variant declaration should perform similarly to any 
other ordinary variants. 

Thanks for the interest.

Regards,

Leo

Re: [Caml-list] [ANN] ocamlopen 1.0.2

[Alain Frisch]

On 04/12/2012 07:07 PM, Leo P White wrote:
> Just like exceptions, they are represented as constructors whose first
> field points to an address that is allocated by the extension definition
> to represent that extension. They have a special tag value so that
> structural equality knows to compare the first fields by address. Note
> that this is exactly what is required to fix the bug with structural
> equality on exceptions (4765).

Do we really need a new special tag?  Why not use Object_tag and 
represent slots as blocks of size 2 (constructor name + unique integer 
as the second field)?  This would have the following consequences (which 
are the expected ones):

  - The generic equality function compares the unique ids.

  - The generic hash function returns the unique id.

  - The generic unmarshaling function allocates a fresh id when 
umarshaling such a block.


Alain

Re: [Caml-list] [ANN] ocamlopen 1.0.2

[Leo P White]

On Apr 12 2012, Alain Frisch wrote:

>Do we really need a new special tag?  Why not use Object_tag and 
>represent slots as blocks of size 2 (constructor name + unique integer 
>as the second field)?  This would have the following consequences (which 
>are the expected ones):
>

I hadn't really considered that option, but it seems like that would 
probably work. It would mean a slightly slower structural comparison, but 
on the other hand it would produce better hashes and might allow slightly 
quicker pattern matching in some specific cases.

Regards,

Leo