[Caml-list] 3.13.0 stricter on interfaces? (building extlib)

[Caml-list] 3.13.0 stricter on interfaces? (building extlib)

["Markus W. Weißmann"]

Hi everyone,

I just tried to build extlib with the most recent checkout of ocaml (rev 11286):
The build fails on the extHashtbl.ml module:

---
Error: The implementation extHashtbl.ml
       does not match the interface extHashtbl.cmi:
       ...
       In module Hashtbl:
       Values do not match:
         val create : ?seed:int -> int -> ('a, 'b) t
       is not included in
         val create : int -> ('a, 'b) t
---

As far as I understand the problem, extlib's Hashtbl claims to implement the Hashtbl-interface from the standard library.
Previous OCaml versions were ok with extlib's "create" function having an additional optional parameter, but 3.13. seems to think different.
Is this on purpose or a bug?


Regards

-Markus

-- 
Markus Weißmann, M.Sc.
Technische Universität München
Institut für Informatik
Boltzmannstr. 3
D-85748 Garching
Germany
http://wwwknoll.in.tum.de/

Re: [Caml-list] 3.13.0 stricter on interfaces? (building extlib)

[Gabriel Scherer]

There is a slighlt misinterpretation in your analysis of the issue:
the problem comes from the fact that OCaml 3.13 standard library has
added a ?seed optional parameter to the Hashtbl.create
method. Extlib's interface still assumes <=3.12 interface with no
parameter (and in particular didn't change this function interface),
and tries to check the new Hashtbl.create standard function with the
old (int -> ('a,'b) t) interface.

In particular, this problem does *not* come from a change of behavior
of the typer, only from libraries changing.
(And the change is on stdlib's side, not extlib's.)

The OCaml manual is not very explicit about implicit coercions
regarding optional arguments:

> Normally the compiler generates a type error if you attempt to pass
> to a function a parameter whose type is different from the expected
> one. However, in the specific case where the expected type is
> a non-labeled function type, and the argument is a function
> expecting optional parameters, the compiler will attempt to
> transform the argument to have it match the expected type, by
> passing None for all optional parameters.

It seems that in some cases it is able to convert from `?x:foo -> bar`
to `bar`. Indeed:

  # (fun (f : int -> int) -> f 3) (fun ?(x=0) y -> x+y);;
  - : int = 3

However, this doesn't seem to work when checking interfaces:

  # module Test : sig val f : int -> int end
                = struct let f ?(x=0) y = x+y end;;
  Error: Signature mismatch: [...]

(This was tested under 3.12.0)

I personally try to avoid adding optional parameters silently as
I have already been bitten by such issues in the past. However I don't
really understand (or try to understand) how they are typed, and have
also been wrong in predicting interface breakage that somehow didn't
happen.

Here are are a few ways this issue could be resolved:

1. the stdlib could change to revert to the old interface, adding
  a `create_seed` function

2. extlib could introduce a conditional compilation trick to support
  either interfaces depending on OCaml version; ugly and painful to
  deal with in ocamldoc

3. instead of repeating `val create : int -> ('a, 'b) t` in its own
  interface, extlib could include (module type of Hashtbl) (referring
  to stdlib's Hashtbl module). This way, their interface would keep in
  synch with stdlib's change, and they would have to change their
  implementation correspondingly.

(3) is not perfect from a compatiblity point of view: if you introduce
this level of coupling, you get the property that the Extlib library
released at time T is always compatible with the OCaml library
released at time T, but you don't necessarily get that it is
compatible with previous Extlib releases (eg. if INRIA's stdlib adds
a function that didn't exist in stdlib but was present in Extlib with
a different signature).

Besides, an issue with (3) is that it requires 3.12 to compile. Extlib
developpers may wish to keep future versions of Extlib usable under
older versions of OCaml.


On Sat, Nov 26, 2011 at 10:52 AM, "Markus W. Weißmann"
<markus.weissmann@in.tum.de> wrote:
> Hi everyone,
>
> I just tried to build extlib with the most recent checkout of ocaml (rev 11286):
> The build fails on the extHashtbl.ml module:
>
> ---
> Error: The implementation extHashtbl.ml
>       does not match the interface extHashtbl.cmi:
>       ...
>       In module Hashtbl:
>       Values do not match:
>         val create : ?seed:int -> int -> ('a, 'b) t
>       is not included in
>         val create : int -> ('a, 'b) t
> ---
>
> As far as I understand the problem, extlib's Hashtbl claims to implement the Hashtbl-interface from the standard library.
> Previous OCaml versions were ok with extlib's "create" function having an additional optional parameter, but 3.13. seems to think different.
> Is this on purpose or a bug?
>
>
> Regards
>
> -Markus
>
> --
> Markus Weißmann, M.Sc.
> Technische Universität München
> Institut für Informatik
> Boltzmannstr. 3
> D-85748 Garching
> Germany
> http://wwwknoll.in.tum.de/
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>

Re: [Caml-list] 3.13.0 stricter on interfaces? (building extlib)

[Jacques Garrigue]

On 2011/11/26, at 19:33, Gabriel Scherer wrote:

> There is a slighlt misinterpretation in your analysis of the issue:
> the problem comes from the fact that OCaml 3.13 standard library has
> added a ?seed optional parameter to the Hashtbl.create
> method. Extlib's interface still assumes <=3.12 interface with no
> parameter (and in particular didn't change this function interface),
> and tries to check the new Hashtbl.create standard function with the
> old (int -> ('a,'b) t) interface.
> 
> In particular, this problem does *not* come from a change of behavior
> of the typer, only from libraries changing.
> (And the change is on stdlib's side, not extlib's.)

Indeed, the compiler never erases optional arguments to match
a specific signature.

> The OCaml manual is not very explicit about implicit coercions
> regarding optional arguments:
> 
>> Normally the compiler generates a type error if you attempt to pass
>> to a function a parameter whose type is different from the expected
>> one. However, in the specific case where the expected type is
>> a non-labeled function type, and the argument is a function
>> expecting optional parameters, the compiler will attempt to
>> transform the argument to have it match the expected type, by
>> passing None for all optional parameters.
> 
> It seems that in some cases it is able to convert from `?x:foo -> bar`
> to `bar`. Indeed:
> 
>  # (fun (f : int -> int) -> f 3) (fun ?(x=0) y -> x+y);;
>  - : int = 3
> 
> However, this doesn't seem to work when checking interfaces:

Indeed again, as written in the manual, this specific case only
concerns functions passed as function argument. This is not
the case of exports from a module.
More precisely, in 3.12 this applies only to function arguments
and record fields, in trunk this applies also to variant arguments
(both normal and polymorphic) and to explicit type annotations.

> Here are are a few ways this issue could be resolved:
> 
> 1. the stdlib could change to revert to the old interface, adding
>  a `create_seed` function
> 
> 2. extlib could introduce a conditional compilation trick to support
>  either interfaces depending on OCaml version; ugly and painful to
>  deal with in ocamldoc
> 
> 3. instead of repeating `val create : int -> ('a, 'b) t` in its own
>  interface, extlib could include (module type of Hashtbl) (referring
>  to stdlib's Hashtbl module). This way, their interface would keep in
>  synch with stdlib's change, and they would have to change their
>  implementation correspondingly.

There is another solution which works with all versions of OCaml

4. In extHashtbl.ml, add the following line

	let create x = create x

   This will discard optional arguments.
   People who need the full version can get it from the standard library.


Jacques Garrigue