[Caml-list] Strange crash after switching to first class modules

[Caml-list] Strange crash after switching to first class modules

[John Carr]

One sentence summary: I added a local module generated by
functor application to an unpacked first class module and
the runtime blew up.

My program segfaults in the OCaml runtime whether compiled with ocamlc
or ocamlopt.  I am not using any constructs I know to be unsafe and I am
not linking C code except for the OCaml runtime.

The proximate cause is a bad channel object passed to caml_ml_output.
It does not point to a valid struct channel.  The bad channel is
supposed to be stdout or stderr.  It is neither.  The channel objects
corresponding to stdout and stderr are still valid.

GC is not responsible.  The crash is consistent regardless of changes to
input data size or minor heap size.

This started after I rewrote part of my program to use first class
modules.  The latest svn version "3.13.0+dev7 (2011-09-22)" fails
exactly the same way.

The last module code I added was a local module generated by applying
a functor to a module argument, like

	let solve (type z) problem =
	  let module CHash = Hashtbl.Make((val problem.cmod : C with type t = z) in
	  ...

(The need for the type parameter is annoying.  In context it could
easily be deduced by the compiler.  Maybe there's a more complicated
example where typing is undecidable without hints.)

The last active code in my source is a function that looks like this
in relevant part:

	let format chan x =
	  let f = Printf.fprintf chan "%s %d" in
	    begin match x with
	      | W (n,false) -> f "string" n
	    end

Note the partial application of fprintf.   I raised an exception from
caml_ml_output and the stack trace points to printf.ml:611, which is

    and cont_s n s i =
      outs out s; doprn n i

The bytecode instruction that fails is C_CALL4 calling caml_ml_output
with a bad channel, a good string, start=0, length=9.

The channel argument is in the young heap (caml_page_table_lookup = 2)
and has tag word 0xb00 (tag=0, color=3, size=2).

Any advice on debugging this?

I can't share full source code but I can try to cut down the program.

Re: [Caml-list] Strange crash after switching to first class modules

[Xavier Leroy]

On 09/29/2011 06:03 PM, John Carr wrote:

> One sentence summary: I added a local module generated by
> functor application to an unpacked first class module and
> the runtime blew up. [...]
> The last active code in my source is a function that looks like this
> in relevant part:
> 
> 	let format chan x =
> 	  let f = Printf.fprintf chan "%s %d" in
> 	    begin match x with
> 	      | W (n,false) -> f "string" n
> 	    end
> 
> Note the partial application of fprintf.

Could you first try to eliminate that partial application of fprintf?
(Inline f at point of call, or eta-expand yourself.)  The Printf
module takes a lot of liberties with the type system (ahem), so it is
one of the usual suspects in such a case.

- Xavier Leroy

Re: [Caml-list] Strange crash after switching to first class modules

[John Carr]

Xavier Leroy <Xavier.Leroy@inria.fr> wrote:

> Could you first try to eliminate that partial application of fprintf?
> (Inline f at point of call, or eta-expand yourself.)  The Printf
> module takes a lot of liberties with the type system (ahem), so it is
> one of the usual suspects in such a case.

The program still crashes if I eta expand.  But it has a different
stack trace due to different tail calls and I think I understand
the problem:

Unpacking a module to provide a functor argument does not permute fields
to match the required signature.

I have a module that I use to represent another module plus another bit
of information paired with the base type of the other module.

	module M : C with type t = t1 * t2 = struct
	  type t = t1 * t2
	  let format chan (a,b) = ... (* format first part then second part *)
	  and f1 (a1,b1) (a2,b2) = ...
	  and f2 (a1,b1) (a2,b2) = ...
	  and equal (a1,b1) (a2,b2) = ...
	  and hash (a1,b2) = ...
	end

This module, packed into a value and unpacked, is the argument to the
Hashtbl.Make functor.

The stack trace goes from

  Hashtbl.MakeSeeded.mem -> M.format -> another format function -> crash

Hashtbl.mem doesn't call the format method because there is no format
type in the functor argument.  It calls equal.  It reads the first
value in the module expecting an equal method and finding something
else.

I think the module value is not being reformatted to the type expected
by Hashtbl.Make.

Is this enough of a clue or do you want me to try cutting it down to
a smaller example?