Exception Unix_error problem in toplevel

Exception Unix_error problem in toplevel

[Bob Williams]

Summary of problem: If you call a function from a script, that
function will raise and catch Unix_error properly.  However, if you
call the same function interactively from an OCaml toplevel, the
function will see an exception but will think it is *not* Unix_error.
A demonstration appears below.


OCaml version :  3.09.2-9
Distribution  :  Debian Linux for i386
Linux kernel  :  2.6.18

Here is file "bug.ml":

	open Unix
	open Printf
	
	let delete_file () =
	  try
	    unlink "no_such"
	  with
	    | Unix_error (_, _, _) -> Printf.eprintf "Unix error.\n%!"
	    | _                    -> Printf.eprintf "Some other error.\n%!"

Here is the script "run.ml":

	Bug.delete_file ()

The following works properly:

	$ ocamlc -c bug.ml 
	$ ocaml unix.cma bug.cmo run.ml
	Unix error.

The output "Unix error" is correct because the file named "no_such"
does not exist.  (By the way, "unix.cma" is OCaml's standard Unix
library.)

Now I omit the script "run.ml" from the ocaml command line and call
Bug.delete_file interactively, as follows:

	$ ocaml unix.cma bug.cmo
	        Objective Caml version 3.09.2
	
	# Bug.delete_file ();;
	Some other error.

The output should be the same as for the first example, but it isn't.
Strange, isn't it?

Re: [Caml-list] Exception Unix_error problem in toplevel

[ls-ocaml-developer-2006]

Bob Williams <a6a37331@telus.net> writes:


Bob Williams <a6a37331@telus.net> writes:

>
> OCaml version :  3.09.2-9
> Distribution  :  Debian Linux for i386
> Linux kernel  :  2.6.18
>



> Summary of problem: If you call a function from a script, that
> function will raise and catch Unix_error properly.  However, if you
> call the same function interactively from an OCaml toplevel, the
> function will see an exception but will think it is *not* Unix_error.
> A demonstration appears below.

<...>

>
> 	$ ocaml unix.cma bug.cmo
> 	        Objective Caml version 3.09.2
> 	
> 	# Bug.delete_file ();;
> 	Some other error.


I couldn't reproduce that on Debian Sarge (Stable is Sarge as of this
date) neither with

 - Objective Caml version 3.08.3 -- as distributed with Sarge.

nor with

 - Objective Caml version 3.09.2 -- as compiled by me from GODI 2
   months ago.

Are you running Testing or something like this? Since I assume that
exceptions are represented just by integer "tags" at the VM level:
Might there be a library mismatch in your case? (This is a question to
the experts here, since my understanding was that that is prevented by
checksums in the interfaces).

Whatever: If your ocaml comes from Debian (and not from GODI or
similar), I suggest you report a bug to the Debian BTS, too. It woul
be terrible, if we had to live with a buggy OCaml for the next 3 years
:-) after the Etch release.

Regards -- Markus

Re: [Caml-list] Exception Unix_error problem in toplevel

[Vu Ngoc San]

I cannot reproduce the error. I have ocaml-3.09.2, Linux debian 
2.6.17-2-686  (unstable)

Re: [Caml-list] Exception Unix_error problem in toplevel

[Luc Maranget]

> > Summary of problem: If you call a function from a script, that
> > function will raise and catch Unix_error properly.  However, if you
> > call the same function interactively from an OCaml toplevel, the
> > function will see an exception but will think it is *not* Unix_error.
> > A demonstration appears below.
> 

> <...>
> 
> >
> > 	$ ocaml unix.cma bug.cmo
> > 	        Objective Caml version 3.09.2
> > 	
> > 	# Bug.delete_file ();;
> > 	Some other error.
> 
> 
> I couldn't reproduce that on Debian Sarge (Stable is Sarge as of this
> date) neither with


May I suggest that Bob Williams reproduces the bug
with the following bug.ml file, and that he reports results.



open Unix
open Printf

let delete_file () =
  try
    unlink "no_such"
  with
  | Unix_error (_, _, _) -> Printf.eprintf "Unix error.\n%!"
  | e                    ->
      Printf.eprintf "Some other exception %s\n" (Printexc.to_string  e)


--Luc

Re: [Caml-list] Exception Unix_error problem in toplevel

[Bob Williams]

On Fri, Feb 02, 2007 at 01:08:20PM +0100, ls-ocaml-developer-2006@m-e-leypold.de wrote:
> Bob Williams <a6a37331@telus.net> writes:
>
> I couldn't reproduce that on Debian Sarge (Stable is Sarge as of this
> date) neither with
> 
>  - Objective Caml version 3.08.3 -- as distributed with Sarge.
> 
> nor with
> 
>  - Objective Caml version 3.09.2 -- as compiled by me from GODI 2
>    months ago.
> 
> Are you running Testing or something like this? Since I assume that
> exceptions are represented just by integer "tags" at the VM level:
> Might there be a library mismatch in your case? (This is a question to
> the experts here, since my understanding was that that is prevented by
> checksums in the interfaces).
> 
> Whatever: If your ocaml comes from Debian (and not from GODI or
> similar), I suggest you report a bug to the Debian BTS, too. It woul
> be terrible, if we had to live with a buggy OCaml for the next 3 years
> :-) after the Etch release.
> 
> Regards -- Markus

Thank you, Markus, and thanks to everyone else who has tried to help.
Special thanks go to Daniel de Rauglaudre, who tried hard to reproduce
the problem.

The explanation for the weird behavior turns out to be partly my
fault, and partly a possible real bug in toplevel.

It turns out that

	ocaml unix.cma bug.cmo

loads "unix.cma" twice, because I have an .ocamlinit (left over from a
previous project) that also loads "unix.cma"!  My stupid fault, mea
culpa.

Now to the possible bug in toplevel.  According to section 6.8.2 of
the OCaml manual,

	exception Unix_error of ...

simply declares another variant of the built-in variant type "exn".  I
haven't looked into the implementation of OCaml, but I assume the
compiler statically assigns a tag number to every variant of a normal
variant type.  For exn, however, the tag numbers must be assigned at
load time or run time.

And that is the problem I stumbled over.  The second copy of
"unix.cma" allocates a new tag number for exception Unix_error.
However, the tag number of the *first* copy of Unix_error has already
been bound into "bug.cmo".  So when the second Unix_error is raised,
"bug.cmo" catches it but fails to see it as a Unix_error. 

My first impulse was to suggest that toplevel disallow the
redefinition of anything.  But Daniel has convinced me that reloading
an object file can be useful during development.  So how about this:
when toplevel redefines an exception, the exception's tag number
should not change.

Daniel mentioned a related problem, with similar pitfalls:

	type t = C;;
	... define function Foo using t ...
	type t = D;;
	... what should we do about Foo? ...

Foo is now clearly obsolete, as are any blocks on the heap that were
constructed by C.  What should be done here?

Re: [Caml-list] Exception Unix_error problem in toplevel

[Gabriel Kerneis]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

Le Fri, 2 Feb 2007 18:29:56 -0800, Bob Williams <a6b37331@telus.net> a
écrit :
> My first impulse was to suggest that toplevel disallow the
> redefinition of anything.  But Daniel has convinced me that reloading
> an object file can be useful during development.  So how about this:
> when toplevel redefines an exception, the exception's tag number
> should not change.
> 
> Daniel mentioned a related problem, with similar pitfalls:
> 
> 	type t = C;;
> 	... define function Foo using t ...
> 	type t = D;;
> 	... what should we do about Foo? ...
> 
> Foo is now clearly obsolete, as are any blocks on the heap that were
> constructed by C.  What should be done here?

As you said, redefinition can be useful sometimes. But in most of the
cases, it's a bug. So why not make the compiler issue a (possibly
optional) warning on redefinitions ? 

Sincerely,
-- 
Gabriel Kerneis


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [Caml-list] Exception Unix_error problem in toplevel

[ls-ocaml-developer-2006]

Bob Williams <a6b37331@telus.net> writes:

> Now to the possible bug in toplevel.  According to section 6.8.2 of
> the OCaml manual,
>
> 	exception Unix_error of ...
>
> simply declares another variant of the built-in variant type "exn".  I
> haven't looked into the implementation of OCaml, but I assume the
> compiler statically assigns a tag number to every variant of a normal
> variant type.  For exn, however, the tag numbers must be assigned at
> load time or run time.
>
> And that is the problem I stumbled over.  The second copy of

Oh cool. :-). There should be RAQ (rarely asked questions + obscure
problems). This should go into it :-).

Regards -- Markus

Re: [Caml-list] Exception Unix_error problem in toplevel

[Bob Williams]

On Sat, Feb 03, 2007 at 10:57:50AM +0100, Gabriel Kerneis wrote:
> Le Fri, 2 Feb 2007 18:29:56 -0800, Bob Williams <a6b37331@telus.net> a
> ?crit :
> > My first impulse was to suggest that toplevel disallow the
> > redefinition of anything.  But Daniel has convinced me that reloading
> > an object file can be useful during development.  So how about this:
> > when toplevel redefines an exception, the exception's tag number
> > should not change.
> > 
> > Daniel mentioned a related problem, with similar pitfalls:
> > 
> > 	type t = C;;
> > 	... define function Foo using t ...
> > 	type t = D;;
> > 	... what should we do about Foo? ...
> > 
> > Foo is now clearly obsolete, as are any blocks on the heap that were
> > constructed by C.  What should be done here?
> 
> As you said, redefinition can be useful sometimes. But in most of the
> cases, it's a bug. So why not make the compiler issue a (possibly
> optional) warning on redefinitions ? 
> 
> Sincerely,
> -- 
> Gabriel Kerneis

I think that is a good idea.  Perhaps the warning should be issued
only when the new type differs from the old type.

Re: [Caml-list] Exception Unix_error problem in toplevel

[Bob Williams]

On Sat, Feb 03, 2007 at 02:48:30PM +0100, ls-ocaml-developer-2006@m-e-leypold.de wrote:
> 
> Bob Williams <a6b37331@telus.net> writes:
> 
> > Now to the possible bug in toplevel.  According to section 6.8.2 of
> > the OCaml manual,
> >
> > 	exception Unix_error of ...
> >
> > simply declares another variant of the built-in variant type "exn".  I
> > haven't looked into the implementation of OCaml, but I assume the
> > compiler statically assigns a tag number to every variant of a normal
> > variant type.  For exn, however, the tag numbers must be assigned at
> > load time or run time.
> >
> > And that is the problem I stumbled over.  The second copy of
> 
> Oh cool. :-). There should be RAQ (rarely asked questions + obscure
> problems). This should go into it :-).
> 
> Regards -- Markus

They are all obscure, Markus.  A careful developer finds and fixes
most of the obvious bugs; the ones left over are nearly always
unusual.

I did something stupid to trigger the exception-renumbering bug, but
there are valid circumstances under which the bug could bite somebody
else.  So it should probably be fixed.

I know almost nothing about OCaml's implementation, so any patch I
contribute will probably hurt more than it will help.  But if nobody
volunteers to fix the bug, I hereby threaten to do it myself.  :)

Re: [Caml-list] Exception Unix_error problem in toplevel

[ls-ocaml-developer-2006]

Bob Williams <a6b37331@telus.net> writes:

> On Sat, Feb 03, 2007 at 02:48:30PM +0100, ls-ocaml-developer-2006@m-e-leypold.de wrote:
>> 
>> Bob Williams <a6b37331@telus.net> writes:
>> 
>> > Now to the possible bug in toplevel.  According to section 6.8.2 of
>> > the OCaml manual,
>> >
>> > 	exception Unix_error of ...
>> >
>> > simply declares another variant of the built-in variant type "exn".  I
>> > haven't looked into the implementation of OCaml, but I assume the
>> > compiler statically assigns a tag number to every variant of a normal
>> > variant type.  For exn, however, the tag numbers must be assigned at
>> > load time or run time.
>> >
>> > And that is the problem I stumbled over.  The second copy of
>> 
>> Oh cool. :-). There should be RAQ (rarely asked questions + obscure
>> problems). This should go into it :-).
>> 
>> Regards -- Markus
>
> They are all obscure, Markus.  A careful developer finds and fixes
> most of the obvious bugs; the ones left over are nearly always
> unusual.
>
> I did something stupid to trigger the exception-renumbering bug, but


But if I understood it right, it's not a bug, but rather a user
error. There is AFAIR a FAQ for a similar situation if people
interactively load byte code files several times and define functions
in between. Your problem is similar, only that you're not redefining
functions but rather execptions.

I also think redefining an exception should not trigger an error or
warning message.

Rather reloading a byte code file of the same name and at the same
time having still loaded byte code files that refer to another version
of the first byte code file -- this should trigger an error or a
warning. Reloading a byte code file to which no other definitions
refer is not a problem and indeed wanted when you do debugging.

> there are valid circumstances under which the bug could bite somebody
> else.  So it should probably be fixed.


> I know almost nothing about OCaml's implementation, so any patch I
> contribute will probably hurt more than it will help.  But if nobody
> volunteers to fix the bug, I hereby threaten to do it myself.  :)

Regards -- Markus

Re: [Caml-list] Exception Unix_error problem in toplevel

[Bob Williams]

On Sun, Feb 04, 2007 at 11:46:55AM +0100, ls-ocaml-developer-2006@m-e-leypold.de wrote:
> 
> > They are all obscure, Markus.  A careful developer finds and fixes
> > most of the obvious bugs; the ones left over are nearly always
> > unusual.
> >
> > I did something stupid to trigger the exception-renumbering bug, but
> 
> 
> But if I understood it right, it's not a bug, but rather a user
> error. There is AFAIR a FAQ for a similar situation if people
> interactively load byte code files several times and define functions
> in between. Your problem is similar, only that you're not redefining
> functions but rather execptions.
> 
> I also think redefining an exception should not trigger an error or
> warning message.
> 
> Rather reloading a byte code file of the same name and at the same
> time having still loaded byte code files that refer to another version
> of the first byte code file -- this should trigger an error or a
> warning. Reloading a byte code file to which no other definitions
> refer is not a problem and indeed wanted when you do debugging.

It can be very useful to reload a cmo file into a running toplevel,
especially if the application is huge (thousands of compilation
units).

I have done some preliminary spelunking into the OCaml sources, and I
think I see an almost general fix for this.  One restriction is that
the replacement cmo file would necessarily need to have the same
interface as the original, but that can be verified by comparing the
interface's MD5 sums.

The idea is to allow any number of #loads or #uses at any time.  You
would be free to replace any compilation unit at any time, or to load
new ones.  Just before evaluating a toplevel expression, toplevel
would check whether there have been any intervening #loads or #uses;
if there have been, toplevel would reapply all the global relocations
(Reloc_setglobal and Reloc_getglobal) and rerun the initialization
code for all the compilation units present in memory, in the order in
which they were loaded.  The repatching and reinitialization should
fix up all the inter-compilation-unit references (including all
exceptions).

What do you all think of this?  Have I missed anything?  The patch
will probably be on the order of 50 lines long; if nobody predicts any
show stoppers, I will start on this in a few days.