Uncaught Unix_error.

Uncaught Unix_error.

[Romain Beauxis]

	Hi all !

We have had a strange bug report on our code. Somehow, an Unix.Unix_error 
exception was not caught while it ought to be. See:
  http://savonet.rastageeks.org/ticket/269

We have precisely characterized the issue in the code, and we would like to 
understand its possible origin.

If we look at the code for raising exceptions in otherlib/unixsupport.c, we 
have:
    if (unix_error_exn == NULL) {
      unix_error_exn = caml_named_value("Unix.Unix_error");
      if (unix_error_exn == NULL)
        invalid_argument("Exception Unix.Unix_error not initialized, please 
link unix.cma");
    }

This code fetches the pointer to the named_value representing an 
Unix.Unix_error, and assigns it to a static variable.

However, the threads module also registers an exception of the same name, 
probably because both modules are mutually exclusive.

We believe that the issue is raised because the module that uses Unix.read is 
compiled with one of unix or threads and the application using that module 
with the other one.

In this case, it should be possible that the static value representing the 
exception is initialized with a wrong pointer and  never updated afterwards.

We were unable yet to reproduce the issue on a minimal code, it was caused by 
a portmap scan on the telnet server, which uses Unix.read calls to read from 
the network socket.

We don't really know if this is a known issue, or if there is something to do 
about this from our side, so we wanted to report it here...


Romain

Re: [Caml-list] Uncaught Unix_error.

[Stéphane Glondu]

Romain Beauxis a écrit :
> We have had a strange bug report on our code. Somehow, an Unix.Unix_error 
> exception was not caught while it ought to be. See:
>   http://savonet.rastageeks.org/ticket/269
> [...]
> We believe that the issue is raised because the module that uses Unix.read is 
> compiled with one of unix or threads and the application using that module 
> with the other one.

Such errors can also happen if you link the (same) Unix module twice:
the exceptions raised by the first instance are not catchable by
anything linked after the second instance. This can happen quite easily,
especially if you use Dynlink. In general, having two modules with the
same name in one application is a bad idea.


Cheers,

-- 
Stéphane

Re: [Caml-list] Uncaught Unix_error.

[Romain Beauxis]

Le jeudi 18 juin 2009 15:17:12, Stéphane Glondu a écrit :
> > We believe that the issue is raised because the module that uses
> > Unix.read is compiled with one of unix or threads and the application
> > using that module with the other one.
>
> Such errors can also happen if you link the (same) Unix module twice:
> the exceptions raised by the first instance are not catchable by
> anything linked after the second instance. This can happen quite easily,
> especially if you use Dynlink. In general, having two modules with the
> same name in one application is a bad idea.

Yes. However, when using external modules, it is probably not always possible 
to restrict this since two dependencies may have been compiled against modules 
with the same name.

It should be possible to mitigate this by making the exn value non-global in 
the C code. It do not think it will have strong consequences in term of 
runtime efficiency.


Romain

Re: [Caml-list] Uncaught Unix_error.

[David Baelde]

On Thu, Jun 18, 2009 at 3:17 PM, Stéphane Glondu<steph@glondu.net> wrote:
> Such errors can also happen if you link the (same) Unix module twice:
> the exceptions raised by the first instance are not catchable by
> anything linked after the second instance.

Both liquidsoap and ocaml-duppy are compiled with -threads, so they
are using the same version of Unix. Duppy is not linked against Unix,
only requires it through ocamlfind. Could it be that ocamlfind (on
some versions) links Unix twice in liquidsoap? I did not know it was
even possible (except with Dynlink).

Also, note that the call to read that raises the error is from the
same module that tries to catch the exception.

I'm lost, I'd like to understand when the exception is registered: at
runtime initialization? In this case, only one should ever be
registered (at least it seems easy to enforce). Or is it statically
registered somehow?

-- 
David

Re: [Caml-list] Uncaught Unix_error.

[Jake Donham]

On Thu, Jun 18, 2009 at 7:41 AM, David Baelde<david.baelde@gmail.com> wrote:
> Also, note that the call to read that raises the error is from the
> same module that tries to catch the exception.
>
> I'm lost, I'd like to understand when the exception is registered: at
> runtime initialization? In this case, only one should ever be
> registered (at least it seems easy to enforce). Or is it statically
> registered somehow?

There are two things going on: First, exception tags in OCaml aren't
registered; they consist only of a block containing a distinguished
string. You can see this if you compile unix.ml with -dlambda:

  (let (Unix_error/197 (makeblock 0 "Unix.Unix_error"))

(Enforcing that there is only one of these seems a little bit delicate
with respect to module abstraction.)

Second, exceptions raised from C use Callback.register_exception,
which registers the exception (that is, the block containing the
distinguished string) globally. If you link Unix twice, loading the
second one overwrites the registration of the first. Now an exception
raised from C, by either Unix, cannot be caught by the first Unix but
only by the second.

I complained about this a while back and suggested that the bytecode
linker prevent you from linking the same module twice, but it occurs
to me now that it's necessary to allow this for the toplevel. (Still
perhaps there could be a flag for strict checking.)

I think the best solution is to be sure you have not linked Unix
twice. The objinfo tool (under tools/ in the OCaml distribution) is
useful for checking libraries; for linked binaries it's probably
easiest just to check the link command.

Jake

Re: [Caml-list] Uncaught Unix_error.

[Stéphane Glondu]

David Baelde a écrit :
> Both liquidsoap and ocaml-duppy are compiled with -threads, so they
> are using the same version of Unix. Duppy is not linked against Unix,
> only requires it through ocamlfind. Could it be that ocamlfind (on
> some versions) links Unix twice in liquidsoap? I did not know it was
> even possible (except with Dynlink).
                 ^^^^^^^^^^^^^^^^^^^

Are you sure Dynlink is not used at all in the circumstances of the bug?

> Also, note that the call to read that raises the error is from the
> same module that tries to catch the exception.

If Jake is right, a call to Unix.read might not be catchable even in the
same module.

> I'm lost, I'd like to understand when the exception is registered: at
> runtime initialization? In this case, only one should ever be
> registered (at least it seems easy to enforce). Or is it statically
> registered somehow?

The exception is registered when the module is evaluated. This can
happen at program startup, or during a call to Dynlink.load_file. Note
that if Unix is linked in a cmxs file and you dynload it, you will face
trouble.

-- 
Stéphane

Re: [Caml-list] Uncaught Unix_error.

[David Baelde]

Thanks for all the details. The application does have support for
dynamically loading plugins but it's basically never used. We should
double check with the user (we can't reproduce) maybe he's loading
something funny...
-- 
David

Re: [Caml-list] Uncaught Unix_error.

[Romain Beauxis]

Le jeudi 18 juin 2009 21:21:20, Stéphane Glondu a écrit :
> > I'm lost, I'd like to understand when the exception is registered: at
> > runtime initialization? In this case, only one should ever be
> > registered (at least it seems easy to enforce). Or is it statically
> > registered somehow?
>
> The exception is registered when the module is evaluated. This can
> happen at program startup, or during a call to Dynlink.load_file. Note
> that if Unix is linked in a cmxs file and you dynload it, you will face
> trouble.

So, if I understand it well, the C code in unixsupport.c does not match the 
expected semantics of exceptions registration.

If exceptions are registered for acess through the C code, if it is assumed 
that one can link against two modules that will provide exceptions in the same 
namespace, and if both modules registers these exceptions dynamically, then it 
should not be assumed that a global value in the C code is sufficient to get the 
exception value.

Current C code says "let's get this value once and for all". This contradicts 
the dynamic loading, hence should be changed to "let's get the *current* value 
for this exception", this being called each time one wants to raise such an 
exception...


Romain