Re: NaN Test in OCaml

[Xavier Leroy <Xavier.Leroy@inria.fr>]

> George Russell <ger@informatik.uni-bremen.de> has suggested on
> comp.lang.ml the following test to find out whether a float is NaN:
>         x is not a NaN <=> (x = x)
> Doing this leads to interesting results with OCaml 3.0:
>     # let nan x = not (x = x);;
>     val nan : 'a -> bool = <fun>
>     # nan (1.0 /. 0.0);;
>     - : bool = false            (* correct *)
>     # nan (0.0 /. 0.0);;
>     - : bool = false            (* should be true *)
> The following definition of nan uses a type annotation and has a
> different result:
>     # let nan (x:float) = not (x = x);;
>     val nan : float -> bool = <fun>
>     # nan (0.0 /. 0.0);;
>     - : bool = true             (* correct *)
>     # nan (1.0 /. 0.0);;
>     - : bool = false            (* correct *)
> Is this a bug or a feature?

It is a bug, more exactly a design error in generic comparisons.

The difference between the two examples is that in the second case
(with the type constraint), the compiler performs type-specialization
on the "=" predicate, turning it into the equality predicate over
floating-point numbers.  This predicate works as specified in IEEE,
in particular NaN is not equal to NaN.

In the first case, no type information is available, so generic
equality is called.  Generic equality is defined in terms of the
"compare" polymorphic comparison function:

        let (=) a b = (compare a b = 0)

and "compare" implements a total ordering relation: either its
arguments a and b are equal, or a is smaller than b, or a is bigger
than b.

But of course IEEE floats are not totally ordered, due to NaN...  
So, rather arbitrarily, "compare NaN NaN" returns 0 -- but any other
return value would be equally wrong!

What is needed is to revamp the polymorphic comparison function so
that it has four possible outcomes: equal, less than, greater than,
and unordered.  "compare" would raise an exception in the "unordered"
case, but generic comparisons (=, <=, <, >=, >) would return "false".
I haven't looked at how to implement this behavior yet, though.

To come back to Archisman's initial question, I'm considering adding an
"fpclassify" function similar to that of ISO C9X, to determine whether
a float is NaN, infinite, zero, exact or denormal.  That should avoid
the confusing "x <> x" test.

- Xavier Leroy