NaN Test in OCaml

NaN Test in OCaml

[Christian Lindig]

George Russell <ger@informatik.uni-bremen.de> has suggested on
comp.lang.ml the following test to find out whether a float is NaN:

        x is not a NaN <=> (x = x)

Doing this leads to interesting results with OCaml 3.0:

    # let nan x = not (x = x);;
    val nan : 'a -> bool = <fun>
    # nan (1.0 /. 0.0);;
    - : bool = false            (* correct *)
    # nan (0.0 /. 0.0);;
    - : bool = false            (* should be true *)

The following definition of nan uses a type annotation and has a
different result:

    # let nan (x:float) = not (x = x);;
    val nan : float -> bool = <fun>
    # nan (0.0 /. 0.0);;
    - : bool = true             (* correct *)
    # nan (1.0 /. 0.0);;
    - : bool = false            (* correct *)

Is this a bug or a feature? Anyway, I guess this again shows the subtleties
of equality.

-- Christian 
        
-- 
Christian Lindig          Harvard University - DEAS
lindig@eecs.harvard.edu   33 Oxford St, MD 242, Cambridge MA 02138
phone: +1 (617) 496-7157  http://www.eecs.harvard.edu/~lindig/

Re: NaN Test in OCaml

[David Mentre]

Hi Christian,

I'm far from being at the level of Caml implementors but:

Christian Lindig <lindig@eecs.harvard.edu> writes:

>     # let nan x = not (x = x);;
>     val nan : 'a -> bool = <fun>
                ^^ here you have a polymorphic equality test
>     # nan (1.0 /. 0.0);;
>     - : bool = false            (* correct *)
>     # nan (0.0 /. 0.0);;
>     - : bool = false            (* should be true *)
> 
> The following definition of nan uses a type annotation and has a
> different result:
> 
>     # let nan (x:float) = not (x = x);;
>     val nan : float -> bool = <fun>
                ^^^^^ here we know that we have floats
>     # nan (0.0 /. 0.0);;
>     - : bool = true             (* correct *)
>     # nan (1.0 /. 0.0);;
>     - : bool = false            (* correct *)
> 
> Is this a bug or a feature? Anyway, I guess this again shows the subtleties
> of equality.

I would say a feature. As Xavier said in his mail[1], if the compiler
knows that in *every case* we have a float, it generates a float
specific comparison code. Otherwise, the compiler takes the safe way by
using a generic comparison operator.

You can see this with an undocumented option of the bytecode compiler:

pochi(mentre):~ [976] ocaml -dinstr
        Objective Caml version 3.00

# let nan x = not (x = x);;
[...]
	ccall equal, 2
[...]
val nan : 'a -> bool = <fun>

# let nan (x:float) = not (x = x);;
[...]
	ccall eq_float, 2
[...]
val nan : float -> bool = <fun>


If you look at assembly code, you'll see the same behavior:

pochi(mentre):/tmp [1008] cat nan.ml
let nan_poly x = not (x = x)

let nan_float (x:float) = not (x = x)

pochi(mentre):/tmp [1009] ocamlopt -S nan.ml
pochi(mentre):/tmp [1010] cat nan.s
[...]
Nan_nan_poly_43:
.L100:
	pushl	%eax
	pushl	%eax
	movl	$equal, %eax
	call	caml_c_call
        ^^^^^^^^^^^^^^^^^^^ the compiler call a generic
                            (i.e. polymorphic equality operator)
[...]
Nan_nan_float_45:
	subl	$8, %esp
.L104:
	fldl	(%eax)
	fldl	(%eax)
	fucompp
        ^^^^^^^ float specific comparison operation, with NaN handling,
                see [2]
[...]


Hope it clarifies,
david

[1] http://caml.inria.fr/archives/200101/msg00195.html
[2] http://webster.cs.ucr.edu/Page_asm/ArtofAssembly/CH14/CH14-5.html#HEADING5-22
-- 
 David.Mentre@inria.fr -- http://www.irisa.fr/prive/dmentre/
 Opinions expressed here are only mine.

Re: NaN Test in OCaml

[Andreas Rossberg]

Christian Lindig wrote:
> 
> George Russell <ger@informatik.uni-bremen.de> has suggested on
> comp.lang.ml the following test to find out whether a float is NaN:
> 
>         x is not a NaN <=> (x = x)
> 
> Doing this leads to interesting results with OCaml 3.0:
> 
>     # let nan x = not (x = x);;
>     val nan : 'a -> bool = <fun>
>     # nan (1.0 /. 0.0);;
>     - : bool = false            (* correct *)
>     # nan (0.0 /. 0.0);;
>     - : bool = false            (* should be true *)
> 
> The following definition of nan uses a type annotation and has a
> different result:
> 
>     # let nan (x:float) = not (x = x);;
>     val nan : float -> bool = <fun>
>     # nan (0.0 /. 0.0);;
>     - : bool = true             (* correct *)
>     # nan (1.0 /. 0.0);;
>     - : bool = false            (* correct *)

Right, because the first example uses polymorphic equality which is
purely structural. The second uses proper floating point comparison.

Actually, what we have here, is a subtle kind of overloading. Subtle in
particular because it is overlapping. IMHO it would be preferable if
floating point comparison used different syntax, probably "=.". In that
case the compiler should probably emit a warning whenever he discovered
the use of structural equality on floats.

-- 
Andreas Rossberg, rossberg@ps.uni-sb.de

:: be declarative. be functional. just be. ::

Re: NaN Test in OCaml

[Xavier Leroy]

> George Russell <ger@informatik.uni-bremen.de> has suggested on
> comp.lang.ml the following test to find out whether a float is NaN:
>         x is not a NaN <=> (x = x)
> Doing this leads to interesting results with OCaml 3.0:
>     # let nan x = not (x = x);;
>     val nan : 'a -> bool = <fun>
>     # nan (1.0 /. 0.0);;
>     - : bool = false            (* correct *)
>     # nan (0.0 /. 0.0);;
>     - : bool = false            (* should be true *)
> The following definition of nan uses a type annotation and has a
> different result:
>     # let nan (x:float) = not (x = x);;
>     val nan : float -> bool = <fun>
>     # nan (0.0 /. 0.0);;
>     - : bool = true             (* correct *)
>     # nan (1.0 /. 0.0);;
>     - : bool = false            (* correct *)
> Is this a bug or a feature?

It is a bug, more exactly a design error in generic comparisons.

The difference between the two examples is that in the second case
(with the type constraint), the compiler performs type-specialization
on the "=" predicate, turning it into the equality predicate over
floating-point numbers.  This predicate works as specified in IEEE,
in particular NaN is not equal to NaN.

In the first case, no type information is available, so generic
equality is called.  Generic equality is defined in terms of the
"compare" polymorphic comparison function:

        let (=) a b = (compare a b = 0)

and "compare" implements a total ordering relation: either its
arguments a and b are equal, or a is smaller than b, or a is bigger
than b.

But of course IEEE floats are not totally ordered, due to NaN...  
So, rather arbitrarily, "compare NaN NaN" returns 0 -- but any other
return value would be equally wrong!

What is needed is to revamp the polymorphic comparison function so
that it has four possible outcomes: equal, less than, greater than,
and unordered.  "compare" would raise an exception in the "unordered"
case, but generic comparisons (=, <=, <, >=, >) would return "false".
I haven't looked at how to implement this behavior yet, though.

To come back to Archisman's initial question, I'm considering adding an
"fpclassify" function similar to that of ISO C9X, to determine whether
a float is NaN, infinite, zero, exact or denormal.  That should avoid
the confusing "x <> x" test.

- Xavier Leroy