Re: catan errors

[Rich Felker <dalias@libc.org>]

On Tue, Apr 10, 2018 at 06:27:24PM -0300, dgutson . wrote:
> On Tue, Apr 10, 2018 at 5:50 PM, Rich Felker <dalias@libc.org> wrote:
> 
> > On Tue, Apr 10, 2018 at 05:41:46PM -0300, dgutson . wrote:
> > > On Tue, Apr 10, 2018 at 5:32 PM, Rich Felker <dalias@libc.org> wrote:
> > >
> > > > On Tue, Apr 10, 2018 at 05:23:12PM -0300, dgutson . wrote:
> > > > > On Tue, Apr 10, 2018 at 4:50 PM, Rich Felker <dalias@libc.org>
> > wrote:
> > > > >
> > > > > > The OpenBSD catan implementation we're using has a number of
> > > > > > nonsensical "overflow" (goto ovrf) conditions that aren't errors,
> > > > > > reported by mepholic on irc. I think the attached patch fixes them
> > > > > > without introducing new problems, but I'm not sure if any other
> > > > > > problems remain.
> > > > > >
> > > > > > Note that, of the three cases removed:
> > > > > >
> > > > > > 1. Is not an exceptional case at all, and made no sense to begin
> > with..
> > > > > >
> > > > > > 2. Is only exceptional if x and a are both zero; atan(2x,0) is
> > > > > >    perfectly well-defined.
> > > > > >
> > > > > > 3. Is only possible if y==1.0 and x==0.0, which is the only real
> > > > > >    exceptional case for atan: z==I.
> > > > > >
> > > > >
> > > > >
> > > > > Besides the trigonometric case, are you considering de-normalized
> > > > numbers,
> > > > > such as 4.94066e-324 as divisor?
> > > > > For example:
> > > > >     double x = 1.0;
> > > > >     double y = 5E-324;
> > > > > x / y is inf, and y != 0.0.
> > > > > Shouldn't 'a' be checked against that number or its absolute value >=
> > > > > minimum?
> > > >
> > > > Can you clarify where you think something goes wrong?
> > > >
> > >
> > > -     if (a == 0.0)
> > > -             goto ovrf;
> > >
> > >       t = y + 1.0;
> > >       a = (x2 + t * t)/a;
> > >
> > >
> > > The check you removed does not look correct for me because what I
> > mentioned..
> > > However, shouldn't you check, before the division, that a is not the
> > > nearest to zero (+ or -) denormalized representable double,
> > > in order to avoid ending in inf?
> >
> > Here a=x²+(y-1)², so unless both x==0 and y==1, the smallest a can be
> >
> 
> I was worried by the case when x is 0 and y is the next (or previous)
> representable value nearest to 1.0; the y == 1.0 check will fail, but the
> division may get big; so I did a small program and verified that the result
> of the division is about 3.24519e+32 when going towards negative and
> 8.11296e+31 when going towards positive, so everything is OK (I didn't dig
> in the atan2 arguments though).

That's basically what I said here, without working out any actual
numbers:

> > is DBL_EPSILON². When a is small, the numerator in the last line is
> > also small (x²+(1+y)² < 2) so dividing by a does not overflow.

The result of a subtraction cannot be smaller than the order of
DBL_EPSILON times the larger-magnitude operand. In particular numbers
like (y-1) are never small in any absolute sense.

Rich