[musl] strtod doesn't set errno to ERANGE for slightly out-of-range hex float

[musl] strtod doesn't set errno to ERANGE for slightly out-of-range hex float

[Lénárd Szolnoki]

Hi,

#include <stdlib.h>
#include <assert.h>
#include <errno.h>

int main() {
   const char* const input = "0x1.fffffffffffff8p+1023";
   double ret = strtod(input, NULL);
   assert(errno == ERANGE);
}

The input is slightly out of range, DBL_MAX is 1.fffffffffffffp+1023 (one fewer binary 1 
in the mantissa).

The assert fails with musl libc, passes with glibc.

musl version: 1.2.5 (as shipped in Debian sid)

Regards,
Lénárd Szolnoki

Re: [musl] strtod doesn't set errno to ERANGE for slightly out-of-range hex float

[Szabolcs Nagy]

* Lénárd Szolnoki <cpp@lenardszolnoki.com> [2025-03-24 10:55:36 +0000]:
> Hi,
> 
> #include <stdlib.h>
> #include <assert.h>
> #include <errno.h>
> 
> int main() {
>   const char* const input = "0x1.fffffffffffff8p+1023";
>   double ret = strtod(input, NULL);
>   assert(errno == ERANGE);
> }
> 
> The input is slightly out of range, DBL_MAX is 1.fffffffffffffp+1023 (one
> fewer binary 1 in the mantissa).
> 
> The assert fails with musl libc, passes with glibc.

this looks valid (might depend on long double
representation).

the overflow threshold is not calculated precisely
for the hexfloat case. __floatscan returns an in
range long double value that is rounded to inf by
strtod, but there is no overflow check there.

i guess hexfloat overflow check should be fixed.

> 
> musl version: 1.2.5 (as shipped in Debian sid)
> 
> Regards,
> Lénárd Szolnoki

Re: [musl] strtod doesn't set errno to ERANGE for slightly out-of-range hex float

[Lénárd Szolnoki]

On 08/04/2025 23:25, Szabolcs Nagy wrote:
> * Lénárd Szolnoki <cpp@lenardszolnoki.com> [2025-03-24 10:55:36 +0000]:
>> Hi,
>>
>> #include <stdlib.h>
>> #include <assert.h>
>> #include <errno.h>
>>
>> int main() {
>>    const char* const input = "0x1.fffffffffffff8p+1023";
>>    double ret = strtod(input, NULL);
>>    assert(errno == ERANGE);
>> }
>>
>> The input is slightly out of range, DBL_MAX is 1.fffffffffffffp+1023 (one
>> fewer binary 1 in the mantissa).
>>
>> The assert fails with musl libc, passes with glibc.
> 
> this looks valid (might depend on long double
> representation).

Good point, maybe I should have used strtof. Anyway, I am testing on x86_64 Linux, with 
80bit long double.

> 
> the overflow threshold is not calculated precisely
> for the hexfloat case. __floatscan returns an in
> range long double value that is rounded to inf by
> strtod, but there is no overflow check there.

An other test case is with a different behaviour is 
"0x1.fffffffffffff00000000000000000000000000000000000000000000001p+1023"

This is even more slightly larger than DBL_MAX, therefore it should still return inf and 
set errno to ERANGE.

For this one strtod returns DBL_MAX. What happens is that this rounds towards the value of 
DBL_MAX within long double, and the conversion from long double to double is exact, so it 
does not overflow. Sort of a double rounding problem, but at the edge of the range of double.

> 
> i guess hexfloat overflow check should be fixed.
> 
>>
>> musl version: 1.2.5 (as shipped in Debian sid)
>>
>> Regards,
>> Lénárd Szolnoki

Re: [musl] strtod doesn't set errno to ERANGE for slightly out-of-range hex float

[Szabolcs Nagy]

* Lénárd Szolnoki <cpp@lenardszolnoki.com> [2025-04-09 11:38:01 +0100]:
> On 08/04/2025 23:25, Szabolcs Nagy wrote:
> > * Lénárd Szolnoki <cpp@lenardszolnoki.com> [2025-03-24 10:55:36 +0000]:
> > > Hi,
> > > 
> > > #include <stdlib.h>
> > > #include <assert.h>
> > > #include <errno.h>
> > > 
> > > int main() {
> > >    const char* const input = "0x1.fffffffffffff8p+1023";
> > >    double ret = strtod(input, NULL);
> > >    assert(errno == ERANGE);
> > > }
> > > 
> > > The input is slightly out of range, DBL_MAX is 1.fffffffffffffp+1023 (one
> > > fewer binary 1 in the mantissa).
> > > 
> > > The assert fails with musl libc, passes with glibc.
...
> > the overflow threshold is not calculated precisely
> > for the hexfloat case. __floatscan returns an in
> > range long double value that is rounded to inf by
> > strtod, but there is no overflow check there.
> 
> An other test case is with a different behaviour is
> "0x1.fffffffffffff00000000000000000000000000000000000000000000001p+1023"
> 
> This is even more slightly larger than DBL_MAX, therefore it should still
> return inf and set errno to ERANGE.
> 

in nearest rounding mode i'd expect DBL_MAX.

in upward rounding mode musl seems to return inf.

> For this one strtod returns DBL_MAX. What happens is that this rounds
> towards the value of DBL_MAX within long double, and the conversion from
> long double to double is exact, so it does not overflow. Sort of a double
> rounding problem, but at the edge of the range of double.

there is only a single rounding, and i think that's fine.


> > 
> > i guess hexfloat overflow check should be fixed.
> > 
> > > 
> > > musl version: 1.2.5 (as shipped in Debian sid)
> > > 
> > > Regards,
> > > Lénárd Szolnoki

Re: [musl] strtod doesn't set errno to ERANGE for slightly out-of-range hex float

[Lénárd Szolnoki]

On 09/04/2025 13:55, Szabolcs Nagy wrote:
> * Lénárd Szolnoki <cpp@lenardszolnoki.com> [2025-04-09 11:38:01 +0100]:
>> On 08/04/2025 23:25, Szabolcs Nagy wrote:
>>> * Lénárd Szolnoki <cpp@lenardszolnoki.com> [2025-03-24 10:55:36 +0000]:
>>>> Hi,
>>>>
>>>> #include <stdlib.h>
>>>> #include <assert.h>
>>>> #include <errno.h>
>>>>
>>>> int main() {
>>>>     const char* const input = "0x1.fffffffffffff8p+1023";
>>>>     double ret = strtod(input, NULL);
>>>>     assert(errno == ERANGE);
>>>> }
>>>>
>>>> The input is slightly out of range, DBL_MAX is 1.fffffffffffffp+1023 (one
>>>> fewer binary 1 in the mantissa).
>>>>
>>>> The assert fails with musl libc, passes with glibc.
> ...
>>> the overflow threshold is not calculated precisely
>>> for the hexfloat case. __floatscan returns an in
>>> range long double value that is rounded to inf by
>>> strtod, but there is no overflow check there.
>>
>> An other test case is with a different behaviour is
>> "0x1.fffffffffffff00000000000000000000000000000000000000000000001p+1023"
>>
>> This is even more slightly larger than DBL_MAX, therefore it should still
>> return inf and set errno to ERANGE.
>>
> 
> in nearest rounding mode i'd expect DBL_MAX.

It appears that you are right. I expected DBL_MAX to be the threshold for all rounding 
modes, but I double checked the IEEE standard and it seems that for nearest rounding mode 
the threshold is exactly my first example.

> 
> in upward rounding mode musl seems to return inf.
> 
>> For this one strtod returns DBL_MAX. What happens is that this rounds
>> towards the value of DBL_MAX within long double, and the conversion from
>> long double to double is exact, so it does not overflow. Sort of a double
>> rounding problem, but at the edge of the range of double.
> 
> there is only a single rounding, and i think that's fine.
> 
> 
>>>
>>> i guess hexfloat overflow check should be fixed.
>>>
>>>>
>>>> musl version: 1.2.5 (as shipped in Debian sid)
>>>>
>>>> Regards,
>>>> Lénárd Szolnoki