cluts - numeric test expectations

cluts - numeric test expectations

[Rich Felker]

I've been reviewing the numeric tests in cluts some more and based on
discussions in the previous thread and on irc, here's where things
seem to stand:

Tests which expect strto* on "0x[junk]" to fail rather than returning
0 with endptr pointing at the 'x': both my interpretation of the
standard and glibc agree that this expectation is wrong, as does at
least one expert I asked. I think these should be changed to accept
the current musl and glibc behavior and treat anything else as a
failure. (Note that the scanf tests, however, seem to be fine.)

Tests which expect *endptr==str after overflow (ERANGE): I believe
this expectation is incorrect, but glibc seems to disagree. I can't
find any language in the standard to support the behavior explicitly,
or to allow it as an interpretation. The definition of "subject
sequence" makes no reference to the value having to fit in a
certain-size integer type, only that it belong to a clearly-defined
regular language, e.g. /[-+]?(0x)?[[:xdigit:]]+/ for base==16.

Otherwise, all the integer tests look okay. I still need to review the
floating point ones.

Rich

Re: cluts - numeric test expectations

[Luka Marčetić]

[-- Attachment #1: Type: text/plain, Size: 3296 bytes --]

On 07/15/2011 05:37 AM, Rich Felker wrote:
> Tests which expect strto* on "0x[junk]" to fail rather than returning
> 0 with endptr pointing at the 'x': both my interpretation of the
> standard and glibc agree that this expectation is wrong, as does at
> least one expert I asked. I think these should be changed to accept
> the current musl and glibc behavior and treat anything else as a
> failure. (Note that the scanf tests, however, seem to be fine.)

For those who missed the chat: We talked about non-base-16 tests which 
do expect endptr to end up pointing at 'x'. Rich's argument against this 
behavior was "longest initial subsequence" (strlen of "0x">"0"), and 
mine was "of the expected form" [see strtol SUSv4 page to see what I 
mean]. We ended up agreeing on the latter. It follows that the above 
objection of Rich's is to hexadecimal "0x*" tests.
Rich: Now that you pointed it out, I do believe that the standard 
applies this logic to base-16 tests as well, because of this one line: 
"If the value of/base /is 16, the characters 0x or 0X may optionally 
precede the sequence of letters and digits". The key word here is 
"optionally". I guess I missed this, expecting base 16 to take only "A 
hexadecimal constant [which] consists of the prefix 0x or 0X followed by 
a sequence of the decimal digits and letters". But the latter definition 
(without optionality), refers to cases where base is 0, not 16. One 
might think that base 0 should then match "0x", but I don't think so - 
it brings us back to "expected from".
In short: I think, like Rich said, absolutely all "0x" tests should 
point to 'x'. Any objections? Speak now or forever hold your peace*.
*where 'forever' is a definite (quite possibly short) amount of time ;-)

> Tests which expect *endptr==str after overflow (ERANGE): I believe
> this expectation is incorrect, but glibc seems to disagree. I can't
> find any language in the standard to support the behavior explicitly,
> or to allow it as an interpretation. The definition of "subject
> sequence" makes no reference to the value having to fit in a
> certain-size integer type,

Though I did remake ERANGE tests (see "> MAX" commit) with the 
endptr-nptr offset strlen of nptr, I've now reviewed the case, and I 
think it should still be endptr==nptr. There is no actual conversion in 
my opinion: *_MAX returned value simply means out of scope. See  "Return 
value" section: "Upon successful completion, these functions shall 
return the converted value". The reason I say it's not a converted value 
is because successful completion it is not, see "Errors" section: "These 
functions shall fail if: (...) [ERANGE] The value to be returned is not 
representable.". Not that I can't see con arguments.

>   only that it belong to a clearly-defined
> regular language, e.g. /[-+]?(0x)?[[:xdigit:]]+/ for base==16.

Is that your regexp or an official one (if there is such a thing)? 
Because I think, in light of what you've said above regarding the first 
issue, it might be wrong: Wouldn't this regexp greedily match "0x" for 
you, instead of the shorter [[:xdigit:]], the latter of which is alone 
of the "expected form"?

> Otherwise, all the integer tests look okay. I still need to review the
> floating point ones.
>
> Rich

I appreciate that.
-Luka

[-- Attachment #2: Type: text/html, Size: 7043 bytes --]

Re: cluts - numeric test expectations

[Rich Felker]

On Fri, Jul 15, 2011 at 01:44:57PM +0200, Luka Marčetić wrote:
> In short: I think, like Rich said, absolutely all "0x" tests should
> point to 'x'. Any objections? Speak now or forever hold your peace*.

I think this is correct.

> >Tests which expect *endptr==str after overflow (ERANGE): I believe
> >this expectation is incorrect, but glibc seems to disagree. I can't
> >find any language in the standard to support the behavior explicitly,
> >or to allow it as an interpretation. The definition of "subject
> >sequence" makes no reference to the value having to fit in a
> >certain-size integer type,
> 
> Though I did remake ERANGE tests (see "> MAX" commit) with the
> endptr-nptr offset strlen of nptr, I've now reviewed the case, and I
> think it should still be endptr==nptr. There is no actual conversion
> in my opinion: *_MAX returned value simply means out of scope. See
> "Return value" section: "Upon successful completion, these functions
> shall return the converted value". The reason I say it's not a
> converted value is because successful completion it is not, see
> "Errors" section: "These functions shall fail if: (...) [ERANGE] The

I agree that the return value may not be a "converted value", but the
text about endptr says nothing about whether a conversion took place.
It merely says:

    If the subject sequence has the expected form...A pointer to the
    final string shall be stored in the object pointed to by endptr,
    provided that endptr is not a null pointer.

The part I elided was about other behavior that happens when the
subject sequence has the correct form (conversion and possible
negation) but there's no text that makes storage to endptr being
conditional on the outcome of that converstion (or conversion
attempt).

> >  only that it belong to a clearly-defined
> >regular language, e.g. /[-+]?(0x)?[[:xdigit:]]+/ for base==16.
> 
> Is that your regexp or an official one (if there is such a thing)?
> Because I think, in light of what you've said above regarding the
> first issue, it might be wrong: Wouldn't this regexp greedily match
> "0x" for you, instead of the shorter [[:xdigit:]], the latter of
> which is alone of the "expected form"?

It's mine, not official. You could probably derive the official one
from some BNF or whatever they have in the C standard. Note however
that the regex I gave does not match "0x" because [[:xdigit:]] has a +
after it (1 or more occurrances). Thus in the case where no xdigits
follow the 0x, the regex matcher would have to take the option of
(0x)? matching "" (empty string) and the [[:xdigit:]] matching the
"0".

Rich