musl's strptime does not support POSIX %U/%W

musl's strptime does not support POSIX %U/%W

[Raphael 'kena' Poss]

Hi all,

we at CockroachDB would like to integrate strptime, but some of our
users on Alpine Linux discovered that musl's strptime doesn't live up to
expectations:

conv_test.go:58: strptime("2018 10 4", "%Y %W %w"): got
"2017-12-31T00:00:00Z", expected "2018-03-08T00:00:00Z"
conv_test.go:58: strptime("2018 10 4", "%Y %U %w"): got
"2017-12-31T00:00:00Z", expected "2018-03-15T00:00:00Z"

Indeed there's a FIXME in there:
https://git.musl-libc.org/cgit/musl/tree/src/time/strptime.c#n123

We've filed this internally as
https://github.com/cockroachdb/cockroach/issues/10130

As of this day this is the only known limitation that prevents
compatibility of CockroachDB with musl. If you have any suggestions /
input we'd be glad to receive them!

Best regards

-- 
Raphael 'kena' Poss

Re: musl's strptime does not support POSIX %U/%W

[Rich Felker]

On Fri, Oct 21, 2016 at 11:00:35AM +0200, Raphael 'kena' Poss wrote:
> Hi all,
> 
> we at CockroachDB would like to integrate strptime, but some of our
> users on Alpine Linux discovered that musl's strptime doesn't live up to
> expectations:
> 
> conv_test.go:58: strptime("2018 10 4", "%Y %W %w"): got
> "2017-12-31T00:00:00Z", expected "2018-03-08T00:00:00Z"
> conv_test.go:58: strptime("2018 10 4", "%Y %U %w"): got
> "2017-12-31T00:00:00Z", expected "2018-03-15T00:00:00Z"
> 
> Indeed there's a FIXME in there:
> https://git.musl-libc.org/cgit/musl/tree/src/time/strptime.c#n123
> 
> We've filed this internally as
> https://github.com/cockroachdb/cockroach/issues/10130
> 
> As of this day this is the only known limitation that prevents
> compatibility of CockroachDB with musl. If you have any suggestions /
> input we'd be glad to receive them!

Indeed, somehow I thought this FIXME had been fixed a long time ago,
but it seems it hasn't. I'll see if we can get it added soon.

Rich

Re: musl's strptime does not support POSIX %U/%W

[Rich Felker]

On Fri, Oct 21, 2016 at 12:18:22PM -0400, Rich Felker wrote:
> On Fri, Oct 21, 2016 at 11:00:35AM +0200, Raphael 'kena' Poss wrote:
> > Hi all,
> > 
> > we at CockroachDB would like to integrate strptime, but some of our
> > users on Alpine Linux discovered that musl's strptime doesn't live up to
> > expectations:
> > 
> > conv_test.go:58: strptime("2018 10 4", "%Y %W %w"): got
> > "2017-12-31T00:00:00Z", expected "2018-03-08T00:00:00Z"
> > conv_test.go:58: strptime("2018 10 4", "%Y %U %w"): got
> > "2017-12-31T00:00:00Z", expected "2018-03-15T00:00:00Z"
> > 
> > Indeed there's a FIXME in there:
> > https://git.musl-libc.org/cgit/musl/tree/src/time/strptime.c#n123
> > 
> > We've filed this internally as
> > https://github.com/cockroachdb/cockroach/issues/10130
> > 
> > As of this day this is the only known limitation that prevents
> > compatibility of CockroachDB with musl. If you have any suggestions /
> > input we'd be glad to receive them!
> 
> Indeed, somehow I thought this FIXME had been fixed a long time ago,
> but it seems it hasn't. I'll see if we can get it added soon.

Looking at this in more detail, I see why it wasn't done before:
there's no clear spec for what output these should produce. struct tm
does not contain a week-number field, so the result would have to be
encoded in other fields -- but which ones? tm_yday? strptime is not
specified to produce a full, consistent struct tm for all inputs,
because many formats may be incomplete and other combinations of
format and input may yield contradictory information.

I'd be really interested in seeing some analysis of this situation by
someone who's studied it and has a viable proposal other than just
"reverse engineer glibc and do whatever it does".

Rich

Re: musl's strptime does not support POSIX %U/%W

[Rich Felker]

On Fri, Oct 21, 2016 at 01:34:27PM -0400, Rich Felker wrote:
> On Fri, Oct 21, 2016 at 12:18:22PM -0400, Rich Felker wrote:
> > On Fri, Oct 21, 2016 at 11:00:35AM +0200, Raphael 'kena' Poss wrote:
> > > Hi all,
> > > 
> > > we at CockroachDB would like to integrate strptime, but some of our
> > > users on Alpine Linux discovered that musl's strptime doesn't live up to
> > > expectations:
> > > 
> > > conv_test.go:58: strptime("2018 10 4", "%Y %W %w"): got
> > > "2017-12-31T00:00:00Z", expected "2018-03-08T00:00:00Z"
> > > conv_test.go:58: strptime("2018 10 4", "%Y %U %w"): got
> > > "2017-12-31T00:00:00Z", expected "2018-03-15T00:00:00Z"
> > > 
> > > Indeed there's a FIXME in there:
> > > https://git.musl-libc.org/cgit/musl/tree/src/time/strptime.c#n123
> > > 
> > > We've filed this internally as
> > > https://github.com/cockroachdb/cockroach/issues/10130
> > > 
> > > As of this day this is the only known limitation that prevents
> > > compatibility of CockroachDB with musl. If you have any suggestions /
> > > input we'd be glad to receive them!
> > 
> > Indeed, somehow I thought this FIXME had been fixed a long time ago,
> > but it seems it hasn't. I'll see if we can get it added soon.
> 
> Looking at this in more detail, I see why it wasn't done before:
> there's no clear spec for what output these should produce. struct tm
> does not contain a week-number field, so the result would have to be
> encoded in other fields -- but which ones? tm_yday? strptime is not
> specified to produce a full, consistent struct tm for all inputs,
> because many formats may be incomplete and other combinations of
> format and input may yield contradictory information.
> 
> I'd be really interested in seeing some analysis of this situation by
> someone who's studied it and has a viable proposal other than just
> "reverse engineer glibc and do whatever it does".

Further, musl's behavior seems to match the documented behavior of
glibc:

%W

    The week number of the current year as a decimal number (range 0
    through 53).

    Leading zeroes are permitted but not required.

    Note: Currently, this is not fully implemented. The format is
    recognized, input is consumed but no field in tm is set.

Source:
https://www.gnu.org/software/libc/manual/html_node/Low_002dLevel-Time-String-Parsing.html

So I'm surprised that the code would work the way you're expecting on
glibc-based systems. On the other hand maybe it's a documentation bug,
because they don't include that "Note:..." for %U.

Rich