strptime() question

strptime() question

[Alba Pompeo]

Hello musl.

I've built R on a musl system and "make check-all" fails because R
does not get correct timezone information from the system.

Here's the error -
http://pastebin.com/raw/32D3ngNZ

This would be correct,

> difftime(
+     as.POSIXct(c("1970-01-01 00:00:00", "1970-01-01 12:00:00"), tz="EST5EDT"),
+     as.POSIXct(c("1970-01-01 00:00:00", "1970-01-01 00:00:00"), tz="UTC"))
Time differences in hours
[1]  5 17

but my system gives   4 16  which would suggest that it thinks that
there is only 4 hours time difference between  UTC  and  EST5EDT and
that is just wrong.

The above can be translated to this (slightly more low-level) R code which
is already somewhat close to the internal C code of R :

  > t0 <- strptime("1970-01-01 00:00:00", "%Y-%m-%d %H:%M:%OS", tz="UTC")
  > as.numeric(t0)
  [1] 0

  > t1 <- strptime("1970-01-01 00:00:00", "%Y-%m-%d %H:%M:%OS", tz="EST5EDT")
  > as.numeric(t1)
  [1] 18000
  > 18000 / 3600
  [1] 5
  >

That's why its 5 hours.
R's  strptime()  is close to the C library  strptime(), so think musl
specialists should have it easy to do the above from C and find out
why you get 4 hour instead of 5 hour difference.

I'm certain this isn't a problem with R and I think it's up to me to
ensure that your system's  C library strptime() function returns the
correct numbers for the above example, but I can't understand why it
doesn't.

Can anyone help here?
Thanks a lot.

Ciao.

Re: strptime() question

[Rich Felker]

On Thu, Feb 04, 2016 at 01:27:40PM -0200, Alba Pompeo wrote:
> Hello musl.
> 
> I've built R on a musl system and "make check-all" fails because R
> does not get correct timezone information from the system.
> 
> Here's the error -
> http://pastebin.com/raw/32D3ngNZ
> 
> This would be correct,
> 
> > difftime(
> +     as.POSIXct(c("1970-01-01 00:00:00", "1970-01-01 12:00:00"), tz="EST5EDT"),
> +     as.POSIXct(c("1970-01-01 00:00:00", "1970-01-01 00:00:00"), tz="UTC"))
> Time differences in hours
> [1]  5 17
> 
> but my system gives   4 16  which would suggest that it thinks that
> there is only 4 hours time difference between  UTC  and  EST5EDT and
> that is just wrong.
> 
> The above can be translated to this (slightly more low-level) R code which
> is already somewhat close to the internal C code of R :
> 
>   > t0 <- strptime("1970-01-01 00:00:00", "%Y-%m-%d %H:%M:%OS", tz="UTC")
>   > as.numeric(t0)
>   [1] 0
> 
>   > t1 <- strptime("1970-01-01 00:00:00", "%Y-%m-%d %H:%M:%OS", tz="EST5EDT")
>   > as.numeric(t1)
>   [1] 18000
>   > 18000 / 3600
>   [1] 5
>   >
> 
> That's why its 5 hours.
> R's  strptime()  is close to the C library  strptime(), so think musl
> specialists should have it easy to do the above from C and find out
> why you get 4 hour instead of 5 hour difference.
> 
> I'm certain this isn't a problem with R and I think it's up to me to
> ensure that your system's  C library strptime() function returns the
> correct numbers for the above example, but I can't understand why it
> doesn't.
> 
> Can anyone help here?
> Thanks a lot.

I think we need to understand how R's strptime function implements its
tz argument, since this is not part of the POSIX strptime API. It's
possible that there's a bug in musl here, but it seems equally
possible that R is doing some dubious hacks with switching timezones.

I'm also wondering what you're expecting from the timezone name
EST5EDT, which is not generally usable; it's in POSIX TZ form rather
than zoneinfo, but it lacks any information about when the switch to
daylight time happens, and the defaults are not correct with respect
to modern policy.

Rich

Re: Fwd: [Rd] [musl] strptime() question

[Rich Felker]

On Thu, Feb 04, 2016 at 04:35:22PM -0200, Alba Pompeo wrote:
> I forwarded our talk on musl mailing list to R mailing list.
> I got this response.
> Does it help?
> Also, what do you think about making a new email that's sent to both
> musl and R mailing list, that way devs from both projects can
> communicate? It's a tad inefficient for me to keep forwarding messages
> like this back and forth.

I'm cc'ing both lists now.

> ---------- Forwarded message ----------
> From: Simon Urbanek <simon.urbanek@r-project.org>
> Date: Thu, Feb 4, 2016 at 3:01 PM
> Subject: Re: [Rd] [musl] strptime() question
> To: Alba Pompeo <albapompeo@gmail.com>
> Cc: r-devel <r-devel@r-project.org>
> 
> 
> 
> On Feb 4, 2016, at 11:20 AM, Alba Pompeo <albapompeo@gmail.com> wrote:
> 
> > There is incompatibility between R strptime and musl libc. I posted
> > about it on their mailing list, but they need more information I can't
> > provide, so I'm forwarding the message here in hope R developers can
> > help. Thanks.
> 
> Generally, it's using the standard tzset() call to set the time zone
> http://pubs.opengroup.org/onlinepubs/009695399/functions/tzset.html

The tzset function doesn't really do much interesting; it just updates
some global state to match the current value of the TZ environment
variable, but most time.h functions do this already anyway. Anyway
this doesn't really answer the question of what R's strptime function
is doing to add timezone functionality onto the underlying system's
strptime. Is R changing the environment (this is highly unsafe in any
potentially-multithreaded process, BTW)? Is it using the globals set
by tzset to compute adjustments to the result?

Rich

Re: Fwd: [Rd] [musl] strptime() question

[Alba Pompeo]

It is setting TZ and using tzset(). R is not multi-threaded so it is safe.

Simon figure out the important settings from the config.log and
config.h files on a musl system:

/* #undef USE_INTERNAL_MKTIME */
#define HAVE_TM_GMTOFF 1
#define HAVE_TM_ZONE 1

Does this help anyone debug the issue? Simon just went on vacation.



On Thu, Feb 4, 2016 at 7:32 PM, Rich Felker <dalias@libc.org> wrote:
> On Thu, Feb 04, 2016 at 04:35:22PM -0200, Alba Pompeo wrote:
>> I forwarded our talk on musl mailing list to R mailing list.
>> I got this response.
>> Does it help?
>> Also, what do you think about making a new email that's sent to both
>> musl and R mailing list, that way devs from both projects can
>> communicate? It's a tad inefficient for me to keep forwarding messages
>> like this back and forth.
>
> I'm cc'ing both lists now.
>
>> ---------- Forwarded message ----------
>> From: Simon Urbanek <simon.urbanek@r-project.org>
>> Date: Thu, Feb 4, 2016 at 3:01 PM
>> Subject: Re: [Rd] [musl] strptime() question
>> To: Alba Pompeo <albapompeo@gmail.com>
>> Cc: r-devel <r-devel@r-project.org>
>>
>>
>>
>> On Feb 4, 2016, at 11:20 AM, Alba Pompeo <albapompeo@gmail.com> wrote:
>>
>> > There is incompatibility between R strptime and musl libc. I posted
>> > about it on their mailing list, but they need more information I can't
>> > provide, so I'm forwarding the message here in hope R developers can
>> > help. Thanks.
>>
>> Generally, it's using the standard tzset() call to set the time zone
>> http://pubs.opengroup.org/onlinepubs/009695399/functions/tzset.html
>
> The tzset function doesn't really do much interesting; it just updates
> some global state to match the current value of the TZ environment
> variable, but most time.h functions do this already anyway. Anyway
> this doesn't really answer the question of what R's strptime function
> is doing to add timezone functionality onto the underlying system's
> strptime. Is R changing the environment (this is highly unsafe in any
> potentially-multithreaded process, BTW)? Is it using the globals set
> by tzset to compute adjustments to the result?
>
> Rich

Re: Re: Fwd: [Rd] [musl] strptime() question

[Rich Felker]

On Sat, Feb 06, 2016 at 10:35:28AM -0200, Alba Pompeo wrote:
> It is setting TZ and using tzset(). R is not multi-threaded so it is safe.
> 
> Simon figure out the important settings from the config.log and
> config.h files on a musl system:
> 
> /* #undef USE_INTERNAL_MKTIME */
> #define HAVE_TM_GMTOFF 1
> #define HAVE_TM_ZONE 1
> 
> Does this help anyone debug the issue? Simon just went on vacation.

R's src/main/datetime.c does A LOT of munging to the output of
strptime and I wouldn't be surprised if they're making nonportable
assumptions about it. Note that the underlying C strptime function has
nothing to do with timezones or time conversion; it's basically just a
time-tailored sscanf that reads integers and strings from the input
string and converts them into values in the fields of struct tm.

It also looks like R isn't even calling the system strptime but
instead using its own R_strptime. I think that supports my theory that
the problem is in the munging being done after the string is parsed.

Rich