[musl] Re: [tz] Weird PST8PDT and EST5EDT behavior on Alpine Linux

[musl] Re: [tz] Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Đoàn Trần Công Danh]

On 2024-03-03 22:51:49-0800, Russ Allbery via tz <tz@iana.org> wrote:
> Why would the system decide that time stamp should use daylight saving
> time?  The system time zone is set to UTC, so it shouldn't be some sort of
> contamination from it.  Or is this some odd bug in the busybox date
> command?  It works correctly with busybox 1.36.1 on a Debian system,
> though:
> 
> % busybox env TZ=PST8PDT date -R -d @1643145780
> Tue, 25 Jan 2022 13:23:00 -0800
> 
> The Alpine Linux system in question does have PST8PDT and EST5EDT files in
> /usr/share/zoneinfo.  The Olson time zone identifiers do work as expected:


Please correct me if I were wrong!
(also added musl-lib list)

I believe it's musl-libc's behaviours.

Both PST8PDT and EST5EDT are timezones in POSIX form.
musl specificly check for that first [1].

Time POSIX form is (space inserted for clarity):

	std offset[dst[offset][,start[/time],end[/time]]]

But seems like nothings was enforced if rule isnot given in the
timezone, which makes it open to intepretion.

Musl inteprete that as no transition at all [2].


1: https://git.musl-libc.org/cgit/musl/tree/src/time/__tz.c?h=v1.2.5&id=0784374d561435f7c787a555aeab8ede699ed298#n159
2: https://git.musl-libc.org/cgit/musl/tree/src/time/__tz.c?h=v1.2.5&id=0784374d561435f7c787a555aeab8ede699ed298#n240

-- 
Danh

[musl] Re: [tz] Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Paul Eggert]

On 3/4/24 19:52, Đoàn Trần Công Danh via tz wrote:
> But seems like nothings was enforced if rule isnot given in the
> timezone, which makes it open to intepretion.
> 
> Musl inteprete that as no transition at all [2].

Thanks, that explains things. TZDB has a TZDEFRULESTRING setting that 
specifies the transitions in this situation; it defaults to US rules, so 
musl and TZDB always disagree about TZ settings like TZ='PST8PDT'.

In TZDB, though, TZDEFRULESTRING is not used for PST8PDT and EST5EDT, as 
these are one of a small set of Zones that emulate the old System V 
behavior except with later US law changes taken into account.

Another way to put it: for musl the two settings TZ='PST8PDT' and 
TZ='pst8pdt' agree except for the case of time zone abbreviations, and 
neither TZ setting ever uses daylight saving time.

For TZDB, though, those two TZ settings agree only back to 2007; for 
earlier timestamps, TZ='PST8PDT' follows historical US behavior for 
standard time (and thus agrees with TZ='America/Los_Angeles' for all 
timestamps after 1883-11-18 at noon) whereas TZ='pst8pdt' disagrees with 
historical behavior by assuming current (2007 and later) rules for DST 
transitions before 2007.

These behaviors all conform to POSIX.

Re: [musl] Re: [tz] Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Rich Felker]

On Tue, Mar 05, 2024 at 10:52:23AM +0700, Đoàn Trần Công Danh wrote:
> On 2024-03-03 22:51:49-0800, Russ Allbery via tz <tz@iana.org> wrote:
> > Why would the system decide that time stamp should use daylight saving
> > time?  The system time zone is set to UTC, so it shouldn't be some sort of
> > contamination from it.  Or is this some odd bug in the busybox date
> > command?  It works correctly with busybox 1.36.1 on a Debian system,
> > though:
> > 
> > % busybox env TZ=PST8PDT date -R -d @1643145780
> > Tue, 25 Jan 2022 13:23:00 -0800
> > 
> > The Alpine Linux system in question does have PST8PDT and EST5EDT files in
> > /usr/share/zoneinfo.  The Olson time zone identifiers do work as expected:
> 
> 
> Please correct me if I were wrong!
> (also added musl-lib list)
> 
> I believe it's musl-libc's behaviours.
> 
> Both PST8PDT and EST5EDT are timezones in POSIX form.
> musl specificly check for that first [1].
> 
> Time POSIX form is (space inserted for clarity):
> 
> 	std offset[dst[offset][,start[/time],end[/time]]]
> 
> But seems like nothings was enforced if rule isnot given in the
> timezone, which makes it open to intepretion.
> 
> Musl inteprete that as no transition at all [2].

Yes, it's not clear to me (POSIX doesn't seem to specify) what the
default rules should be when none are specified. However, it is clear
that these strings meet the syntax for the POSIX TZ form, and thus
musl does not attempt to interpret them as filenames, which could
result in behavior contrary to the specified meaning.

If you want to use a file by that name, the safe way to request that
is by prefixing it with a colon, as in TZ=:PST8PDT

Rich

Re: [musl] Re: [tz] Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Russ Allbery]

Rich Felker <dalias@libc.org> writes:

> If you want to use a file by that name, the safe way to request that is
> by prefixing it with a colon, as in TZ=:PST8PDT

We're going to change the test to use America/Los_Angeles by default and
only fall back if that doesn't work (probably should have done that about
fifteen years ago), but for the record, PST8PDT was for support for very
old commercial UNIXes where I believe that was the only recognized style
of TZ setting, and I'm pretty dubious that the POSIX-introduced :PST8PDT
syntax would work there.

One can quite reasonably argue that any system that doesn't understand
America/Los_Angeles is at this point a museum piece, so I don't think this
is a hugely important issue, but I don't think it has that obvious of a
solution.

-- 
Russ Allbery (eagle@eyrie.org)             <https://www.eyrie.org/~eagle/>

[musl] Re: [tz] [musl] Re: Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Guy Harris]

On Mar 5, 2024, at 12:56 PM, Russ Allbery via tz <tz@iana.org> wrote:

> but for the record, PST8PDT was for support for very
> old commercial UNIXes where I believe that was the only recognized style
> of TZ setting, and I'm pretty dubious that the POSIX-introduced :PST8PDT
> syntax would work there.

That TZ syntax dates back at least to System III:

	http://www.bitsavers.org/pdf/att/unix/System_III/UNIX_Users_Manual_Release_3_Jun80.pdf

(see CTIME(3C)), which did not support anything tzdb-related, and didn't even support the more elaborate POSIX TZ syntax to include rules.  At least according to the CTIME(3C) page, System V Release 1 doesn't appear to support the POSIX syntax (or tzdb), and the same is true of SVR2 and SVR3.

Full POSIX support - but not tzdb support, may have shown up in SVR3.2; SVR4 had a tzcode-based implementation and shipped with the tzdb files.

[musl] Re: [tz] [musl] Re: Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Russ Allbery]

Guy Harris <gharris@sonic.net> writes:
> On Mar 5, 2024, at 12:56 PM, Russ Allbery via tz <tz@iana.org> wrote:

>> but for the record, PST8PDT was for support for very old commercial
>> UNIXes where I believe that was the only recognized style of TZ
>> setting, and I'm pretty dubious that the POSIX-introduced :PST8PDT
>> syntax would work there.

> That TZ syntax dates back at least to System III:

> 	http://www.bitsavers.org/pdf/att/unix/System_III/UNIX_Users_Manual_Release_3_Jun80.pdf

> (see CTIME(3C)), which did not support anything tzdb-related, and didn't
> even support the more elaborate POSIX TZ syntax to include rules.

I don't see any reference to the leading colon in that documentation.  Am
I just missing something?  It seems to support what I said: PST8PDT was
supported, but :PST8PDT was not.

    If an environment variable named TZ is present, asctime uses the
    contents of the variable to override the default time zone. The value
    of TZ must be a three-letter time zone name, followed by a number
    representing the difference between local time and Greenwich time in
    hours, followed by an optional three-letter name for a daylight time
    zone. For example, the setting for New Jersey would be EST5EDT.

-- 
Russ Allbery (eagle@eyrie.org)             <https://www.eyrie.org/~eagle/>

[musl] Re: [tz] [musl] Re: Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Guy Harris]

On Mar 5, 2024, at 2:03 PM, Russ Allbery <eagle@eyrie.org> wrote:

> I don't see any reference to the leading colon in that documentation.  Am
> I just missing something?

No, you aren't.

> It seems to support what I said: PST8PDT was supported, but :PST8PDT was not.

Yes.  As I indicated, they didn't have tzdb code (if they did, that would be a surprise, as the tzdb project didn't exist in 1979/1980), and they didn't support any POSIX extensions to TZ (if they did, that would be a surprise as, at least according to https://en.wikipedia.org/wiki/POSIX, the first POSIX standard came out in 1988).

[musl] Re: [tz] [musl] Re: Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Russ Allbery]

Guy Harris <gharris@sonic.net> writes:
> On Mar 5, 2024, at 2:03 PM, Russ Allbery <eagle@eyrie.org> wrote:

>> It seems to support what I said: PST8PDT was supported, but :PST8PDT was not.

> Yes.  As I indicated, they didn't have tzdb code (if they did, that
> would be a surprise, as the tzdb project didn't exist in 1979/1980), and
> they didn't support any POSIX extensions to TZ (if they did, that would
> be a surprise as, at least according to
> https://en.wikipedia.org/wiki/POSIX, the first POSIX standard came out
> in 1988).

Ah, sorry, I had just misunderstood you.  Thank you for the confirmation
and reference!

Basically, to summarize, up until now I think it was possible to set TZ to
PST8PDT and have it work essentially everywhere, no matter how old the
edition of UNIX, thanks to the backward compatibility file in tzdata.  I
think musl may have been the first place where this broke somewhat
intentionally.  This is not a big deal and I'm not sure I'm arguing that
musl should change; I expect interfaces to change over time and it's
probably time to assume everyone supports the Olson identifiers.  But it
is a change in the portability landscape, if a minor one.  (Probably one
that's been around for a while and I just never noticed because I hadn't
been running the INN test suite on Alpine Linux.)

-- 
Russ Allbery (eagle@eyrie.org)             <https://www.eyrie.org/~eagle/>

[musl] Re: [tz] [musl] Re: Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Paul Eggert]

On 3/5/24 12:12, Rich Felker via tz wrote:
> Yes, it's not clear to me (POSIX doesn't seem to specify) what the
> default rules should be when none are specified. However, it is clear
> that these strings meet the syntax for the POSIX TZ form, and thus
> musl does not attempt to interpret them as filenames, which could
> result in behavior contrary to the specified meaning.

Although TZDB interprets them as filenames, that is OK because the TZDB 
filenames have the specified meaning. Each TZDB Zone whose name has the 
form of a POSIX.1-2017 TZ string implements a set of transitions 
compatible with the corresponding POSIX TZ setting.

Long ago that wasn't the case: there were Zones like "GMT-12" that 
weren't compatible with POSIX even though their names took the POSIX 
forms. But that bug was fixed decades ago, in tzdata 94d (1994-02-24), 
and we no longer need to worry about that possibility.

Re: [musl] Re: [tz] [musl] Re: Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Rich Felker]

On Tue, Mar 05, 2024 at 04:47:02PM -0800, Paul Eggert wrote:
> On 3/5/24 12:12, Rich Felker via tz wrote:
> >Yes, it's not clear to me (POSIX doesn't seem to specify) what the
> >default rules should be when none are specified. However, it is clear
> >that these strings meet the syntax for the POSIX TZ form, and thus
> >musl does not attempt to interpret them as filenames, which could
> >result in behavior contrary to the specified meaning.
> 
> Although TZDB interprets them as filenames, that is OK because the
> TZDB filenames have the specified meaning. Each TZDB Zone whose name
> has the form of a POSIX.1-2017 TZ string implements a set of
> transitions compatible with the corresponding POSIX TZ setting.
> 
> Long ago that wasn't the case: there were Zones like "GMT-12" that
> weren't compatible with POSIX even though their names took the POSIX
> forms. But that bug was fixed decades ago, in tzdata 94d
> (1994-02-24), and we no longer need to worry about that possibility.

Interesting idea we could consider: if the TZ string matches POSIX
format for a zone with DST but without any rule portion, load a file
and accept it if the zones are consistent (all name/offset pairs
match) -- so that it's effectively just defining the transition date
rules.

Rich

[musl] Re: Weird PST8PDT and EST5EDT behavior on Alpine Linux

[James Cloos]

>>>>> "RA" == Russ Allbery <eagle@eyrie.org> writes:

RA> and it's probably time to assume everyone supports the Olson
RA> identifiers.

Operwrt, at least in general, does not.  (Nor does aix, if anyone still
cares.)

I have a clock (actual hw clock) which uses ntp by way of an embedded
mips board running a version of openwrt.  It lack the ram and storage
to use tzdb.  So I have to use EST5EDT,M3.2.0/2:00:00,M11.1.0/2:00:00
for the timezone.  (The explicit /2:00:00 might be avoidable, but the
use of self documenting data can be beneficial.)

I mention aix above because their docs were the top of the search
results when I needed a reminder of the details of non-Olson syntax.
It had, of course, been more than a decade since I'd last used or
even seen EST5EDT anywhere before I had to work out the above.

(I must say it was a bit anoying that the library kept the original
definition of EST5EDT et alia rathar than moving with the legislation.
But only a bit.)

-JimC
-- 
James Cloos <cloos@jhcloos.com>
            OpenPGP: https://jhcloos.com/0x997A9F17ED7DAEA6.asc

[musl] Re: [tz] Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Paul Eggert]

On 2024-03-06 07:39, James Cloos via tz wrote:

> (I must say it was a bit anoying that the library kept the original
> definition of EST5EDT et alia rathar than moving with the legislation.
> But only a bit.)

So Openwrt uses pre-2007 US daylight saving rules for TZ='EST5EDT'? 
That's puzzling. I thought Openwrt is based on musl, which ignores DST 
for that TZ setting (a behavior that POSIX allows). I'm a bit curious as 
to what's going on there.


> So I have to use EST5EDT,M3.2.0/2:00:00,M11.1.0/2:00:00
> for the timezone.  (The explicit /2:00:00 might be avoidable, but the
> use of self documenting data can be beneficial.)

For a fully self-documented TZ setting you could use this:

<EST>+05:00:00<EDT>+04:00:00,M3.2.0/+02:00:00,M11.1.0/+02:00:00

However, this sort of thing tends to be error-prone; e.g., suppose I 
forgot a colon?

'zic' attempts to generate the shortest equivalent TZ setting. On my 
machine /usr/share/zoneinfo/America/New_York ends with this equivalent:

EST5EDT,M3.2.0,M11.1.0

which I find more readable. I'm used to TZ strings, though, so that 
colors my opinion.

[musl] Re: Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Guy Harris]

On Mar 6, 2024, at 7:39 AM, James Cloos <cloos@jhcloos.com> wrote:

>>>>>> "RA" == Russ Allbery <eagle@eyrie.org> writes:
> 
> RA> and it's probably time to assume everyone supports the Olson
> RA> identifiers.
> 
> Operwrt, at least in general, does not.

I.e., it doesn't provide tzdb files, to save storage space on small embedded devices?

> (Nor does aix, if anyone still cares.)

According to at least one page on IBM's support web site, "The default timezone format for AIX 6.1 and AIX 7 is Olson Time":

	https://www.ibm.com/support/pages/managing-time-zone-variable-posix

Is that no longer the case?

> I have a clock (actual hw clock) which uses ntp by way of an embedded
> mips board running a version of openwrt.  It lack the ram and storage
> to use tzdb.  So I have to use EST5EDT,M3.2.0/2:00:00,M11.1.0/2:00:00
> for the timezone.  (The explicit /2:00:00 might be avoidable, but the
> use of self documenting data can be beneficial.)

If the desire here is to allow all users of a system to specify time zones using old-style TZ strings such as EST5EDT rather than either POSIX-style or tzdb-style strings, if the system in question can be used outside of a particular hardwired polity, that'll require some way for time zone rules to be specified.

If the desire here is to allow old-style TZ strings within a given polity - said polity probably being the US - to use old-style TZ strings, that could be done by hardcoding the rules for the US and use them, and, if the US changes the rules, distribute an update to the system.

> (I must say it was a bit anoying that the library kept the original
> definition of EST5EDT et alia rathar than moving with the legislation.
> But only a bit.)

To which library are you referring here?

Re: [musl] Re: [tz] Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Rich Felker]

On Fri, Mar 08, 2024 at 09:46:24AM -0800, Paul Eggert wrote:
> On 2024-03-06 07:39, James Cloos via tz wrote:
> 
> >(I must say it was a bit anoying that the library kept the original
> >definition of EST5EDT et alia rathar than moving with the legislation.
> >But only a bit.)
> 
> So Openwrt uses pre-2007 US daylight saving rules for TZ='EST5EDT'?
> That's puzzling. I thought Openwrt is based on musl, which ignores
> DST for that TZ setting (a behavior that POSIX allows). I'm a bit
> curious as to what's going on there.

It's not so much that we ignore it (it's present in tzname[1], and
would get applied if you use mktime with tm_isdst=1), just that, in
the absence of a transition rule for it, it's never considered active.
I don't think this is a particularly good behavior (even though it is
allowed by POSIX) but nothing great has been proposed as an
alternative.

> >So I have to use EST5EDT,M3.2.0/2:00:00,M11.1.0/2:00:00
> >for the timezone.  (The explicit /2:00:00 might be avoidable, but the
> >use of self documenting data can be beneficial.)
> 
> For a fully self-documented TZ setting you could use this:
> 
> <EST>+05:00:00<EDT>+04:00:00,M3.2.0/+02:00:00,M11.1.0/+02:00:00
> 
> However, this sort of thing tends to be error-prone; e.g., suppose I
> forgot a colon?
> 
> 'zic' attempts to generate the shortest equivalent TZ setting. On my
> machine /usr/share/zoneinfo/America/New_York ends with this
> equivalent:
> 
> EST5EDT,M3.2.0,M11.1.0

Yes, this i the form I've used in the past before we had zoneinfo
support.

Rich

[musl] Re: [tz] Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Paul Eggert]

On 2024-03-08 10:39, Guy Harris via tz wrote:
> According to at least one page on IBM's support web site, "The default timezone format for AIX 6.1 and AIX 7 is Olson Time":
> 
> 	https://www.ibm.com/support/pages/managing-time-zone-variable-posix
> 
> Is that no longer the case?

I just checked, and AIX 7.3 (the latest release) supports TZ settings 
like TZ='America/Los_Angeles' and has files like 
/usr/share/lib/zoneinfo/America/Los_Angeles.

[musl] Re: Weird PST8PDT and EST5EDT behavior on Alpine Linux

[James Cloos]

>>>>> "GH" == Guy Harris <gharris@sonic.net> writes:

>> Operwrt, at least in general, does not.

GH> I.e., it doesn't provide tzdb files, to save storage space on small embedded devices?

more likely it supports hw that is too small, and so only offers tzdb as optional.

>> (Nor does aix, if anyone still cares.)

GH> According to at least one page on IBM's support web site, "The default timezone format for AIX 6.1 and AIX 7 is Olson Time":

GH> 	https://www.ibm.com/support/pages/managing-time-zone-variable-posix

GH> Is that no longer the case?

probably the other way 'round.

It must be >5 years since i got that clock.

back then every web search i tried about posix time strings pointed to
aix's docs.  and i recall some discussion at that time that aix lacked
tzdb.

if, as you write, they've since corrected the omission, kudos to them.

>> (I must say it was a bit anoying that the library kept the original
>> definition of EST5EDT et alia rathar than moving with the legislation.
>> But only a bit.)

GH> To which library are you referring here?

the clock's (openwrt's) libc.  which reports:

# /lib/libc.so 
musl libc (mipsel-sf)
Version 1.1.16
Dynamic Program Loader
Usage: /lib/libc.so [options] [--] pathname [args]

-JimC
-- 
James Cloos <cloos@jhcloos.com>
            OpenPGP: https://jhcloos.com/0x997A9F17ED7DAEA6.asc

Re: [musl] Re: [tz] Weird PST8PDT and EST5EDT behavior on Alpine Linux

[James Cloos]

>>>>> "PE" == Paul Eggert writes:
>>>>> "RF" == Rich Felker writes:

PE> EST5EDT,M3.2.0,M11.1.0

RF> Yes, this i the form I've used in the past before we had zoneinfo
RF> support.

now that i think about it (it has been >5 years since i got that clock)
i ended up with that form, too.  but then changed the setting to match
the manufacturer's subsequent recommendation.

(it was long enough ago that i had to ssh into the clock and grep(1) /etc
to remind myself of its tz configuration...)

-JimC
-- 
James Cloos <cloos@jhcloos.com>
            OpenPGP: https://jhcloos.com/0x997A9F17ED7DAEA6.asc

[musl] Re: [tz] [musl] Re: Weird PST8PDT and EST5EDT behavior on Alpine Linux

[Guy Harris]

On Mar 8, 2024, at 11:02 AM, Rich Felker via tz <tz@iana.org> wrote:

> It's not so much that we ignore it (it's present in tzname[1], and
> would get applied if you use mktime with tm_isdst=1), just that, in
> the absence of a transition rule for it, it's never considered active.
> I don't think this is a particularly good behavior (even though it is
> allowed by POSIX) but nothing great has been proposed as an
> alternative.

The executive summary of this thread appears to be:

	Use XXXnnnYYY TZ settings at your own risk, as there's no guarantee how they'll behave.  If you want predictable behavior, either use POSIX settings or, if tzdb is present, tzdb settings.