[musl] Broken mktime calculations when crossing DST boundary

[musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

[-- Attachment #1: Type: text/plain, Size: 763 bytes --]

Consider this struct tm for  in CET (Europe/Prague):
tm_sec: 3
tm_min: 59
tm_hour: 1
tm_mday: 31
tm_mon: 2
tm_year: 124
tm_wday: 0
tm_yday: 90
tm_isdst: 0
Representing:
2024-03-31 01:59:02

Add a minute and call mktime (crossing DST boundary).

Instead of getting:
2024-03-31 03:00:02
You get:
2024-03-31 01:00:02

tm_sec: 3
tm_min: 0
tm_hour: 1
tm_mday: 31
tm_mon: 2
tm_year: 124
tm_wday: 0
tm_yday: 90tm_isdst: 0

Not only We are going backwards, DST flag is not even set.

Glibc behaves correctly:
tm_sec: 3
tm_min: 0
tm_hour: 3
tm_mday: 31
tm_mon: 2
tm_year: 124
tm_wday: 0
tm_yday: 90tm_isdst: 1

tm_hour = 3 and tm_isdst = 1

This pretty messes with my cron tool that rely on mktime being able to correctly calculate struct tm after incrementing fields.

AW

[-- Attachment #2: Type: text/html, Size: 2905 bytes --]

Re: [musl] Broken mktime calculations when crossing DST boundary

[Markus Wichmann]

[-- Attachment #1: Type: text/plain, Size: 423 bytes --]

Hi,

can you send a reproducer? I have written the attached program, and
cannot see the issue. I have tried both the zoneinfo file and the POSIX
TZ.


$ TZ=Europe/Prague ./foo 2024 3 31 1 58 00
before: 2024-03-31 01:58:00 CET
after: 2024-03-31 03:00:00 CEST
$ TZ=CET-1CEST,M3.5.0,M10.5.0/3 ./foo 2024 3 31 1 58 00
before: 2024-03-31 01:58:00 CET
after: 2024-03-31 03:00:00 CEST

What are you doing different?

Ciao,
Markus

[-- Attachment #2: foo.c --]
[-- Type: text/x-csrc, Size: 712 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main(int argc, char *argv[])
{
    time_t t;
    struct tm tm = {0};
    char buf[64];

    if (argc < 7) {
        fprintf(stderr, "Usage: %s year month day hour minute second\n");
        return EXIT_FAILURE;
    }

    tm.tm_year = atoi(argv[1]) - 1900;
    tm.tm_mon = atoi(argv[2]) - 1;
    tm.tm_mday = atoi(argv[3]);
    tm.tm_hour = atoi(argv[4]);
    tm.tm_min = atoi(argv[5]);
    tm.tm_sec = atoi(argv[6]);

    mktime(&tm);
    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("before: %s\n", buf);
    tm.tm_min += 2;
    mktime(&tm);
    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after: %s\n", buf);
    return 0;
}

Re: [musl] Broken mktime calculations when crossing DST boundary

[Markus Wichmann]

Am Sat, Mar 23, 2024 at 09:26:00AM +0000 schrieb Alexander Weps:
> This works for me as well even after changes to match struct tm in my
> app (setting tm_yday, __tm_gtmoff a __tm_zone):

You don't need to set yday or the others before calling mktime().
mktime() is defined to only use year, mon, mday, hour, min, sec, and
isdst as input, normalize them, and calculate the others (and also the
UNIX time stamp).

>
> Any idea how could a previous calculation mess with musl internals so
> it would start producing bad results? Because otherwise, I don't see
> how this could be happening if you completely extract it out of the
> code and it works.
>

The only thing that means is that the isolated code works, and the bug
is elsewhere. I'm sorry, but you are going to have to debug this
yourself. There may be some static memory getting corrupted (e.g. the TZ
and rule caches), but honestly this is just speculation.

> And when I compile under glibc, everything is fine.

That tends to indicate some undefined behavior. Not that that helps you
find the reason. You are going to have to debug it. Finding a minimum
reproducer may help in that, or you directly apply liberal amounts of
gdb.

You seem to have dropped the list from CC, so I'm adding it back.

Ciao,
Markus

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

> You don't need to set yday or the others before calling mktime().

I thought that too, but wanted to test it on exactly the same struct tm. No change in behavior.

So I have found a minimal example:

void test()
{
    time_t t;
    struct tm tm = {0};
    char buf[64];

    tm.tm_year = 2024 - 1900;
    tm.tm_mon = 3 - 1;
    tm.tm_mday = 31;
    tm.tm_hour = 1;
    tm.tm_min = 59;
    tm.tm_sec = 2;
    print_tm(&tm);

    mktime(&tm);
    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("before: %s\n", buf);
    tm.tm_isdst = -1; // This is the cause.
    tm.tm_min += 1;
    mktime(&tm);
    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after: %s\n", buf);
}

Setting tm_isdst to -1 after first mktime and before second mktime causes the issue.

AW


On Saturday, March 23rd, 2024 at 11:38, Markus Wichmann <nullplan@gmx.net> wrote:

> Am Sat, Mar 23, 2024 at 09:26:00AM +0000 schrieb Alexander Weps:
>
> > This works for me as well even after changes to match struct tm in my
> > app (setting tm_yday, __tm_gtmoff a __tm_zone):
>
>
> You don't need to set yday or the others before calling mktime().
> mktime() is defined to only use year, mon, mday, hour, min, sec, and
> isdst as input, normalize them, and calculate the others (and also the
> UNIX time stamp).
>
> > Any idea how could a previous calculation mess with musl internals so
> > it would start producing bad results? Because otherwise, I don't see
> > how this could be happening if you completely extract it out of the
> > code and it works.
>
>
> The only thing that means is that the isolated code works, and the bug
> is elsewhere. I'm sorry, but you are going to have to debug this
> yourself. There may be some static memory getting corrupted (e.g. the TZ
> and rule caches), but honestly this is just speculation.
>
> > And when I compile under glibc, everything is fine.
>
>
> That tends to indicate some undefined behavior. Not that that helps you
> find the reason. You are going to have to debug it. Finding a minimum
> reproducer may help in that, or you directly apply liberal amounts of
> gdb.
>
> You seem to have dropped the list from CC, so I'm adding it back.
>
> Ciao,
> Markus

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

This is how it should work as far as I am concerned. After manipulating the date, the tm_isdst is set to -1.

From https://cplusplus.com/reference/ctime/tm/:
The Daylight Saving Time flag (tm_isdst) is greater than zero if Daylight Saving Time is in effect, zero if Daylight Saving Time is not in effect, and less than zero if the information is not available.

And the date should be correctly set as DST or STD in mktime.

I have a date. I make change in fields, I set tm_isdst = -1, I call mktime.

I see only on place where tm_isdst is checked and that's:
time_t mktime(struct tm *tm)
{
    struct tm new;
    long opp;
    long long t = __tm_to_secs(tm);

    __secs_to_zone(t, 1, &new.tm_isdst, &new.__tm_gmtoff, &opp, &new.__tm_zone);

    if (tm->tm_isdst>=0 && new.tm_isdst!=tm->tm_isdst)
        t -= opp - new.__tm_gmtoff;

    t -= new.__tm_gmtoff;
    if ((time_t)t != t) goto error;

    __secs_to_zone(t, 0, &new.tm_isdst, &new.__tm_gmtoff, &opp, &new.__tm_zone);

    if (__secs_to_tm(t + new.__tm_gmtoff, &new) < 0) goto error;

    *tm = new;
    return t;

error:
    errno = EOVERFLOW;
    return -1;
}

So tm->tm_isdst>=0 seems to be the cause here.

AW

On Saturday, March 23rd, 2024 at 13:00, Alexander Weps <exander77@pm.me> wrote:

> > You don't need to set yday or the others before calling mktime().
> 
> 
> I thought that too, but wanted to test it on exactly the same struct tm. No change in behavior.
> 
> So I have found a minimal example:
> 
> void test()
> {
> time_t t;
> struct tm tm = {0};
> char buf[64];
> 
> tm.tm_year = 2024 - 1900;
> tm.tm_mon = 3 - 1;
> tm.tm_mday = 31;
> tm.tm_hour = 1;
> tm.tm_min = 59;
> tm.tm_sec = 2;
> print_tm(&tm);
> 
> mktime(&tm);
> strftime(buf, sizeof buf, "%F %T %Z", &tm);
> printf("before: %s\n", buf);
> tm.tm_isdst = -1; // This is the cause.
> tm.tm_min += 1;
> mktime(&tm);
> strftime(buf, sizeof buf, "%F %T %Z", &tm);
> printf("after: %s\n", buf);
> }
> 
> Setting tm_isdst to -1 after first mktime and before second mktime causes the issue.
> 
> AW
> 
> 
> 
> 
> On Saturday, March 23rd, 2024 at 11:38, Markus Wichmann nullplan@gmx.net wrote:
> 
> > Am Sat, Mar 23, 2024 at 09:26:00AM +0000 schrieb Alexander Weps:
> > 
> > > This works for me as well even after changes to match struct tm in my
> > > app (setting tm_yday, __tm_gtmoff a __tm_zone):
> > 
> > You don't need to set yday or the others before calling mktime().
> > mktime() is defined to only use year, mon, mday, hour, min, sec, and
> > isdst as input, normalize them, and calculate the others (and also the
> > UNIX time stamp).
> > 
> > > Any idea how could a previous calculation mess with musl internals so
> > > it would start producing bad results? Because otherwise, I don't see
> > > how this could be happening if you completely extract it out of the
> > > code and it works.
> > 
> > The only thing that means is that the isolated code works, and the bug
> > is elsewhere. I'm sorry, but you are going to have to debug this
> > yourself. There may be some static memory getting corrupted (e.g. the TZ
> > and rule caches), but honestly this is just speculation.
> > 
> > > And when I compile under glibc, everything is fine.
> > 
> > That tends to indicate some undefined behavior. Not that that helps you
> > find the reason. You are going to have to debug it. Finding a minimum
> > reproducer may help in that, or you directly apply liberal amounts of
> > gdb.
> > 
> > You seem to have dropped the list from CC, so I'm adding it back.
> > 
> > Ciao,
> > Markus

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Sat, Mar 23, 2024 at 12:00:52PM +0000, Alexander Weps wrote:
> This is how it should work as far as I am concerned. After
> manipulating the date, the tm_isdst is set to -1.
> 
> >From https://cplusplus.com/reference/ctime/tm/:

This site is notoriously sloppy if not outright wrong about pretty
much everything.

> The Daylight Saving Time flag (tm_isdst) is greater than zero if
> Daylight Saving Time is in effect, zero if Daylight Saving Time is
> not in effect, and less than zero if the information is not
> available.

The actual specification is at
https://pubs.opengroup.org/onlinepubs/9699919799/functions/mktime.html
(POSIX version; the C version is not independently linkable but says
basically the same thing anyway). The relevant text, which differs
from the above, is:

    "A negative value for tm_isdst shall cause mktime() to attempt to
    determine whether Daylight Savings Time is in effect for the
    specified time."

No details are specified for how this determination is made, so it may
differ between implementations. At DST transition boundaries, it's
impossible to define unambiguously. There are either times that don't
exist, or that could mean two different things.

If there are cases where our current behavior isn't *consistent* (the
input time does not match the output in either DST or non-DST), it
seems like those should be treated as bugs and fixed. But if you're
just expecting the implementation to guess whether your nonexistant
time came from moving forward from non-DST or backward from DST, it
can't; it just had to make an arbitrary choice, and that choice is not
going to agree across different systems.

> And the date should be correctly set as DST or STD in mktime.
> 
> I have a date. I make change in fields, I set tm_isdst = -1, I call
> mktime.

If you already have a date and want a new date offset by some fixed
amount of time from it, setting tm_isdst is wrong. You leave tm_isdst
alone and adjust whichever field(s) (e.g. tm_min) you want to offset
by, then call mktime. The resulting tm_isdst (and correspondingly
other fields) may have changed if you crossed a DST boundary by making
the offset.

When you set tm_isdst=-1, you're throwing away information and
explicitly asking the implementation to guess what an ambiguous
timestamp meant.

Does this help? Do you think there's an actual bug to be investigated
on our side here?

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

I don't think time can go backwards by incrementing field under any conditions.

Going from:
tm_sec: 2
tm_min: 60
tm_hour: 1
tm_mday: 31
tm_mon: 2
tm_year: 124
tm_wday: 0
tm_yday: 90
tm_isdst: -1

To:
tm_sec: 2
tm_min: 0
tm_hour: 1
tm_mday: 31
tm_mon: 2
tm_year: 124
tm_wday: 0
tm_yday: 90
tm_isdst: 0

Seems to be plain wrong. I cannot come up with any argument for this being correct under any conditions.

mktime was given a struct tm with uncertain STD/DST, it deduced it is STD and then thrown away 60 minute information. The minutes got reset from 60 to 0 and no other change was done.

It claims that 01:00:00 + 60 minutes is 01:00:00

It should be:
02:00:00 (wrong, but at least makes sense)
Or it should be:
03:00:00 (correct)

How it could be?
01:00:00

This leads to:
01:59:00 + 1 minute to be 01:00:00

Time is going backwards by incrementing fields.

I looked when the tm_isdst = -1 was introduced in the library I am working on and it comments with this:
---
 CRON_USE_LOCAL_TIME: use DST offsets

mktime(3) will automatically account for daylight savings time offsets
if the tm_isdst field of the struct tm is set to -1:

    http://pubs.opengroup.org/onlinepubs/009695399/functions/strptime.html

~~~
tm.tm_isdst = -1;      /* Not set by strptime(); tells mktime()
                          to determine whether daylight saving time
                          is in effect */
~~~

ctime(3) also mentions:

    tm_isdst A flag that indicates whether daylight saving time is in
             effect at the time described. The value is positive if daylight
             saving time is in effect, zero if it is not, and negative if the
             information is not available.
---

Looking at this, I think musl completely ignores the detection part.

If minutes are set to 60 (or above), the struct tm was created by incrementing (going forward in time).
If minutes are set to -1 (or below), the struct tm was created by decrementing (going backward in time).

This applies to all fields. So there is enough information to know how was DST boundary crossed.

I am just looking into glibc code and there is a pretty significant part that handles what I assume is this detection.

I don't see any code in musl that handles this, except for some comparison in __secs_to_zone.

It has enough information to correctly deduce STD/DST, but it does not do it at all.

AW


On Saturday, March 23rd, 2024 at 13:31, Rich Felker <dalias@libc.org> wrote:

> On Sat, Mar 23, 2024 at 12:00:52PM +0000, Alexander Weps wrote:
>
> > This is how it should work as far as I am concerned. After
> > manipulating the date, the tm_isdst is set to -1.
> >
> > > From https://cplusplus.com/reference/ctime/tm/:
>
>
> This site is notoriously sloppy if not outright wrong about pretty
> much everything.
>
> > The Daylight Saving Time flag (tm_isdst) is greater than zero if
> > Daylight Saving Time is in effect, zero if Daylight Saving Time is
> > not in effect, and less than zero if the information is not
> > available.
>
>
> The actual specification is at
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/mktime.html
> (POSIX version; the C version is not independently linkable but says
> basically the same thing anyway). The relevant text, which differs
> from the above, is:
>
> "A negative value for tm_isdst shall cause mktime() to attempt to
> determine whether Daylight Savings Time is in effect for the
> specified time."
>
> No details are specified for how this determination is made, so it may
> differ between implementations. At DST transition boundaries, it's
> impossible to define unambiguously. There are either times that don't
> exist, or that could mean two different things.
>
> If there are cases where our current behavior isn't consistent (the
> input time does not match the output in either DST or non-DST), it
> seems like those should be treated as bugs and fixed. But if you're
> just expecting the implementation to guess whether your nonexistant
> time came from moving forward from non-DST or backward from DST, it
> can't; it just had to make an arbitrary choice, and that choice is not
> going to agree across different systems.
>
> > And the date should be correctly set as DST or STD in mktime.
> >
> > I have a date. I make change in fields, I set tm_isdst = -1, I call
> > mktime.
>
>
> If you already have a date and want a new date offset by some fixed
> amount of time from it, setting tm_isdst is wrong. You leave tm_isdst
> alone and adjust whichever field(s) (e.g. tm_min) you want to offset
> by, then call mktime. The resulting tm_isdst (and correspondingly
> other fields) may have changed if you crossed a DST boundary by making
> the offset.
>
> When you set tm_isdst=-1, you're throwing away information and
> explicitly asking the implementation to guess what an ambiguous
> timestamp meant.
>
> Does this help? Do you think there's an actual bug to be investigated
> on our side here?
>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Sat, Mar 23, 2024 at 01:49:48PM +0000, Alexander Weps wrote:
> I don't think time can go backwards by incrementing field under any conditions.
> 
> Going from:
> tm_sec: 2
> tm_min: 60
> tm_hour: 1
> tm_mday: 31
> tm_mon: 2
> tm_year: 124
> tm_wday: 0
> tm_yday: 90
> tm_isdst: -1
> 
> To:
> tm_sec: 2
> tm_min: 0
> tm_hour: 1
> tm_mday: 31
> tm_mon: 2
> tm_year: 124
> tm_wday: 0
> tm_yday: 90
> tm_isdst: 0
> 
> Seems to be plain wrong. I cannot come up with any argument for this
> being correct under any conditions.

The above broke-down time is 2:00:02, which does not exist on that day
as a normalized time. If interpreted as non-DST, it would be just a
couple seconds past the end of non-DST (1:59:59.99999..). If
interpreted as DST, it would be just under an hour before the start of
DST (3:00:00), which, after normalization, is 1:00:02 non-DST.

mktime cannot tell whether your non-normalized input was the result of
you starting with 01:00:02 and adding 1 hour (in which case, our
output does not reflect your intent) or of you starting with 3:00:02
and subtracting 1 hour (in which case, our output does reflect your
intent).

> mktime was given a struct tm with uncertain STD/DST, it deduced it
> is STD and then thrown away 60 minute information. The minutes got
> reset from 60 to 0 and no other change was done.

It did not deduce it was STD. It deduced it was non-normalized DST
rather than non-normalized STD (this is an arbitrary choice), then
normalized it and got STD.

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

One of the main purposes of struct tm is to calculate date and time, by adding and substracting it's fields.

> mktime cannot tell whether your non-normalized input was the result of
> you starting with 01:00:02 and adding 1 hour (in which case, our
> output does not reflect your intent) or of you starting with 3:00:02
> and subtracting 1 hour (in which case, our output does reflect your
> intent).

We are not adding hours here, your example is completely unrelated.

We are adding or subtracting minutes that changes hours.

tm_sec: 2
tm_min: 60
tm_hour: 1

vs

tm_sec: 2
tm_min: 0
tm_hour: 2

And:

tm_sec: 2
tm_min: 59
tm_hour: 1

vs

tm_sec: 2
tm_min: -1
tm_hour: 2

AW


On Saturday, March 23rd, 2024 at 16:31, Rich Felker <dalias@libc.org> wrote:

> On Sat, Mar 23, 2024 at 01:49:48PM +0000, Alexander Weps wrote:
>
> > I don't think time can go backwards by incrementing field under any conditions.
> >
> > Going from:
> > tm_sec: 2
> > tm_min: 60
> > tm_hour: 1
> > tm_mday: 31
> > tm_mon: 2
> > tm_year: 124
> > tm_wday: 0
> > tm_yday: 90
> > tm_isdst: -1
> >
> > To:
tm_sec: 1
tm_min: 59
tm_hour: 2
> > tm_sec: 2
> > tm_min: 0
> > tm_hour: 1
> > tm_mday: 31
> > tm_mon: 2
> > tm_year: 124
> > tm_wday: 0
> > tm_yday: 90
> > tm_isdst: 0
> >
> > Seems to be plain wrong. I cannot come up with any argument for this
> > being correct under any conditions.
>
>
> The above broke-down time is 2:00:02, which does not exist on that day
> as a normalized time. If interpreted as non-DST, it would be just a
> couple seconds past the end of non-DST (1:59:59.99999..). If
> interpreted as DST, it would be just under an hour before the start of
> DST (3:00:00), which, after normalization, is 1:00:02 non-DST.
>
> mktime cannot tell whether your non-normalized input was the result of
> you starting with 01:00:02 and adding 1 hour (in which case, our
> output does not reflect your intent) or of you starting with 3:00:02
> and subtracting 1 hour (in which case, our output does reflect your
> intent).
>
> > mktime was given a struct tm with uncertain STD/DST, it deduced it
> > is STD and then thrown away 60 minute information. The minutes got
> > reset from 60 to 0 and no other change was done.
>
>
> It did not deduce it was STD. It deduced it was non-normalized DST
> rather than non-normalized STD (this is an arbitrary choice), then
> normalized it and got STD.
>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

So, in the meantime, I was debugging with not setting tm_isdst = -1;

This causes pretty annoying behavior:

before: 2010-10-31 14:00:00

tm_sec: 0
tm_min: 0
tm_hour: 14
tm_mday: 31
tm_mon: 9
tm_year: 110
tm_wday: 0
tm_yday: 0
tm_isdst: 0
tm_gmtoff: 3600
tm_zone: CET

tm->tm_hour = 0; <-- reset hour field
mktime(&tm);

after: 2010-10-31 01:00:00 CEST <-- 10:00:00 instead of 00:00:00
tm_sec: 0
tm_min: 0
tm_hour: 1
tm_mday: 31
tm_mon: 9
tm_year: 110
tm_wday: 0
tm_yday: 303
tm_isdst: 1
tm_gmtoff: 7200
tm_zone: CEST

tm->tm_hour = 0;
mktime(&tm);

after: 2010-10-31 00:00:00 CEST <-- second run gives a correct value
tm_sec: 0
tm_min: 0
tm_hour: 0
tm_mday: 31
tm_mon: 9
tm_year: 110
tm_wday: 0
tm_yday: 303
tm_isdst: 1
tm_gmtoff: 7200
tm_zone: CEST

This basically means that setting field twice produces different value each time:

AW


On Saturday, March 23rd, 2024 at 17:54, Alexander Weps <exander77@pm.me> wrote:

> One of the main purposes of struct tm is to calculate date and time, by adding and substracting it's fields.
>
> > mktime cannot tell whether your non-normalized input was the result of
> > you starting with 01:00:02 and adding 1 hour (in which case, our
> > output does not reflect your intent) or of you starting with 3:00:02
> > and subtracting 1 hour (in which case, our output does reflect your
> > intent).
>
>
> We are not adding hours here, your example is completely unrelated.
>
> We are adding or subtracting minutes that changes hours.
>
> tm_sec: 2
> tm_min: 60
> tm_hour: 1
>
> vs
>
> tm_sec: 2
> tm_min: 0
> tm_hour: 2
>
> And:
>
> tm_sec: 2
> tm_min: 59
> tm_hour: 1
>
> vs
>
> tm_sec: 2
> tm_min: -1
> tm_hour: 2
>
> AW
>
>
>
>
> On Saturday, March 23rd, 2024 at 16:31, Rich Felker dalias@libc.org wrote:
>
> > On Sat, Mar 23, 2024 at 01:49:48PM +0000, Alexander Weps wrote:
> >
> > > I don't think time can go backwards by incrementing field under any conditions.
> > >
> > > Going from:
> > > tm_sec: 2
> > > tm_min: 60
> > > tm_hour: 1
> > > tm_mday: 31
> > > tm_mon: 2
> > > tm_year: 124
> > > tm_wday: 0
> > > tm_yday: 90
> > > tm_isdst: -1
> > >
> > > To:
>
> tm_sec: 1
> tm_min: 59
> tm_hour: 2
>
> > > tm_sec: 2
> > > tm_min: 0
> > > tm_hour: 1
> > > tm_mday: 31
> > > tm_mon: 2
> > > tm_year: 124
> > > tm_wday: 0
> > > tm_yday: 90
> > > tm_isdst: 0
> > >
> > > Seems to be plain wrong. I cannot come up with any argument for this
> > > being correct under any conditions.
> >
> > The above broke-down time is 2:00:02, which does not exist on that day
> > as a normalized time. If interpreted as non-DST, it would be just a
> > couple seconds past the end of non-DST (1:59:59.99999..). If
> > interpreted as DST, it would be just under an hour before the start of
> > DST (3:00:00), which, after normalization, is 1:00:02 non-DST.
> >
> > mktime cannot tell whether your non-normalized input was the result of
> > you starting with 01:00:02 and adding 1 hour (in which case, our
> > output does not reflect your intent) or of you starting with 3:00:02
> > and subtracting 1 hour (in which case, our output does reflect your
> > intent).
> >
> > > mktime was given a struct tm with uncertain STD/DST, it deduced it
> > > is STD and then thrown away 60 minute information. The minutes got
> > > reset from 60 to 0 and no other change was done.
> >
> > It did not deduce it was STD. It deduced it was non-normalized DST
> > rather than non-normalized STD (this is an arbitrary choice), then
> > normalized it and got STD.
> >
> > Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

What is the reliable way to get start of this day and similar?

Seems there is just no way if you say tm_isdst = -1 is not a proper way?

For cron calculations I have to move between seconds, minutes, hours, days, months etc. reliably.

AW

On Saturday, March 23rd, 2024 at 19:57, Alexander Weps <exander77@pm.me> wrote:

> So, in the meantime, I was debugging with not setting tm_isdst = -1;
>
> This causes pretty annoying behavior:
>
> before: 2010-10-31 14:00:00
>
> tm_sec: 0
> tm_min: 0
> tm_hour: 14
> tm_mday: 31
> tm_mon: 9
> tm_year: 110
> tm_wday: 0
> tm_yday: 0
> tm_isdst: 0
> tm_gmtoff: 3600
> tm_zone: CET
>
> tm->tm_hour = 0; <-- reset hour field
>
> mktime(&tm);
>
> after: 2010-10-31 01:00:00 CEST <-- 10:00:00 instead of 00:00:00
> tm_sec: 0
> tm_min: 0
> tm_hour: 1
> tm_mday: 31
> tm_mon: 9
> tm_year: 110
> tm_wday: 0
> tm_yday: 303
> tm_isdst: 1
> tm_gmtoff: 7200
> tm_zone: CEST
>
> tm->tm_hour = 0;
>
> mktime(&tm);
>
> after: 2010-10-31 00:00:00 CEST <-- second run gives a correct value
> tm_sec: 0
> tm_min: 0
> tm_hour: 0
> tm_mday: 31
> tm_mon: 9
> tm_year: 110
> tm_wday: 0
> tm_yday: 303
> tm_isdst: 1
> tm_gmtoff: 7200
> tm_zone: CEST
>
> This basically means that setting field twice produces different value each time:
>
> AW
>
>
>
>
> On Saturday, March 23rd, 2024 at 17:54, Alexander Weps exander77@pm.me wrote:
>
> > One of the main purposes of struct tm is to calculate date and time, by adding and substracting it's fields.
> >
> > > mktime cannot tell whether your non-normalized input was the result of
> > > you starting with 01:00:02 and adding 1 hour (in which case, our
> > > output does not reflect your intent) or of you starting with 3:00:02
> > > and subtracting 1 hour (in which case, our output does reflect your
> > > intent).
> >
> > We are not adding hours here, your example is completely unrelated.
> >
> > We are adding or subtracting minutes that changes hours.
> >
> > tm_sec: 2
> > tm_min: 60
> > tm_hour: 1
> >
> > vs
> >
> > tm_sec: 2
> > tm_min: 0
> > tm_hour: 2
> >
> > And:
> >
> > tm_sec: 2
> > tm_min: 59
> > tm_hour: 1
> >
> > vs
> >
> > tm_sec: 2
> > tm_min: -1
> > tm_hour: 2
> >
> > AW
> >
> > On Saturday, March 23rd, 2024 at 16:31, Rich Felker dalias@libc.org wrote:
> >
> > > On Sat, Mar 23, 2024 at 01:49:48PM +0000, Alexander Weps wrote:
> > >
> > > > I don't think time can go backwards by incrementing field under any conditions.
> > > >
> > > > Going from:
> > > > tm_sec: 2
> > > > tm_min: 60
> > > > tm_hour: 1
> > > > tm_mday: 31
> > > > tm_mon: 2
> > > > tm_year: 124
> > > > tm_wday: 0
> > > > tm_yday: 90
> > > > tm_isdst: -1
> > > >
> > > > To:
> >
> > tm_sec: 1
> > tm_min: 59
> > tm_hour: 2
> >
> > > > tm_sec: 2
> > > > tm_min: 0
> > > > tm_hour: 1
> > > > tm_mday: 31
> > > > tm_mon: 2
> > > > tm_year: 124
> > > > tm_wday: 0
> > > > tm_yday: 90
> > > > tm_isdst: 0
> > > >
> > > > Seems to be plain wrong. I cannot come up with any argument for this
> > > > being correct under any conditions.
> > >
> > > The above broke-down time is 2:00:02, which does not exist on that day
> > > as a normalized time. If interpreted as non-DST, it would be just a
> > > couple seconds past the end of non-DST (1:59:59.99999..). If
> > > interpreted as DST, it would be just under an hour before the start of
> > > DST (3:00:00), which, after normalization, is 1:00:02 non-DST.
> > >
> > > mktime cannot tell whether your non-normalized input was the result of
> > > you starting with 01:00:02 and adding 1 hour (in which case, our
> > > output does not reflect your intent) or of you starting with 3:00:02
> > > and subtracting 1 hour (in which case, our output does reflect your
> > > intent).
> > >
> > > > mktime was given a struct tm with uncertain STD/DST, it deduced it
> > > > is STD and then thrown away 60 minute information. The minutes got
> > > > reset from 60 to 0 and no other change was done.
> > >
> > > It did not deduce it was STD. It deduced it was non-normalized DST
> > > rather than non-normalized STD (this is an arbitrary choice), then
> > > normalized it and got STD.
> > >
> > > Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Sat, Mar 23, 2024 at 06:57:21PM +0000, Alexander Weps wrote:
> So, in the meantime, I was debugging with not setting tm_isdst = -1;
> 
> This causes pretty annoying behavior:
> 
> before: 2010-10-31 14:00:00
> 
> tm_sec: 0
> tm_min: 0
> tm_hour: 14
> tm_mday: 31
> tm_mon: 9
> tm_year: 110
> tm_wday: 0
> tm_yday: 0
> tm_isdst: 0
> tm_gmtoff: 3600
> tm_zone: CET
> 
> tm->tm_hour = 0; <-- reset hour field
> mktime(&tm);
> 
> after: 2010-10-31 01:00:00 CEST <-- 10:00:00 instead of 00:00:00

I guess you meant 01:00:00 not 10:00:00. This is expected. You asked
mktime to normalize a time expressed in standard (non-DST, CET) time
but referring to a time at which DST is in effect. After
normalization, it expresses that time in DST (CEST). Since there is no
tm_isdst<0 (the only source of arbitrary implementation choices)
involved, you will find glibc and all other implementations do exactly
the same thing here.

> tm_sec: 0
> tm_min: 0
> tm_hour: 1
> tm_mday: 31
> tm_mon: 9
> tm_year: 110
> tm_wday: 0
> tm_yday: 303
> tm_isdst: 1
> tm_gmtoff: 7200
> tm_zone: CEST
> 
> tm->tm_hour = 0;
> mktime(&tm);
> 
> after: 2010-10-31 00:00:00 CEST <-- second run gives a correct value
> tm_sec: 0
> tm_min: 0
> tm_hour: 0
> tm_mday: 31
> tm_mon: 9
> tm_year: 110
> tm_wday: 0
> tm_yday: 303
> tm_isdst: 1
> tm_gmtoff: 7200
> tm_zone: CEST
> 
> This basically means that setting field twice produces different
> value each time:

No it does not. After the first time, tm_isdst is 1. Now when you
change the hour to 0, you are giving it a time expressed in DST. Since
DST is in effect at this time, it's already normalized, and you get
back what you put in.

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

Yes, the behavior is the same here glibc and musl and it can't reliably determine start of the day etc. Which is I assume expected.

That's why there is tm_isdst = -1.

I don't see any reliable way to determine beginning of the day without it.

If I want to get beginning of the day I do it this way:

before: 2010-10-31 14:00:00 CET
tm_sec: 0
tm_min: 0
tm_hour: 14
tm_mday: 31
tm_mon: 9
tm_year: 110
tm_wday: 0
tm_yday: 303
tm_isdst: 0
tm_gmtoff: 3600
tm_zone: CET

tm.tm_isdst = -1; <-- setting tm_isdst = -1
tm.tm_hour = 0;
mktime(&tm);

after: 2010-10-31 00:00:00 CEST
tm_sec: 0
tm_min: 0
tm_hour: 0
tm_mday: 31
tm_mon: 9
tm_year: 110
tm_wday: 0
tm_yday: 303
tm_isdst: 1
tm_gmtoff: 7200
tm_zone: CEST

Is there a way, how to reliable get beginning of day etc. without tm_isdst = -1.

AW


On Saturday, March 23rd, 2024 at 21:18, Rich Felker <dalias@libc.org> wrote:

> On Sat, Mar 23, 2024 at 06:57:21PM +0000, Alexander Weps wrote:
>
> > So, in the meantime, I was debugging with not setting tm_isdst = -1;
> >
> > This causes pretty annoying behavior:
> >
> > before: 2010-10-31 14:00:00
> >
> > tm_sec: 0
> > tm_min: 0
> > tm_hour: 14
> > tm_mday: 31
> > tm_mon: 9
> > tm_year: 110
> > tm_wday: 0
> > tm_yday: 0
> > tm_isdst: 0
> > tm_gmtoff: 3600
> > tm_zone: CET
> >
> > tm->tm_hour = 0; <-- reset hour field
> > mktime(&tm);
> >
> > after: 2010-10-31 01:00:00 CEST <-- 10:00:00 instead of 00:00:00
>
>
> I guess you meant 01:00:00 not 10:00:00. This is expected. You asked
> mktime to normalize a time expressed in standard (non-DST, CET) time
> but referring to a time at which DST is in effect. After
> normalization, it expresses that time in DST (CEST). Since there is no
> tm_isdst<0 (the only source of arbitrary implementation choices)
> involved, you will find glibc and all other implementations do exactly
> the same thing here.
>
> > tm_sec: 0
> > tm_min: 0
> > tm_hour: 1
> > tm_mday: 31
> > tm_mon: 9
> > tm_year: 110
> > tm_wday: 0
> > tm_yday: 303
> > tm_isdst: 1
> > tm_gmtoff: 7200
> > tm_zone: CEST
> >
> > tm->tm_hour = 0;
> > mktime(&tm);
> >
> > after: 2010-10-31 00:00:00 CEST <-- second run gives a correct value
> > tm_sec: 0
> > tm_min: 0
> > tm_hour: 0
> > tm_mday: 31
> > tm_mon: 9
> > tm_year: 110
> > tm_wday: 0
> > tm_yday: 303
> > tm_isdst: 1
> > tm_gmtoff: 7200
> > tm_zone: CEST
> >
> > This basically means that setting field twice produces different
> > value each time:
>
>
> No it does not. After the first time, tm_isdst is 1. Now when you
> change the hour to 0, you are giving it a time expressed in DST. Since
> DST is in effect at this time, it's already normalized, and you get
> back what you put in.
>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Eric Pruitt]

On Sat, Mar 23, 2024 at 08:40:50PM +0000, Alexander Weps wrote:
> Is there a way, how to reliable get beginning of day etc. without tm_isdst = -1.

I needed to do something similar in a program that involved sunset and
sunrise calculations. Depending on what you're doing and the information
you have, maybe the approach I used could work:

    static time_t round_down_to_midnight(time_t when)
    {
        char buf[20];
        struct tm *in;
        struct tm out;

        when -= when % 60;
        in = localtime(&when);

        if (in->tm_hour == 0 && in->tm_min == 0) {
            return when;
        }

        sprintf(buf, "%d-%02d-%02d 00:00:00", in->tm_year + 1900, in->tm_mon + 1,
            in->tm_mday);
        strptime(buf, "%Y-%m-%d %H:%M:%S", &out);
        return mktime(&out);
    }

The performance of this is not great, though.

Eric

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Sat, Mar 23, 2024 at 08:40:50PM +0000, Alexander Weps wrote:
> Yes, the behavior is the same here glibc and musl and it can't
> reliably determine start of the day etc. Which is I assume expected.
> 
> That's why there is tm_isdst = -1.
> 
> I don't see any reliable way to determine beginning of the day without it.

It's rather inherent to the horribleness of DST that determining the
"beginning of the day" is not easy. In fact, it might not even be
well-defined, depending on where your timezone puts its transition (it
could happen right at midnight just to be evil; that it doesn't is
only a matter of polite convention).

> If I want to get beginning of the day I do it this way:
> 
> before: 2010-10-31 14:00:00 CET
> tm_sec: 0
> tm_min: 0
> tm_hour: 14
> tm_mday: 31
> tm_mon: 9
> tm_year: 110
> tm_wday: 0
> tm_yday: 303
> tm_isdst: 0
> tm_gmtoff: 3600
> tm_zone: CET
> 
> tm.tm_isdst = -1; <-- setting tm_isdst = -1
> tm.tm_hour = 0;
> mktime(&tm);
> 
> after: 2010-10-31 00:00:00 CEST
> tm_sec: 0
> tm_min: 0
> tm_hour: 0
> tm_mday: 31
> tm_mon: 9
> tm_year: 110
> tm_wday: 0
> tm_yday: 303
> tm_isdst: 1
> tm_gmtoff: 7200
> tm_zone: CEST
> 
> Is there a way, how to reliable get beginning of day etc. without
> tm_isdst = -1.

Depending on how you want to define it, yes, but it may need a second
call to mktime. First, call mktime with 00:00:00 and tm_isdst=0. If
the result has tm_isdst==0, you're done. If not, try again with the
original struct tm input but tm_isdst changed to 1. The only way this
procedure will fail is if the time 00:00:00 *does not exist* on that
particular day.

Note that if DST ends such that there are two times 00:00:00 on a
particular day, this will pick the second (non-DST) one, as the first
one might only be the new day temporarily, then jump back to the old
day, which does not strike me as a good "beginning of the day". You
could flip the order you try them around if you prefer to count it.

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Daniel Gutson]

[-- Attachment #1: Type: text/plain, Size: 2743 bytes --]

El sáb, 23 mar 2024, 23:04, Rich Felker <dalias@libc.org> escribió:

> On Sat, Mar 23, 2024 at 08:40:50PM +0000, Alexander Weps wrote:
> > Yes, the behavior is the same here glibc and musl and it can't
> > reliably determine start of the day etc. Which is I assume expected.
> >
> > That's why there is tm_isdst = -1.
> >
> > I don't see any reliable way to determine beginning of the day without
> it.
>
> It's rather inherent to the horribleness of DST that determining the
> "beginning of the day" is not easy. In fact, it might not even be
> well-defined, depending on where your timezone puts its transition (it
> could happen right at midnight just to be evil; that it doesn't is
> only a matter of polite convention).
>
> > If I want to get beginning of the day I do it this way:
> >
> > before: 2010-10-31 14:00:00 CET
> > tm_sec: 0
> > tm_min: 0
> > tm_hour: 14
> > tm_mday: 31
> > tm_mon: 9
> > tm_year: 110
> > tm_wday: 0
> > tm_yday: 303
> > tm_isdst: 0
> > tm_gmtoff: 3600
> > tm_zone: CET
> >
> > tm.tm_isdst = -1; <-- setting tm_isdst = -1
> > tm.tm_hour = 0;
> > mktime(&tm);
> >
> > after: 2010-10-31 00:00:00 CEST
> > tm_sec: 0
> > tm_min: 0
> > tm_hour: 0
> > tm_mday: 31
> > tm_mon: 9
> > tm_year: 110
> > tm_wday: 0
> > tm_yday: 303
> > tm_isdst: 1
> > tm_gmtoff: 7200
> > tm_zone: CEST
> >
> > Is there a way, how to reliable get beginning of day etc. without
> > tm_isdst = -1.
>
> Depending on how you want to define it, yes, but it may need a second
> call to mktime. First, call mktime with 00:00:00 and tm_isdst=0. If
> the result has tm_isdst==0, you're done. If not, try again with the
> original struct tm input but tm_isdst changed to 1. The only way this
> procedure will fail is if the time 00:00:00 *does not exist* on that
> particular day.
>

I was so curious about this that I asked Perplexity for an example:

"For example, in the case of Samoa as mentioned in the search results, on
December 30, 2011, the time went from 23:59:59 on December 29 straight to
00:00:00 on December 31, effectively skipping December 30 entirely as the
country moved west of the International Date Line and also changed its time
zone from UTC-11 to UTC+13"
(
https://www.caktusgroup.com/blog/2019/03/21/coding-time-zones-and-daylight-saving-time/
)

This is sadly entertaining.


> Note that if DST ends such that there are two times 00:00:00 on a
> particular day, this will pick the second (non-DST) one, as the first
> one might only be the new day temporarily, then jump back to the old
> day, which does not strike me as a good "beginning of the day". You
> could flip the order you try them around if you prefer to count it.
>
> Rich
>

[-- Attachment #2: Type: text/html, Size: 5540 bytes --]

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

[-- Attachment #1: Type: text/plain, Size: 3323 bytes --]

My head cannon for Israeli-Palestinian war is that it's a war about superiority of ones timezone.

Single region, two timezone, two different DST changes. Depending on your nationality, you have different timezone, your job may have different timezone, you bus may have different timezone, your mobile phone may serve different timezone... and there can be a daylight saving on each of these timezone and it has different start and end for each one.

It's pure insanity.

https://english.news.cn/20220328/8a1ad2ddbde14941a6f208f1f175b550/c.html

AW

On Sunday, March 24th, 2024 at 04:32, Daniel Gutson <danielgutson@gmail.com> wrote:

> El sáb, 23 mar 2024, 23:04, Rich Felker <dalias@libc.org> escribió:
>
>> On Sat, Mar 23, 2024 at 08:40:50PM +0000, Alexander Weps wrote:
>>> Yes, the behavior is the same here glibc and musl and it can't
>>> reliably determine start of the day etc. Which is I assume expected.
>>>
>>> That's why there is tm_isdst = -1.
>>>
>>> I don't see any reliable way to determine beginning of the day without it.
>>
>> It's rather inherent to the horribleness of DST that determining the
>> "beginning of the day" is not easy. In fact, it might not even be
>> well-defined, depending on where your timezone puts its transition (it
>> could happen right at midnight just to be evil; that it doesn't is
>> only a matter of polite convention).
>>
>>> If I want to get beginning of the day I do it this way:
>>>
>>> before: 2010-10-31 14:00:00 CET
>>> tm_sec: 0
>>> tm_min: 0
>>> tm_hour: 14
>>> tm_mday: 31
>>> tm_mon: 9
>>> tm_year: 110
>>> tm_wday: 0
>>> tm_yday: 303
>>> tm_isdst: 0
>>> tm_gmtoff: 3600
>>> tm_zone: CET
>>>
>>> tm.tm_isdst = -1; <-- setting tm_isdst = -1
>>> tm.tm_hour = 0;
>>> mktime(&tm);
>>>
>>> after: 2010-10-31 00:00:00 CEST
>>> tm_sec: 0
>>> tm_min: 0
>>> tm_hour: 0
>>> tm_mday: 31
>>> tm_mon: 9
>>> tm_year: 110
>>> tm_wday: 0
>>> tm_yday: 303
>>> tm_isdst: 1
>>> tm_gmtoff: 7200
>>> tm_zone: CEST
>>>
>>> Is there a way, how to reliable get beginning of day etc. without
>>> tm_isdst = -1.
>>
>> Depending on how you want to define it, yes, but it may need a second
>> call to mktime. First, call mktime with 00:00:00 and tm_isdst=0. If
>> the result has tm_isdst==0, you're done. If not, try again with the
>> original struct tm input but tm_isdst changed to 1. The only way this
>> procedure will fail is if the time 00:00:00 *does not exist* on that
>> particular day.
>
> I was so curious about this that I asked Perplexity for an example:
>
> "For example, in the case of Samoa as mentioned in the search results, on December 30, 2011, the time went from 23:59:59 on December 29 straight to 00:00:00 on December 31, effectively skipping December 30 entirely as the country moved west of the International Date Line and also changed its time zone from UTC-11 to UTC+13"
> (https://www.caktusgroup.com/blog/2019/03/21/coding-time-zones-and-daylight-saving-time/)
>
> This is sadly entertaining.
>
>> Note that if DST ends such that there are two times 00:00:00 on a
>> particular day, this will pick the second (non-DST) one, as the first
>> one might only be the new day temporarily, then jump back to the old
>> day, which does not strike me as a good "beginning of the day". You
>> could flip the order you try them around if you prefer to count it.
>>
>> Rich

[-- Attachment #2: Type: text/html, Size: 7402 bytes --]

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

[-- Attachment #1: Type: text/plain, Size: 3900 bytes --]

Also thanks for the Pacific/Apia example. Not only that it fails for that date:
Pattern: * * * * * *
Initial: 2011-12-29_23:59:59
Expected: 2011-32-31_00:00:00
Actual: 2011-12-29_00:00:00
My cron tool again going back in time.

It fails one other test.

I have to run my tests on multiple timezones.

And it works in glibc.

And that's after I removed tm_isdst and rewrote half the code to accommodate.

AW

On Sunday, March 24th, 2024 at 12:05, Alexander Weps <exander77@pm.me> wrote:

> My head cannon for Israeli-Palestinian war is that it's a war about superiority of ones timezone.
>
> Single region, two timezone, two different DST changes. Depending on your nationality, you have different timezone, your job may have different timezone, you bus may have different timezone, your mobile phone may serve different timezone... and there can be a daylight saving on each of these timezone and it has different start and end for each one.
>
> It's pure insanity.
>
> https://english.news.cn/20220328/8a1ad2ddbde14941a6f208f1f175b550/c.html
>
> AW
>
> On Sunday, March 24th, 2024 at 04:32, Daniel Gutson <danielgutson@gmail.com> wrote:
>
>> El sáb, 23 mar 2024, 23:04, Rich Felker <dalias@libc.org> escribió:
>>
>>> On Sat, Mar 23, 2024 at 08:40:50PM +0000, Alexander Weps wrote:
>>>> Yes, the behavior is the same here glibc and musl and it can't
>>>> reliably determine start of the day etc. Which is I assume expected.
>>>>
>>>> That's why there is tm_isdst = -1.
>>>>
>>>> I don't see any reliable way to determine beginning of the day without it.
>>>
>>> It's rather inherent to the horribleness of DST that determining the
>>> "beginning of the day" is not easy. In fact, it might not even be
>>> well-defined, depending on where your timezone puts its transition (it
>>> could happen right at midnight just to be evil; that it doesn't is
>>> only a matter of polite convention).
>>>
>>>> If I want to get beginning of the day I do it this way:
>>>>
>>>> before: 2010-10-31 14:00:00 CET
>>>> tm_sec: 0
>>>> tm_min: 0
>>>> tm_hour: 14
>>>> tm_mday: 31
>>>> tm_mon: 9
>>>> tm_year: 110
>>>> tm_wday: 0
>>>> tm_yday: 303
>>>> tm_isdst: 0
>>>> tm_gmtoff: 3600
>>>> tm_zone: CET
>>>>
>>>> tm.tm_isdst = -1; <-- setting tm_isdst = -1
>>>> tm.tm_hour = 0;
>>>> mktime(&tm);
>>>>
>>>> after: 2010-10-31 00:00:00 CEST
>>>> tm_sec: 0
>>>> tm_min: 0
>>>> tm_hour: 0
>>>> tm_mday: 31
>>>> tm_mon: 9
>>>> tm_year: 110
>>>> tm_wday: 0
>>>> tm_yday: 303
>>>> tm_isdst: 1
>>>> tm_gmtoff: 7200
>>>> tm_zone: CEST
>>>>
>>>> Is there a way, how to reliable get beginning of day etc. without
>>>> tm_isdst = -1.
>>>
>>> Depending on how you want to define it, yes, but it may need a second
>>> call to mktime. First, call mktime with 00:00:00 and tm_isdst=0. If
>>> the result has tm_isdst==0, you're done. If not, try again with the
>>> original struct tm input but tm_isdst changed to 1. The only way this
>>> procedure will fail is if the time 00:00:00 *does not exist* on that
>>> particular day.
>>
>> I was so curious about this that I asked Perplexity for an example:
>>
>> "For example, in the case of Samoa as mentioned in the search results, on December 30, 2011, the time went from 23:59:59 on December 29 straight to 00:00:00 on December 31, effectively skipping December 30 entirely as the country moved west of the International Date Line and also changed its time zone from UTC-11 to UTC+13"
>> (https://www.caktusgroup.com/blog/2019/03/21/coding-time-zones-and-daylight-saving-time/)
>>
>> This is sadly entertaining.
>>
>>> Note that if DST ends such that there are two times 00:00:00 on a
>>> particular day, this will pick the second (non-DST) one, as the first
>>> one might only be the new day temporarily, then jump back to the old
>>> day, which does not strike me as a good "beginning of the day". You
>>> could flip the order you try them around if you prefer to count it.
>>>
>>> Rich

[-- Attachment #2: Type: text/html, Size: 9221 bytes --]

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

This is how it should work as far as I am concerned. After manipulating the date, the tm_isdst is set to -1.

From https://cplusplus.com/reference/ctime/tm/:
The Daylight Saving Time flag (tm_isdst) is greater than zero if Daylight Saving Time is in effect, zero if Daylight Saving Time is not in effect, and less than zero if the information is not available.

And the date should be correctly set as DST or STD in mktime.

I have a date. I make change in fields, I set tm_isdst = -1, I call mktime.

I see only on place where tm_isdst is checked and that's:
time_t mktime(struct tm *tm)
{
    struct tm new;
    long opp;
    long long t = __tm_to_secs(tm);

    __secs_to_zone(t, 1, &new.tm_isdst, &new.__tm_gmtoff, &opp, &new.__tm_zone);

    if (tm->tm_isdst>=0 && new.tm_isdst!=tm->tm_isdst)
        t -= opp - new.__tm_gmtoff;

    t -= new.__tm_gmtoff;
    if ((time_t)t != t) goto error;

    __secs_to_zone(t, 0, &new.tm_isdst, &new.__tm_gmtoff, &opp, &new.__tm_zone);

    if (__secs_to_tm(t + new.__tm_gmtoff, &new) < 0) goto error;

    *tm = new;
    return t;

error:
    errno = EOVERFLOW;
    return -1;
}

So tm->tm_isdst>=0 seems to be the cause here.

AW

On Saturday, March 23rd, 2024 at 13:00, Alexander Weps <exander77@pm.me> wrote:

> > You don't need to set yday or the others before calling mktime().
>
>
> I thought that too, but wanted to test it on exactly the same struct tm. No change in behavior.
>
> So I have found a minimal example:
>
> void test()
> {
> time_t t;
> struct tm tm = {0};
> char buf[64];
>
> tm.tm_year = 2024 - 1900;
> tm.tm_mon = 3 - 1;
> tm.tm_mday = 31;
> tm.tm_hour = 1;
> tm.tm_min = 59;
> tm.tm_sec = 2;
> print_tm(&tm);
>
> mktime(&tm);
> strftime(buf, sizeof buf, "%F %T %Z", &tm);
> printf("before: %s\n", buf);
> tm.tm_isdst = -1; // This is the cause.
> tm.tm_min += 1;
> mktime(&tm);
> strftime(buf, sizeof buf, "%F %T %Z", &tm);
> printf("after: %s\n", buf);
> }
>
> Setting tm_isdst to -1 after first mktime and before second mktime causes the issue.
>
> AW
>
>
>
>
> On Saturday, March 23rd, 2024 at 11:38, Markus Wichmann nullplan@gmx.net wrote:
>
> > Am Sat, Mar 23, 2024 at 09:26:00AM +0000 schrieb Alexander Weps:
> >
> > > This works for me as well even after changes to match struct tm in my
> > > app (setting tm_yday, __tm_gtmoff a __tm_zone):
> >
> > You don't need to set yday or the others before calling mktime().
> > mktime() is defined to only use year, mon, mday, hour, min, sec, and
> > isdst as input, normalize them, and calculate the others (and also the
> > UNIX time stamp).
> >
> > > Any idea how could a previous calculation mess with musl internals so
> > > it would start producing bad results? Because otherwise, I don't see
> > > how this could be happening if you completely extract it out of the
> > > code and it works.
> >
> > The only thing that means is that the isolated code works, and the bug
> > is elsewhere. I'm sorry, but you are going to have to debug this
> > yourself. There may be some static memory getting corrupted (e.g. the TZ
> > and rule caches), but honestly this is just speculation.
> >
> > > And when I compile under glibc, everything is fine.
> >
> > That tends to indicate some undefined behavior. Not that that helps you
> > find the reason. You are going to have to debug it. Finding a minimum
> > reproducer may help in that, or you directly apply liberal amounts of
> > gdb.
> >
> > You seem to have dropped the list from CC, so I'm adding it back.
> >
> > Ciao,
> > Markus

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

[-- Attachment #1: Type: text/plain, Size: 4897 bytes --]

Also thanks for the Pacific/Apia example. Not only that it fails for that date:
Pattern: * * * * * *
Initial: 2011-12-29_23:59:59
Expected: 2011-32-31_00:00:00
Actual: 2011-12-29_00:00:00
My cron tool again going back in time.

It fails one other test.

I have to run my tests on multiple timezones.

And it works in glibc.

And that's after I removed tm_isdst and rewrote half the code to accommodate.

Can We agree on some simple premise that with no uncertain STD/DST settings (tm_isdst = 0 or tm_isdst = 1), incrementing seconds by 1 and calling mktime should never cause time to go back?

Well, behold Pacific/Apia:

I set 2011-12-29 23:59:59:

tm_sec: 59
tm_min: 59
tm_hour: 23
tm_mday: 29
tm_mon: 11
tm_year: 111
tm_wday: 0
tm_yday: 0
tm_isdst: 1
tm_gmtoff: 0
tm_zone: (null)

Calling mktime to see if everything is correct:
mktime(&tm);

before: 2011-12-29 23:59:59 -10
tm_sec: 59
tm_min: 59
tm_hour: 23
tm_mday: 29
tm_mon: 11
tm_year: 111
tm_wday: 4
tm_yday: 362
tm_isdst: 1
tm_gmtoff: -36000
tm_zone: -10

Incrementing seconds and calling mktime:
tm.tm_sec += 1;
mktime(&tm);

after: 2011-12-29 00:00:00 -10
tm_sec: 0
tm_min: 0
tm_hour: 0
tm_mday: 29
tm_mon: 11
tm_year: 111
tm_wday: 4
tm_yday: 362
tm_isdst: 1
tm_gmtoff: -36000
tm_zone: -10
We went from:
2011-12-29 23:59:59 -10
To:
2011-12-29 00:00:00 -10

By adding 1 second. The tm_isdst was not set to -1.

This is totally unreliable.

AW

On Sunday, March 24th, 2024 at 12:05, Alexander Weps <exander77@pm.me> wrote:

> My head cannon for Israeli-Palestinian war is that it's a war about superiority of ones timezone.
>
> Single region, two timezone, two different DST changes. Depending on your nationality, you have different timezone, your job may have different timezone, you bus may have different timezone, your mobile phone may serve different timezone... and there can be a daylight saving on each of these timezone and it has different start and end for each one.
>
> It's pure insanity.
>
> https://english.news.cn/20220328/8a1ad2ddbde14941a6f208f1f175b550/c.html
>
> AW
>
> On Sunday, March 24th, 2024 at 04:32, Daniel Gutson <danielgutson@gmail.com> wrote:
>
>> El sáb, 23 mar 2024, 23:04, Rich Felker <dalias@libc.org> escribió:
>>
>>> On Sat, Mar 23, 2024 at 08:40:50PM +0000, Alexander Weps wrote:
>>>> Yes, the behavior is the same here glibc and musl and it can't
>>>> reliably determine start of the day etc. Which is I assume expected.
>>>>
>>>> That's why there is tm_isdst = -1.
>>>>
>>>> I don't see any reliable way to determine beginning of the day without it.
>>>
>>> It's rather inherent to the horribleness of DST that determining the
>>> "beginning of the day" is not easy. In fact, it might not even be
>>> well-defined, depending on where your timezone puts its transition (it
>>> could happen right at midnight just to be evil; that it doesn't is
>>> only a matter of polite convention).
>>>
>>>> If I want to get beginning of the day I do it this way:
>>>>
>>>> before: 2010-10-31 14:00:00 CET
>>>> tm_sec: 0
>>>> tm_min: 0
>>>> tm_hour: 14
>>>> tm_mday: 31
>>>> tm_mon: 9
>>>> tm_year: 110
>>>> tm_wday: 0
>>>> tm_yday: 303
>>>> tm_isdst: 0
>>>> tm_gmtoff: 3600
>>>> tm_zone: CET
>>>>
>>>> tm.tm_isdst = -1; <-- setting tm_isdst = -1
>>>> tm.tm_hour = 0;
>>>> mktime(&tm);
>>>>
>>>> after: 2010-10-31 00:00:00 CEST
>>>> tm_sec: 0
>>>> tm_min: 0
>>>> tm_hour: 0
>>>> tm_mday: 31
>>>> tm_mon: 9
>>>> tm_year: 110
>>>> tm_wday: 0
>>>> tm_yday: 303
>>>> tm_isdst: 1
>>>> tm_gmtoff: 7200
>>>> tm_zone: CEST
>>>>
>>>> Is there a way, how to reliable get beginning of day etc. without
>>>> tm_isdst = -1.
>>>
>>> Depending on how you want to define it, yes, but it may need a second
>>> call to mktime. First, call mktime with 00:00:00 and tm_isdst=0. If
>>> the result has tm_isdst==0, you're done. If not, try again with the
>>> original struct tm input but tm_isdst changed to 1. The only way this
>>> procedure will fail is if the time 00:00:00 *does not exist* on that
>>> particular day.
>>
>> I was so curious about this that I asked Perplexity for an example:
>>
>> "For example, in the case of Samoa as mentioned in the search results, on December 30, 2011, the time went from 23:59:59 on December 29 straight to 00:00:00 on December 31, effectively skipping December 30 entirely as the country moved west of the International Date Line and also changed its time zone from UTC-11 to UTC+13"
>> (https://www.caktusgroup.com/blog/2019/03/21/coding-time-zones-and-daylight-saving-time/)
>>
>> This is sadly entertaining.
>>
>>> Note that if DST ends such that there are two times 00:00:00 on a
>>> particular day, this will pick the second (non-DST) one, as the first
>>> one might only be the new day temporarily, then jump back to the old
>>> day, which does not strike me as a good "beginning of the day". You
>>> could flip the order you try them around if you prefer to count it.
>>>
>>> Rich

[-- Attachment #2: Type: text/html, Size: 13353 bytes --]

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

[-- Attachment #1: Type: text/plain, Size: 6163 bytes --]

I see other weird things in Pacific/Apia, but these are at least consistent with glibc (and not an utter nonsense like going back in time):

tm_sec: 0
tm_min: 0
tm_hour: 1
tm_mday: 26
tm_mon: 8
tm_year: 110
tm_wday: 0
tm_yday: 0
tm_isdst: 1
tm_gmtoff: 0
tm_zone: (null)

mktime(&tm);

before: 2010-09-26 01:00:00 -10
tm_sec: 0
tm_min: 0
tm_hour: 1
tm_mday: 26
tm_mon: 8
tm_year: 110
tm_wday: 0
tm_yday: 268
tm_isdst: 1
tm_gmtoff: -36000
tm_zone: -10

tm.tm_hour = 0; mktime(&tm);

after: 2010-09-25 23:00:00 -11
tm_sec: 0
tm_min: 0
tm_hour: 23
tm_mday: 25
tm_mon: 8
tm_year: 110
tm_wday: 6
tm_yday: 267
tm_isdst: 0
tm_gmtoff: -39600
tm_zone: -11
But here even doubly calling:
tm.tm_hour = 0; mktime(&tm);
tm.tm_hour = 0; mktime(&tm);

Doesn't fix the issue, as you get:
2010-09-25 00:00:00 -11

There should be a reliable way how to determine a beginning of a day without tm_isdst = -1. I used that before. I can use that in glibc, but what can I do in musl?

AW

On Sunday, March 24th, 2024 at 14:36, Alexander Weps <exander77@pm.me> wrote:

> Also thanks for the Pacific/Apia example. Not only that it fails for that date:
> Pattern: * * * * * *
> Initial: 2011-12-29_23:59:59
> Expected: 2011-32-31_00:00:00
> Actual: 2011-12-29_00:00:00
> My cron tool again going back in time.
>
> It fails one other test.
>
> I have to run my tests on multiple timezones.
>
> And it works in glibc.
>
> And that's after I removed tm_isdst and rewrote half the code to accommodate.
>
> Can We agree on some simple premise that with no uncertain STD/DST settings (tm_isdst = 0 or tm_isdst = 1), incrementing seconds by 1 and calling mktime should never cause time to go back?
>
> Well, behold Pacific/Apia:
>
> I set 2011-12-29 23:59:59:
>
> tm_sec: 59
> tm_min: 59
> tm_hour: 23
> tm_mday: 29
> tm_mon: 11
> tm_year: 111
> tm_wday: 0
> tm_yday: 0
> tm_isdst: 1
> tm_gmtoff: 0
> tm_zone: (null)
>
> Calling mktime to see if everything is correct:
> mktime(&tm);
>
> before: 2011-12-29 23:59:59 -10
> tm_sec: 59
> tm_min: 59
> tm_hour: 23
> tm_mday: 29
> tm_mon: 11
> tm_year: 111
> tm_wday: 4
> tm_yday: 362
> tm_isdst: 1
> tm_gmtoff: -36000
> tm_zone: -10
>
> Incrementing seconds and calling mktime:
> tm.tm_sec += 1;
> mktime(&tm);
>
> after: 2011-12-29 00:00:00 -10
> tm_sec: 0
> tm_min: 0
> tm_hour: 0
> tm_mday: 29
> tm_mon: 11
> tm_year: 111
> tm_wday: 4
> tm_yday: 362
> tm_isdst: 1
> tm_gmtoff: -36000
> tm_zone: -10
> We went from:
> 2011-12-29 23:59:59 -10
> To:
> 2011-12-29 00:00:00 -10
>
> By adding 1 second. The tm_isdst was not set to -1.
>
> This is totally unreliable.
>
> AW
>
> On Sunday, March 24th, 2024 at 12:05, Alexander Weps <exander77@pm.me> wrote:
>
>> My head cannon for Israeli-Palestinian war is that it's a war about superiority of ones timezone.
>>
>> Single region, two timezone, two different DST changes. Depending on your nationality, you have different timezone, your job may have different timezone, you bus may have different timezone, your mobile phone may serve different timezone... and there can be a daylight saving on each of these timezone and it has different start and end for each one.
>>
>> It's pure insanity.
>>
>> https://english.news.cn/20220328/8a1ad2ddbde14941a6f208f1f175b550/c.html
>>
>> AW
>>
>> On Sunday, March 24th, 2024 at 04:32, Daniel Gutson <danielgutson@gmail.com> wrote:
>>
>>> El sáb, 23 mar 2024, 23:04, Rich Felker <dalias@libc.org> escribió:
>>>
>>>> On Sat, Mar 23, 2024 at 08:40:50PM +0000, Alexander Weps wrote:
>>>>> Yes, the behavior is the same here glibc and musl and it can't
>>>>> reliably determine start of the day etc. Which is I assume expected.
>>>>>
>>>>> That's why there is tm_isdst = -1.
>>>>>
>>>>> I don't see any reliable way to determine beginning of the day without it.
>>>>
>>>> It's rather inherent to the horribleness of DST that determining the
>>>> "beginning of the day" is not easy. In fact, it might not even be
>>>> well-defined, depending on where your timezone puts its transition (it
>>>> could happen right at midnight just to be evil; that it doesn't is
>>>> only a matter of polite convention).
>>>>
>>>>> If I want to get beginning of the day I do it this way:
>>>>>
>>>>> before: 2010-10-31 14:00:00 CET
>>>>> tm_sec: 0
>>>>> tm_min: 0
>>>>> tm_hour: 14
>>>>> tm_mday: 31
>>>>> tm_mon: 9
>>>>> tm_year: 110
>>>>> tm_wday: 0
>>>>> tm_yday: 303
>>>>> tm_isdst: 0
>>>>> tm_gmtoff: 3600
>>>>> tm_zone: CET
>>>>>
>>>>> tm.tm_isdst = -1; <-- setting tm_isdst = -1
>>>>> tm.tm_hour = 0;
>>>>> mktime(&tm);
>>>>>
>>>>> after: 2010-10-31 00:00:00 CEST
>>>>> tm_sec: 0
>>>>> tm_min: 0
>>>>> tm_hour: 0
>>>>> tm_mday: 31
>>>>> tm_mon: 9
>>>>> tm_year: 110
>>>>> tm_wday: 0
>>>>> tm_yday: 303
>>>>> tm_isdst: 1
>>>>> tm_gmtoff: 7200
>>>>> tm_zone: CEST
>>>>>
>>>>> Is there a way, how to reliable get beginning of day etc. without
>>>>> tm_isdst = -1.
>>>>
>>>> Depending on how you want to define it, yes, but it may need a second
>>>> call to mktime. First, call mktime with 00:00:00 and tm_isdst=0. If
>>>> the result has tm_isdst==0, you're done. If not, try again with the
>>>> original struct tm input but tm_isdst changed to 1. The only way this
>>>> procedure will fail is if the time 00:00:00 *does not exist* on that
>>>> particular day.
>>>
>>> I was so curious about this that I asked Perplexity for an example:
>>>
>>> "For example, in the case of Samoa as mentioned in the search results, on December 30, 2011, the time went from 23:59:59 on December 29 straight to 00:00:00 on December 31, effectively skipping December 30 entirely as the country moved west of the International Date Line and also changed its time zone from UTC-11 to UTC+13"
>>> (https://www.caktusgroup.com/blog/2019/03/21/coding-time-zones-and-daylight-saving-time/)
>>>
>>> This is sadly entertaining.
>>>
>>>> Note that if DST ends such that there are two times 00:00:00 on a
>>>> particular day, this will pick the second (non-DST) one, as the first
>>>> one might only be the new day temporarily, then jump back to the old
>>>> day, which does not strike me as a good "beginning of the day". You
>>>> could flip the order you try them around if you prefer to count it.
>>>>
>>>> Rich

[-- Attachment #2: Type: text/html, Size: 17632 bytes --]

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Sun, Mar 24, 2024 at 01:36:42PM +0000, Alexander Weps wrote:
> Also thanks for the Pacific/Apia example. Not only that it fails for that date:
> Pattern: * * * * * *
> Initial: 2011-12-29_23:59:59
> Expected: 2011-32-31_00:00:00
> Actual: 2011-12-29_00:00:00
> My cron tool again going back in time.
> 
> It fails one other test.
> 
> I have to run my tests on multiple timezones.
> 
> And it works in glibc.
> 
> And that's after I removed tm_isdst and rewrote half the code to accommodate.
> 
> Can We agree on some simple premise that with no uncertain STD/DST
> settings (tm_isdst = 0 or tm_isdst = 1), incrementing seconds by 1
> and calling mktime should never cause time to go back?
> 
> Well, behold Pacific/Apia:
> 
> I set 2011-12-29 23:59:59:
> 
> tm_sec: 59
> tm_min: 59
> tm_hour: 23
> tm_mday: 29
> tm_mon: 11
> tm_year: 111
> tm_wday: 0
> tm_yday: 0
> tm_isdst: 1
> tm_gmtoff: 0
> tm_zone: (null)
> 
> Calling mktime to see if everything is correct:
> mktime(&tm);
> 
> before: 2011-12-29 23:59:59 -10
> tm_sec: 59
> tm_min: 59
> tm_hour: 23
> tm_mday: 29
> tm_mon: 11
> tm_year: 111
> tm_wday: 4
> tm_yday: 362
> tm_isdst: 1
> tm_gmtoff: -36000
> tm_zone: -10
> 
> Incrementing seconds and calling mktime:
> tm.tm_sec += 1;
> mktime(&tm);
> 
> after: 2011-12-29 00:00:00 -10
> tm_sec: 0
> tm_min: 0
> tm_hour: 0
> tm_mday: 29
> tm_mon: 11
> tm_year: 111
> tm_wday: 4
> tm_yday: 362
> tm_isdst: 1
> tm_gmtoff: -36000
> tm_zone: -10
> We went from:
> 2011-12-29 23:59:59 -10
> To:
> 2011-12-29 00:00:00 -10
> 
> By adding 1 second. The tm_isdst was not set to -1.
> 
> This is totally unreliable.

From what I understand, you've set the input to a time that doesn't
exist in the local timezone: 2011-12-30 00:00:00. This could be either
1 second after 2011-12-29 23:59:59, as you intended, or 1 day before
2011-12-31 00:00:00. The latter is how it was interpreted. However, it
does not seem to have correctly set tm_gmtoff to reflect how it was
interpreted. We should check this out because that's probably an
actual bug.

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

Adding seconds cannot make time go backwards. If that is how it was interpreted, than the interpretation is erroneous.

I think I will do some test.

If you start with 1900-01-01 00:00:00 and start adding seconds and calling mktime, you should reliable get to 2200-12-31 59:59:59 in all timezones. Same goes for minutes, hours, days, years.

If that's not the case then the interpretation in musl is wrong.

This is a basic elementary functionality of mktime.

There can't be any going back in time a cycles.

AW

On Sunday, March 24th, 2024 at 18:04, Rich Felker <dalias@libc.org> wrote:

> On Sun, Mar 24, 2024 at 01:36:42PM +0000, Alexander Weps wrote:
>
> > Also thanks for the Pacific/Apia example. Not only that it fails for that date:
> > Pattern: * * * * * *
> > Initial: 2011-12-29_23:59:59
> > Expected: 2011-32-31_00:00:00
> > Actual: 2011-12-29_00:00:00
> > My cron tool again going back in time.
> >
> > It fails one other test.
> >
> > I have to run my tests on multiple timezones.
> >
> > And it works in glibc.
> >
> > And that's after I removed tm_isdst and rewrote half the code to accommodate.
> >
> > Can We agree on some simple premise that with no uncertain STD/DST
> > settings (tm_isdst = 0 or tm_isdst = 1), incrementing seconds by 1
> > and calling mktime should never cause time to go back?
> >
> > Well, behold Pacific/Apia:
> >
> > I set 2011-12-29 23:59:59:
> >
> > tm_sec: 59
> > tm_min: 59
> > tm_hour: 23
> > tm_mday: 29
> > tm_mon: 11
> > tm_year: 111
> > tm_wday: 0
> > tm_yday: 0
> > tm_isdst: 1
> > tm_gmtoff: 0
> > tm_zone: (null)
> >
> > Calling mktime to see if everything is correct:
> > mktime(&tm);
> >
> > before: 2011-12-29 23:59:59 -10
> > tm_sec: 59
> > tm_min: 59
> > tm_hour: 23
> > tm_mday: 29
> > tm_mon: 11
> > tm_year: 111
> > tm_wday: 4
> > tm_yday: 362
> > tm_isdst: 1
> > tm_gmtoff: -36000
> > tm_zone: -10
> >
> > Incrementing seconds and calling mktime:
> > tm.tm_sec += 1;
> > mktime(&tm);
> >
> > after: 2011-12-29 00:00:00 -10
> > tm_sec: 0
> > tm_min: 0
> > tm_hour: 0
> > tm_mday: 29
> > tm_mon: 11
> > tm_year: 111
> > tm_wday: 4
> > tm_yday: 362
> > tm_isdst: 1
> > tm_gmtoff: -36000
> > tm_zone: -10
> > We went from:
> > 2011-12-29 23:59:59 -10
> > To:
> > 2011-12-29 00:00:00 -10
> >
> > By adding 1 second. The tm_isdst was not set to -1.
> >
> > This is totally unreliable.
>
>
> From what I understand, you've set the input to a time that doesn't
> exist in the local timezone: 2011-12-30 00:00:00. This could be either
> 1 second after 2011-12-29 23:59:59, as you intended, or 1 day before
> 2011-12-31 00:00:00. The latter is how it was interpreted. However, it
> does not seem to have correctly set tm_gmtoff to reflect how it was
> interpreted. We should check this out because that's probably an
> actual bug.
>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

So I started from:
1900-01-01 00:00:00
Adding 59 seconds each cycle (to speed things up).

And I already have some cycles in Musl even for Europe/Prague.
1946-12-01 02:59:17 CET
1946-12-01 01:00:16 CET <-- There should be: 1946-12-01 02:00:16 GMT

This looks like an issue involving negative DST change.

There were 3 STD/DST changes in 1946 in Europe/Prague:

1945-10-01 01:00:00Z +01:00:00 standard CET

1946-05-06 01:00:00Z +02:00:00 daylight CEST
1946-10-06 01:00:00Z +01:00:00 standard CET
1946-12-01 02:00:00Z +00:00:00 daylight GMT

1947-02-23 02:00:00Z +01:00:00 standard CET

And it works in glibc. I was able to run it all the way from 1900 to 2200.

Glibc correctly went from CET to GMT and then back:
1946-12-01 02:59:57 CET
1946-12-01 02:00:56 GMT
...

AW

On Sunday, March 24th, 2024 at 18:12, Alexander Weps <exander77@pm.me> wrote:

> Adding seconds cannot make time go backwards. If that is how it was interpreted, than the interpretation is erroneous.
>
> I think I will do some test.
>
> If you start with 1900-01-01 00:00:00 and start adding seconds and calling mktime, you should reliable get to 2200-12-31 59:59:59 in all timezones. Same goes for minutes, hours, days, years.
>
> If that's not the case then the interpretation in musl is wrong.
>
> This is a basic elementary functionality of mktime.
>
> There can't be any going back in time a cycles.
>
> AW
>
>
>
> On Sunday, March 24th, 2024 at 18:04, Rich Felker dalias@libc.org wrote:
>
> > On Sun, Mar 24, 2024 at 01:36:42PM +0000, Alexander Weps wrote:
> >
> > > Also thanks for the Pacific/Apia example. Not only that it fails for that date:
> > > Pattern: * * * * * *
> > > Initial: 2011-12-29_23:59:59
> > > Expected: 2011-32-31_00:00:00
> > > Actual: 2011-12-29_00:00:00
> > > My cron tool again going back in time.
> > >
> > > It fails one other test.
> > >
> > > I have to run my tests on multiple timezones.
> > >
> > > And it works in glibc.
> > >
> > > And that's after I removed tm_isdst and rewrote half the code to accommodate.
> > >
> > > Can We agree on some simple premise that with no uncertain STD/DST
> > > settings (tm_isdst = 0 or tm_isdst = 1), incrementing seconds by 1
> > > and calling mktime should never cause time to go back?
> > >
> > > Well, behold Pacific/Apia:
> > >
> > > I set 2011-12-29 23:59:59:
> > >
> > > tm_sec: 59
> > > tm_min: 59
> > > tm_hour: 23
> > > tm_mday: 29
> > > tm_mon: 11
> > > tm_year: 111
> > > tm_wday: 0
> > > tm_yday: 0
> > > tm_isdst: 1
> > > tm_gmtoff: 0
> > > tm_zone: (null)
> > >
> > > Calling mktime to see if everything is correct:
> > > mktime(&tm);
> > >
> > > before: 2011-12-29 23:59:59 -10
> > > tm_sec: 59
> > > tm_min: 59
> > > tm_hour: 23
> > > tm_mday: 29
> > > tm_mon: 11
> > > tm_year: 111
> > > tm_wday: 4
> > > tm_yday: 362
> > > tm_isdst: 1
> > > tm_gmtoff: -36000
> > > tm_zone: -10
> > >
> > > Incrementing seconds and calling mktime:
> > > tm.tm_sec += 1;
> > > mktime(&tm);
> > >
> > > after: 2011-12-29 00:00:00 -10
> > > tm_sec: 0
> > > tm_min: 0
> > > tm_hour: 0
> > > tm_mday: 29
> > > tm_mon: 11
> > > tm_year: 111
> > > tm_wday: 4
> > > tm_yday: 362
> > > tm_isdst: 1
> > > tm_gmtoff: -36000
> > > tm_zone: -10
> > > We went from:
> > > 2011-12-29 23:59:59 -10
> > > To:
> > > 2011-12-29 00:00:00 -10
> > >
> > > By adding 1 second. The tm_isdst was not set to -1.
> > >
> > > This is totally unreliable.
> >
> > From what I understand, you've set the input to a time that doesn't
> > exist in the local timezone: 2011-12-30 00:00:00. This could be either
> > 1 second after 2011-12-29 23:59:59, as you intended, or 1 day before
> > 2011-12-31 00:00:00. The latter is how it was interpreted. However, it
> > does not seem to have correctly set tm_gmtoff to reflect how it was
> > interpreted. We should check this out because that's probably an
> > actual bug.
> >
> > Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Sun, Mar 24, 2024 at 05:12:35PM +0000, Alexander Weps wrote:
> Adding seconds cannot make time go backwards. If that is how it was
> interpreted, than the interpretation is erroneous.

And subtracting seconds can't make time go forwards, but that's what
would happen with the alternate interpretation you want.

There is fundamentally no way for the interface to read your mind and
know that the time you requested is "move forward from a past valid
normalized time" and not "move backward from a future valid normalized
time".

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

> And subtracting seconds can't make time go forwards, but that's what would happen with the alternate interpretation you want.

That's just nonsense.

I can go from 1900 to 2200 by adding seconds.
And from 2200 to 1900 by subtracting seconds.

I just did that using glibc.

This is because each addition to struct tz fields leads to time going forward and each subtraction from struct tz fields leads to time going backwards. As it should.

There is clear ordering of struct tz contents.

AW

On Sunday, March 24th, 2024 at 19:02, Rich Felker <dalias@libc.org> wrote:

> On Sun, Mar 24, 2024 at 05:12:35PM +0000, Alexander Weps wrote:
>
> > Adding seconds cannot make time go backwards. If that is how it was
> > interpreted, than the interpretation is erroneous.
>
>
> And subtracting seconds can't make time go forwards, but that's what
> would happen with the alternate interpretation you want.
>
> There is fundamentally no way for the interface to read your mind and
> know that the time you requested is "move forward from a past valid
> normalized time" and not "move backward from a future valid normalized
> time".
>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Sun, Mar 24, 2024 at 06:16:20PM +0000, Alexander Weps wrote:
> > And subtracting seconds can't make time go forwards, but that's
> > what would happen with the alternate interpretation you want.
> 
> That's just nonsense.
> 
> I can go from 1900 to 2200 by adding seconds.
> And from 2200 to 1900 by subtracting seconds.
> 
> I just did that using glibc.
> 
> This is because each addition to struct tz fields leads to time
> going forward and each subtraction from struct tz fields leads to
> time going backwards.. As it should.
> 
> There is clear ordering of struct tz contents.

This is getting really tiring.

In the presence of times which do not exist, the properties you want
are not mathematically consistent.

EITHER you get cases where "start from time T, add something,
normalize" gives a broken-down time that looks like it's before T (but
isn't, because it's in a different zone rule),

OR you get cases where "start from time T, subtract something,
normalize" gives a broken-down time that looks like it's after T (bit
isn't, because it's in a different zone rule).

Preferring one of these nasty behaviors over the other is entirely
arbitrary.

Time zones are nasty. Local time is nasty. If you want to do things
with it, you have to deal with that nastiness.

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

It is tiring, because you are not correct.

You are also talking about a slightly different thing.

If you have normalized time T1 in struct tm and you add something, normalize, you should always get normalized time T2, what is higher than T1.
If you have normalized time T2 in struct tm and you subtract something, normalize, you should always get normalized time T1, which is lower than T2.

I agree than for non normalized time (tm_isdst = -1 etc.) this would not apply. I agree that the decision how to deduce it is implementation specific and I don't really hold it against musl. I rewrote everything without tm_isdst = -1.

But there cannot be a case where you have normalized time add something, normalize and create normalized time that is lower and vice versa.

If you claim otherwise, provide counter example.

I have done pretty extensive testing.

AW

On Sunday, March 24th, 2024 at 19:24, Rich Felker <dalias@libc.org> wrote:

> On Sun, Mar 24, 2024 at 06:16:20PM +0000, Alexander Weps wrote:
>
> > > And subtracting seconds can't make time go forwards, but that's
> > > what would happen with the alternate interpretation you want.
> >
> > That's just nonsense.
> >
> > I can go from 1900 to 2200 by adding seconds.
> > And from 2200 to 1900 by subtracting seconds.
> >
> > I just did that using glibc.
> >
> > This is because each addition to struct tz fields leads to time
> > going forward and each subtraction from struct tz fields leads to
> > time going backwards.. As it should.
> >
> > There is clear ordering of struct tz contents.
>
>
> This is getting really tiring.
>
> In the presence of times which do not exist, the properties you want
> are not mathematically consistent.
>
> EITHER you get cases where "start from time T, add something,
> normalize" gives a broken-down time that looks like it's before T (but
> isn't, because it's in a different zone rule),
>
> OR you get cases where "start from time T, subtract something,
> normalize" gives a broken-down time that looks like it's after T (bit
> isn't, because it's in a different zone rule).
>
> Preferring one of these nasty behaviors over the other is entirely
> arbitrary.
>
> Time zones are nasty. Local time is nasty. If you want to do things
> with it, you have to deal with that nastiness.
>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Joakim Sindholt]

On Sun, 24 Mar 2024 18:36:40 +0000, Alexander Weps <exander77@pm.me> wrote:
> But there cannot be a case where you have normalized time add
> something, normalize and create normalized time that is lower and vice
> versa.

I'm curious. How do you imagine mktime, which takes only 1 parameter and
has no memory, deduces that to get the presently invalid 02:00:00
timestamp it's looking at you added 1 second to a valid 01:59:59
timestamp rather than say, subtracting 1 hour from a valid 03:00:00
timestamp?

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

Provide struct tm that was passed as an argument.

AW

On Sunday, March 24th, 2024 at 20:01, Joakim Sindholt <opensource@zhasha.com> wrote:

> On Sun, 24 Mar 2024 18:36:40 +0000, Alexander Weps exander77@pm.me wrote:
>
> > But there cannot be a case where you have normalized time add
> > something, normalize and create normalized time that is lower and vice
> > versa.
>
>
> I'm curious. How do you imagine mktime, which takes only 1 parameter and
> has no memory, deduces that to get the presently invalid 02:00:00
> timestamp it's looking at you added 1 second to a valid 01:59:59
> timestamp rather than say, subtracting 1 hour from a valid 03:00:00
> timestamp?

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

I ran Asia/Omsk from 1900 to 2200 there and back again (59 seconds increments/decrements).

Everything ok in glibc.

Fails in musl very early:
1919-11-13 23:59:32 LMT
1919-11-13 23:54:01 LMT

No idea what is even happening there.

Glibc shows nothing interesting there:
1911-12-13 23:59:32 LMT
1911-12-14 00:00:31 LMT

???

AW

On Sunday, March 24th, 2024 at 19:36, Alexander Weps <exander77@pm.me> wrote:

> It is tiring, because you are not correct.
>
> You are also talking about a slightly different thing.
>
> If you have normalized time T1 in struct tm and you add something, normalize, you should always get normalized time T2, what is higher than T1.
> If you have normalized time T2 in struct tm and you subtract something, normalize, you should always get normalized time T1, which is lower than T2.
>
> I agree than for non normalized time (tm_isdst = -1 etc.) this would not apply. I agree that the decision how to deduce it is implementation specific and I don't really hold it against musl. I rewrote everything without tm_isdst = -1.
>
> But there cannot be a case where you have normalized time add something, normalize and create normalized time that is lower and vice versa.
>
> If you claim otherwise, provide counter example.
>
> I have done pretty extensive testing.
>
> AW
>
>
>
> On Sunday, March 24th, 2024 at 19:24, Rich Felker dalias@libc.org wrote:
>
> > On Sun, Mar 24, 2024 at 06:16:20PM +0000, Alexander Weps wrote:
> >
> > > > And subtracting seconds can't make time go forwards, but that's
> > > > what would happen with the alternate interpretation you want.
> > >
> > > That's just nonsense.
> > >
> > > I can go from 1900 to 2200 by adding seconds.
> > > And from 2200 to 1900 by subtracting seconds.
> > >
> > > I just did that using glibc.
> > >
> > > This is because each addition to struct tz fields leads to time
> > > going forward and each subtraction from struct tz fields leads to
> > > time going backwards.. As it should.
> > >
> > > There is clear ordering of struct tz contents.
> >
> > This is getting really tiring.
> >
> > In the presence of times which do not exist, the properties you want
> > are not mathematically consistent.
> >
> > EITHER you get cases where "start from time T, add something,
> > normalize" gives a broken-down time that looks like it's before T (but
> > isn't, because it's in a different zone rule),
> >
> > OR you get cases where "start from time T, subtract something,
> > normalize" gives a broken-down time that looks like it's after T (bit
> > isn't, because it's in a different zone rule).
> >
> > Preferring one of these nasty behaviors over the other is entirely
> > arbitrary.
> >
> > Time zones are nasty. Local time is nasty. If you want to do things
> > with it, you have to deal with that nastiness.
> >
> > Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

Sorry, found it, looked at wrong date in glibc:

So Asia/Omsk:

So glibc is:
1919-11-13 23:59:23 LMT
1919-11-14 00:06:52 +05

And musl is:
1919-11-13 23:59:32 LMT
1919-11-13 23:54:01 LMT

AW


On Sunday, March 24th, 2024 at 20:06, Alexander Weps <exander77@pm.me> wrote:

> I ran Asia/Omsk from 1900 to 2200 there and back again (59 seconds increments/decrements).
>
> Everything ok in glibc.
>
> Fails in musl very early:
> 1919-11-13 23:59:32 LMT
> 1919-11-13 23:54:01 LMT
>
> No idea what is even happening there.
>
> Glibc shows nothing interesting there:
> 1911-12-13 23:59:32 LMT
> 1911-12-14 00:00:31 LMT
>
> ???
>
> AW
>
>
>
> On Sunday, March 24th, 2024 at 19:36, Alexander Weps exander77@pm.me wrote:
>
> > It is tiring, because you are not correct.
> >
> > You are also talking about a slightly different thing.
> >
> > If you have normalized time T1 in struct tm and you add something, normalize, you should always get normalized time T2, what is higher than T1.
> > If you have normalized time T2 in struct tm and you subtract something, normalize, you should always get normalized time T1, which is lower than T2.
> >
> > I agree than for non normalized time (tm_isdst = -1 etc.) this would not apply. I agree that the decision how to deduce it is implementation specific and I don't really hold it against musl. I rewrote everything without tm_isdst = -1.
> >
> > But there cannot be a case where you have normalized time add something, normalize and create normalized time that is lower and vice versa.
> >
> > If you claim otherwise, provide counter example.
> >
> > I have done pretty extensive testing.
> >
> > AW
> >
> > On Sunday, March 24th, 2024 at 19:24, Rich Felker dalias@libc.org wrote:
> >
> > > On Sun, Mar 24, 2024 at 06:16:20PM +0000, Alexander Weps wrote:
> > >
> > > > > And subtracting seconds can't make time go forwards, but that's
> > > > > what would happen with the alternate interpretation you want.
> > > >
> > > > That's just nonsense.
> > > >
> > > > I can go from 1900 to 2200 by adding seconds.
> > > > And from 2200 to 1900 by subtracting seconds.
> > > >
> > > > I just did that using glibc.
> > > >
> > > > This is because each addition to struct tz fields leads to time
> > > > going forward and each subtraction from struct tz fields leads to
> > > > time going backwards.. As it should.
> > > >
> > > > There is clear ordering of struct tz contents.
> > >
> > > This is getting really tiring.
> > >
> > > In the presence of times which do not exist, the properties you want
> > > are not mathematically consistent.
> > >
> > > EITHER you get cases where "start from time T, add something,
> > > normalize" gives a broken-down time that looks like it's before T (but
> > > isn't, because it's in a different zone rule),
> > >
> > > OR you get cases where "start from time T, subtract something,
> > > normalize" gives a broken-down time that looks like it's after T (bit
> > > isn't, because it's in a different zone rule).
> > >
> > > Preferring one of these nasty behaviors over the other is entirely
> > > arbitrary.
> > >
> > > Time zones are nasty. Local time is nasty. If you want to do things
> > > with it, you have to deal with that nastiness.
> > >
> > > Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

Sorry, found it, looked at wrong date in glibc:

So Asia/Omsk:

So glibc is:
1919-11-13 23:59:23 LMT
1919-11-14 00:06:52 +05

And musl is:
1919-11-13 23:59:32 LMT
1919-11-13 23:54:01 LMT

AW


On Sunday, March 24th, 2024 at 20:06, Alexander Weps <exander77@pm.me> wrote:

> I ran Asia/Omsk from 1900 to 2200 there and back again (59 seconds increments/decrements).
>
> Everything ok in glibc.
>
> Fails in musl very early:
> 1919-11-13 23:59:32 LMT
> 1919-11-13 23:54:01 LMT
>
> No idea what is even happening there.
>
> Glibc shows nothing interesting there:
> 1911-12-13 23:59:32 LMT
> 1911-12-14 00:00:31 LMT
>
> ???
>
> AW
>
>
>
> On Sunday, March 24th, 2024 at 19:36, Alexander Weps exander77@pm.me wrote:
>
> > It is tiring, because you are not correct.
> >
> > You are also talking about a slightly different thing.
> >
> > If you have normalized time T1 in struct tm and you add something, normalize, you should always get normalized time T2, what is higher than T1.
> > If you have normalized time T2 in struct tm and you subtract something, normalize, you should always get normalized time T1, which is lower than T2.
> >
> > I agree than for non normalized time (tm_isdst = -1 etc.) this would not apply. I agree that the decision how to deduce it is implementation specific and I don't really hold it against musl. I rewrote everything without tm_isdst = -1.
> >
> > But there cannot be a case where you have normalized time add something, normalize and create normalized time that is lower and vice versa.
> >
> > If you claim otherwise, provide counter example.
> >
> > I have done pretty extensive testing.
> >
> > AW
> >
> > On Sunday, March 24th, 2024 at 19:24, Rich Felker dalias@libc.org wrote:
> >
> > > On Sun, Mar 24, 2024 at 06:16:20PM +0000, Alexander Weps wrote:
> > >
> > > > > And subtracting seconds can't make time go forwards, but that's
> > > > > what would happen with the alternate interpretation you want.
> > > >
> > > > That's just nonsense.
> > > >
> > > > I can go from 1900 to 2200 by adding seconds.
> > > > And from 2200 to 1900 by subtracting seconds.
> > > >
> > > > I just did that using glibc.
> > > >
> > > > This is because each addition to struct tz fields leads to time
> > > > going forward and each subtraction from struct tz fields leads to
> > > > time going backwards.. As it should.
> > > >
> > > > There is clear ordering of struct tz contents.
> > >
> > > This is getting really tiring.
> > >
> > > In the presence of times which do not exist, the properties you want
> > > are not mathematically consistent.
> > >
> > > EITHER you get cases where "start from time T, add something,
> > > normalize" gives a broken-down time that looks like it's before T (but
> > > isn't, because it's in a different zone rule),
> > >
> > > OR you get cases where "start from time T, subtract something,
> > > normalize" gives a broken-down time that looks like it's after T (bit
> > > isn't, because it's in a different zone rule).
> > >
> > > Preferring one of these nasty behaviors over the other is entirely
> > > arbitrary.
> > >
> > > Time zones are nasty. Local time is nasty. If you want to do things
> > > with it, you have to deal with that nastiness.
> > >
> > > Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Sun, Mar 24, 2024 at 06:36:40PM +0000, Alexander Weps wrote:
> It is tiring, because you are not correct.
> 
> You are also talking about a slightly different thing.
> 
> If you have normalized time T1 in struct tm and you add something,
> normalize, you should always get normalized time T2, what is higher
> than T1.
> If you have normalized time T2 in struct tm and you subtract
> something, normalize, you should always get normalized time T1,
> which is lower than T2.
> 
> I agree than for non normalized time (tm_isdst = -1 etc.) this would
> not apply. I agree that the decision how to deduce it is
> implementation specific and I don't really hold it against musl. I
> rewrote everything without tm_isdst = -1.

You're mixing up what non-normalized means. There are basically two
meanings, neither of which has to do with tm_isdst=-1 (forget it
because it's not relevant to any of this).

1. The value of one of the tm_* values it outside of its calendar
   range (e.g. tm_min=70). These are reduced prior to any
   consideration of timezone mess, producing a nominally valid
   calendar date.

2. The normalized (in sense 1 above) time in the tm_* values does not
   exist due to daylight time change (spring-forward) or change in the
   timezone rule for the territory.

You're making test cases which involve both 1 and 2 above, which makes
them more confusing to reason about.

> But there cannot be a case where you have normalized time add
> something, normalize and create normalized time that is lower and
> vice versa.
> 
> If you claim otherwise, provide counter example.

What I've told you is that, if you compare the broken-down tm element
by element ignoring what zone rule it's under, there will always be
instances where mktime is non order preserving, *regardless of what
choices the implementation makes*. One way of writing this precisely
is that there will always exist tm1 and tm2 where

    timegm(tm1) < timegm(tm2)

but after mktime(tm1) and mktime(tm2):

    timegm(tm1) > timegm(tm2)

This is really not profound. It's just a case of "local times are
lossy in the absence of also taking into account the associated UTC
offset or local time rule in effect".

I think you've found one real bug where something goes wrong with the
2011-12-29 corner case, but digging in on other things you think are
wrong, that are just fundamental to how local time works, is
distracting from actually investigating that. Can we try to actually
figure out what's going on there?

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

See below.

AW


On Sunday, March 24th, 2024 at 20:22, Rich Felker <dalias@libc.org> wrote:

> On Sun, Mar 24, 2024 at 06:36:40PM +0000, Alexander Weps wrote:
>
> > It is tiring, because you are not correct.
> >
> > You are also talking about a slightly different thing.
> >
> > If you have normalized time T1 in struct tm and you add something,
> > normalize, you should always get normalized time T2, what is higher
> > than T1.
> > If you have normalized time T2 in struct tm and you subtract
> > something, normalize, you should always get normalized time T1,
> > which is lower than T2.
> >
> > I agree than for non normalized time (tm_isdst = -1 etc.) this would
> > not apply. I agree that the decision how to deduce it is
> > implementation specific and I don't really hold it against musl. I
> > rewrote everything without tm_isdst = -1.
>
>
> You're mixing up what non-normalized means. There are basically two
> meanings, neither of which has to do with tm_isdst=-1 (forget it
> because it's not relevant to any of this).

I use normalized tm struct in a sense that calling mktime on that stuct tm doesn't do any change it.
Struct tm with tm_isdst = -1 is inherently non-normalized by my definition.
But I am up to use your definition.

>
> 1. The value of one of the tm_* values it outside of its calendar
> range (e.g. tm_min=70). These are reduced prior to any
> consideration of timezone mess, producing a nominally valid
> calendar date.

You are describing the musl behavior, more specifically what I see in mktime & __tm_to_secs.
I don't think this is correct behavior.
You basically throw away important information and later claim that you don't have it and it's impossible to deduce it.

If this is what you call normalization than normalization is what breaks it.

>
> 2. The normalized (in sense 1 above) time in the tm_* values does not
> exist due to daylight time change (spring-forward) or change in the
> timezone rule for the territory.

If you consider normalization in 1. a correct behavior and you have some notion that normalized tm_* values represent a specific date time that could be present or not present within a timezone.

>
> You're making test cases which involve both 1 and 2 above, which makes
> them more confusing to reason about.
>
> > But there cannot be a case where you have normalized time add
> > something, normalize and create normalized time that is lower and
> > vice versa.
> >
> > If you claim otherwise, provide counter example.
>
>
> What I've told you is that, if you compare the broken-down tm element
> by element ignoring what zone rule it's under, there will always be
> instances where mktime is non order preserving, regardless of what
> choices the implementation makes. One way of writing this precisely
> is that there will always exist tm1 and tm2 where

You made it non order preserving by your choices. You have just shown that the implementation is broken by choices that were made.
You can make valid ordering of all struct tm if you consider all of the fields.

This is not even relevant to normalization. You can do it on all struct tm just as they are.
Normalization should be there to make it easier to do it, not make it impossible to do it.

>
> timegm(tm1) < timegm(tm2)
>
> but after mktime(tm1) and mktime(tm2):
>
> timegm(tm1) > timegm(tm2)

This is not related. So far everything discussed by me related to localtime of a single timezone.

I have not made any claims about time being consistent while converting between timezones.
I claim that time is consistent within a timezone like Europe/Prague with regard to all changes described by zonefile (CET, CEST, GMT...).

>
>
> This is really not profound. It's just a case of "local times are
> lossy in the absence of also taking into account the associated UTC
> offset or local time rule in effect".
>
> I think you've found one real bug where something goes wrong with the
> 2011-12-29 corner case, but digging in on other things you think are
> wrong, that are just fundamental to how local time works, is
> distracting from actually investigating that. Can we try to actually
> figure out what's going on there?

Sure. But that's not the only bug.

>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Sun, Mar 24, 2024 at 07:57:39PM +0000, Alexander Weps wrote:
> > 1. The value of one of the tm_* values it outside of its calendar
> > range (e.g. tm_min=70). These are reduced prior to any
> > consideration of timezone mess, producing a nominally valid
> > calendar date.
> 
> You are describing the musl behavior, more specifically what I see
> in mktime & __tm_to_secs.
> I don't think this is correct behavior.
> You basically throw away important information and later claim that
> you don't have it and it's impossible to deduce it.

This "important information" does not tell us what the caller did to
get the non-normalized input we received, *even if you assume the
caller just made a single change*.

For example if you see tm_mday=31 in a month with only 30 days, you
don't know if the caller was trying to move one day forward from the
last day of the month, or was trying to move one month back from the
next month.

The reasonable, consistent, least-surprise thing to do is not to try
to make guesses based on the individual fields and how you think the
caller might have gotten to them, but instead to normalize completely
to the ranges before even considering timezone shenanigans.

> > You're making test cases which involve both 1 and 2 above, which makes
> > them more confusing to reason about.
> >
> > > But there cannot be a case where you have normalized time add
> > > something, normalize and create normalized time that is lower and
> > > vice versa.
> > >
> > > If you claim otherwise, provide counter example.
> >
> >
> > What I've told you is that, if you compare the broken-down tm element
> > by element ignoring what zone rule it's under, there will always be
> > instances where mktime is non order preserving, regardless of what
> > choices the implementation makes. One way of writing this precisely
> > is that there will always exist tm1 and tm2 where
> 
> You made it non order preserving by your choices. You have just
> shown that the implementation is broken by choices that were made.
> You can make valid ordering of all struct tm if you consider all of
> the fields.
> 
> This is not even relevant to normalization. You can do it on all
> struct tm just as they are.
> Normalization should be there to make it easier to do it, not make
> it impossible to do it.

No, this happens regardless of the above.

> > This is really not profound. It's just a case of "local times are
> > lossy in the absence of also taking into account the associated UTC
> > offset or local time rule in effect".
> >
> > I think you've found one real bug where something goes wrong with the
> > 2011-12-29 corner case, but digging in on other things you think are
> > wrong, that are just fundamental to how local time works, is
> > distracting from actually investigating that. Can we try to actually
> > figure out what's going on there?
> 
> Sure. But that's not the only bug.

Well I haven't seen any other credible claims of a bug in this thread.

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

See below.

AW

On Sunday, March 24th, 2024 at 21:22, Rich Felker <dalias@libc.org> wrote:

> On Sun, Mar 24, 2024 at 07:57:39PM +0000, Alexander Weps wrote:
>
> > > 1. The value of one of the tm_* values it outside of its calendar
> > > range (e.g. tm_min=70). These are reduced prior to any
> > > consideration of timezone mess, producing a nominally valid
> > > calendar date.
> >
> > You are describing the musl behavior, more specifically what I see
> > in mktime & __tm_to_secs.
> > I don't think this is correct behavior.
> > You basically throw away important information and later claim that
> > you don't have it and it's impossible to deduce it.
>
>
> This "important information" does not tell us what the caller did to
> get the non-normalized input we received, even if you assume the
> caller just made a single change.
>
> For example if you see tm_mday=31 in a month with only 30 days, you
> don't know if the caller was trying to move one day forward from the
> last day of the month, or was trying to move one month back from the
> next month.

This is what is called limitations.
I actually investigated this limitation very closely.

And it doesn't influence ordering if you don't handle it at all.

>
> The reasonable, consistent, least-surprise thing to do is not to try
> to make guesses based on the individual fields and how you think the
> caller might have gotten to them, but instead to normalize completely
> to the ranges before even considering timezone shenanigans.

No.

The reasonable, consistent, least-surprising thing to do is to take a minimal subset of assumptions to make viable ordering.
So time can be predictably incremented and decremented and doesn't go backwards to run into loops like these:

1946-12-01 02:59:17 CET
1946-12-01 01:00:16 CET

1919-11-13 23:59:32 LMT
1919-11-13 23:54:01 LMT

Reasonable is also to behave similarly as other implementations.

>
> > > You're making test cases which involve both 1 and 2 above, which makes
> > > them more confusing to reason about.
> > >
> > > > But there cannot be a case where you have normalized time add
> > > > something, normalize and create normalized time that is lower and
> > > > vice versa.
> > > >
> > > > If you claim otherwise, provide counter example.
> > >
> > > What I've told you is that, if you compare the broken-down tm element
> > > by element ignoring what zone rule it's under, there will always be
> > > instances where mktime is non order preserving, regardless of what
> > > choices the implementation makes. One way of writing this precisely
> > > is that there will always exist tm1 and tm2 where
> >
> > You made it non order preserving by your choices. You have just
> > shown that the implementation is broken by choices that were made.
> > You can make valid ordering of all struct tm if you consider all of
> > the fields.
> >
> > This is not even relevant to normalization. You can do it on all
> > struct tm just as they are.
> > Normalization should be there to make it easier to do it, not make
> > it impossible to do it.
>
>
> No, this happens regardless of the above.

Then provide an example.

>
> > > This is really not profound. It's just a case of "local times are
> > > lossy in the absence of also taking into account the associated UTC
> > > offset or local time rule in effect".
> > >
> > > I think you've found one real bug where something goes wrong with the
> > > 2011-12-29 corner case, but digging in on other things you think are
> > > wrong, that are just fundamental to how local time works, is
> > > distracting from actually investigating that. Can we try to actually
> > > figure out what's going on there?
> >
> > Sure. But that's not the only bug.
>
>
> Well I haven't seen any other credible claims of a bug in this thread.

I am not sure if this is a serious conversation.

I have a struct tm created by mktime. So this is a valid structure that was produced by mktime.
I increment a field (be it second, minute, hour...) in that structure.
I call mktime to give me result of that calculation.
Result of mktime is a s struct tm that has earlier time than the original struct tm.

One of the primary purposes of struct tm and mktime is to make calculations.

Currently the implementation in musl cannot take a date represented in struct tm and iteratively increment it in second intervals to generated all dates up to some date. This basically means, that some calculations are impossible to make. It cannot reliably take a date and add seconds, minutes, hours, days... to that date and get the result.

>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

Not to mention this breaks a tonne of existing functions and programs. If these were just bad dates, it would be one thing...

Bud due to such cycles it causes anything from infinite looping to stack overflows.

Sure it only does that in some cases, but if it happens it can crash a program.

Not to mention that there is no recovery.

It fails on tasks like create 10 consecutive days if it hits wrong spot. If a programmer is creative enough he/she can check if incrementing date lead to an earlier date and something is wrong, but what is correct recovery to get that that consecutive date?

Show me a recovery from this issue.
Show me how I can reliably generate consecutive times/dates under such conditions.
Show me how to generate start of each day.

There is basically no reliable way how to do it under musl.

I do not mind a complicated function to get these.

But just simple enumerating all dates seems an impossible task in musl. Or what do you suggest as a solution? Can you provide me with a code that enumerates all dates up to seconds, minutes, hours, days, months, years resolutions?

AW

On Sunday, March 24th, 2024 at 21:50, Alexander Weps <exander77@pm.me> wrote:

> See below.
>
> AW
>
>
>
> On Sunday, March 24th, 2024 at 21:22, Rich Felker dalias@libc.org wrote:
>
> > On Sun, Mar 24, 2024 at 07:57:39PM +0000, Alexander Weps wrote:
> >
> > > > 1. The value of one of the tm_* values it outside of its calendar
> > > > range (e.g. tm_min=70). These are reduced prior to any
> > > > consideration of timezone mess, producing a nominally valid
> > > > calendar date.
> > >
> > > You are describing the musl behavior, more specifically what I see
> > > in mktime & __tm_to_secs.
> > > I don't think this is correct behavior.
> > > You basically throw away important information and later claim that
> > > you don't have it and it's impossible to deduce it.
> >
> > This "important information" does not tell us what the caller did to
> > get the non-normalized input we received, even if you assume the
> > caller just made a single change.
> >
> > For example if you see tm_mday=31 in a month with only 30 days, you
> > don't know if the caller was trying to move one day forward from the
> > last day of the month, or was trying to move one month back from the
> > next month.
>
>
> This is what is called limitations.
> I actually investigated this limitation very closely.
>
> And it doesn't influence ordering if you don't handle it at all.
>
> > The reasonable, consistent, least-surprise thing to do is not to try
> > to make guesses based on the individual fields and how you think the
> > caller might have gotten to them, but instead to normalize completely
> > to the ranges before even considering timezone shenanigans.
>
>
> No.
>
> The reasonable, consistent, least-surprising thing to do is to take a minimal subset of assumptions to make viable ordering.
> So time can be predictably incremented and decremented and doesn't go backwards to run into loops like these:
>
> 1946-12-01 02:59:17 CET
> 1946-12-01 01:00:16 CET
>
> 1919-11-13 23:59:32 LMT
> 1919-11-13 23:54:01 LMT
>
> Reasonable is also to behave similarly as other implementations.
>
> > > > You're making test cases which involve both 1 and 2 above, which makes
> > > > them more confusing to reason about.
> > > >
> > > > > But there cannot be a case where you have normalized time add
> > > > > something, normalize and create normalized time that is lower and
> > > > > vice versa.
> > > > >
> > > > > If you claim otherwise, provide counter example.
> > > >
> > > > What I've told you is that, if you compare the broken-down tm element
> > > > by element ignoring what zone rule it's under, there will always be
> > > > instances where mktime is non order preserving, regardless of what
> > > > choices the implementation makes. One way of writing this precisely
> > > > is that there will always exist tm1 and tm2 where
> > >
> > > You made it non order preserving by your choices. You have just
> > > shown that the implementation is broken by choices that were made.
> > > You can make valid ordering of all struct tm if you consider all of
> > > the fields.
> > >
> > > This is not even relevant to normalization. You can do it on all
> > > struct tm just as they are.
> > > Normalization should be there to make it easier to do it, not make
> > > it impossible to do it.
> >
> > No, this happens regardless of the above.
>
>
> Then provide an example.
>
> > > > This is really not profound. It's just a case of "local times are
> > > > lossy in the absence of also taking into account the associated UTC
> > > > offset or local time rule in effect".
> > > >
> > > > I think you've found one real bug where something goes wrong with the
> > > > 2011-12-29 corner case, but digging in on other things you think are
> > > > wrong, that are just fundamental to how local time works, is
> > > > distracting from actually investigating that. Can we try to actually
> > > > figure out what's going on there?
> > >
> > > Sure. But that's not the only bug.
> >
> > Well I haven't seen any other credible claims of a bug in this thread.
>
>
> I am not sure if this is a serious conversation.
>
> I have a struct tm created by mktime. So this is a valid structure that was produced by mktime.
> I increment a field (be it second, minute, hour...) in that structure.
> I call mktime to give me result of that calculation.
> Result of mktime is a s struct tm that has earlier time than the original struct tm.
>
> One of the primary purposes of struct tm and mktime is to make calculations.
>
> Currently the implementation in musl cannot take a date represented in struct tm and iteratively increment it in second intervals to generated all dates up to some date. This basically means, that some calculations are impossible to make. It cannot reliably take a date and add seconds, minutes, hours, days... to that date and get the result.
>
> > Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Thorsten Glaser]

Alexander Weps dixit:

>You are describing the musl behavior, more specifically what I see in mktime & __tm_to_secs.
>I don't think this is correct behavior.

This is what POSIX (Issue 8) and AFAIR also the next ISO C standard
mandate, though:

1.–6.	struct tm is normalised from seconds or minutes up to year
7.	struct tm is converted to time_t (wrongly written down as
	“the number of seconds since the epoch” as it omits leap
	seconds)
8.	timezone corrections for standard time at the moment in
	time calculated in step 7 is applied
9.	if the timezone has DST:
	+ if tm_isdst is positive, the time is adjusted by the offset
	+ if tm_isdst is negative, the result is either the same as
	  if it were 0 or the same as if it were 1; if the struct tm
	  specifies a gap or repeated segment, which of the two is
	  used is explicitly unspecified, i.e. the caller cannot rely
	  on the libc to guess his intent if he sets tm_isdst to -1.
10. (not numbered) for gaps or repeats, mktime uses either the value
	from before the gap/repeat or the one after, choice again
	unspecified

Tough luck there.

The wording in this part is interesting though:

| If tm_isdst is positive, mktime() shall further adjust the seconds
| since the Epoch by the DST offset.

But I guess that if you call with tm_isdst=1 and a broken-down time
that clearly corresponds to nōn-DST, the DST offset for it is just 0
and it’ll work out the obvious way.

bye,
//mirabilos
-- 
“It is inappropriate to require that a time represented as
 seconds since the Epoch precisely represent the number of
 seconds between the referenced time and the Epoch.”
	-- IEEE Std 1003.1b-1993 (POSIX) Section B.2.2.2

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

I have no problem with the POSIX (Issue 8) or ISO C standard.

I agree it doesn't mandate mktime making correct calculations, but I would assume it is expected.

AW

On Monday, March 25th, 2024 at 00:51, Thorsten Glaser <tg@mirbsd.de> wrote:

> Alexander Weps dixit:
>
> > You are describing the musl behavior, more specifically what I see in mktime & __tm_to_secs.
> > I don't think this is correct behavior.
>
>
> This is what POSIX (Issue 8) and AFAIR also the next ISO C standard
> mandate, though:
>
> 1.–6. struct tm is normalised from seconds or minutes up to year
> 7. struct tm is converted to time_t (wrongly written down as
> “the number of seconds since the epoch” as it omits leap
> seconds)
> 8. timezone corrections for standard time at the moment in
> time calculated in step 7 is applied
> 9. if the timezone has DST:
> + if tm_isdst is positive, the time is adjusted by the offset
> + if tm_isdst is negative, the result is either the same as
> if it were 0 or the same as if it were 1; if the struct tm
> specifies a gap or repeated segment, which of the two is
> used is explicitly unspecified, i.e. the caller cannot rely
> on the libc to guess his intent if he sets tm_isdst to -1.
> 10. (not numbered) for gaps or repeats, mktime uses either the value
> from before the gap/repeat or the one after, choice again
> unspecified
>
> Tough luck there.
>
> The wording in this part is interesting though:
>
> | If tm_isdst is positive, mktime() shall further adjust the seconds
> | since the Epoch by the DST offset.
>
> But I guess that if you call with tm_isdst=1 and a broken-down time
> that clearly corresponds to nōn-DST, the DST offset for it is just 0
> and it’ll work out the obvious way.
>
> bye,
> //mirabilos
> --
> “It is inappropriate to require that a time represented as
> seconds since the Epoch precisely represent the number of
> seconds between the referenced time and the Epoch.”
> -- IEEE Std 1003.1b-1993 (POSIX) Section B.2.2.2

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

This is the simplest and most obvious example how broken the calculation in musl is:

void test10()
{
    time_t t = 0;
    struct tm tm = {0};
    char buf[64];

    tm.tm_year = 2011 - 1900;
    tm.tm_mon = 12 - 1;
    tm.tm_mday = 29;
    tm.tm_hour = 0;
    tm.tm_min = 0;
    tm.tm_sec = 0;
    tm.tm_isdst = 0;

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("before: %s %ld %ld\n", buf, t, calc(&tm));

    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after1: %s %ld %ld\n", buf, t, calc(&tm));

    tm.tm_mday += 1;
    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after2: %s %ld %ld\n", buf, t, calc(&tm));
}

TZ=Pacific/Apia
Year is greater than 1970.

Input:
2011-12-29 01:00:00 -10

Add a day:
    tm.tm_mday += 1;
    t = mktime(&tm);

Output:
2011-12-29 01:00:00 -10

Musl cannot reliably increment date by a day. Incrementing struct tm representing 2011-12-29 01:00:00 -10 by one day leads to the same date.

Causing a program to loop or stack overflow.

AW


On Monday, March 25th, 2024 at 01:36, Alexander Weps <exander77@pm.me> wrote:

> I have no problem with the POSIX (Issue 8) or ISO C standard.
>
> I agree it doesn't mandate mktime making correct calculations, but I would assume it is expected.
>
> AW
>
>
>
> On Monday, March 25th, 2024 at 00:51, Thorsten Glaser tg@mirbsd.de wrote:
>
> > Alexander Weps dixit:
> >
> > > You are describing the musl behavior, more specifically what I see in mktime & __tm_to_secs.
> > > I don't think this is correct behavior.
> >
> > This is what POSIX (Issue 8) and AFAIR also the next ISO C standard
> > mandate, though:
> >
> > 1.–6. struct tm is normalised from seconds or minutes up to year
> > 7. struct tm is converted to time_t (wrongly written down as
> > “the number of seconds since the epoch” as it omits leap
> > seconds)
> > 8. timezone corrections for standard time at the moment in
> > time calculated in step 7 is applied
> > 9. if the timezone has DST:
> > + if tm_isdst is positive, the time is adjusted by the offset
> > + if tm_isdst is negative, the result is either the same as
> > if it were 0 or the same as if it were 1; if the struct tm
> > specifies a gap or repeated segment, which of the two is
> > used is explicitly unspecified, i.e. the caller cannot rely
> > on the libc to guess his intent if he sets tm_isdst to -1.
> > 10. (not numbered) for gaps or repeats, mktime uses either the value
> > from before the gap/repeat or the one after, choice again
> > unspecified
> >
> > Tough luck there.
> >
> > The wording in this part is interesting though:
> >
> > | If tm_isdst is positive, mktime() shall further adjust the seconds
> > | since the Epoch by the DST offset.
> >
> > But I guess that if you call with tm_isdst=1 and a broken-down time
> > that clearly corresponds to nōn-DST, the DST offset for it is just 0
> > and it’ll work out the obvious way.
> >
> > bye,
> > //mirabilos
> > --
> > “It is inappropriate to require that a time represented as
> > seconds since the Epoch precisely represent the number of
> > seconds between the referenced time and the Epoch.”
> > -- IEEE Std 1003.1b-1993 (POSIX) Section B.2.2.2

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Mon, Mar 25, 2024 at 11:52:00AM +0000, Alexander Weps wrote:
> This is the simplest and most obvious example how broken the
> calculation in musl is:
> 
> void test10()
> {
>     time_t t = 0;
>     struct tm tm = {0};
>     char buf[64];
> 
>     tm.tm_year = 2011 - 1900;
>     tm.tm_mon = 12 - 1;
>     tm.tm_mday = 29;
>     tm.tm_hour = 0;
>     tm.tm_min = 0;
>     tm.tm_sec = 0;
>     tm.tm_isdst = 0;
> 
>     strftime(buf, sizeof buf, "%F %T %Z", &tm);
>     printf("before: %s %ld %ld\n", buf, t, calc(&tm));
> 
>     t = mktime(&tm);
> 
>     strftime(buf, sizeof buf, "%F %T %Z", &tm);
>     printf("after1: %s %ld %ld\n", buf, t, calc(&tm));
> 
>     tm.tm_mday += 1;
>     t = mktime(&tm);
> 
>     strftime(buf, sizeof buf, "%F %T %Z", &tm);
>     printf("after2: %s %ld %ld\n", buf, t, calc(&tm));
> }
> 
> TZ=Pacific/Apia
> Year is greater than 1970.
> 
> Input:
> 2011-12-29 01:00:00 -10
> 
> Add a day:
>     tm.tm_mday += 1;
>     t = mktime(&tm);
> 
> Output:
> 2011-12-29 01:00:00 -10
> 
> Musl cannot reliably increment date by a day. Incrementing struct tm
> representing 2011-12-29 01:00:00 -10 by one day leads to the same
> date.
> 
> Causing a program to loop or stack overflow.

I thought you had found a real bug here, and spent some time working
out the math by hand on paper because local time is so headbangingly
awful and confusing. In the end, the conclusion I'm left with is that
it's working just as expected.

A "spring forward" like this is just like the start of DST, except
that you can't disambiguate the does-not-exist time with an explicit
tm_isdst. So all reasoning about what happens is equivalent to the
much more familar case of start-of-DST with tm_isdst=-1.

If you take your test program and switch it to initialize with
tm_mday=31, then do -=1 instead of +=1, you'll find that it gives 
2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
expected thing to happen. Any change to "fix" the case you're
complaining about would *necessarily* break this case.

You cannot iterate days by making relative changes to struct tm and
calling mktime. This just does not work. You could instead iterate
calendar day inputs yourself, throwing away duplicate outputs
(resulting from nonexistent days like this one) but that would miss
days that exist in duplicate on the calendar, where the change happens
in the opposite direction. What's probably a better approach is
iterating time_t values (or a struct tm in UTC, using timegm) then,
for each day, converting to localtime and picking a "start of day"
time in localtime.

In any case, the core issue you're hitting here is that time zones are
HARD to work with and that there is inherent complexity that libc
cannot save you from. You only got lucky that what you were trying to
do "worked" with glibc because you were iterating days forward; if you
were doing reverse, it would break exactly the same way.

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

See below.

AW

On Monday, March 25th, 2024 at 13:21, Rich Felker <dalias@libc.org> wrote:

> On Mon, Mar 25, 2024 at 11:52:00AM +0000, Alexander Weps wrote:
>
> > This is the simplest and most obvious example how broken the
> > calculation in musl is:
> >
> > void test10()
> > {
> > time_t t = 0;
> > struct tm tm = {0};
> > char buf[64];
> >
> > tm.tm_year = 2011 - 1900;
> > tm.tm_mon = 12 - 1;
> > tm.tm_mday = 29;
> > tm.tm_hour = 0;
> > tm.tm_min = 0;
> > tm.tm_sec = 0;
> > tm.tm_isdst = 0;
> >
> > strftime(buf, sizeof buf, "%F %T %Z", &tm);
> > printf("before: %s %ld %ld\n", buf, t, calc(&tm));
> >
> > t = mktime(&tm);
> >
> > strftime(buf, sizeof buf, "%F %T %Z", &tm);
> > printf("after1: %s %ld %ld\n", buf, t, calc(&tm));
> >
> > tm.tm_mday += 1;
> > t = mktime(&tm);
> >
> > strftime(buf, sizeof buf, "%F %T %Z", &tm);
> > printf("after2: %s %ld %ld\n", buf, t, calc(&tm));
> > }
> >
> > TZ=Pacific/Apia
> > Year is greater than 1970.
> >
> > Input:
> > 2011-12-29 01:00:00 -10
> >
> > Add a day:
> > tm.tm_mday += 1;
> > t = mktime(&tm);
> >
> > Output:
> > 2011-12-29 01:00:00 -10
> >
> > Musl cannot reliably increment date by a day. Incrementing struct tm
> > representing 2011-12-29 01:00:00 -10 by one day leads to the same
> > date.
> >
> > Causing a program to loop or stack overflow.
>
>
> I thought you had found a real bug here, and spent some time working
> out the math by hand on paper because local time is so headbangingly
> awful and confusing. In the end, the conclusion I'm left with is that
> it's working just as expected.

It isn't.

Output from musl:

2011-12-29 01:00:00 -10

    tm.tm_mday += 1;
    t = mktime(&tm);

2011-12-29 01:00:00 -10 <-- date is the same after incrementing

    tm.tm_mday -= 1;
    t = mktime(&tm);

2011-12-28 01:00:00 -10 <-- going below the original date while decrementing

Output from glibc:

2011-12-29 01:00:00 -10

    tm.tm_mday += 1;
    t = mktime(&tm);

2011-12-30 01:00:00 -10 <-- ok

    tm.tm_mday -= 1;
    t = mktime(&tm);

2011-12-29 01:00:00 -10 <-- ok

Hour earlier (same calculations):

Output from musl:

2011-12-29 00:00:00 -10
2011-12-29 00:00:00 -10 <-- date is the same after incrementing
2011-12-28 00:00:00 -10 <-- going below the original date while decrementing

Output from glibc:
2011-12-29 00:00:00 -10
2011-12-30 00:00:00 -10 <-- ok
2011-12-29 00:00:00 -10 <-- ok

Hour after (same calculations).

Output from musl:

2011-12-29 02:00:00 -10
2011-12-29 02:00:00 -10 <-- date is the same after incrementing
2011-12-28 02:00:00 -10 <-- going below the original date while decrementing

Output from glibc:
2011-12-29 02:00:00 -10
2011-12-30 02:00:00 -10 <-- ok
2011-12-29 02:00:00 -10 <-- ok

What are you talking about?


>
> A "spring forward" like this is just like the start of DST, except
> that you can't disambiguate the does-not-exist time with an explicit
> tm_isdst. So all reasoning about what happens is equivalent to the
> much more familar case of start-of-DST with tm_isdst=-1.
>

As I said proclaimed, the issue never was about tm_isdst=-1. Issue is with musl.

> If you take your test program and switch it to initialize with
> tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> expected thing to happen. Any change to "fix" the case you're
> complaining about would necessarily break this case.

So (- day, +day):

Musl:
2011-12-31 01:00:00 +14
2011-12-29 01:00:00 -10
2011-12-29 01:00:00 -10

Glibc:
2012-01-01 01:00:00 +14
2011-12-31 01:00:00 +14
2012-01-01 01:00:00 +14

Seems like musl doesn't even interpret the initial struct tm correctly in that case. It is off by day.

Because December only had 30 days, 31s day after normalization is January 1st.

So no, musl is not correct, it is even more incorrect.

Jesus Christ.

>
> You cannot iterate days by making relative changes to struct tm and
> calling mktime. This just does not work. You could instead iterate
> calendar day inputs yourself, throwing away duplicate outputs
> (resulting from nonexistent days like this one) but that would miss
> days that exist in duplicate on the calendar, where the change happens
> in the opposite direction. What's probably a better approach is
> iterating time_t values (or a struct tm in UTC, using timegm) then,
> for each day, converting to localtime and picking a "start of day"
> time in localtime.
>
> In any case, the core issue you're hitting here is that time zones are
> HARD to work with and that there is inherent complexity that libc
> cannot save you from. You only got lucky that what you were trying to
> do "worked" with glibc because you were iterating days forward; if you
> were doing reverse, it would break exactly the same way.

I am not really commenting on this, until you sort out the above inconsistencies.

>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> See below.
> 
> AW
> 
> On Monday, March 25th, 2024 at 13:21, Rich Felker <dalias@libc.org> wrote:
> 
> > On Mon, Mar 25, 2024 at 11:52:00AM +0000, Alexander Weps wrote:
> >
> > > This is the simplest and most obvious example how broken the
> > > calculation in musl is:
> > >
> > > void test10()
> > > {
> > > time_t t = 0;
> > > struct tm tm = {0};
> > > char buf[64];
> > >
> > > tm.tm_year = 2011 - 1900;
> > > tm.tm_mon = 12 - 1;
> > > tm.tm_mday = 29;
> > > tm.tm_hour = 0;
> > > tm.tm_min = 0;
> > > tm.tm_sec = 0;
> > > tm.tm_isdst = 0;
> > >
> > > strftime(buf, sizeof buf, "%F %T %Z", &tm);
> > > printf("before: %s %ld %ld\n", buf, t, calc(&tm));
> > >
> > > t = mktime(&tm);
> > >
> > > strftime(buf, sizeof buf, "%F %T %Z", &tm);
> > > printf("after1: %s %ld %ld\n", buf, t, calc(&tm));
> > >
> > > tm.tm_mday += 1;
> > > t = mktime(&tm);
> > >
> > > strftime(buf, sizeof buf, "%F %T %Z", &tm);
> > > printf("after2: %s %ld %ld\n", buf, t, calc(&tm));
> > > }
> > >
> > > TZ=Pacific/Apia
> > > Year is greater than 1970.
> > >
> > > Input:
> > > 2011-12-29 01:00:00 -10
> > >
> > > Add a day:
> > > tm.tm_mday += 1;
> > > t = mktime(&tm);
> > >
> > > Output:
> > > 2011-12-29 01:00:00 -10
> > >
> > > Musl cannot reliably increment date by a day. Incrementing struct tm
> > > representing 2011-12-29 01:00:00 -10 by one day leads to the same
> > > date.
> > >
> > > Causing a program to loop or stack overflow.
> >
> >
> > I thought you had found a real bug here, and spent some time working
> > out the math by hand on paper because local time is so headbangingly
> > awful and confusing. In the end, the conclusion I'm left with is that
> > it's working just as expected.
> 
> It isn't.
> 
> Output from musl:
> 
> 2011-12-29 01:00:00 -10
> 
>     tm.tm_mday += 1;
>     t = mktime(&tm);
> 
> 2011-12-29 01:00:00 -10 <-- date is the same after incrementing
> 
>     tm.tm_mday -= 1;
>     t = mktime(&tm);

Read my mail again. I'm talking about starting from 2011-12-31 and
decrementing.

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

Have you read that e-mail? That case is addressed there as well...

So I am putting it here again, so you don't need to search for it:

> If you take your test program and switch it to initialize with
> tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> expected thing to happen. Any change to "fix" the case you're
> complaining about would necessarily break this case.

So (-day, +day):

Musl:
2011-12-31 01:00:00 +14
2011-12-29 01:00:00 -10
2011-12-29 01:00:00 -10

Glibc:
2012-01-01 01:00:00 +14
2011-12-31 01:00:00 +14
2012-01-01 01:00:00 +14

Seems like musl doesn't even interpret the initial struct tm correctly in that case. It is off by day.

Because December only had 30 days, 31s day after normalization is January 1st.

So no, musl is not correct, it is even more incorrect.

Jesus Christ.

AW

On Monday, March 25th, 2024 at 14:07, Rich Felker <dalias@libc.org> wrote:

> On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
>
> > See below.
> >
> > AW
> >
> > On Monday, March 25th, 2024 at 13:21, Rich Felker dalias@libc.org wrote:
> >
> > > On Mon, Mar 25, 2024 at 11:52:00AM +0000, Alexander Weps wrote:
> > >
> > > > This is the simplest and most obvious example how broken the
> > > > calculation in musl is:
> > > >
> > > > void test10()
> > > > {
> > > > time_t t = 0;
> > > > struct tm tm = {0};
> > > > char buf[64];
> > > >
> > > > tm.tm_year = 2011 - 1900;
> > > > tm.tm_mon = 12 - 1;
> > > > tm.tm_mday = 29;
> > > > tm.tm_hour = 0;
> > > > tm.tm_min = 0;
> > > > tm.tm_sec = 0;
> > > > tm.tm_isdst = 0;
> > > >
> > > > strftime(buf, sizeof buf, "%F %T %Z", &tm);
> > > > printf("before: %s %ld %ld\n", buf, t, calc(&tm));
> > > >
> > > > t = mktime(&tm);
> > > >
> > > > strftime(buf, sizeof buf, "%F %T %Z", &tm);
> > > > printf("after1: %s %ld %ld\n", buf, t, calc(&tm));
> > > >
> > > > tm.tm_mday += 1;
> > > > t = mktime(&tm);
> > > >
> > > > strftime(buf, sizeof buf, "%F %T %Z", &tm);
> > > > printf("after2: %s %ld %ld\n", buf, t, calc(&tm));
> > > > }
> > > >
> > > > TZ=Pacific/Apia
> > > > Year is greater than 1970.
> > > >
> > > > Input:
> > > > 2011-12-29 01:00:00 -10
> > > >
> > > > Add a day:
> > > > tm.tm_mday += 1;
> > > > t = mktime(&tm);
> > > >
> > > > Output:
> > > > 2011-12-29 01:00:00 -10
> > > >
> > > > Musl cannot reliably increment date by a day. Incrementing struct tm
> > > > representing 2011-12-29 01:00:00 -10 by one day leads to the same
> > > > date.
> > > >
> > > > Causing a program to loop or stack overflow.
> > >
> > > I thought you had found a real bug here, and spent some time working
> > > out the math by hand on paper because local time is so headbangingly
> > > awful and confusing. In the end, the conclusion I'm left with is that
> > > it's working just as expected.
> >
> > It isn't.
> >
> > Output from musl:
> >
> > 2011-12-29 01:00:00 -10
> >
> > tm.tm_mday += 1;
> > t = mktime(&tm);
> >
> > 2011-12-29 01:00:00 -10 <-- date is the same after incrementing
> >
> > tm.tm_mday -= 1;
> > t = mktime(&tm);
>
>
> Read my mail again. I'm talking about starting from 2011-12-31 and
> decrementing.
>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> > If you take your test program and switch it to initialize with
> > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > expected thing to happen. Any change to "fix" the case you're
> > complaining about would necessarily break this case.
> 
> So (- day, +day):
> 
> Musl:
> 2011-12-31 01:00:00 +14
> 2011-12-29 01:00:00 -10
> 2011-12-29 01:00:00 -10
> 
> Glibc:
> 2012-01-01 01:00:00 +14
> 2011-12-31 01:00:00 +14
> 2012-01-01 01:00:00 +14
> 
> Seems like musl doesn't even interpret the initial struct tm
> correctly in that case. It is off by day.
> 
> Because December only had 30 days, 31s day after normalization is
> January 1st.

This is nonsense. December has a day 31, which you can clearly see
from the glibc output. For this particular year in this zone, with the
zone rule change, there are "only 30 days" in December, but they are
numbered 1-29 and 31, not 1-30.

What did you do that got glibc to output 2012-01-01? I guess you wrote
code to do some wacky arithmetic *after* the original code you already
had, rather than *changing* the code to start with 2011-12-31 as I
suggested to get a look at what's happening.

> > In any case, the core issue you're hitting here is that time zones are
> > HARD to work with and that there is inherent complexity that libc
> > cannot save you from. You only got lucky that what you were trying to
> > do "worked" with glibc because you were iterating days forward; if you
> > were doing reverse, it would break exactly the same way.
> 
> I am not really commenting on this, until you sort out the above
> inconsistencies.

I already have but you refuse to look.

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

See below.

AW


On Monday, March 25th, 2024 at 14:13, Rich Felker <dalias@libc.org> wrote:

> On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> 
> > > If you take your test program and switch it to initialize with
> > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > expected thing to happen. Any change to "fix" the case you're
> > > complaining about would necessarily break this case.
> > 
> > So (- day, +day):
> > 
> > Musl:
> > 2011-12-31 01:00:00 +14
> > 2011-12-29 01:00:00 -10
> > 2011-12-29 01:00:00 -10
> > 
> > Glibc:
> > 2012-01-01 01:00:00 +14
> > 2011-12-31 01:00:00 +14
> > 2012-01-01 01:00:00 +14
> > 
> > Seems like musl doesn't even interpret the initial struct tm
> > correctly in that case. It is off by day.
> > 
> > Because December only had 30 days, 31s day after normalization is
> > January 1st.
> 
> 
> This is nonsense. December has a day 31, which you can clearly see
> from the glibc output. For this particular year in this zone, with the
> zone rule change, there are "only 30 days" in December, but they are
> numbered 1-29 and 31, not 1-30.

You confuse day of month which is represented in tm_mday with calendar day that is interpreted by strftime.

You said to set tm_mday = 31, which would be January 1st after normalization.
December 31s is 30th day of month represented as tm_mday = 30.

> 
> What did you do that got glibc to output 2012-01-01? I guess you wrote
> code to do some wacky arithmetic after the original code you already
> had, rather than changing the code to start with 2011-12-31 as I
> suggested to get a look at what's happening.
> 
> > > In any case, the core issue you're hitting here is that time zones are
> > > HARD to work with and that there is inherent complexity that libc
> > > cannot save you from. You only got lucky that what you were trying to
> > > do "worked" with glibc because you were iterating days forward; if you
> > > were doing reverse, it would break exactly the same way.
> > 
> > I am not really commenting on this, until you sort out the above
> > inconsistencies.
> 
> 
> I already have but you refuse to look.

It was addressed, do didn't scroll at the end of the e-mail.

> 
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

[-- Attachment #1: Type: text/plain, Size: 3172 bytes --]

On Mon, Mar 25, 2024 at 01:24:57PM +0000, Alexander Weps wrote:
> See below.
> 
> AW
> 
> 
> On Monday, March 25th, 2024 at 14:13, Rich Felker <dalias@libc.org> wrote:
> 
> > On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> > 
> > > > If you take your test program and switch it to initialize with
> > > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > > expected thing to happen. Any change to "fix" the case you're
> > > > complaining about would necessarily break this case.
> > > 
> > > So (- day, +day):
> > > 
> > > Musl:
> > > 2011-12-31 01:00:00 +14
> > > 2011-12-29 01:00:00 -10
> > > 2011-12-29 01:00:00 -10
> > > 
> > > Glibc:
> > > 2012-01-01 01:00:00 +14
> > > 2011-12-31 01:00:00 +14
> > > 2012-01-01 01:00:00 +14
> > > 
> > > Seems like musl doesn't even interpret the initial struct tm
> > > correctly in that case. It is off by day.
> > > 
> > > Because December only had 30 days, 31s day after normalization is
> > > January 1st.
> > 
> > 
> > This is nonsense. December has a day 31, which you can clearly see
> > from the glibc output. For this particular year in this zone, with the
> > zone rule change, there are "only 30 days" in December, but they are
> > numbered 1-29 and 31, not 1-30.
> 
> You confuse day of month which is represented in tm_mday with
> calendar day that is interpreted by strftime.
> 
> You said to set tm_mday = 31, which would be January 1st after normalization.
> December 31s is 30th day of month represented as tm_mday = 30.

OK, I meant tm_mday=31-1.

> > What did you do that got glibc to output 2012-01-01? I guess you wrote
> > code to do some wacky arithmetic after the original code you already
> > had, rather than changing the code to start with 2011-12-31 as I
> > suggested to get a look at what's happening.
> > 
> > > > In any case, the core issue you're hitting here is that time zones are
> > > > HARD to work with and that there is inherent complexity that libc
> > > > cannot save you from. You only got lucky that what you were trying to
> > > > do "worked" with glibc because you were iterating days forward; if you
> > > > were doing reverse, it would break exactly the same way.
> > > 
> > > I am not really commenting on this, until you sort out the above
> > > inconsistencies.
> > 
> > 
> > I already have but you refuse to look.
> 
> It was addressed, do didn't scroll at the end of the e-mail.

Run the attached passing the starting date to check as the first/only
argument, and these test dates:

- "2011-12-29 00:00:00"
- "2011-12-31 00:00:00"

Hopefully that will clarify things for you. On musl you will see:

normalized input: 2011-12-29 00:00:00 -10
+1day per mktime: 2011-12-29 00:00:00 -10
+1day via time_t: 2011-12-31 00:00:00 +14
-1day per mktime: 2011-12-28 00:00:00 -10
-1day via time_t: 2011-12-28 00:00:00 -10

normalized input: 2011-12-31 00:00:00 +14
+1day per mktime: 2012-01-01 00:00:00 +14
+1day via time_t: 2012-01-01 00:00:00 +14
-1day per mktime: 2011-12-29 00:00:00 -10
-1day via time_t: 2011-12-29 00:00:00 -10

You can see what you get on glibc.

Rich

[-- Attachment #2: mktime_rel.c --]
[-- Type: text/plain, Size: 1161 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <errno.h>

int main(int argc, char **argv)
{
	struct tm tm = { .tm_isdst = -1 };

	if (!strptime(argv[1], "%Y-%m-%d %H:%M:%S", &tm)) {
		perror("strptime");
		return 1;
	}
	errno = 0;
	time_t t = mktime(&tm);
	if (t==-1 && errno) {
		perror("mktime");
		return 1;
	}

	char buf[100];
	strftime(buf, sizeof buf, "%Y-%m-%d %H:%M:%S %Z", &tm);
	printf("normalized input: %s\n", buf);

	struct tm tm2 = tm;
	tm2.tm_mday++;
	errno = 0;
	if (mktime(&tm2)==-1 && errno) {
		perror("mktime day+1");
	} else {
		strftime(buf, sizeof buf, "%Y-%m-%d %H:%M:%S %Z", &tm2);
		printf("+1day per mktime: %s\n", buf);
	}
	localtime_r(&(time_t){t+86400}, &tm2);
	strftime(buf, sizeof buf, "%Y-%m-%d %H:%M:%S %Z", &tm2);
	printf("+1day via time_t: %s\n", buf);

	tm2 = tm;
	tm2.tm_mday--;
	errno = 0;
	if (mktime(&tm2)==-1 && errno) {
		perror("mktime day-1");
	} else {
		strftime(buf, sizeof buf, "%Y-%m-%d %H:%M:%S %Z", &tm2);
		printf("-1day per mktime: %s\n", buf);
	}
	localtime_r(&(time_t){t-86400}, &tm2);
	strftime(buf, sizeof buf, "%Y-%m-%d %H:%M:%S %Z", &tm2);
	printf("-1day via time_t: %s\n", buf);
}

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

For a change you requested 31 - 1:
    tm.tm_mday = 31 - 1;

Complete code to run:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

void test10()
{
    time_t t = 0;
    struct tm tm = {0};
    char buf[64];

    tm.tm_year = 2011 - 1900;
    tm.tm_mon = 12 - 1;
    tm.tm_mday = 31 - 1; // <-- here is the change
    tm.tm_hour = 0;
    tm.tm_min = 0;
    tm.tm_sec = 0;
    tm.tm_isdst = 0;

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("before: %s %ld\n", buf, t);

    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after1: %s %ld\n", buf, t);

    tm.tm_mday -= 1;
    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after2: %s %ld\n", buf, t);

    tm.tm_mday += 1;
    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after3: %s %ld\n", buf, t);
}

int main(int argc, char *argv[])
{
    test10();
    return 0;
}

$  musl-gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-30 00:00:00  0
after1: 2011-12-29 01:00:00 -10 1325156400
after2: 2011-12-28 01:00:00 -10 1325070000
after3: 2011-12-29 01:00:00 -10 1325156400


$  gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-30 00:00:00 +13 0
after1: 2011-12-31 00:00:00 +14 1325239200
after2: 2011-12-30 00:00:00 +14 -1
after3: 2011-12-31 00:00:00 +14 1325239200


So this is a also a bug in struct tm interpretation.

Behavior is consistent with tm_mday = 31;

AW


On Monday, March 25th, 2024 at 14:42, Rich Felker <dalias@libc.org> wrote:

> On Mon, Mar 25, 2024 at 01:24:57PM +0000, Alexander Weps wrote:
>
> > See below.
> >
> > AW
> >
> > On Monday, March 25th, 2024 at 14:13, Rich Felker dalias@libc.org wrote:
> >
> > > On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> > >
> > > > > If you take your test program and switch it to initialize with
> > > > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > > > expected thing to happen. Any change to "fix" the case you're
> > > > > complaining about would necessarily break this case.
> > > >
> > > > So (- day, +day):
> > > >
> > > > Musl:
> > > > 2011-12-31 01:00:00 +14
> > > > 2011-12-29 01:00:00 -10
> > > > 2011-12-29 01:00:00 -10
> > > >
> > > > Glibc:
> > > > 2012-01-01 01:00:00 +14
> > > > 2011-12-31 01:00:00 +14
> > > > 2012-01-01 01:00:00 +14
> > > >
> > > > Seems like musl doesn't even interpret the initial struct tm
> > > > correctly in that case. It is off by day.
> > > >
> > > > Because December only had 30 days, 31s day after normalization is
> > > > January 1st.
> > >
> > > This is nonsense. December has a day 31, which you can clearly see
> > > from the glibc output. For this particular year in this zone, with the
> > > zone rule change, there are "only 30 days" in December, but they are
> > > numbered 1-29 and 31, not 1-30.
> >
> > You confuse day of month which is represented in tm_mday with
> > calendar day that is interpreted by strftime.
> >
> > You said to set tm_mday = 31, which would be January 1st after normalization.
> > December 31s is 30th day of month represented as tm_mday = 30.
>
>
> OK, I meant tm_mday=31-1.
>
> > > What did you do that got glibc to output 2012-01-01? I guess you wrote
> > > code to do some wacky arithmetic after the original code you already
> > > had, rather than changing the code to start with 2011-12-31 as I
> > > suggested to get a look at what's happening.
> > >
> > > > > In any case, the core issue you're hitting here is that time zones are
> > > > > HARD to work with and that there is inherent complexity that libc
> > > > > cannot save you from. You only got lucky that what you were trying to
> > > > > do "worked" with glibc because you were iterating days forward; if you
> > > > > were doing reverse, it would break exactly the same way.
> > > >
> > > > I am not really commenting on this, until you sort out the above
> > > > inconsistencies.
> > >
> > > I already have but you refuse to look.
> >
> > It was addressed, do didn't scroll at the end of the e-mail.
>
>
> Run the attached passing the starting date to check as the first/only
> argument, and these test dates:
>
> - "2011-12-29 00:00:00"
> - "2011-12-31 00:00:00"
>
> Hopefully that will clarify things for you. On musl you will see:
>
> normalized input: 2011-12-29 00:00:00 -10
> +1day per mktime: 2011-12-29 00:00:00 -10
> +1day via time_t: 2011-12-31 00:00:00 +14
> -1day per mktime: 2011-12-28 00:00:00 -10
> -1day via time_t: 2011-12-28 00:00:00 -10
>
> normalized input: 2011-12-31 00:00:00 +14
> +1day per mktime: 2012-01-01 00:00:00 +14
> +1day via time_t: 2012-01-01 00:00:00 +14
> -1day per mktime: 2011-12-29 00:00:00 -10
> -1day via time_t: 2011-12-29 00:00:00 -10
>
> You can see what you get on glibc.
>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

I also run it with tm_isdst = 1 & tm_mday = 31 - 1:

Complete code to run:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

void test10()
{
    time_t t = 0;
    struct tm tm = {0};
    char buf[64];

    tm.tm_year = 2011 - 1900;
    tm.tm_mon = 12 - 1;
    tm.tm_mday = 31 - 1; // <-- here is the change
    tm.tm_hour = 0;
    tm.tm_min = 0;
    tm.tm_sec = 0;
    tm.tm_isdst = 1; // <-- here is the change

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("before: %s %ld\n", buf, t);

    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after1: %s %ld\n", buf, t);

    tm.tm_mday -= 1;
    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after2: %s %ld\n", buf, t);

    tm.tm_mday += 1;
    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after3: %s %ld\n", buf, t);
}

int main(int argc, char *argv[])
{
    test10();
    return 0;
}

$  musl-gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-30 00:00:00  0
after1: 2011-12-29 00:00:00 -10 1325152800
after2: 2011-12-28 00:00:00 -10 1325066400
after3: 2011-12-29 00:00:00 -10 1325152800


$  gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-30 00:00:00 +14 0
after1: 2011-12-30 00:00:00 +14 -1
after2: 2011-12-29 00:00:00 -10 1325152800
after3: 2011-12-30 00:00:00 -10 -1


So this is a also a bug in struct tm interpretation.

Behavior is consistent with tm_mday = 31;

AW


On Monday, March 25th, 2024 at 14:48, Alexander Weps <exander77@pm.me> wrote:

> For a change you requested 31 - 1:
> tm.tm_mday = 31 - 1;
>
> Complete code to run:
>
> #include <stdio.h>
>
> #include <stdlib.h>
>
> #include <time.h>
>
>
> void test10()
> {
> time_t t = 0;
> struct tm tm = {0};
> char buf[64];
>
> tm.tm_year = 2011 - 1900;
> tm.tm_mon = 12 - 1;
> tm.tm_mday = 31 - 1; // <-- here is the change
> tm.tm_hour = 0;
> tm.tm_min = 0;
> tm.tm_sec = 0;
> tm.tm_isdst = 0;
>
> strftime(buf, sizeof buf, "%F %T %Z", &tm);
> printf("before: %s %ld\n", buf, t);
>
> t = mktime(&tm);
>
> strftime(buf, sizeof buf, "%F %T %Z", &tm);
> printf("after1: %s %ld\n", buf, t);
>
> tm.tm_mday -= 1;
> t = mktime(&tm);
>
> strftime(buf, sizeof buf, "%F %T %Z", &tm);
> printf("after2: %s %ld\n", buf, t);
>
> tm.tm_mday += 1;
> t = mktime(&tm);
>
> strftime(buf, sizeof buf, "%F %T %Z", &tm);
> printf("after3: %s %ld\n", buf, t);
> }
>
> int main(int argc, char *argv[])
> {
> test10();
> return 0;
> }
>
> $ musl-gcc foo.c -o foo && TZ=Pacific/Apia ./foo
> before: 2011-12-30 00:00:00 0
> after1: 2011-12-29 01:00:00 -10 1325156400
> after2: 2011-12-28 01:00:00 -10 1325070000
> after3: 2011-12-29 01:00:00 -10 1325156400
>
>
> $ gcc foo.c -o foo && TZ=Pacific/Apia ./foo
> before: 2011-12-30 00:00:00 +13 0
> after1: 2011-12-31 00:00:00 +14 1325239200
> after2: 2011-12-30 00:00:00 +14 -1
> after3: 2011-12-31 00:00:00 +14 1325239200
>
>
> So this is a also a bug in struct tm interpretation.
>
> Behavior is consistent with tm_mday = 31;
>
> AW
>
>
>
>
> On Monday, March 25th, 2024 at 14:42, Rich Felker dalias@libc.org wrote:
>
> > On Mon, Mar 25, 2024 at 01:24:57PM +0000, Alexander Weps wrote:
> >
> > > See below.
> > >
> > > AW
> > >
> > > On Monday, March 25th, 2024 at 14:13, Rich Felker dalias@libc.org wrote:
> > >
> > > > On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> > > >
> > > > > > If you take your test program and switch it to initialize with
> > > > > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > > > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > > > > expected thing to happen. Any change to "fix" the case you're
> > > > > > complaining about would necessarily break this case.
> > > > >
> > > > > So (- day, +day):
> > > > >
> > > > > Musl:
> > > > > 2011-12-31 01:00:00 +14
> > > > > 2011-12-29 01:00:00 -10
> > > > > 2011-12-29 01:00:00 -10
> > > > >
> > > > > Glibc:
> > > > > 2012-01-01 01:00:00 +14
> > > > > 2011-12-31 01:00:00 +14
> > > > > 2012-01-01 01:00:00 +14
> > > > >
> > > > > Seems like musl doesn't even interpret the initial struct tm
> > > > > correctly in that case. It is off by day.
> > > > >
> > > > > Because December only had 30 days, 31s day after normalization is
> > > > > January 1st.
> > > >
> > > > This is nonsense. December has a day 31, which you can clearly see
> > > > from the glibc output. For this particular year in this zone, with the
> > > > zone rule change, there are "only 30 days" in December, but they are
> > > > numbered 1-29 and 31, not 1-30.
> > >
> > > You confuse day of month which is represented in tm_mday with
> > > calendar day that is interpreted by strftime.
> > >
> > > You said to set tm_mday = 31, which would be January 1st after normalization.
> > > December 31s is 30th day of month represented as tm_mday = 30.
> >
> > OK, I meant tm_mday=31-1.
> >
> > > > What did you do that got glibc to output 2012-01-01? I guess you wrote
> > > > code to do some wacky arithmetic after the original code you already
> > > > had, rather than changing the code to start with 2011-12-31 as I
> > > > suggested to get a look at what's happening.
> > > >
> > > > > > In any case, the core issue you're hitting here is that time zones are
> > > > > > HARD to work with and that there is inherent complexity that libc
> > > > > > cannot save you from. You only got lucky that what you were trying to
> > > > > > do "worked" with glibc because you were iterating days forward; if you
> > > > > > were doing reverse, it would break exactly the same way.
> > > > >
> > > > > I am not really commenting on this, until you sort out the above
> > > > > inconsistencies.
> > > >
> > > > I already have but you refuse to look.
> > >
> > > It was addressed, do didn't scroll at the end of the e-mail.
> >
> > Run the attached passing the starting date to check as the first/only
> > argument, and these test dates:
> >
> > - "2011-12-29 00:00:00"
> > - "2011-12-31 00:00:00"
> >
> > Hopefully that will clarify things for you. On musl you will see:
> >
> > normalized input: 2011-12-29 00:00:00 -10
> > +1day per mktime: 2011-12-29 00:00:00 -10
> > +1day via time_t: 2011-12-31 00:00:00 +14
> > -1day per mktime: 2011-12-28 00:00:00 -10
> > -1day via time_t: 2011-12-28 00:00:00 -10
> >
> > normalized input: 2011-12-31 00:00:00 +14
> > +1day per mktime: 2012-01-01 00:00:00 +14
> > +1day via time_t: 2012-01-01 00:00:00 +14
> > -1day per mktime: 2011-12-29 00:00:00 -10
> > -1day via time_t: 2011-12-29 00:00:00 -10
> >
> > You can see what you get on glibc.
> >
> > Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Mon, Mar 25, 2024 at 09:42:53AM -0400, Rich Felker wrote:
> On Mon, Mar 25, 2024 at 01:24:57PM +0000, Alexander Weps wrote:
> > See below.
> > 
> > AW
> > 
> > 
> > On Monday, March 25th, 2024 at 14:13, Rich Felker <dalias@libc.org> wrote:
> > 
> > > On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> > > 
> > > > > If you take your test program and switch it to initialize with
> > > > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > > > expected thing to happen. Any change to "fix" the case you're
> > > > > complaining about would necessarily break this case.
> > > > 
> > > > So (- day, +day):
> > > > 
> > > > Musl:
> > > > 2011-12-31 01:00:00 +14
> > > > 2011-12-29 01:00:00 -10
> > > > 2011-12-29 01:00:00 -10
> > > > 
> > > > Glibc:
> > > > 2012-01-01 01:00:00 +14
> > > > 2011-12-31 01:00:00 +14
> > > > 2012-01-01 01:00:00 +14
> > > > 
> > > > Seems like musl doesn't even interpret the initial struct tm
> > > > correctly in that case. It is off by day.
> > > > 
> > > > Because December only had 30 days, 31s day after normalization is
> > > > January 1st.
> > > 
> > > 
> > > This is nonsense. December has a day 31, which you can clearly see
> > > from the glibc output. For this particular year in this zone, with the
> > > zone rule change, there are "only 30 days" in December, but they are
> > > numbered 1-29 and 31, not 1-30.
> > 
> > You confuse day of month which is represented in tm_mday with
> > calendar day that is interpreted by strftime.
> > 
> > You said to set tm_mday = 31, which would be January 1st after normalization.
> > December 31s is 30th day of month represented as tm_mday = 30.
> 
> OK, I meant tm_mday=31-1.

Um, no, where did you get that idea? I just assumed you were right
because I always forget which tm_* are off-by-1, but tm_mday is
one-based not zero-based:

           int    tm_mday;          //   day of the month -- [1, 31]

(per the standard). So how did you end up getting the wrong thing? Are
you even running the code you say you are?

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

On Monday, March 25th, 2024 at 19:02, Rich Felker <dalias@libc.org> wrote:

> On Mon, Mar 25, 2024 at 09:42:53AM -0400, Rich Felker wrote:
>
> > On Mon, Mar 25, 2024 at 01:24:57PM +0000, Alexander Weps wrote:
> >
> > > See below.
> > >
> > > AW
> > >
> > > On Monday, March 25th, 2024 at 14:13, Rich Felker dalias@libc.org wrote:
> > >
> > > > On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> > > >
> > > > > > If you take your test program and switch it to initialize with
> > > > > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > > > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > > > > expected thing to happen. Any change to "fix" the case you're
> > > > > > complaining about would necessarily break this case.
> > > > >
> > > > > So (- day, +day):
> > > > >
> > > > > Musl:
> > > > > 2011-12-31 01:00:00 +14
> > > > > 2011-12-29 01:00:00 -10
> > > > > 2011-12-29 01:00:00 -10
> > > > >
> > > > > Glibc:
> > > > > 2012-01-01 01:00:00 +14
> > > > > 2011-12-31 01:00:00 +14
> > > > > 2012-01-01 01:00:00 +14
> > > > >
> > > > > Seems like musl doesn't even interpret the initial struct tm
> > > > > correctly in that case. It is off by day.
> > > > >
> > > > > Because December only had 30 days, 31s day after normalization is
> > > > > January 1st.
> > > >
> > > > This is nonsense. December has a day 31, which you can clearly see
> > > > from the glibc output. For this particular year in this zone, with the
> > > > zone rule change, there are "only 30 days" in December, but they are
> > > > numbered 1-29 and 31, not 1-30.
> > >
> > > You confuse day of month which is represented in tm_mday with
> > > calendar day that is interpreted by strftime.
> > >
> > > You said to set tm_mday = 31, which would be January 1st after normalization.
> > > December 31s is 30th day of month represented as tm_mday = 30.
> >
> > OK, I meant tm_mday=31-1.
>
>
> Um, no, where did you get that idea? I just assumed you were right
> because I always forget which tm_* are off-by-1, but tm_mday is
> one-based not zero-based:
>
> int tm_mday; // day of the month -- [1, 31]
>
> (per the standard). So how did you end up getting the wrong thing? Are
> you even running the code you say you are?
>

I have to sincerely ask if you are feeling ok?
You seem not able to follow this conversation.

What idea do you mean?
Also you have the codes. You can like "I don't know" run them yourself?
You question I run those codes without trying to run them yourself? Again?!
What is going on?

Maybe I reiterate some basic facts for you and that will put you back on track.

This was an example from an article provided earlier in this thread (by somebody).
We are in TZ=Pacific/Apia.
The 30th December was skipped in 2011. There was no December 30th.
So, there were only 30 days in December.
30th day of the month December was December 31st.

And run those examples yourself. I have no idea why I am being questioned if they generate the output when you can easily verify it yourself.

> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Mon, Mar 25, 2024 at 06:28:14PM +0000, Alexander Weps wrote:
> On Monday, March 25th, 2024 at 19:02, Rich Felker <dalias@libc.org> wrote:
> 
> > On Mon, Mar 25, 2024 at 09:42:53AM -0400, Rich Felker wrote:
> >
> > > On Mon, Mar 25, 2024 at 01:24:57PM +0000, Alexander Weps wrote:
> > >
> > > > See below.
> > > >
> > > > AW
> > > >
> > > > On Monday, March 25th, 2024 at 14:13, Rich Felker dalias@libc.org wrote:
> > > >
> > > > > On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> > > > >
> > > > > > > If you take your test program and switch it to initialize with
> > > > > > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > > > > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > > > > > expected thing to happen. Any change to "fix" the case you're
> > > > > > > complaining about would necessarily break this case.
> > > > > >
> > > > > > So (- day, +day):
> > > > > >
> > > > > > Musl:
> > > > > > 2011-12-31 01:00:00 +14
> > > > > > 2011-12-29 01:00:00 -10
> > > > > > 2011-12-29 01:00:00 -10
> > > > > >
> > > > > > Glibc:
> > > > > > 2012-01-01 01:00:00 +14
> > > > > > 2011-12-31 01:00:00 +14
> > > > > > 2012-01-01 01:00:00 +14
> > > > > >
> > > > > > Seems like musl doesn't even interpret the initial struct tm
> > > > > > correctly in that case. It is off by day.
> > > > > >
> > > > > > Because December only had 30 days, 31s day after normalization is
> > > > > > January 1st.
> > > > >
> > > > > This is nonsense. December has a day 31, which you can clearly see
> > > > > from the glibc output. For this particular year in this zone, with the
> > > > > zone rule change, there are "only 30 days" in December, but they are
> > > > > numbered 1-29 and 31, not 1-30.
> > > >
> > > > You confuse day of month which is represented in tm_mday with
> > > > calendar day that is interpreted by strftime.
> > > >
> > > > You said to set tm_mday = 31, which would be January 1st after normalization.
> > > > December 31s is 30th day of month represented as tm_mday = 30.
> > >
> > > OK, I meant tm_mday=31-1.
> >
> >
> > Um, no, where did you get that idea? I just assumed you were right
> > because I always forget which tm_* are off-by-1, but tm_mday is
> > one-based not zero-based:
> >
> > int tm_mday; // day of the month -- [1, 31]
> >
> > (per the standard). So how did you end up getting the wrong thing? Are
> > you even running the code you say you are?
> >
> 
> I have to sincerely ask if you are feeling ok?
> You seem not able to follow this conversation.
> 
> What idea do you mean?
> Also you have the codes. You can like "I don't know" run them yourself?
> You question I run those codes without trying to run them yourself? Again?!
> What is going on?

The first few pieces of code you posted did not work because they
depended on other code you did not include, so I stopped trying to run
them.

> Maybe I reiterate some basic facts for you and that will put you
> back on track.
> 
> This was an example from an article provided earlier in this thread (by somebody).
> We are in TZ=Pacific/Apia.
> The 30th December was skipped in 2011. There was no December 30th.
> So, there were only 30 days in December.
> 30th day of the month December was December 31st.
> 
> And run those examples yourself. I have no idea why I am being
> questioned if they generate the output when you can easily verify it
> yourself.

Which piece of self-contained, actually-runnable code would you like
me to look at that demonstrates something wrong? (i.e. not something I
have already said is behaving as expected)

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

I am not sure which one you mean, all latest codes even includes headers and main...

I have no idea what to tell you.

AW


On Monday, March 25th, 2024 at 19:53, Rich Felker <dalias@libc.org> wrote:

> On Mon, Mar 25, 2024 at 06:28:14PM +0000, Alexander Weps wrote:
>
> > On Monday, March 25th, 2024 at 19:02, Rich Felker dalias@libc.org wrote:
> >
> > > On Mon, Mar 25, 2024 at 09:42:53AM -0400, Rich Felker wrote:
> > >
> > > > On Mon, Mar 25, 2024 at 01:24:57PM +0000, Alexander Weps wrote:
> > > >
> > > > > See below.
> > > > >
> > > > > AW
> > > > >
> > > > > On Monday, March 25th, 2024 at 14:13, Rich Felker dalias@libc.org wrote:
> > > > >
> > > > > > On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> > > > > >
> > > > > > > > If you take your test program and switch it to initialize with
> > > > > > > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > > > > > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > > > > > > expected thing to happen. Any change to "fix" the case you're
> > > > > > > > complaining about would necessarily break this case.
> > > > > > >
> > > > > > > So (- day, +day):
> > > > > > >
> > > > > > > Musl:
> > > > > > > 2011-12-31 01:00:00 +14
> > > > > > > 2011-12-29 01:00:00 -10
> > > > > > > 2011-12-29 01:00:00 -10
> > > > > > >
> > > > > > > Glibc:
> > > > > > > 2012-01-01 01:00:00 +14
> > > > > > > 2011-12-31 01:00:00 +14
> > > > > > > 2012-01-01 01:00:00 +14
> > > > > > >
> > > > > > > Seems like musl doesn't even interpret the initial struct tm
> > > > > > > correctly in that case. It is off by day.
> > > > > > >
> > > > > > > Because December only had 30 days, 31s day after normalization is
> > > > > > > January 1st.
> > > > > >
> > > > > > This is nonsense. December has a day 31, which you can clearly see
> > > > > > from the glibc output. For this particular year in this zone, with the
> > > > > > zone rule change, there are "only 30 days" in December, but they are
> > > > > > numbered 1-29 and 31, not 1-30.
> > > > >
> > > > > You confuse day of month which is represented in tm_mday with
> > > > > calendar day that is interpreted by strftime.
> > > > >
> > > > > You said to set tm_mday = 31, which would be January 1st after normalization.
> > > > > December 31s is 30th day of month represented as tm_mday = 30.
> > > >
> > > > OK, I meant tm_mday=31-1.
> > >
> > > Um, no, where did you get that idea? I just assumed you were right
> > > because I always forget which tm_* are off-by-1, but tm_mday is
> > > one-based not zero-based:
> > >
> > > int tm_mday; // day of the month -- [1, 31]
> > >
> > > (per the standard). So how did you end up getting the wrong thing? Are
> > > you even running the code you say you are?
> >
> > I have to sincerely ask if you are feeling ok?
> > You seem not able to follow this conversation.
> >
> > What idea do you mean?
> > Also you have the codes. You can like "I don't know" run them yourself?
> > You question I run those codes without trying to run them yourself? Again?!
> > What is going on?
>
>
> The first few pieces of code you posted did not work because they
> depended on other code you did not include, so I stopped trying to run
> them.
>
> > Maybe I reiterate some basic facts for you and that will put you
> > back on track.
> >
> > This was an example from an article provided earlier in this thread (by somebody).
> > We are in TZ=Pacific/Apia.
> > The 30th December was skipped in 2011. There was no December 30th.
> > So, there were only 30 days in December.
> > 30th day of the month December was December 31st.
> >
> > And run those examples yourself. I have no idea why I am being
> > questioned if they generate the output when you can easily verify it
> > yourself.
>
>
> Which piece of self-contained, actually-runnable code would you like
> me to look at that demonstrates something wrong? (i.e. not something I
> have already said is behaving as expected)
>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Mon, Mar 25, 2024 at 06:57:49PM +0000, Alexander Weps wrote:
> I am not sure which one you mean, all latest codes even includes
> headers and main...

https://www.openwall.com/lists/musl/2024/03/25/3

> I have no idea what to tell you.

The first version I found that's actually compilable is:

https://www.openwall.com/lists/musl/2024/03/25/11

It roughly behaves as expected on musl, except possibly not applying
the tm_isdst=0, which is what was making the output confusing on
glibc -- that threw the input back across the rule change cutoff.

With tm_isdst=1 and tm_mday=31, on glibc, I get:

before: 2011-12-31 00:00:00 WSDT 0
after1: 2011-12-31 00:00:00 WSDT 1325239200
after2: 2011-12-30 00:00:00 WSDT -1
after3: 2011-12-31 00:00:00 WSDT 1325239200

The -1 in the after2 line indicates that mktime failed with an error
(and should not have modified tm; that's arguably a bug in glibc). The
partial modification that it made reflects the initial normalization
(type 1 in my notation) but not the rule change normalization (type 2
in my notation) since glibc has failed the operation for an input date
that does not exist on the calendar (it does not do type 2
normalization at all; it just rejects it).

Running this same change on musl, I get:

before: 2011-12-31 00:00:00  0
after1: 2011-12-31 00:00:00 +14 1325239200
after2: 2011-12-29 00:00:00 -10 1325152800
after3: 2011-12-29 00:00:00 -10 1325152800

which again is what I expect. From one side, the move-by-1-day changes
the time to the next calendar day in that direction. From the other
side, it's unable to change it.

I'll look into why the tm_isdst=0 application was not happening.

Rich






> On Monday, March 25th, 2024 at 19:53, Rich Felker <dalias@libc.org> wrote:
> 
> > On Mon, Mar 25, 2024 at 06:28:14PM +0000, Alexander Weps wrote:
> >
> > > On Monday, March 25th, 2024 at 19:02, Rich Felker dalias@libc.org wrote:
> > >
> > > > On Mon, Mar 25, 2024 at 09:42:53AM -0400, Rich Felker wrote:
> > > >
> > > > > On Mon, Mar 25, 2024 at 01:24:57PM +0000, Alexander Weps wrote:
> > > > >
> > > > > > See below.
> > > > > >
> > > > > > AW
> > > > > >
> > > > > > On Monday, March 25th, 2024 at 14:13, Rich Felker dalias@libc.org wrote:
> > > > > >
> > > > > > > On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> > > > > > >
> > > > > > > > > If you take your test program and switch it to initialize with
> > > > > > > > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > > > > > > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > > > > > > > expected thing to happen. Any change to "fix" the case you're
> > > > > > > > > complaining about would necessarily break this case.
> > > > > > > >
> > > > > > > > So (- day, +day):
> > > > > > > >
> > > > > > > > Musl:
> > > > > > > > 2011-12-31 01:00:00 +14
> > > > > > > > 2011-12-29 01:00:00 -10
> > > > > > > > 2011-12-29 01:00:00 -10
> > > > > > > >
> > > > > > > > Glibc:
> > > > > > > > 2012-01-01 01:00:00 +14
> > > > > > > > 2011-12-31 01:00:00 +14
> > > > > > > > 2012-01-01 01:00:00 +14
> > > > > > > >
> > > > > > > > Seems like musl doesn't even interpret the initial struct tm
> > > > > > > > correctly in that case. It is off by day.
> > > > > > > >
> > > > > > > > Because December only had 30 days, 31s day after normalization is
> > > > > > > > January 1st.
> > > > > > >
> > > > > > > This is nonsense. December has a day 31, which you can clearly see
> > > > > > > from the glibc output. For this particular year in this zone, with the
> > > > > > > zone rule change, there are "only 30 days" in December, but they are
> > > > > > > numbered 1-29 and 31, not 1-30.
> > > > > >
> > > > > > You confuse day of month which is represented in tm_mday with
> > > > > > calendar day that is interpreted by strftime.
> > > > > >
> > > > > > You said to set tm_mday = 31, which would be January 1st after normalization.
> > > > > > December 31s is 30th day of month represented as tm_mday = 30.
> > > > >
> > > > > OK, I meant tm_mday=31-1.
> > > >
> > > > Um, no, where did you get that idea? I just assumed you were right
> > > > because I always forget which tm_* are off-by-1, but tm_mday is
> > > > one-based not zero-based:
> > > >
> > > > int tm_mday; // day of the month -- [1, 31]
> > > >
> > > > (per the standard). So how did you end up getting the wrong thing? Are
> > > > you even running the code you say you are?
> > >
> > > I have to sincerely ask if you are feeling ok?
> > > You seem not able to follow this conversation.
> > >
> > > What idea do you mean?
> > > Also you have the codes. You can like "I don't know" run them yourself?
> > > You question I run those codes without trying to run them yourself? Again?!
> > > What is going on?
> >
> >
> > The first few pieces of code you posted did not work because they
> > depended on other code you did not include, so I stopped trying to run
> > them.
> >
> > > Maybe I reiterate some basic facts for you and that will put you
> > > back on track.
> > >
> > > This was an example from an article provided earlier in this thread (by somebody).
> > > We are in TZ=Pacific/Apia.
> > > The 30th December was skipped in 2011. There was no December 30th.
> > > So, there were only 30 days in December.
> > > 30th day of the month December was December 31st.
> > >
> > > And run those examples yourself. I have no idea why I am being
> > > questioned if they generate the output when you can easily verify it
> > > yourself.
> >
> >
> > Which piece of self-contained, actually-runnable code would you like
> > me to look at that demonstrates something wrong? (i.e. not something I
> > have already said is behaving as expected)

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Mon, Mar 25, 2024 at 03:38:13PM -0400, Rich Felker wrote:
> On Mon, Mar 25, 2024 at 06:57:49PM +0000, Alexander Weps wrote:
> > I am not sure which one you mean, all latest codes even includes
> > headers and main...
> 
> https://www.openwall.com/lists/musl/2024/03/25/3
> 
> > I have no idea what to tell you.
> 
> The first version I found that's actually compilable is:
> 
> https://www.openwall.com/lists/musl/2024/03/25/11
> 
> It roughly behaves as expected on musl, except possibly not applying
> the tm_isdst=0, which is what was making the output confusing on
> glibc -- that threw the input back across the rule change cutoff.

No, it's deeper than this. glibc is offsetting the input by an entire
day when tm_isdst=0, and I don't know why. It looks like a bug in
glibc.

> With tm_isdst=1 and tm_mday=31, on glibc, I get:
> 
> before: 2011-12-31 00:00:00 WSDT 0
> after1: 2011-12-31 00:00:00 WSDT 1325239200
> after2: 2011-12-30 00:00:00 WSDT -1
> after3: 2011-12-31 00:00:00 WSDT 1325239200
> 
> The -1 in the after2 line indicates that mktime failed with an error
> (and should not have modified tm; that's arguably a bug in glibc). The
> partial modification that it made reflects the initial normalization
> (type 1 in my notation) but not the rule change normalization (type 2
> in my notation) since glibc has failed the operation for an input date
> that does not exist on the calendar (it does not do type 2
> normalization at all; it just rejects it).
> 
> Running this same change on musl, I get:
> 
> before: 2011-12-31 00:00:00  0
> after1: 2011-12-31 00:00:00 +14 1325239200
> after2: 2011-12-29 00:00:00 -10 1325152800
> after3: 2011-12-29 00:00:00 -10 1325152800
> 
> which again is what I expect. From one side, the move-by-1-day changes
> the time to the next calendar day in that direction. From the other
> side, it's unable to change it.
> 
> I'll look into why the tm_isdst=0 application was not happening.

Hmm, I must have misread the output. It seems to be correct with
tm_isdst=0 too:

before: 2011-12-31 00:00:00  0
after1: 2011-12-31 01:00:00 +14 1325242800
after2: 2011-12-29 01:00:00 -10 1325156400
after3: 2011-12-29 01:00:00 -10 1325156400

(If it's 00:00:00 in standard time, it's 01:00:00 in DST, so the
initial time seems to have been interpreted correctly.)

I also went back and tested both with tm_isdst=-1, and both glibc and
musl do the same thing as they do with tm_isdst=1 (which is correct).

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

Sorry my bad, that was isdst = 0;

$  gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-31 00:00:00 +14 0
after1: 2011-12-31 00:00:00 +14 1325239200
after2: 2011-12-30 00:00:00 +14 -1
after3: 2011-12-31 00:00:00 +14 1325239200

$  musl-gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-31 00:00:00  0
after1: 2011-12-31 00:00:00 +14 1325239200
after2: 2011-12-29 00:00:00 -10 1325152800
after3: 2011-12-29 00:00:00 -10 1325152800

I agree with isdst = 1; reuslts.

AW


On Monday, March 25th, 2024 at 20:47, Rich Felker <dalias@libc.org> wrote:

> On Mon, Mar 25, 2024 at 03:38:13PM -0400, Rich Felker wrote:
>
> > On Mon, Mar 25, 2024 at 06:57:49PM +0000, Alexander Weps wrote:
> >
> > > I am not sure which one you mean, all latest codes even includes
> > > headers and main...
> >
> > https://www.openwall.com/lists/musl/2024/03/25/3
> >
> > > I have no idea what to tell you.
> >
> > The first version I found that's actually compilable is:
> >
> > https://www.openwall.com/lists/musl/2024/03/25/11
> >
> > It roughly behaves as expected on musl, except possibly not applying
> > the tm_isdst=0, which is what was making the output confusing on
> > glibc -- that threw the input back across the rule change cutoff.
>
>
> No, it's deeper than this. glibc is offsetting the input by an entire
> day when tm_isdst=0, and I don't know why. It looks like a bug in
> glibc.
>
> > With tm_isdst=1 and tm_mday=31, on glibc, I get:
> >
> > before: 2011-12-31 00:00:00 WSDT 0
> > after1: 2011-12-31 00:00:00 WSDT 1325239200
> > after2: 2011-12-30 00:00:00 WSDT -1
> > after3: 2011-12-31 00:00:00 WSDT 1325239200
> >
> > The -1 in the after2 line indicates that mktime failed with an error
> > (and should not have modified tm; that's arguably a bug in glibc). The
> > partial modification that it made reflects the initial normalization
> > (type 1 in my notation) but not the rule change normalization (type 2
> > in my notation) since glibc has failed the operation for an input date
> > that does not exist on the calendar (it does not do type 2
> > normalization at all; it just rejects it).
> >
> > Running this same change on musl, I get:
> >
> > before: 2011-12-31 00:00:00 0
> > after1: 2011-12-31 00:00:00 +14 1325239200
> > after2: 2011-12-29 00:00:00 -10 1325152800
> > after3: 2011-12-29 00:00:00 -10 1325152800
> >
> > which again is what I expect. From one side, the move-by-1-day changes
> > the time to the next calendar day in that direction. From the other
> > side, it's unable to change it.
> >
> > I'll look into why the tm_isdst=0 application was not happening.
>
>
> Hmm, I must have misread the output. It seems to be correct with
> tm_isdst=0 too:
>
> before: 2011-12-31 00:00:00 0
> after1: 2011-12-31 01:00:00 +14 1325242800
> after2: 2011-12-29 01:00:00 -10 1325156400
> after3: 2011-12-29 01:00:00 -10 1325156400
>
> (If it's 00:00:00 in standard time, it's 01:00:00 in DST, so the
> initial time seems to have been interpreted correctly.)
>
> I also went back and tested both with tm_isdst=-1, and both glibc and
> musl do the same thing as they do with tm_isdst=1 (which is correct).
>
> Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

Analyzing this and Glibc behavior is perfectly valid for that case:

+ day, + day, - day, - day

$  gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-31 00:00:00 +14 0
after1: 2011-12-31 00:00:00 +14 1325239200
after1: 2011-12-30 00:00:00 +14 -1
after2: 2011-12-29 00:00:00 -10 1325152800
after2: 2011-12-30 00:00:00 -10 -1
after3: 2011-12-31 00:00:00 +14 1325239200

I can go forward and backward in time.

$  musl-gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-31 00:00:00  0
after1: 2011-12-31 00:00:00 +14 1325239200
after1: 2011-12-29 00:00:00 -10 1325152800
after2: 2011-12-28 00:00:00 -10 1325066400
after2: 2011-12-29 00:00:00 -10 1325152800
after3: 2011-12-29 00:00:00 -10 1325152800

Musl is off by 2 days and ran into a cycle.

AW


On Monday, March 25th, 2024 at 21:05, Alexander Weps <exander77@pm.me> wrote:

> Sorry my bad, that was isdst = 0;
>
> $ gcc foo.c -o foo && TZ=Pacific/Apia ./foo
> before: 2011-12-31 00:00:00 +14 0
> after1: 2011-12-31 00:00:00 +14 1325239200
> after2: 2011-12-30 00:00:00 +14 -1
> after3: 2011-12-31 00:00:00 +14 1325239200
>
> $ musl-gcc foo.c -o foo && TZ=Pacific/Apia ./foo
> before: 2011-12-31 00:00:00 0
> after1: 2011-12-31 00:00:00 +14 1325239200
> after2: 2011-12-29 00:00:00 -10 1325152800
> after3: 2011-12-29 00:00:00 -10 1325152800
>
> I agree with isdst = 1; reuslts.
>
> AW
>
>
>
>
> On Monday, March 25th, 2024 at 20:47, Rich Felker dalias@libc.org wrote:
>
> > On Mon, Mar 25, 2024 at 03:38:13PM -0400, Rich Felker wrote:
> >
> > > On Mon, Mar 25, 2024 at 06:57:49PM +0000, Alexander Weps wrote:
> > >
> > > > I am not sure which one you mean, all latest codes even includes
> > > > headers and main...
> > >
> > > https://www.openwall.com/lists/musl/2024/03/25/3
> > >
> > > > I have no idea what to tell you.
> > >
> > > The first version I found that's actually compilable is:
> > >
> > > https://www.openwall.com/lists/musl/2024/03/25/11
> > >
> > > It roughly behaves as expected on musl, except possibly not applying
> > > the tm_isdst=0, which is what was making the output confusing on
> > > glibc -- that threw the input back across the rule change cutoff.
> >
> > No, it's deeper than this. glibc is offsetting the input by an entire
> > day when tm_isdst=0, and I don't know why. It looks like a bug in
> > glibc.
> >
> > > With tm_isdst=1 and tm_mday=31, on glibc, I get:
> > >
> > > before: 2011-12-31 00:00:00 WSDT 0
> > > after1: 2011-12-31 00:00:00 WSDT 1325239200
> > > after2: 2011-12-30 00:00:00 WSDT -1
> > > after3: 2011-12-31 00:00:00 WSDT 1325239200
> > >
> > > The -1 in the after2 line indicates that mktime failed with an error
> > > (and should not have modified tm; that's arguably a bug in glibc). The
> > > partial modification that it made reflects the initial normalization
> > > (type 1 in my notation) but not the rule change normalization (type 2
> > > in my notation) since glibc has failed the operation for an input date
> > > that does not exist on the calendar (it does not do type 2
> > > normalization at all; it just rejects it).
> > >
> > > Running this same change on musl, I get:
> > >
> > > before: 2011-12-31 00:00:00 0
> > > after1: 2011-12-31 00:00:00 +14 1325239200
> > > after2: 2011-12-29 00:00:00 -10 1325152800
> > > after3: 2011-12-29 00:00:00 -10 1325152800
> > >
> > > which again is what I expect. From one side, the move-by-1-day changes
> > > the time to the next calendar day in that direction. From the other
> > > side, it's unable to change it.
> > >
> > > I'll look into why the tm_isdst=0 application was not happening.
> >
> > Hmm, I must have misread the output. It seems to be correct with
> > tm_isdst=0 too:
> >
> > before: 2011-12-31 00:00:00 0
> > after1: 2011-12-31 01:00:00 +14 1325242800
> > after2: 2011-12-29 01:00:00 -10 1325156400
> > after3: 2011-12-29 01:00:00 -10 1325156400
> >
> > (If it's 00:00:00 in standard time, it's 01:00:00 in DST, so the
> > initial time seems to have been interpreted correctly.)
> >
> > I also went back and tested both with tm_isdst=-1, and both glibc and
> > musl do the same thing as they do with tm_isdst=1 (which is correct).
> >
> > Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

On Monday, March 25th, 2024 at 20:38, Rich Felker <dalias@libc.org> wrote:

> On Mon, Mar 25, 2024 at 06:57:49PM +0000, Alexander Weps wrote:
>
> > I am not sure which one you mean, all latest codes even includes
> > headers and main...
>
>
> https://www.openwall.com/lists/musl/2024/03/25/3
>
> > I have no idea what to tell you.
>
>
> The first version I found that's actually compilable is:
>
> https://www.openwall.com/lists/musl/2024/03/25/11
>
> It roughly behaves as expected on musl, except possibly not applying
> the tm_isdst=0, which is what was making the output confusing on
> glibc -- that threw the input back across the rule change cutoff.
>
> With tm_isdst=1 and tm_mday=31, on glibc, I get:
>
> before: 2011-12-31 00:00:00 WSDT 0
> after1: 2011-12-31 00:00:00 WSDT 1325239200
> after2: 2011-12-30 00:00:00 WSDT -1
> after3: 2011-12-31 00:00:00 WSDT 1325239200
>
> The -1 in the after2 line indicates that mktime failed with an error
> (and should not have modified tm; that's arguably a bug in glibc).The

No, -1 means that was not able to make seconds since beginning of epoch.
It has nothing to do with modifying tm...

Also, can you share the whole code, you did some changes and I don't reproduce the result. Which sample it is based on?

I set tm_isdst=1 and tm_mday=31 in the example above and I get:

$  gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-31 00:00:00 +13 0
after1: 2012-01-01 01:00:00 +14 1325329200
after2: 2011-12-31 01:00:00 +14 1325242800
after3: 2012-01-01 01:00:00 +14 1325329200

I do not get any -1 results.

$  musl-gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-31 00:00:00  0
after1: 2011-12-31 01:00:00 +14 1325242800
after2: 2011-12-29 01:00:00 -10 1325156400
after3: 2011-12-29 01:00:00 -10 1325156400

> partial modification that it made reflects the initial normalization
> (type 1 in my notation) but not the rule change normalization (type 2
> in my notation) since glibc has failed the operation for an input date
> that does not exist on the calendar (it does not do type 2
> normalization at all; it just rejects it).
>
> Running this same change on musl, I get:
>
> before: 2011-12-31 00:00:00 0
> after1: 2011-12-31 00:00:00 +14 1325239200
> after2: 2011-12-29 00:00:00 -10 1325152800
> after3: 2011-12-29 00:00:00 -10 1325152800
>
> which again is what I expect. From one side, the move-by-1-day changes
> the time to the next calendar day in that direction. From the other
> side, it's unable to change it.

Glibc works without any issues as shown above on my machine. Both forward and backward.

Your results are off by one hour. And you start with WSDT (+14). Without mktime. The initial date produced by glibc should be +13 as Pacific/Apia is +13.

>
> I'll look into why the tm_isdst=0 application was not happening.
>
> Rich
>
>
>
>
>
> > On Monday, March 25th, 2024 at 19:53, Rich Felker dalias@libc.org wrote:
> >
> > > On Mon, Mar 25, 2024 at 06:28:14PM +0000, Alexander Weps wrote:
> > >
> > > > On Monday, March 25th, 2024 at 19:02, Rich Felker dalias@libc.org wrote:
> > > >
> > > > > On Mon, Mar 25, 2024 at 09:42:53AM -0400, Rich Felker wrote:
> > > > >
> > > > > > On Mon, Mar 25, 2024 at 01:24:57PM +0000, Alexander Weps wrote:
> > > > > >
> > > > > > > See below.
> > > > > > >
> > > > > > > AW
> > > > > > >
> > > > > > > On Monday, March 25th, 2024 at 14:13, Rich Felker dalias@libc.org wrote:
> > > > > > >
> > > > > > > > On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> > > > > > > >
> > > > > > > > > > If you take your test program and switch it to initialize with
> > > > > > > > > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > > > > > > > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > > > > > > > > expected thing to happen. Any change to "fix" the case you're
> > > > > > > > > > complaining about would necessarily break this case.
> > > > > > > > >
> > > > > > > > > So (- day, +day):
> > > > > > > > >
> > > > > > > > > Musl:
> > > > > > > > > 2011-12-31 01:00:00 +14
> > > > > > > > > 2011-12-29 01:00:00 -10
> > > > > > > > > 2011-12-29 01:00:00 -10
> > > > > > > > >
> > > > > > > > > Glibc:
> > > > > > > > > 2012-01-01 01:00:00 +14
> > > > > > > > > 2011-12-31 01:00:00 +14
> > > > > > > > > 2012-01-01 01:00:00 +14
> > > > > > > > >
> > > > > > > > > Seems like musl doesn't even interpret the initial struct tm
> > > > > > > > > correctly in that case. It is off by day.
> > > > > > > > >
> > > > > > > > > Because December only had 30 days, 31s day after normalization is
> > > > > > > > > January 1st.
> > > > > > > >
> > > > > > > > This is nonsense. December has a day 31, which you can clearly see
> > > > > > > > from the glibc output. For this particular year in this zone, with the
> > > > > > > > zone rule change, there are "only 30 days" in December, but they are
> > > > > > > > numbered 1-29 and 31, not 1-30.
> > > > > > >
> > > > > > > You confuse day of month which is represented in tm_mday with
> > > > > > > calendar day that is interpreted by strftime.
> > > > > > >
> > > > > > > You said to set tm_mday = 31, which would be January 1st after normalization.
> > > > > > > December 31s is 30th day of month represented as tm_mday = 30.
> > > > > >
> > > > > > OK, I meant tm_mday=31-1.
> > > > >
> > > > > Um, no, where did you get that idea? I just assumed you were right
> > > > > because I always forget which tm_* are off-by-1, but tm_mday is
> > > > > one-based not zero-based:
> > > > >
> > > > > int tm_mday; // day of the month -- [1, 31]
> > > > >
> > > > > (per the standard). So how did you end up getting the wrong thing? Are
> > > > > you even running the code you say you are?
> > > >
> > > > I have to sincerely ask if you are feeling ok?
> > > > You seem not able to follow this conversation.
> > > >
> > > > What idea do you mean?
> > > > Also you have the codes. You can like "I don't know" run them yourself?
> > > > You question I run those codes without trying to run them yourself? Again?!
> > > > What is going on?
> > >
> > > The first few pieces of code you posted did not work because they
> > > depended on other code you did not include, so I stopped trying to run
> > > them.
> > >
> > > > Maybe I reiterate some basic facts for you and that will put you
> > > > back on track.
> > > >
> > > > This was an example from an article provided earlier in this thread (by somebody).
> > > > We are in TZ=Pacific/Apia.
> > > > The 30th December was skipped in 2011. There was no December 30th.
> > > > So, there were only 30 days in December.
> > > > 30th day of the month December was December 31st.
> > > >
> > > > And run those examples yourself. I have no idea why I am being
> > > > questioned if they generate the output when you can easily verify it
> > > > yourself.
> > >
> > > Which piece of self-contained, actually-runnable code would you like
> > > me to look at that demonstrates something wrong? (i.e. not something I
> > > have already said is behaving as expected)

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Mon, Mar 25, 2024 at 08:00:19PM +0000, Alexander Weps wrote:
> On Monday, March 25th, 2024 at 20:38, Rich Felker <dalias@libc.org> wrote:
> 
> > On Mon, Mar 25, 2024 at 06:57:49PM +0000, Alexander Weps wrote:
> >
> > > I am not sure which one you mean, all latest codes even includes
> > > headers and main...
> >
> >
> > https://www.openwall.com/lists/musl/2024/03/25/3
> >
> > > I have no idea what to tell you.
> >
> >
> > The first version I found that's actually compilable is:
> >
> > https://www.openwall.com/lists/musl/2024/03/25/11
> >
> > It roughly behaves as expected on musl, except possibly not applying
> > the tm_isdst=0, which is what was making the output confusing on
> > glibc -- that threw the input back across the rule change cutoff.
> >
> > With tm_isdst=1 and tm_mday=31, on glibc, I get:
> >
> > before: 2011-12-31 00:00:00 WSDT 0
> > after1: 2011-12-31 00:00:00 WSDT 1325239200
> > after2: 2011-12-30 00:00:00 WSDT -1
> > after3: 2011-12-31 00:00:00 WSDT 1325239200
> >
> > The -1 in the after2 line indicates that mktime failed with an error
> > (and should not have modified tm; that's arguably a bug in glibc).The
> 
> No, -1 means that was not able to make seconds since beginning of epoch.
> It has nothing to do with modifying tm...

That might actually be the correct interpretation. However, in that
case glibc should not be returning -1, since the time is representable
as time_t..

> Also, can you share the whole code, you did some changes and I don't
> reproduce the result. Which sample it is based on?

https://www.openwall.com/lists/musl/2024/03/25/11

The only change I made was tm_isdst=1 (you already had tm_mday=31
there). I'll attach the file anyway though.

> I set tm_isdst=1 and tm_mday=31 in the example above and I get:
> 
> $  gcc foo.c -o foo && TZ=Pacific/Apia ./foo
> before: 2011-12-31 00:00:00 +13 0
> after1: 2012-01-01 01:00:00 +14 1325329200
> after2: 2011-12-31 01:00:00 +14 1325242800
> after3: 2012-01-01 01:00:00 +14 1325329200

That's close to what I'm seeing with tm_isdst=0:

before: 2011-12-31 00:00:00 WSST 0
after1: 2012-01-01 01:00:00 WSDT 1325329200
after2: 2011-12-31 01:00:00 WSDT 1325242800
after3: 2012-01-01 01:00:00 WSDT 1325329200

With tm_isdst=1 I get:

before: 2011-12-31 00:00:00 WSDT 0
after1: 2011-12-31 00:00:00 WSDT 1325239200
after2: 2011-12-30 00:00:00 WSDT -1
after3: 2011-12-31 00:00:00 WSDT 1325239200

This is on the very old glibc on a test system I had lying around,
2.28. It probably has different bugs (and seems unable to read the
current zoneinfo file which is a newer version; thus the WSDT zone
name from whatever old one it came with).

> I do not get any -1 results.

That's presumably because the bug that threw it into 2012 made it so
there are no invalid dates appearing.

> $  musl-gcc foo.c -o foo && TZ=Pacific/Apia ./foo
> before: 2011-12-31 00:00:00  0
> after1: 2011-12-31 01:00:00 +14 1325242800
> after2: 2011-12-29 01:00:00 -10 1325156400
> after3: 2011-12-29 01:00:00 -10 1325156400

Are you sure you're not testing with tm_isdst=0? This is the output I
get for tm_isdst=0 too.

> 
> > partial modification that it made reflects the initial normalization
> > (type 1 in my notation) but not the rule change normalization (type 2
> > in my notation) since glibc has failed the operation for an input date
> > that does not exist on the calendar (it does not do type 2
> > normalization at all; it just rejects it).
> >
> > Running this same change on musl, I get:
> >
> > before: 2011-12-31 00:00:00 0
> > after1: 2011-12-31 00:00:00 +14 1325239200
> > after2: 2011-12-29 00:00:00 -10 1325152800
> > after3: 2011-12-29 00:00:00 -10 1325152800
> >
> > which again is what I expect. From one side, the move-by-1-day changes
> > the time to the next calendar day in that direction. From the other
> > side, it's unable to change it.
> 
> Glibc works without any issues as shown above on my machine. Both
> forward and backward.

No it doesn't. It jumped to completely the wrong day, 2012-01-01. In
the case where it doesn't do that (tm_isdst=1), it emits a completely
nonexistent date in the reverse direction:

before: 2011-12-31 00:00:00 WSDT 0
after1: 2011-12-31 00:00:00 WSDT 1325239200
after2: 2011-12-30 00:00:00 WSDT -1
after3: 2011-12-31 00:00:00 WSDT 1325239200

This is because it *did* error out before finishing updating the tm
fields. It performed normalization type 1, then said "nope!" and left
the partial work behind.

> Your results are off by one hour. And you start with WSDT (+14).
> Without mktime. The initial date produced by glibc should be +13 as
> Pacific/Apia is +13.

I don't follow. It should never produce +13 in output, as non-DST is
not active anywhere near this date.

Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

[-- Attachment #1: Type: text/plain, Size: 392 bytes --]

On Mon, Mar 25, 2024 at 04:23:29PM -0400, Rich Felker wrote:
> > Also, can you share the whole code, you did some changes and I don't
> > reproduce the result. Which sample it is based on?
> 
> https://www.openwall.com/lists/musl/2024/03/25/11
> 
> The only change I made was tm_isdst=1 (you already had tm_mday=31
> there). I'll attach the file anyway though.

Forgot to attach. Here it is.

[-- Attachment #2: test10.c --]
[-- Type: text/plain, Size: 847 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

void test10()
{
    time_t t = 0;
    struct tm tm = {0};
    char buf[64];

    tm.tm_year = 2011 - 1900;
    tm.tm_mon = 12 - 1;
    tm.tm_mday = 31; // <-- here is the change
    tm.tm_hour = 0;
    tm.tm_min = 0;
    tm.tm_sec = 0;
    tm.tm_isdst = 1;

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("before: %s %ld\n", buf, t);

    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after1: %s %ld\n", buf, t);

    tm.tm_mday -= 1;
    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after2: %s %ld\n", buf, t);

    tm.tm_mday += 1;
    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after3: %s %ld\n", buf, t);
}

int main(int argc, char *argv[])
{
    test10();
    return 0;
}

Re: [musl] Broken mktime calculations when crossing DST boundary

[Thorsten Glaser]

Alexander Weps dixit:

>30th day of the month December was December 31st.

If counting days by counting them from the beginning of the
month, which POSIX is famously not doing even for seconds
since the epoch.

tm_mday is defined as day of month, not days since the
beginning of the month. So 31ˢᵗ December always has tm_mday
of 31, independent of whether 30ᵗʰ December existed in that
particular month.

bye,
//mirabilos
-- 
“It is inappropriate to require that a time represented as
 seconds since the Epoch precisely represent the number of
 seconds between the referenced time and the Epoch.”
	-- IEEE Std 1003.1b-1993 (POSIX) Section B.2.2.2

Re: [musl] Broken mktime calculations when crossing DST boundary

[Thorsten Glaser]

I wrote (in another mail):

> tbh I’d expect this to end up in 1325239199=2011-12-29 23:59:59
> instead of 2011-12-29 01:00:00 though, at least from reading the
> latest Issue 8 proofreading draft. WDYT dalias?

Looking at the spec again, my expectation was wrong, the…

| […] and the broken-down time corresponds to a time that was (or will
| be) skipped over or repeated due to the occurrence of such a change,
| mktime() shall calculate the time since the Epoch value using either
| the offset in effect before the change or the offset in effect after
| the change.

… doesn’t yield this. In fact, further down, we have:

| If a geographical timezone changes its UTC offset such that “old
| 00:00” becomes “new 00:30” and mktime() is given 00:20, it treats that
| as either 20 minutes after “old 00:00” or 10 minutes before “new
| 00:30”, and gives back appropriately altered struct tm fields.

This entirely specifies how this is to be handled… (more below).


Rich Felker dixit:

>Hopefully that will clarify things for you. On musl you will see:
>
>normalized input: 2011-12-29 00:00:00 -10
>+1day per mktime: 2011-12-29 00:00:00 -10
>+1day via time_t: 2011-12-31 00:00:00 +14
>-1day per mktime: 2011-12-28 00:00:00 -10
>-1day via time_t: 2011-12-28 00:00:00 -10
>
>normalized input: 2011-12-31 00:00:00 +14
>+1day per mktime: 2012-01-01 00:00:00 +14
>+1day via time_t: 2012-01-01 00:00:00 +14
>-1day per mktime: 2011-12-29 00:00:00 -10
>-1day via time_t: 2011-12-29 00:00:00 -10
>
>You can see what you get on glibc.

Debian 11 (glibc 2.31) amd64:

$ TZ=Pacific/Apia ./mktime_rel "2011-12-29 00:00:00"
normalized input: 2011-12-29 00:00:00 -10
mktime day+1: Value too large for defined data type
+1day via time_t: 2011-12-31 00:00:00 +14
-1day per mktime: 2011-12-28 00:00:00 -10
-1day via time_t: 2011-12-28 00:00:00 -10
$ TZ=Pacific/Apia ./mktime_rel "2011-12-31 00:00:00"
normalized input: 2011-12-31 00:00:00 +14
+1day per mktime: 2012-01-01 00:00:00 +14
+1day via time_t: 2012-01-01 00:00:00 +14
mktime day-1: Value too large for defined data type
-1day via time_t: 2011-12-29 00:00:00 -10

Same on sid.

Looking at the spec excerpt from above, I think returning EOVERFLOW
is not permitted here, so glibc has something to fix…

MirBSD/i386:

$ TZ=Pacific/Apia ./mktime_rel "2011-12-29 00:00:00"
normalized input: 2011-12-29 00:00:00 -10
+1day per mktime: 2011-12-30 00:00:00 -10   ← how? probably bug in old tzcode
+1day via time_t: 2011-12-31 00:00:00 +14
-1day per mktime: 2011-12-28 00:00:00 -10
-1day via time_t: 2011-12-28 00:00:00 -10
$ TZ=Pacific/Apia ./mktime_rel "2011-12-31 00:00:00"
normalized input: 2011-12-31 00:00:00 +14
+1day per mktime: 2012-01-01 00:00:00 +14
+1day via time_t: 2012-01-01 00:00:00 +14
-1day per mktime: 2011-12-30 00:00:00 +14   ← how? probably bug in old tzcode
-1day via time_t: 2011-12-29 00:00:00 -10

I think I’ll have a bug to fix there ;)
But that code is ancient and doesn’t yet know
about all those new standards…


Looking at the spec excerpt from above again, what we have
is a discontinuity…

$ TZ=Pacific/Apia date -d @1325239199
Thu Dec 29 23:59:59 -10 2011
$ TZ=Pacific/Apia date -d @1325239200
Sat Dec 31 00:00:00 +14 2011

… and an incoming request via struct tm of, let’s say:

2011-12-30 01:00:00

From the excerpt above, mktime can now treat this as either
1 hour after “old 2011-12-30 00:00:00” or 23 hours before
“new 2011-12-31 00:00:00”.

Old 2011-12-30 00:00:00 becomes new 2011-12-31 00:00:00,
so in the first case, the expected result is indeed
2011-12-31 01:00:00 (independent of whether one is going
up or down!), and 23 hours before new 2011-12-31 00:00:00
becomes 23 hours before old 2011-12-30 00:00:00 becomes
2011-12-29 01:00:00. Which is what musl returns, which
means indeed musl DTRT here.

bye,
//mirabilos
-- 
“It is inappropriate to require that a time represented as
 seconds since the Epoch precisely represent the number of
 seconds between the referenced time and the Epoch.”
	-- IEEE Std 1003.1b-1993 (POSIX) Section B.2.2.2

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

>
> What did you do that got glibc to output 2012-01-01? I guess you wrote
> code to do some wacky arithmetic after the original code you already
> had, rather than changing the code to start with 2011-12-31 as I
> suggested to get a look at what's happening.
>

To address this. I would appreciate you not to lie or make some nonsense up.

There was only change related to calculations in the code and it was a change you requested:
    tm.tm_mday = 31;

Complete code to run:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

void test10()
{
    time_t t = 0;
    struct tm tm = {0};
    char buf[64];

    tm.tm_year = 2011 - 1900;
    tm.tm_mon = 12 - 1;
    tm.tm_mday = 31; // <-- here is the change
    tm.tm_hour = 0;
    tm.tm_min = 0;
    tm.tm_sec = 0;
    tm.tm_isdst = 0;

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("before: %s %ld\n", buf, t);

    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after1: %s %ld\n", buf, t);

    tm.tm_mday -= 1;
    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after2: %s %ld\n", buf, t);

    tm.tm_mday += 1;
    t = mktime(&tm);

    strftime(buf, sizeof buf, "%F %T %Z", &tm);
    printf("after3: %s %ld\n", buf, t);
}

int main(int argc, char *argv[])
{
    test10();
    return 0;
}

$  musl-gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-31 00:00:00  0 <-- before first mktime call to verify
after1: 2011-12-31 01:00:00 +14 1325242800 <-- mktime to normalize
after2: 2011-12-29 01:00:00 -10 1325156400 <-- -day & mktime
after3: 2011-12-29 01:00:00 -10 1325156400 <-- +day & mktime

$  gcc foo.c -o foo && TZ=Pacific/Apia ./foo
before: 2011-12-31 00:00:00 +13 0 <-- before first mktime call to verify
after1: 2012-01-01 01:00:00 +14 1325329200 <-- mktime to normalize
after2: 2011-12-31 01:00:00 +14 1325242800 <-- -day & mktime
after3: 2012-01-01 01:00:00 +14 1325329200 <-- +day & mktime

So this is a bug in struct tm interpretation.

AW

On Monday, March 25th, 2024 at 14:24, Alexander Weps <exander77@pm.me> wrote:

> See below.
>
> AW
>
>
>
>
> On Monday, March 25th, 2024 at 14:13, Rich Felker dalias@libc.org wrote:
>
> > On Mon, Mar 25, 2024 at 12:55:28PM +0000, Alexander Weps wrote:
> >
> > > > If you take your test program and switch it to initialize with
> > > > tm_mday=31, then do -=1 instead of +=1, you'll find that it gives
> > > > 2011-12-29 01:00:00 -10 as well, only now it seems like the correct,
> > > > expected thing to happen. Any change to "fix" the case you're
> > > > complaining about would necessarily break this case.
> > >
> > > So (- day, +day):
> > >
> > > Musl:
> > > 2011-12-31 01:00:00 +14
> > > 2011-12-29 01:00:00 -10
> > > 2011-12-29 01:00:00 -10
> > >
> > > Glibc:
> > > 2012-01-01 01:00:00 +14
> > > 2011-12-31 01:00:00 +14
> > > 2012-01-01 01:00:00 +14
> > >
> > > Seems like musl doesn't even interpret the initial struct tm
> > > correctly in that case. It is off by day.
> > >
> > > Because December only had 30 days, 31s day after normalization is
> > > January 1st.
> >
> > This is nonsense. December has a day 31, which you can clearly see
> > from the glibc output. For this particular year in this zone, with the
> > zone rule change, there are "only 30 days" in December, but they are
> > numbered 1-29 and 31, not 1-30.
>
>
> You confuse day of month which is represented in tm_mday with calendar day that is interpreted by strftime.
>
> You said to set tm_mday = 31, which would be January 1st after normalization.
> December 31s is 30th day of month represented as tm_mday = 30.
>
> > What did you do that got glibc to output 2012-01-01? I guess you wrote
> > code to do some wacky arithmetic after the original code you already
> > had, rather than changing the code to start with 2011-12-31 as I
> > suggested to get a look at what's happening.
> >
> > > > In any case, the core issue you're hitting here is that time zones are
> > > > HARD to work with and that there is inherent complexity that libc
> > > > cannot save you from. You only got lucky that what you were trying to
> > > > do "worked" with glibc because you were iterating days forward; if you
> > > > were doing reverse, it would break exactly the same way.
> > >
> > > I am not really commenting on this, until you sort out the above
> > > inconsistencies.
> >
> > I already have but you refuse to look.
>
>
> It was addressed, do didn't scroll at the end of the e-mail.
>
> > Rich

Re: [musl] Broken mktime calculations when crossing DST boundary

[Thorsten Glaser]

Alexander Weps dixit:
>On Monday, March 25th, 2024 at 13:21, Rich Felker <dalias@libc.org> wrote:
>> On Mon, Mar 25, 2024 at 11:52:00AM +0000, Alexander Weps wrote:

>> > Musl cannot reliably increment date by a day. Incrementing struct tm
>> > representing 2011-12-29 01:00:00 -10 by one day leads to the same
>> > date.

No, that’s correct.

Your chosen timezone has a discontinuity:

$ TZ=Pacific/Apia date -d @1325239199
Thu Dec 29 23:59:59 -10 2011
$ TZ=Pacific/Apia date -d @1325239200
Sat Dec 31 00:00:00 +14 2011

This means any time between this simply does not exist
in broken-down time.

>> > Causing a program to loop or stack overflow.

That’s because your application violates the constraints
that bind both, not just the libc, to the spec.

>Output from musl:
>
>2011-12-29 01:00:00 -10
>
>    tm.tm_mday += 1;
>    t = mktime(&tm);
>
>2011-12-29 01:00:00 -10 <-- date is the same after incrementing

This is… not as incorrect as you state.

The steps here are:

• 2011-12-30 01:00:00 ← input
• 2011-12-30 01:00:00 ← input after normalisation (!)
• conversion to time_t (1325206800), application of timezone offset
• detection of the discontinuity between 2011-12-29 23:59:59 and
      2011-12-31 00:00:00
• arbitrary choice of selecting either endpoint

tbh I’d expect this to end up in 1325239199=2011-12-29 23:59:59
instead of 2011-12-29 01:00:00 though, at least from reading the
latest Issue 8 proofreading draft. WDYT dalias?

>    tm.tm_mday -= 1;
>    t = mktime(&tm);
>
>2011-12-28 01:00:00 -10 <-- going below the original date while decrementing

This is entirely correct, again. You’re starting from
2011-12-28 01:00:00, you’re getting it back.

bye,
//mirabilos
-- 
“It is inappropriate to require that a time represented as
 seconds since the Epoch precisely represent the number of
 seconds between the referenced time and the Epoch.”
	-- IEEE Std 1003.1b-1993 (POSIX) Section B.2.2.2

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

See below.

AW


On Monday, March 25th, 2024 at 23:40, Thorsten Glaser <tg@mirbsd.de> wrote:

> Alexander Weps dixit:
>
> > On Monday, March 25th, 2024 at 13:21, Rich Felker dalias@libc.org wrote:
> >
> > > On Mon, Mar 25, 2024 at 11:52:00AM +0000, Alexander Weps wrote:
>
> > > > Musl cannot reliably increment date by a day. Incrementing struct tm
> > > > representing 2011-12-29 01:00:00 -10 by one day leads to the same
> > > > date.
>
>
> No, that’s correct.
>
> Your chosen timezone has a discontinuity:
>
> $ TZ=Pacific/Apia date -d @1325239199
> Thu Dec 29 23:59:59 -10 2011
> $ TZ=Pacific/Apia date -d @1325239200
> Sat Dec 31 00:00:00 +14 2011
>
> This means any time between this simply does not exist
> in broken-down time.

I am not calculating any time between.

I am using struct tm and mktime to calculated a date with specific offset from initial date.
When I add 1 to tm_sec it means I am calculating a date that is one second after the initial date.

And all of this works in glibc.

Is the argument that glibc being able to do basic struct tm calculations is an incorrect behavior?

>
> > > > Causing a program to loop or stack overflow.
>
>
> That’s because your application violates the constraints
> that bind both, not just the libc, to the spec.

Specify those constraints.

>
> > Output from musl:
> >
> > 2011-12-29 01:00:00 -10
> >
> > tm.tm_mday += 1;
> > t = mktime(&tm);
> >
> > 2011-12-29 01:00:00 -10 <-- date is the same after incrementing
>
>
> This is… not as incorrect as you state.
>
> The steps here are:
>
> • 2011-12-30 01:00:00 ← input
> • 2011-12-30 01:00:00 ← input after normalisation (!)
> • conversion to time_t (1325206800), application of timezone offset
> • detection of the discontinuity between 2011-12-29 23:59:59 and
> 2011-12-31 00:00:00
> • arbitrary choice of selecting either endpoint
>
> tbh I’d expect this to end up in 1325239199=2011-12-29 23:59:59
> instead of 2011-12-29 01:00:00 though, at least from reading the
> latest Issue 8 proofreading draft. WDYT dalias?
>
> > tm.tm_mday -= 1;
> > t = mktime(&tm);
> >
> > 2011-12-28 01:00:00 -10 <-- going below the original date while decrementing
>
>
> This is entirely correct, again. You’re starting from
> 2011-12-28 01:00:00, you’re getting it back.

Show me a function implementation that produces same time next day under this behavior you assume to be correct.

>
> bye,
> //mirabilos
> --
> “It is inappropriate to require that a time represented as
> seconds since the Epoch precisely represent the number of
> seconds between the referenced time and the Epoch.”
> -- IEEE Std 1003.1b-1993 (POSIX) Section B.2.2.2

Re: [musl] Broken mktime calculations when crossing DST boundary

[Thorsten Glaser]

Alexander Weps dixit:

>And all of this works in glibc.

Not at all, glibc’s mktime just throws the towel with
EOVERFLOW saying that the requested time does not exist.

(It is only not permitted to change the input struct tm
member tm_wday if it returns EOVERFLOW but that doesn’t
mean that the output struct tm is correct.)

>> That’s because your application violates the constraints
>> that bind both, not just the libc, to the spec.
>
>Specify those constraints.

My wording here was badly, as dalias mentioned:

>It's not a constraint violation. mktime is required to accept inputs
>where the fields of the broken-down time don't fall within their
>respective ranges, and normalize them.
>
>Rather, the calling code is just making incorrect assumptions about
>the unspecified way that happens, […]

The “constraint” (wrong word) I meant was that your program
must not make assumptions about which of the two possible
conflict resolutions the libc chooses.

>Show me a function implementation that produces same time next day
>under this behavior you assume to be correct.

It is not possible to do that with mktime. You’ll have to do
that yourself. POSIX even says so.

It does indicate that on implementations (their word for libcs
here) that follow its recommendation to not normalise tm_sec,
you can achieve the desired effect by adding 86400 to it, though
that will not work right in the presence of a leap second on
systems honouring them (which is a deviation from POSIX, of
course).

Adding 86400 to the time_t value, under the same leap second
caveat, can work if your code can rely on POSIX (ISO C does not
specify the internal structure of time_t).

bye,
//mirabilos
-- 
22:20⎜<asarch> The crazy that persists in his craziness becomes a master
22:21⎜<asarch> And the distance between the craziness and geniality is
only measured by the success 18:35⎜<asarch> "Psychotics are consistently
inconsistent. The essence of sanity is to be inconsistently inconsistent

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

See below.

AW


On Tuesday, March 26th, 2024 at 00:34, Thorsten Glaser <tg@mirbsd.de> wrote:

> Alexander Weps dixit:
>
> > And all of this works in glibc.
>
>
> Not at all, glibc’s mktime just throws the towel with
> EOVERFLOW saying that the requested time does not exist.

How is this not compliant with POSIX?

This is perfectly valid behavior, that is both expected and can be handled in code easily.

I have to ask, but have you actually used mktime from the application end?

> (It is only not permitted to change the input struct tm
> member tm_wday if it returns EOVERFLOW but that doesn’t
> mean that the output struct tm is correct.)
>

I am not sure what you mean by correct. Struct tm is neither correct nor incorrect. It can be in three states:
1) Set by user.
2) Normalized by mktime.
3) Not fully normalized by mktime.

If I get -1, I know the struct tm does not represent valid time_t and I handle it and move on.

This is perfect example (TZ=Pacific/Apia):

before: 2011-12-31 00:00:00 +14 0
after1: 2011-12-31 00:00:00 +14 1325239200
after1: 2011-12-30 00:00:00 +14 -1
after1: 2011-12-29 00:00:00 -10 1325152800
after2: 2011-12-28 00:00:00 -10 1325066400
after2: 2011-12-29 00:00:00 -10 1325152800
after2: 2011-12-30 00:00:00 -10 -1
after3: 2011-12-31 00:00:00 +14 1325239200

I am enumerating midnighta of each day. If I get -1, I ignore that date and move along.

Musl instead of giving sane results starts running in the circle at some point:
after2: 2011-12-29 00:00:00 -10 1325152800
after3: 2011-12-29 00:00:00 -10 1325152800

> > > That’s because your application violates the constraints
> > > that bind both, not just the libc, to the spec.
> >
> > Specify those constraints.
>
>
> My wording here was badly, as dalias mentioned:
>
> > It's not a constraint violation. mktime is required to accept inputs
> > where the fields of the broken-down time don't fall within their
> > respective ranges, and normalize them.
> >
> > Rather, the calling code is just making incorrect assumptions about
> > the unspecified way that happens, […]
>
>
> The “constraint” (wrong word) I meant was that your program
> must not make assumptions about which of the two possible
> conflict resolutions the libc chooses.
>
> > Show me a function implementation that produces same time next day
> > under this behavior you assume to be correct.
>
>
> It is not possible to do that with mktime. You’ll have to do
> that yourself. POSIX even says so.
>
> It does indicate that on implementations (their word for libcs
> here) that follow its recommendation to not normalise tm_sec,
> you can achieve the desired effect by adding 86400 to it, though
> that will not work right in the presence of a leap second on
> systems honouring them (which is a deviation from POSIX, of
> course).
>
> Adding 86400 to the time_t value, under the same leap second
> caveat, can work if your code can rely on POSIX (ISO C does not
> specify the internal structure of time_t).
>
> bye,
> //mirabilos
> --
> 22:20⎜<asarch> The crazy that persists in his craziness becomes a master
>
> 22:21⎜<asarch> And the distance between the craziness and geniality is
>
> only measured by the success 18:35⎜<asarch> "Psychotics are consistently
>
> inconsistent. The essence of sanity is to be inconsistently inconsistent

Re: [musl] Broken mktime calculations when crossing DST boundary

[Thorsten Glaser]

Alexander Weps dixit:

>> Not at all, glibc’s mktime just throws the towel with
>> EOVERFLOW saying that the requested time does not exist.
>
>How is this not compliant with POSIX?

POSIX indicates that this is valid only if the date is not
representable in time_t, and it has different handling for
dates that fall into gaps, see the other mails from me, as
well as below.

>This is perfectly valid behavior, that is both expected and can be
>handled in code easily.

No, it’s a bug in glibc.

>I have to ask, but have you actually used mktime from the application end?

Of course.

>I am not sure what you mean by correct. Struct tm is neither correct
>nor incorrect. It can be in three states:
>1) Set by user.
>2) Normalized by mktime.
>3) Not fully normalized by mktime.

Huh? No.

>If I get -1, I know the struct tm does not represent valid time_t and I
>handle it and move on.

Define “move on”. With POSIX’ mktime interface, if you get -1 and
EOVERFLOW, then moving further into the same direction will never
give you not -1 again, because -1 is what you get when your tm_year
was too far out of the representation (e.g. 2039 on a system with
a 32-bit time_t).

EOVERFLOW means that the time cannot be represented in time_t, not
that the time cannot be represented in struct tm. And for these
gaps, the time_t values are consecutive (1325239199/1325239200).

>This is perfect example (TZ=Pacific/Apia):
>
>before: 2011-12-31 00:00:00 +14 0
>after1: 2011-12-31 00:00:00 +14 1325239200
>after1: 2011-12-30 00:00:00 +14 -1

No, this cannot give -1 per POSIX.

>Musl instead of giving sane results starts running in the circle at some point:
>after2: 2011-12-29 00:00:00 -10 1325152800
>after3: 2011-12-29 00:00:00 -10 1325152800

That’s because it does this correctly.

>Doesn't work, this will not give the same time next day, this fails on
>STD/DST changes.
>
>Because same time next day is not always 86400 apart.

I know. But the basic assumption that there even is such a
thing as “same time next day” made by you is invalid. POSIX
listed several examples (29ᵗʰ February next year as well as
gaps in timezone offsets).

One thing you can do is to add 86400, localtime(), then check
that at least tm_mday, tm_hour and tm_min (Issue 8d4, line
48052) are what you expect, and handle cases where they aren’t
manually. But having added 86400, you have two starting points
from which to manually approach this (the original value and
the newer value). (Perhaps a location could even skip more than
24 hours in a discontinuity.)

bye,
//mirabilos
-- 
  “Having a smoking section in a restaurant is like having
          a peeing section in a swimming pool.”
						-- Edward Burr

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

See below.

AW


On Tuesday, March 26th, 2024 at 22:59, Thorsten Glaser <tg@mirbsd.de> wrote:

> Alexander Weps dixit:
>
> > > Not at all, glibc’s mktime just throws the towel with
> > > EOVERFLOW saying that the requested time does not exist.
> >
> > How is this not compliant with POSIX?
>
>
> POSIX indicates that this is valid only if the date is not
> representable in time_t, and it has different handling for
> dates that fall into gaps, see the other mails from me, as
> well as below.
>

I think you confuse two things here:

1. mktime returning -1
2. mktime setting errno to EOVERFLOW

The 2. maybe debatable in obscure circles, but that's entirely unrelated.

But mktime can return -1 if and when it deems that struct tm cannot be represented in epoch seconds.

That is entirely compliant with POSIX and I would need to see some hard evidence for it not being the case.

From Issue 7:

> RETURN VALUE
>
>     The mktime() function shall return the specified time since the Epoch encoded as a value of type time_t. If the time since the Epoch cannot be represented, the function shall return the value (time_t)-1 [CX] [Option Start]  and set errno to indicate the error. [Option End]
>
> ERRORS
>
>     The mktime() function shall fail if:
>
>     [EOVERFLOW]
>         [CX] [Option Start] The result cannot be represented. [Option End]


First and foremost mktime may fail and when it fails it returns -1.

Issue 6: If the time since the Epoch cannot be represented, the function shall return the value (time_t)-1 and may set errno to indicate the error.
Isuse 7: If the time since the Epoch cannot be represented, the function shall return the value (time_t)-1 and set errno to indicate the error.

Only errno offered is EOVERFLOW.
So I interpret it that since Issue 7, the correct behavior is to set errno to EOVERFLOW.

Also from:
Minutes of the 27th July 2023 Teleconference    Austin-1332 Page 1 of 1
Submitted by Andrew Josey, The Open Group.         28th July 2023

> Bug 1614: XSH 3/mktime does not specify EINVAL and should  Accepted as Marked
> https://austingroupbugs.net/view.php?id=1614
>
> An interpretation is required.
>
> Interpretation response:
> The standard clearly states that when an unsuccessful call to
> mktime() returns (time_t)-1 it sets errno to [EOVERFLOW], and
> conforming implementations must conform to this.
>
> Rationale:
>
> The RETURN VALUE section on the mktime() page states:
>     If the time since the Epoch cannot be represented, the function
>     shall return the value (time_t)-1 [CX]and set errno to indicate
>     the error[/CX].
>
> This requires that errno is set to indicate "the error", and the
> beginning of the sentence states the nature of the error condition
> to which "the error" refers: the time since the Epoch (i.e. the
> integer value to be returned) cannot be represented. The ERRORS
> section requires that the error number [EOVERFLOW] is used for this
> condition.
>
> Thus the standard requires that errno is set to [EOVERFLOW] when
> an unsuccessful call to mktime() returns (time_t)-1 and an
> implementation that sets it to [EINVAL] does not conform.
>
> The mktime() function does not have any way to indicate to the
> caller that an error other than [EOVERFLOW] occurred.

> > This is perfectly valid behavior, that is both expected and can be
> > handled in code easily.
>
>
> No, it’s a bug in glibc.
>
> > I have to ask, but have you actually used mktime from the application end?
>
>
> Of course.
>
> > I am not sure what you mean by correct. Struct tm is neither correct
> > nor incorrect. It can be in three states:
> > 1) Set by user.
> > 2) Normalized by mktime.
> > 3) Not fully normalized by mktime.
>
>
> Huh? No.

Then explain what is incorrect struct tm.

>
> > If I get -1, I know the struct tm does not represent valid time_t and I
> > handle it and move on.
>
>
> Define “move on”. With POSIX’ mktime interface, if you get -1 and
> EOVERFLOW, then moving further into the same direction will never
> give you not -1 again, because -1 is what you get when your tm_year
> was too far out of the representation (e.g. 2039 on a system with
> a 32-bit time_t).

Not true as shown above.

>
> EOVERFLOW means that the time cannot be represented in time_t, not
> that the time cannot be represented in struct tm. And for these
> gaps, the time_t values are consecutive (1325239199/1325239200).

No, return value -1 means that the time cannot represented as epoch seconds (for any number of reasons).

Required errno is Issue 7 change that should be used to indicate type of error, but only EOVERFLOW is listed.

>
> > This is perfect example (TZ=Pacific/Apia):
> >
> > before: 2011-12-31 00:00:00 +14 0
> > after1: 2011-12-31 00:00:00 +14 1325239200
> > after1: 2011-12-30 00:00:00 +14 -1
>
>
> No, this cannot give -1 per POSIX.

Not true as shown above.

>
> > Musl instead of giving sane results starts running in the circle at some point:
> > after2: 2011-12-29 00:00:00 -10 1325152800
> > after3: 2011-12-29 00:00:00 -10 1325152800
>
>
> That’s because it does this correctly.
>
> > Doesn't work, this will not give the same time next day, this fails on
> > STD/DST changes.
> >
> > Because same time next day is not always 86400 apart.
>
>
> I know. But the basic assumption that there even is such a
> thing as “same time next day” made by you is invalid. POSIX
> listed several examples (29ᵗʰ February next year as well as
> gaps in timezone offsets).
>
> One thing you can do is to add 86400, localtime(), then check
> that at least tm_mday, tm_hour and tm_min (Issue 8d4, line
> 48052) are what you expect, and handle cases where they aren’t
> manually. But having added 86400, you have two starting points
> from which to manually approach this (the original value and
> the newer value). (Perhaps a location could even skip more than
> 24 hours in a discontinuity.)
>
> bye,
> //mirabilos
> --
> “Having a smoking section in a restaurant is like having
> a peeing section in a swimming pool.”
> -- Edward Burr

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

[-- Attachment #1: Type: text/plain, Size: 6836 bytes --]

Attaching whole:
Minutes of the 27th July 2023 Teleconference    Austin-1332 Page 1 of 1
Submitted by Andrew Josey, The Open Group.         28th July 2023

Also go through:
https://www.austingroupbugs.net/view.php?id=1614

Seems most of what We talk here was already addressed.

AW


On Wednesday, March 27th, 2024 at 01:14, Alexander Weps <exander77@pm.me> wrote:

> See below.
>
> AW
>
>
>
>
> On Tuesday, March 26th, 2024 at 22:59, Thorsten Glaser tg@mirbsd.de wrote:
>
> > Alexander Weps dixit:
> >
> > > > Not at all, glibc’s mktime just throws the towel with
> > > > EOVERFLOW saying that the requested time does not exist.
> > >
> > > How is this not compliant with POSIX?
> >
> > POSIX indicates that this is valid only if the date is not
> > representable in time_t, and it has different handling for
> > dates that fall into gaps, see the other mails from me, as
> > well as below.
>
>
> I think you confuse two things here:
>
> 1. mktime returning -1
> 2. mktime setting errno to EOVERFLOW
>
> The 2. maybe debatable in obscure circles, but that's entirely unrelated.
>
> But mktime can return -1 if and when it deems that struct tm cannot be represented in epoch seconds.
>
> That is entirely compliant with POSIX and I would need to see some hard evidence for it not being the case.
>
> From Issue 7:
>
> > RETURN VALUE
> >
> > The mktime() function shall return the specified time since the Epoch encoded as a value of type time_t. If the time since the Epoch cannot be represented, the function shall return the value (time_t)-1 [CX] [Option Start] and set errno to indicate the error. [Option End]
> >
> > ERRORS
> >
> > The mktime() function shall fail if:
> >
> > [EOVERFLOW]
> > [CX] [Option Start] The result cannot be represented. [Option End]
>
>
>
> First and foremost mktime may fail and when it fails it returns -1.
>
> Issue 6: If the time since the Epoch cannot be represented, the function shall return the value (time_t)-1 and may set errno to indicate the error.
> Isuse 7: If the time since the Epoch cannot be represented, the function shall return the value (time_t)-1 and set errno to indicate the error.
>
> Only errno offered is EOVERFLOW.
> So I interpret it that since Issue 7, the correct behavior is to set errno to EOVERFLOW.
>
> Also from:
> Minutes of the 27th July 2023 Teleconference Austin-1332 Page 1 of 1
> Submitted by Andrew Josey, The Open Group. 28th July 2023
>
> > Bug 1614: XSH 3/mktime does not specify EINVAL and should Accepted as Marked
> > https://austingroupbugs.net/view.php?id=1614
> >
> > An interpretation is required.
> >
> > Interpretation response:
> > The standard clearly states that when an unsuccessful call to
> > mktime() returns (time_t)-1 it sets errno to [EOVERFLOW], and
> > conforming implementations must conform to this.
> >
> > Rationale:
> >
> > The RETURN VALUE section on the mktime() page states:
> > If the time since the Epoch cannot be represented, the function
> > shall return the value (time_t)-1 [CX]and set errno to indicate
> > the error[/CX].
> >
> > This requires that errno is set to indicate "the error", and the
> > beginning of the sentence states the nature of the error condition
> > to which "the error" refers: the time since the Epoch (i.e. the
> > integer value to be returned) cannot be represented. The ERRORS
> > section requires that the error number [EOVERFLOW] is used for this
> > condition.
> >
> > Thus the standard requires that errno is set to [EOVERFLOW] when
> > an unsuccessful call to mktime() returns (time_t)-1 and an
> > implementation that sets it to [EINVAL] does not conform.
> >
> > The mktime() function does not have any way to indicate to the
> > caller that an error other than [EOVERFLOW] occurred.
>
> > > This is perfectly valid behavior, that is both expected and can be
> > > handled in code easily.
> >
> > No, it’s a bug in glibc.
> >
> > > I have to ask, but have you actually used mktime from the application end?
> >
> > Of course.
> >
> > > I am not sure what you mean by correct. Struct tm is neither correct
> > > nor incorrect. It can be in three states:
> > > 1) Set by user.
> > > 2) Normalized by mktime.
> > > 3) Not fully normalized by mktime.
> >
> > Huh? No.
>
>
> Then explain what is incorrect struct tm.
>
> > > If I get -1, I know the struct tm does not represent valid time_t and I
> > > handle it and move on.
> >
> > Define “move on”. With POSIX’ mktime interface, if you get -1 and
> > EOVERFLOW, then moving further into the same direction will never
> > give you not -1 again, because -1 is what you get when your tm_year
> > was too far out of the representation (e.g. 2039 on a system with
> > a 32-bit time_t).
>
>
> Not true as shown above.
>
> > EOVERFLOW means that the time cannot be represented in time_t, not
> > that the time cannot be represented in struct tm. And for these
> > gaps, the time_t values are consecutive (1325239199/1325239200).
>
>
> No, return value -1 means that the time cannot represented as epoch seconds (for any number of reasons).
>
> Required errno is Issue 7 change that should be used to indicate type of error, but only EOVERFLOW is listed.
>
> > > This is perfect example (TZ=Pacific/Apia):
> > >
> > > before: 2011-12-31 00:00:00 +14 0
> > > after1: 2011-12-31 00:00:00 +14 1325239200
> > > after1: 2011-12-30 00:00:00 +14 -1
> >
> > No, this cannot give -1 per POSIX.
>
>
> Not true as shown above.
>
> > > Musl instead of giving sane results starts running in the circle at some point:
> > > after2: 2011-12-29 00:00:00 -10 1325152800
> > > after3: 2011-12-29 00:00:00 -10 1325152800
> >
> > That’s because it does this correctly.
> >
> > > Doesn't work, this will not give the same time next day, this fails on
> > > STD/DST changes.
> > >
> > > Because same time next day is not always 86400 apart.
> >
> > I know. But the basic assumption that there even is such a
> > thing as “same time next day” made by you is invalid. POSIX
> > listed several examples (29ᵗʰ February next year as well as
> > gaps in timezone offsets).
> >
> > One thing you can do is to add 86400, localtime(), then check
> > that at least tm_mday, tm_hour and tm_min (Issue 8d4, line
> > 48052) are what you expect, and handle cases where they aren’t
> > manually. But having added 86400, you have two starting points
> > from which to manually approach this (the original value and
> > the newer value). (Perhaps a location could even skip more than
> > 24 hours in a discontinuity.)
> >
> > bye,
> > //mirabilos
> > --
> > “Having a smoking section in a restaurant is like having
> > a peeing section in a swimming pool.”
> > -- Edward Burr

[-- Attachment #2: austin_1332.txt --]
[-- Type: text/plain, Size: 12402 bytes --]

Minutes of the 27th July 2023 Teleconference    Austin-1332 Page 1 of 1
Submitted by Andrew Josey, The Open Group.         28th July 2023

Attendees:
    Andrew Josey, The Open Group 
    Don Cragun,  IEEE PASC OR
    Mark Ziegast, SHware Systems Dev.
    Geoff Clare, The Open Group
    Eric Ackermann, HPI, University of Potsdam
    Eric Blake, Red Hat, The Open Group OR

Apologies
    Nick Stoughton, Logitech/USENIX, ISO/IEC JTC 1/SC 22 OR 
    Tom Thompson, IEEE

* General news

The meeting on July 31 will be a webex.
The meeting logistics for August 3rd are undecided as to whether it
is a zoom or webex meeting. We will check with Nick on Monday.

Andrew noted that draft planning is still to be done. 
It is expected that the cutoff for the next draft will be determined
by the closing date for open interpretations.

* Current Business

Note for issue resolution all items are tagged for Issue 8 unless
noted otherwise or disposition is reject or duplicate.


Bug 1766: catgets: quotation in "Change History" lacks closing quotes Accepted
https://austingroupbugs.net/view.php?id=1766

This item is tagged for TC3-2008.

Bug 1769: CX shading needed for fputwc() EILSEQ error indicator requirement Accepted
https://austingroupbugs.net/view.php?id=1769


Bug 1614: XSH 3/mktime does not specify EINVAL and should  Accepted as Marked
https://austingroupbugs.net/view.php?id=1614

An interpretation is required.

Interpretation response:
The standard clearly states that when an unsuccessful call to
mktime() returns (time_t)-1 it sets errno to [EOVERFLOW], and
conforming implementations must conform to this.

Rationale:

The RETURN VALUE section on the mktime() page states:
    If the time since the Epoch cannot be represented, the function
    shall return the value (time_t)-1 [CX]and set errno to indicate
    the error[/CX].

This requires that errno is set to indicate "the error", and the
beginning of the sentence states the nature of the error condition
to which "the error" refers: the time since the Epoch (i.e. the
integer value to be returned) cannot be represented. The ERRORS
section requires that the error number [EOVERFLOW] is used for this
condition.

Thus the standard requires that errno is set to [EOVERFLOW] when
an unsuccessful call to mktime() returns (time_t)-1 and an
implementation that sets it to [EINVAL] does not conform.

The mktime() function does not have any way to indicate to the
caller that an error other than [EOVERFLOW] occurred.

Notes to the Editor (not part of this interpretation):
On page 425 line 14451 section <time.h>, after applying bug 1253 change:
    The value of tm_isdst shall be positive if Daylight Saving Time
    is in effect, 0 if Daylight Saving Time is not in effect, and
    negative if the information is not available.
to:
    When tm_isdst is set by an interface defined in this standard,
    its value shall be positive if Daylight Saving Time (DST) is
    in effect and 0 if DST is not in effect. [CX]It shall not be
    set to a negative value by any interface defined in this standard.
    When tm_isdst is passed to the mktime() function, it specifies
    how mktime() is to handle DST when calculating the time since
    the Epoch value; see [xref to mktime()].[/CX]

On page 1331 line 44310 section mktime(), delete:
    A positive or 0 value for tm_isdst shall cause mktime() to
    presume initially that Daylight Savings Time, respectively, is
    or is not in effect for the specified time. A negative value
    for tm_isdst shall cause mktime() to attempt to determine whether
    Daylight Savings Time is in effect for the specified time.

On page 1331 line 44317 section mktime(), change:
    corrected for timezone and any seasonal time adjustments
to:
    corrected for the offset of the timezone's standard time from
    Coordinated Universal Time and further corrected (if applicable--see
    below) for Daylight Saving Time

After page 1331 line 44321 section mktime(), add these new paragraphs:
    [CX]If the timezone is one that includes Daylight Saving Time
    (DST) adjustments, the value of tm_isdst in the tm structure
    controls whether or not mktime() adjusts the calculated seconds
    since the Epoch value by the DST offset (after it has made the
    timezone adjustment), as follows:

	If tm_isdst is zero, mktime() shall not further adjust the
	seconds since the Epoch by the DST offset.

	If tm_isdst is positive, mktime() shall further adjust the
	seconds since the Epoch by the DST offset.

	If tm_isdst is negative, mktime() shall attempt to determine
	whether DST is in effect for the specified time; if it
	determines that DST is in effect it shall produce the same
	result as an equivalent call with a positive tm_isdst value,
	otherwise it shall produce the same result as an equivalent
	call with a tm_isdst value of zero. If the broken-down time
	specifies a time that is either skipped over or repeated
	when a transition to or from DST occurs, it is unspecified
	whether mktime() produces the same result as an equivalent
	call with a positive tm_isdst value or as an equivalent
	call with a tm_isdst value of zero.

    If the TZ environment variable specifies a geographical timezone
    for which the implementation's timezone database includes
    historical or future changes to the offset from Coordinated
    Universal Time of the timezone's standard time, and the broken-down
    time corresponds to a time that was (or will be) skipped over
    or repeated due to the occurrence of such a change, mktime()
    shall calculate the time since the Epoch value using either the
    offset in effect before the change or the offset in effect after
    the change.[/CX]

On page 1331 line 44323 section mktime(), after applying bug 1613 change:
    with the specified time since the Epoch as its argument
to:
    with the calculated time since the Epoch as its argument

On page 1331 line 44327 section mktime(), change:
    The mktime() function shall return the specified time since the
    Epoch encoded as a value of type time_t. If the time since the
    Epoch cannot be represented, the function shall return the value
    (time_t)-1 [CX]and set errno to indicate the error[/CX].
to:
    The mktime() function shall return the calculated time since
    the Epoch encoded as a value of type time_t. If the time since
    the Epoch cannot be represented as a time_t [CX]or the value
    to be returned in the tm_year member of the structure pointed
    to by timeptr cannot be represented as an int[/CX], the function
    shall return the value (time_t)-1 [CX]and set errno to [EOVERFLOW],
    and shall not change the value of the tm_wday component of the
    structure.[/CX]

    [CX]Since (time_t)-1 is a valid return value for a successful
    call to mktime(), an application wishing to check for error
    situations should set tm_wday to a value less than 0 or greater
    than 6 before calling mktime(). On return, if tm_wday has not
    changed an error has occurred.[/CX]

On page 1332 line 44348 section mktime(), change:
    if (mktime(&time_str) == -1)
to:
    time_str.tm_wday = -1;
    if (mktime(&time_str) == (time_t)-1 && time_str.tm_wday == -1)

On page 1332 line 44359 section mktime(), change RATIONALE from "None" to:
    In order to allow applications to distinguish between a successful
    return of (time_t)-1 and an [EOVERFLOW] error, mktime() is
    required not to change tm_wday on error. This mechanism is used
    rather than the convention used for other functions whereby the
    application sets errno to zero before the call and the call
    does not change errno on error because the ISO C standard does
    not require mktime() to set errno on error. The next revision
    of the ISO C standard is expected to require that mktime() does
    not change tm_wday when returning (time_t)-1 to indicate an
    error, and that this return convention is used both for the
    case where the value to be returned by the function cannot be
    represented as a time_t and the case where the value to be
    returned in the tm_year member of the tm structure cannot be
    represented as an int.

    The DESCRIPTION section says that mktime() converts the specified
    broken-down time into a time since the Epoch value. The use of
    the indefinite article here is necessary because, when tm_isdst
    is negative and the timezone has Daylight Saving Time transitions,
    there is not a one-to-one correspondence between broken-down
    times and time since the Epoch values.

    The description of how the value of tm_isdst affects the behavior
    of mktime() is shaded CX because the requirements in the ISO C
    standard are unclear. The next revision of the ISO C standard
    is expected to state the requirements using wording equivalent
    to the wording in this standard.

Bug 1647: printf("%lc", (wint_t)0) can't output NUL byte Accepted as Marked
https://austingroupbugs.net/view.php?id=1647

Now the C committee has responded to our feedback, we can approve
the proposed interpretation. (closed after meeting)


Bug 1768: catgets: incomplete paragraph in "Change History" Accepted as Marked
https://austingroupbugs.net/view.php?id=1768

This item is tagged for TC3-2008

The text of Austin Group Interpretation 1003.1-2001 #148 can be found here:

https://collaboration.opengroup.org/austin/interps/documents/14539/AI-148.txt [^]

It doesn't contain any changes to catgets().

We considered whether there might be a typo in the 148 number, but we
have been unable to find any other interpretation (besides #044
that is already listed) affecting catgets(), and as far as it can
be seen, all of the changes to catgets() between Issue 6 and Issue 7
are described in the other Change History entries.

The other possibility is that this entry should have been on a
different page, but all of the functions affected by #148 have a
Change History entry for it.

All we can do at this point is treat this line as spurious and delete it.


Bug 1770: Typo re iswblank_l Accepted
https://austingroupbugs.net/view.php?id=1770 

This item is tagged for TC3-2008

Bug 1767: sed: clarify behavior of c (change) command Accepted
https://austingroupbugs.net/view.php?id=1767

Bug 1747: mailx: document alias expansion prevention Accepted
https://austingroupbugs.net/view.php?id=1747

Bug 1746: fuser output format clarification
https://austingroupbugs.net/view.php?id=1746

This item is tagged for TC3-2008

An interpretation is required.

Interpretation response:
The standard states that the process ID is written using the format
"%d", and conforming implementations must conform to this. However,
concerns have been raised about this which are being referred to
the sponsor.

Rationale:
Format "%d" allows, but does not require a space or tab before the
process ID. The standard should require separation between process
IDs in order for the output to be usable.

Notes to the Editor (not part of this interpretation):

Make the changes in bugnote:6341

Bug 1745: tsort input and output format clarifications Accepted as Marked
https://austingroupbugs.net/view.php?id=1745

This item is tagged for TC3-2008.
An interpretation is required.

Interpretation response:
The standard is unclear on this issue, and no conformance distinction
can be made between alternative implementations based on this. This
is being referred to the sponsor.

Rationale:
None.

Notes to the Editor (not part of this interpretation):
Make the changes in the Desired Action.

We will continue next time on:

Bug 1732: cp and mv EXIT STATUS does not account for -i
https://austingroupbugs.net/view.php?id=1732

Next Steps
----------
The next call is on:

   Mon 2023-07-31 (Webex meeting - general bugs/ballot resolution)
   Thu 2023-08-03 (TBD Zoom or Webex meeting - general bugs/ballot resolution)

The calls are for 90 minutes

Calls are anchored on US time. (8am Pacific)

Please check the calendar invites for dial in details.

Apologies in advance:
    Tom Thompson 2023-07-31
    Andrew Josey 2023-08-07, 2023-08-10


Bugs are at:
https://austingroupbugs.net

An etherpad is usually up for the meeting, with a URL using the date format as below:

https://posix.rhansen.org/p/20xx-mm-dd

(For write access this uses The Open Group single sign on,
for those individuals with gitlab.opengroup.org accounts.
Please contact Andrew if you need to be setup)

Re: [musl] Broken mktime calculations when crossing DST boundary

[Thorsten Glaser]

Alexander Weps dixit:

>But mktime can return -1 if and when it deems that struct tm cannot be
>represented in epoch seconds.

But it CAN be represented in epoch seconds.

It can only not be represented in epoch seconds if it’s
outside of the range of time_t, e.g. [LONG_MIN; LONG_MAX].

POSIX even explicitly documents how these discontinuities
are mapped, which I already quoted:

| If a geographical timezone changes its UTC offset such that “old
| 00:00” becomes “new 00:30” and mktime() is given 00:20, it treats that
| as either 20 minutes after “old 00:00” or 10 minutes before “new
| 00:30”, and gives back appropriately altered struct tm fields.

This makes it clear that there are two ways the struct tm can
be represented in time_t, and that the implementation must
choose one of them.

bye,
//mirabilos
-- 
[...] if maybe ext3fs wasn't a better pick, or jfs, or maybe reiserfs, oh but
what about xfs, and if only i had waited until reiser4 was ready... in the be-
ginning, there was ffs, and in the middle, there was ffs, and at the end, there
was still ffs, and the sys admins knew it was good. :)  -- Ted Unangst über *fs

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

On Wednesday, March 27th, 2024 at 02:35, Thorsten Glaser <tg@mirbsd.de> wrote:

> Alexander Weps dixit:
>
> > But mktime can return -1 if and when it deems that struct tm cannot be
> > represented in epoch seconds.
>
>
> But it CAN be represented in epoch seconds.
>
> It can only not be represented in epoch seconds if it’s
> outside of the range of time_t, e.g. [LONG_MIN; LONG_MAX].
>
> POSIX even explicitly documents how these discontinuities
> are mapped, which I already quoted:
>
> | If a geographical timezone changes its UTC offset such that “old
> | 00:00” becomes “new 00:30” and mktime() is given 00:20, it treats that
> | as either 20 minutes after “old 00:00” or 10 minutes before “new
> | 00:30”, and gives back appropriately altered struct tm fields.

Provide source.

>
> This makes it clear that there are two ways the struct tm can
> be represented in time_t, and that the implementation must
> choose one of them.
>
> bye,
> //mirabilos
> --
> [...] if maybe ext3fs wasn't a better pick, or jfs, or maybe reiserfs, oh but
> what about xfs, and if only i had waited until reiser4 was ready... in the be-
> ginning, there was ffs, and in the middle, there was ffs, and at the end, there
> was still ffs, and the sys admins knew it was good. :) -- Ted Unangst über *fs

Re: [musl] Broken mktime calculations when crossing DST boundary

[Thorsten Glaser]

Alexander Weps dixit:

>> | If a geographical timezone changes its UTC offset such that “old
>> | 00:00” becomes “new 00:30” and mktime() is given 00:20, it treats that
>> | as either 20 minutes after “old 00:00” or 10 minutes before “new
>> | 00:30”, and gives back appropriately altered struct tm fields.
>
>Provide source.

POSIX Issue 8d4, lines 48038ff, page 1430.

bye,
//mirabilos
-- 
> Wish I had pine to hand :-( I'll give lynx a try, thanks.

Michael Schmitz on nntp://news.gmane.org/gmane.linux.debian.ports.68k
a.k.a. {news.gmane.org/nntp}#news.gmane.linux.debian.ports.68k in pine

Re: [musl] Broken mktime calculations when crossing DST boundary

[Alexander Weps]

>
> > Show me a function implementation that produces same time next day
> > under this behavior you assume to be correct.
>
>
> It is not possible to do that with mktime. You’ll have to do
> that yourself. POSIX even says so.
>
> It does indicate that on implementations (their word for libcs
> here) that follow its recommendation to not normalise tm_sec,
> you can achieve the desired effect by adding 86400 to it, though
> that will not work right in the presence of a leap second on
> systems honouring them (which is a deviation from POSIX, of
> course).
>
> Adding 86400 to the time_t value, under the same leap second
> caveat, can work if your code can rely on POSIX (ISO C does not
> specify the internal structure of time_t).

Doesn't work, this will not give the same time next day, this fails on STD/DST changes.

Because same time next day is not always 86400 apart.

>
> bye,
> //mirabilos
> --
> 22:20⎜<asarch> The crazy that persists in his craziness becomes a master
>
> 22:21⎜<asarch> And the distance between the craziness and geniality is
>
> only measured by the success 18:35⎜<asarch> "Psychotics are consistently
>
> inconsistent. The essence of sanity is to be inconsistently inconsistent

Re: [musl] Broken mktime calculations when crossing DST boundary

[Rich Felker]

On Mon, Mar 25, 2024 at 10:40:01PM +0000, Thorsten Glaser wrote:
> >> > Causing a program to loop or stack overflow.
> 
> That’s because your application violates the constraints
> that bind both, not just the libc, to the spec.

It's not a constraint violation. mktime is required to accept inputs
where the fields of the broken-down time don't fall within their
respective ranges, and normalize them.

Rather, the calling code is just making incorrect assumptions about
the unspecified way that happens, and it only happens to "work" on
glibc because glibc is erroring out in the middle of a computation,
giving a partial result, rather than fully normalizing it.

> >Output from musl:
> >
> >2011-12-29 01:00:00 -10
> >
> >    tm.tm_mday += 1;
> >    t = mktime(&tm);
> >
> >2011-12-29 01:00:00 -10 <-- date is the same after incrementing
> 
> This is… not as incorrect as you state.
> 
> The steps here are:
> 
> • 2011-12-30 01:00:00 ← input
> • 2011-12-30 01:00:00 ← input after normalisation (!)
> • conversion to time_t (1325206800), application of timezone offset
> • detection of the discontinuity between 2011-12-29 23:59:59 and
>       2011-12-31 00:00:00
> • arbitrary choice of selecting either endpoint
> 
> tbh I’d expect this to end up in 1325239199=2011-12-29 23:59:59
> instead of 2011-12-29 01:00:00 though, at least from reading the
> latest Issue 8 proofreading draft. WDYT dalias?

No, there is no clamping involved and I don't see how that would be
useful or conforming. This is all underspecified, but basically,
mktime is required to interpret out-of-range values for some fields as
a denormalized time of sort and normalize it via a sort of division
algorithm to a valid calendar time.

What happens is that, when the local time is in the gap not covered by
either adjacent timezone rule, we make an arbitrary choice to
interpret it as a denormal in the later of the two adjacent rules,
which then normalizes into a time in the earlier range. Here,
2011-12-30 gets interpreted as "1 day below 2011-12-31", which is
2011-12-29.

Rich