libdate

libdate

[ori]

Date handling on plan 9 is almost adequate today if you don't
have to parse dates or deal with timezones, and don't do
multithreading. Otherwise, it's difficult to get right, and
we often don't.

We've got a crappy home-rolled date parser in seconds(1),
a few in the upas source tree to deal with mail formats,
and git9 has a few hacks around this as well.

Out of tree, joe9 has been trying to write code that takes
stock information in one timezone and moves them to another,
and our APIs there are completely inadequate.

So, I tried to write a library that is adequate, without
being complicated.

The code lives here:

https://git.eigenstate.org/ori/date.git

I'll probably be merging in the changes between Tmd and Tm
soon, and committing to 9front, possibly even as part of libc.

Some additional work is probably going to be needed to convert
from IANA zoneinfo to actually bring our timezone data up to
date. We may also need some timezone info format changes to
handle political (and leap second) changes.


The manpage is attached below for review:

     TMDATE(2)                                               TMDATE(2)

     NAME
          tmnow, tmgetzone, tmtime, tmstime, tmparse, tmfmt, tmnorm, -
          convert date and time

     SYNOPSIS
          #include <u.h>
          #include <libc.h>

          typedef struct Tmd Tmd;
          struct Tmd {
                vlong    abs; /* seconds since Jan 1 1970, GMT */
                int sec; /* seconds (range 0..59) */
                int min; /* minutes (0..59) */
                int hour;     /* hours (0..23) */
                int mday;     /* day of the month (1..31) */
                int mon; /* month of the year (0..11) */
                int year;     /* year A.D. - 1900 */
                int wday;     /* day of week (0..6, Sunday = 0) */
                int yday;     /* day of year (0..365) */
                char     zone[];   /* time zone name */
                int tzoff;    /* time   zone delta from GMT */
          };

          Tzone *tmgetzone(char *name);
          Tmd    *tmnow(Tmd *tm, char *tz);
          Tmd    *tmtime(Tmd *tm, vlong abs, Tzone *tz);
          Tmd    *tmstime(Tmd *tm, vlong sec, Tzone *tz);
          Tmd    *tmparse(char *fmt, char *tm, Tzone *zone, Tmd *dst);
          int   tmfmt(char *buf, usize nbuf, char *fmt, Tmd *tm);
          void tmnorm(Tmd *tm);
          void tmfmtinstall(char *fmt);

     DESCRIPTION
          This family of functions handles simple date and time manpu-
          lation.  Times are represented as an absolute instant in
          time, combined with a time zone.

          Time zones are loaded by as name.  They can be specified as
          the abbreviated timezone name, the full timezone name, the
          path to a timezone file, or an absolute offset in the HHMM
          form.

          When given as a timezone, any instant-dependent adjustments
          such as leap seconds and daylight savings time will be
          applied to the derived fields of struct tm, but will not
          affect the absolute time.  The time zone name local always
          refers to the time in /env/timezone.  The nil timezone
          always refers to GMT.

          Tmgetzone loads a timezone by name. The returned timezone is
          cached for the lifetime of the program, and should not be
          freed.  Loading a timezone repeatedly by name loads from the
          cache, and does not leak.

          Tmnow gets the current time of day in the requested time
          zone.

          Tmtime converts the millisecond-resolution timestamp 'abs'
          into a Tm struct in the requested timezone.

          Tmstime is identical to tmtime, but accepts the time in sec-
          onds.

          Tmparse parses a time from a string according to the format
          argument. The result is returned in the timezone requested.
          If there is a timezone in the date, then we tzshift to the
          local timezone.

          The format argument takes contains zero or more of the fol-
          lowing components:

          Y, YY, YYYY
               Represents the year.  YY prints the year in 2 digit
               form.

          M, MM, MMM, MMMM
               The month of the year, in unpadded numeric, padded
               numeric, short name, or long name, respectively.

          D, DD
               The day of month in unpadded or padded numeric form,
               respectively.

          W, WW
               The day of week in short or long name form, respec-
               tively.

          h, hh
               The hour in unpadded or padded form, respectively

          m, mm
               The minute in unpadded or padded form, respectively

          s, ss
               The second in unpadded or padded form, respectively

          z, Z, ZZ
               The timezone in named, [+-]HHMM and [+-]HH:MM form,
               respectively

          a, A Lower and uppercase 'am' and 'pm' specifiers, respec-
               tively.

          [...]
               Quoted text, copied directly to the output.

          Any characters not specified above are copied directly to
          output, without modification.

          If the format argument is nil, it makes an attempt to parse
          common human readable date formats.  These formats include
          ISO-8601,RFC-3339 and RFC-2822 dates.

          Tmfmt formats a Tm struct according to the format fmt. If
          fmt is nil, we format as in ctime(2). At most characters are
          written into buf, including the terminator.  The format is
          identical to tmparse.

          When parsing, any amount of whitespace is treated as a sin-
          gle token.  All string matches are case insensitive, and
          zero padding is optional.

          Tmrecalc takes a manually adjusted Tm structure, and recal-
          culates the absolute time from the year, mon, mday, hr, min
          and sec fields. Other fields are ignored.  This recalcula-
          tion respects the time zone stored in struct tm.  Out of
          range values are wrapped. For example, December 32nd becomes
          January 1st.

          Tmfmtinstall installs a time format specifier %τ. The time
          format behaves as in tmfmt

     Examples
          All examples assume tmfmtinstall has been called.

          Get the current date in the local timezone, GMT, and
          US_Pacific time. Print it using the default format.

               Tm t;
               Tzone *zl, *zp;
               if((zl = tmgetzone("local") == nil)
                    sysfatal("load zone: %r");
               if((zp = tmgetzone("US_Pacific") == nil)
                    sysfatal("load zone: %r");
               print("local: %τ\n", tmnow(&t, zl));
               print("gmt: %τ\n", tmnow(&t, nil));
               print("eastern: %τ\n", tmnow(&t, zp));

          Compare if two times are the same, regardless of timezone.

               Tm a, b;

               tmparse(&a, nil, "Tue Dec 10 12:36:00 PST 2019");
               tmparse(&b, nil, "Tue Dec 10 15:36:00 EST 2019");
               if(a.abs == b.abs)
                    print("same\n");
               else
                    print("different\n");

          Convert from one timezone to another.

               Tm here, there;
               Tzone *zl, *zp;
               if((zl = tmgetzone("local") == nil)
                    sysfatal("load zone: %r");
               if((zp = tmgetzone("US_Pacific") == nil)
                    sysfatal("load zone: %r");
               if(tmnow(&here, zl) == nil)
                    sysfatal("get time: %r");
               if(tmtime(&there, here.abs, zp) == nil)
                    sysfatal("shift time: %r");

          Add a day to two times. Because we picked daylight savings
          time to adjust over, only 23 hours are added.

          Tm t;
          tmparse(&t, "W MMM D hh:mm:ss z YYYY, "Sun Nov 2 13:11:11 PST 2019");
          tm.day++;
          tmrecalc(&t);
          print("%τ", &t); /*  Mon Nov 3 13:11:11 PST 2019 */

Re: [9fans] libdate

[tlaronde]

Hello,

On Sat, Apr 25, 2020 at 05:54:30PM -0700, ori@eigenstate.org wrote:
> Date handling on plan 9 is almost adequate today if you don't
> have to parse dates or deal with timezones, and don't do
> multithreading. Otherwise, it's difficult to get right, and
> we often don't.
> 
> We've got a crappy home-rolled date parser in seconds(1),
> a few in the upas source tree to deal with mail formats,
> and git9 has a few hacks around this as well.
> 
> Out of tree, joe9 has been trying to write code that takes
> stock information in one timezone and moves them to another,
> and our APIs there are completely inadequate.
> 
> So, I tried to write a library that is adequate, without
> being complicated.
> 

Just out of curiosity (I may have missed the point): since this is not
heavily system dependent, and more user related, and for the sake of
APE, did you consider the standard C and the POSIX interfaces?

> The code lives here:
> 
> https://git.eigenstate.org/ori/date.git
> 
> I'll probably be merging in the changes between Tmd and Tm
> soon, and committing to 9front, possibly even as part of libc.
> 
> Some additional work is probably going to be needed to convert
> from IANA zoneinfo to actually bring our timezone data up to
> date. We may also need some timezone info format changes to
> handle political (and leap second) changes.
> 
> 
> The manpage is attached below for review:

FWIW, a typo in the sample code, the closing bracket in the assignation
before comparison is missing.

>                if((zl = tmgetzone("local") == nil)
                                            ^

(in all the chunks).

Thank you for the work!

Best regards,
-- 
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                     http://www.kergis.com/
                       http://www.sbfa.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9fans] libdate

[ori]

> Just out of curiosity (I may have missed the point): since this is not
> heavily system dependent, and more user related, and for the sake of
> APE, did you consider the standard C and the POSIX interfaces?

First, the posix interfaces share inadequacies with plan 9.
They only work for local timezones and GMT.  So, I see this
as a place where we can get our APIs right, and not just
copying poorly considered interface rot from Unix.

Second, ape can't call this code anyways -- ape lives in its
own world, with its own libc.  Adopting posix interfaces
here doesn't help ape.

Still, here's probably code that could be dropped into ape,
and adapted to support more of the posix APIs. I'd consider
committing patches if they came across my inbox.[1]

>> The manpage is attached below for review:
> 
> FWIW, a typo in the sample code, the closing bracket in the assignation
> before comparison is missing.
> 
>>                if((zl = tmgetzone("local") == nil)
>                                             ^
> 
> (in all the chunks).

Thanks, fixed.

> Thank you for the work!

[1] A note on my approach here: I see ape as a tool for porting programs,
    so following standards in a vaccuum isn't useful -- I'd prefer to make
    ape changes lazily -- eg, as a result of porting netsurf.

Re: [9fans] libdate

[Calvin Morrison]

> Time zones are loaded by as name.  

Extra word?


On Sat, Apr 25, 2020, at 8:54 PM, ori@eigenstate.org wrote:
> Date handling on plan 9 is almost adequate today if you don't
> have to parse dates or deal with timezones, and don't do
> multithreading. Otherwise, it's difficult to get right, and
> we often don't.
> 
> We've got a crappy home-rolled date parser in seconds(1),
> a few in the upas source tree to deal with mail formats,
> and git9 has a few hacks around this as well.
> 
> Out of tree, joe9 has been trying to write code that takes
> stock information in one timezone and moves them to another,
> and our APIs there are completely inadequate.
> 
> So, I tried to write a library that is adequate, without
> being complicated.
> 
> The code lives here:
> 
> https://git.eigenstate.org/ori/date.git
> 
> I'll probably be merging in the changes between Tmd and Tm
> soon, and committing to 9front, possibly even as part of libc.
> 
> Some additional work is probably going to be needed to convert
> from IANA zoneinfo to actually bring our timezone data up to
> date. We may also need some timezone info format changes to
> handle political (and leap second) changes.
> 
> 
> The manpage is attached below for review:
> 
>      TMDATE(2)                                               TMDATE(2)
> 
>      NAME
>           tmnow, tmgetzone, tmtime, tmstime, tmparse, tmfmt, tmnorm, -
>           convert date and time
> 
>      SYNOPSIS
>           #include <u.h>
>           #include <libc.h>
> 
>           typedef struct Tmd Tmd;
>           struct Tmd {
>                 vlong    abs; /* seconds since Jan 1 1970, GMT */
>                 int sec; /* seconds (range 0..59) */
>                 int min; /* minutes (0..59) */
>                 int hour;     /* hours (0..23) */
>                 int mday;     /* day of the month (1..31) */
>                 int mon; /* month of the year (0..11) */
>                 int year;     /* year A.D. - 1900 */
>                 int wday;     /* day of week (0..6, Sunday = 0) */
>                 int yday;     /* day of year (0..365) */
>                 char     zone[];   /* time zone name */
>                 int tzoff;    /* time   zone delta from GMT */
>           };
> 
>           Tzone *tmgetzone(char *name);
>           Tmd    *tmnow(Tmd *tm, char *tz);
>           Tmd    *tmtime(Tmd *tm, vlong abs, Tzone *tz);
>           Tmd    *tmstime(Tmd *tm, vlong sec, Tzone *tz);
>           Tmd    *tmparse(char *fmt, char *tm, Tzone *zone, Tmd *dst);
>           int   tmfmt(char *buf, usize nbuf, char *fmt, Tmd *tm);
>           void tmnorm(Tmd *tm);
>           void tmfmtinstall(char *fmt);
> 
>      DESCRIPTION
>           This family of functions handles simple date and time manpu-
>           lation.  Times are represented as an absolute instant in
>           time, combined with a time zone.
> 
>           Time zones are loaded by as name.  They can be specified as
>           the abbreviated timezone name, the full timezone name, the
>           path to a timezone file, or an absolute offset in the HHMM
>           form.
> 
>           When given as a timezone, any instant-dependent adjustments
>           such as leap seconds and daylight savings time will be
>           applied to the derived fields of struct tm, but will not
>           affect the absolute time.  The time zone name local always
>           refers to the time in /env/timezone.  The nil timezone
>           always refers to GMT.
> 
>           Tmgetzone loads a timezone by name. The returned timezone is
>           cached for the lifetime of the program, and should not be
>           freed.  Loading a timezone repeatedly by name loads from the
>           cache, and does not leak.
> 
>           Tmnow gets the current time of day in the requested time
>           zone.
> 
>           Tmtime converts the millisecond-resolution timestamp 'abs'
>           into a Tm struct in the requested timezone.
> 
>           Tmstime is identical to tmtime, but accepts the time in sec-
>           onds.
> 
>           Tmparse parses a time from a string according to the format
>           argument. The result is returned in the timezone requested.
>           If there is a timezone in the date, then we tzshift to the
>           local timezone.
> 
>           The format argument takes contains zero or more of the fol-
>           lowing components:
> 
>           Y, YY, YYYY
>                Represents the year.  YY prints the year in 2 digit
>                form.
> 
>           M, MM, MMM, MMMM
>                The month of the year, in unpadded numeric, padded
>                numeric, short name, or long name, respectively.
> 
>           D, DD
>                The day of month in unpadded or padded numeric form,
>                respectively.
> 
>           W, WW
>                The day of week in short or long name form, respec-
>                tively.
> 
>           h, hh
>                The hour in unpadded or padded form, respectively
> 
>           m, mm
>                The minute in unpadded or padded form, respectively
> 
>           s, ss
>                The second in unpadded or padded form, respectively
> 
>           z, Z, ZZ
>                The timezone in named, [+-]HHMM and [+-]HH:MM form,
>                respectively
> 
>           a, A Lower and uppercase 'am' and 'pm' specifiers, respec-
>                tively.
> 
>           [...]
>                Quoted text, copied directly to the output.
> 
>           Any characters not specified above are copied directly to
>           output, without modification.
> 
>           If the format argument is nil, it makes an attempt to parse
>           common human readable date formats.  These formats include
>           ISO-8601,RFC-3339 and RFC-2822 dates.
> 
>           Tmfmt formats a Tm struct according to the format fmt. If
>           fmt is nil, we format as in ctime(2). At most characters are
>           written into buf, including the terminator.  The format is
>           identical to tmparse.
> 
>           When parsing, any amount of whitespace is treated as a sin-
>           gle token.  All string matches are case insensitive, and
>           zero padding is optional.
> 
>           Tmrecalc takes a manually adjusted Tm structure, and recal-
>           culates the absolute time from the year, mon, mday, hr, min
>           and sec fields. Other fields are ignored.  This recalcula-
>           tion respects the time zone stored in struct tm.  Out of
>           range values are wrapped. For example, December 32nd becomes
>           January 1st.
> 
>           Tmfmtinstall installs a time format specifier %τ. The time
>           format behaves as in tmfmt
> 
>      Examples
>           All examples assume tmfmtinstall has been called.
> 
>           Get the current date in the local timezone, GMT, and
>           US_Pacific time. Print it using the default format.
> 
>                Tm t;
>                Tzone *zl, *zp;
>                if((zl = tmgetzone("local") == nil)
>                     sysfatal("load zone: %r");
>                if((zp = tmgetzone("US_Pacific") == nil)
>                     sysfatal("load zone: %r");
>                print("local: %τ\n", tmnow(&t, zl));
>                print("gmt: %τ\n", tmnow(&t, nil));
>                print("eastern: %τ\n", tmnow(&t, zp));
> 
>           Compare if two times are the same, regardless of timezone.
> 
>                Tm a, b;
> 
>                tmparse(&a, nil, "Tue Dec 10 12:36:00 PST 2019");
>                tmparse(&b, nil, "Tue Dec 10 15:36:00 EST 2019");
>                if(a.abs == b.abs)
>                     print("same\n");
>                else
>                     print("different\n");
> 
>           Convert from one timezone to another.
> 
>                Tm here, there;
>                Tzone *zl, *zp;
>                if((zl = tmgetzone("local") == nil)
>                     sysfatal("load zone: %r");
>                if((zp = tmgetzone("US_Pacific") == nil)
>                     sysfatal("load zone: %r");
>                if(tmnow(&here, zl) == nil)
>                     sysfatal("get time: %r");
>                if(tmtime(&there, here.abs, zp) == nil)
>                     sysfatal("shift time: %r");
> 
>           Add a day to two times. Because we picked daylight savings
>           time to adjust over, only 23 hours are added.
> 
>           Tm t;
>           tmparse(&t, "W MMM D hh:mm:ss z YYYY, "Sun Nov 2 13:11:11 PST 2019");
>           tm.day++;
>           tmrecalc(&t);
>           print("%τ", &t); /*  Mon Nov 3 13:11:11 PST 2019 */
> 
> 
> ------------------------------------------
> 9fans: 9fans
> Permalink: 
> https://9fans.topicbox.com/groups/9fans/T5b9f56a5fac852c2-Mca6c8f44f1a159d02b37302b
> Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
>

Re: [9fans] libdate

[cinap_lenrek]

in the Tmd struct:

>                vlong    abs; /* seconds since Jan 1 1970, GMT */

in the description of tmtime():

>          Tmtime converts the millisecond-resolution timestamp 'abs'
>          into a Tm struct in the requested timezone.

the example:

>               if(tmtime(&there, here.abs, zp) == nil)

should Tmd.abs not be documented as milliseconds instead of seconds?

--
cinap

Re: [9fans] libdate

[ori]

> Date handling on plan 9 is almost adequate today if you don't
> have to parse dates or deal with timezones, and don't do
> multithreading. Otherwise, it's difficult to get right, and
> we often don't.

I should have said -- I'm hoping to get some feedback.

The best feedback would be "I don't see how I'd do <thing>",
or "It seems like <manipulations>" would be error prone or
fragile. 

Another problem that I'm not sure how to deal with is that
right now, our timezone database format has two issues:
First, it has no way of mapping from an abbreviated zone.
For example, EST should map to /adm/timezone/US_Eastern,
but that information requires a search of all timezones.

Second, even if we did search all timezones, we're hard
coding a timezone name length of 3, which means that
EST is ambiguous: It maps to US_Eastern, but it also
is used as the name for AEST (Australian East), BRT
(Brazil), and Jamaica.

As a result, if we have a timestamp like this:

	Jan 1 2013, 7:00 PM EST

will not be able to deal with the timezone component.

Extending the length of the name in our current timezone
files will break old binaries that use localtime, since
they'll return an error on timezone naming.

I'm wondering if there are clean approaches to deal
with this.

Re: [9fans] libdate

[ori]

> As a result, if we have a timestamp like this:
> 
> 	Jan 1 2013, 7:00 PM EST
> 
> will not be able to deal with the timezone component.

And now that I've started poking at converting code like
seconds(1), it looks like I'm going to need to solve it.
I'm thinking that I want to add /adm/timezone/abbrev,
which would look like:

	EST	US_Eastern
	PST US_Pacific
	AEST Australia_Tasmania

Obviously, this is a lossy mapping: There are multiple
named timezones that map to one abbreviation, and I think
that's fine.

Re: [9fans] libdate

[ori]

>> As a result, if we have a timestamp like this:
>> 
>>       Jan 1 2013, 7:00 PM EST
>> 
>> will not be able to deal with the timezone component.
> 
> And now that I've started poking at converting code like
> seconds(1), it looks like I'm going to need to solve it.
> I'm thinking that I want to add /adm/timezone/abbrev,
> which would look like:
> 
>         EST     US_Eastern
>         PST US_Pacific
>         AEST Australia_Tasmania
> 
> Obviously, this is a lossy mapping: There are multiple
> named timezones that map to one abbreviation, and I think
> that's fine.

Hm. And looking again at the relevant standards, rfc5322
only allows a short list of US timezone names, and rfc8601
doesn't allow any.

Perhaps the appropriate solution is to hard code the American
timezones, and reject all other abbreviations, as in rfc5322.

I don't like the lack of generality, but it's an option.