[9fans] silly question

[9fans] silly question

[Steve Simon]

I want to process some dated logfiles in awk.

gawk has date, strftime and mktime but Brian's does not.

plan9 has date(1) but there is no tm2sec(1), unless it
is called somthing I didn't expect.

Anyone found somting I could not in the plan9 distribution?

-Steve

Re: [9fans] silly question

[lucio]

> gawk has date, strftime and mktime but Brian's does not.

I hacked a version of strftime() for my own use, I don't know if it
helps.  It may not be the very latest version, I keep messing with it:

#include <u.h>
#include <libc.h>

static char *awday[7] = { "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat" };
static char *wday[7] = { "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday",
			"Friday", "Saturday"};
static char *amon[12] = { "Jan", "Feb", "Mar", "Apr", "May", "Jun",
			    "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" };
static char *mon[12] = {"January", "February", "March", "April", "May", "June",
		"July", "August", "September", "October", "November", "December"};
static char *ampm[2] = {"AM", "PM"};
static char *tz[2] = {"EST", "EDT"};

static int jan1(int);
static char *strval(char *, char *, char **, int, int);
static char *dval(char *, char *, int, int);

long
strftime (char *s, long maxsize, const char *format, const Tm *t)
{
	char *sp, *se, *fp;
	int i;

	sp = s;
	se = s+maxsize;
	for(fp=(char *)format; *fp && sp<se; fp++){
		if(*fp != '%')
			*sp++ = *fp;
		else switch(*++fp){
			case 'a':
				sp = strval(sp, se, awday, t->wday, 7);
				break;
			case 'A':
				sp = strval(sp, se, wday, t->wday, 7);
				break;
			case 'b':
				sp = strval(sp, se, amon, t->mon, 12);
				break;
			case 'B':
				sp = strval(sp, se, mon, t->mon, 12);
				break;
			case 'c':
				sp += strftime(sp, se-sp, "%a %b %d %H:%M:%S %Y", t);
				break;
			case 'd':
				sp = dval(sp, se, t->mday, 2);
				break;
			case 'H':
				sp = dval(sp, se, t->hour, 2);
				break;
			case 'I':
				i = t->hour;
				if(i == 0)
					i = 12;
				else if(i > 12)
					i -= 12;
				sp = dval(sp, se, i, 2);
				break;
			case 'j':
				sp = dval(sp, se, t->yday+1, 3);
				break;
			case 'm':
				sp = dval(sp, se, t->mon+1, 2);
				break;
			case 'M':
				sp = dval(sp, se, t->min, 2);
				break;
			case 'p':
				i = (t->hour < 12)? 0 : 1;
				sp = strval(sp, se, ampm, i, 2);
				break;
			case 'S':
				sp = dval(sp, se, t->sec, 2);
				break;
			case 'U':
				i = 7-jan1(t->year);
				if(i == 7)
					i = 0;
				/* Now i is yday number of first sunday in year */
				if(t->yday < i)
					i = 0;
				else
					i = (t->yday-i)/7 + 1;
				sp = dval(sp, se, i, 2);
				break;
			case 'w':
				sp = dval(sp, se, t->wday, 1);
				break;
			case 'W':
				i = 8-jan1(t->year);
				if(i >= 7)
					i -= 7;
				/* Now i is yday number of first monday in year */
				if(t->yday < i)
					i = 0;
				else
					i = (t->yday-i)/7 + 1;
				sp = dval(sp, se, i, 2);
				break;
			case 'x':
				sp += strftime(sp, se-sp, "%a %b %d, %Y", t);
				break;
			case 'X':
				sp += strftime(sp, se-sp, "%H:%M:%S", t);
				break;
			case 'y':
				sp = dval(sp, se, t->year%100, 2);
				break;
			case 'Y':
				sp = dval(sp, se, t->year+1900, 4);
				break;
			case 'Z':
				/* hack for now: assume eastern time zone */
				i = /* t->isdst? 1 : */ 0;
				sp = strval(sp, se, tz, i, 2);
				break;
			case 0:
				fp--; /* stop loop after next fp incr */
				break;
			default:
				*sp++ = *fp;
			}
	}
	if(*fp)
		sp = s; /* format string didn't end: no room for conversion */
	if(sp<se)
		*sp = 0;
	return sp-s;
}

static char *
strval(char *start, char *end, char **array, int index, int alen)
{
	int n;

	if(index<0 || index>=alen){
		*start = '?';
		return start+1;
	}
	n = strlen(array[index]);
	if(n > end-start)
		n = end-start;
	memcpy(start, array[index], n);
	return start+n;
}

static char *
dval(char *start, char *end, int val, int width)
{
	char *p;

	if(val<0 || end-start<width){
		*start = '?';
		return start+1;
	}
	p = start+width-1;
	while(p>=start){
		*p-- = val%10 + '0';
		val /= 10;
	}
	if(val>0)
		*start = '*';
	return start+width;
}

/*
 *	return day of the week
 *	of jan 1 of given year
 */
static int
jan1(int yr)
{
	int y, d;

/*
 *	normal gregorian calendar
 *	one extra day per four years
 */

	y = yr+1900;
	d = 4+y+(y+3)/4;

/*
 *	julian calendar
 *	regular gregorian
 *	less three days per 400
 */

	if(y > 1800) {
		d -= (y-1701)/100;
		d += (y-1601)/400;
	}

/*
 *	great calendar changeover instant
 */

	if(y > 1752)
		d += 3;

	return(d%7);
}

Re: [9fans] silly question

[arnold]

"Steve Simon" <steve@quintile.net> wrote:

> I want to process some dated logfiles in awk.
>
> gawk has date, strftime and mktime but Brian's does not.
>
> plan9 has date(1) but there is no tm2sec(1), unless it
> is called somthing I didn't expect.
>
> Anyone found somting I could not in the plan9 distribution?
>
> -Steve

I'd be happy to know the results of attempting a gawk port via APE. :-)

Thanks,

Arnold

Re: [9fans] silly question

[Jens Staal]

On Tuesday 02 September 2014 02:46:12 arnold@skeeve.com wrote:
> "Steve Simon" <steve@quintile.net> wrote:
> > I want to process some dated logfiles in awk.
> >
> > gawk has date, strftime and mktime but Brian's does not.
> >
> > plan9 has date(1) but there is no tm2sec(1), unless it
> > is called somthing I didn't expect.
> >
> > Anyone found somting I could not in the plan9 distribution?
> >
> > -Steve
>
> I'd be happy to know the results of attempting a gawk port via APE. :-)
>
> Thanks,
>
> Arnold

There is one....

http://ports2plan9.googlecode.com/files/gawk-4.0.0b.pkg.tbz

or
/n/sources/contrib/staal1978/pkg/gawk-4.0.0b.pkg.tbz

Re: [9fans] silly question

[arnold]

> > I'd be happy to know the results of attempting a gawk port via APE. :-)
>
> There is one....
>
> http://ports2plan9.googlecode.com/files/gawk-4.0.0b.pkg.tbz
>
> or
> /n/sources/contrib/staal1978/pkg/gawk-4.0.0b.pkg.tbz

4.0.0 is around 3 years old. Current version is 4.1.1.
Although this one would do for Steve who wanted time functions.

Thanks,

Arnold

Re: [9fans] silly question

[Steve Simon]

> I'd be happy to know the results of attempting a gawk port via APE. :-)

Not sure Al, Peter, or Brian would forgive me :-)
Though if memory serves it has been done already.

-Steve

Re: [9fans] silly question

[arnold]

"Steve Simon" <steve@quintile.net> wrote:

> > I'd be happy to know the results of attempting a gawk port via APE. :-)
>
> Not sure Al, Peter, or Brian would forgive me :-)
> Though if memory serves it has been done already.
>
> -Steve

Brian is a good friend of mine. He (at least) wouldn't mind. :-)

Arnold

Re: [9fans] silly question

[erik quanstrom]

On Tue Sep  2 03:07:56 EDT 2014, steve@quintile.net wrote:
> I want to process some dated logfiles in awk.
>
> gawk has date, strftime and mktime but Brian's does not.

i'm not sure what your particular problem domain is since you don't say, but i've
always just used the standard awk functions to convert dates.  with string manipulation,
regular expressions, and associative arrays, i have not found a date conversion issue that
awk can't handle.  this has the advantage that a second awk isn't introduced to the system,
with all the resulting incompatibilites, and not requiring porting something from linux.

the operations that strftime the unix fn provides are implementable directly in awk.

speaking as one who has fallen into this trap, i think this might be an example of negative
pattern, too.  it's easy to get hung up on missing specialized function, when in fact the task
at hand doesn't really need it. stand up and shout if you remember the days when packages
would often fail to compile because they were looking for a function that wasn't really
necessary, and the test for that function failed in some unexpected way.  i wish i could
remember some of the examples.  they were spectacular.  and not in a good way.

- erik

Re: [9fans] silly question

[Steve Simon]

> i'm not sure what your particular problem domain is since you don't say

True.

Strftime is a red herring (sorry), I can use  and "date" | getline
to generate pretty much any date string I need.

The issue is more going the other way. tm2sec in awk is quite complex
and hids many pitfalls if you want to do it correctly.

My problem is parsing logfiles which contain dates in the form
of date(1) / ctime(2).

I want to graph stuff over time and so I want a monotonically incrementing
number (secs sinc 1/1/70 would be ideal). I have coded this in awk but
for one year leap years break - though not by much.

This is why I was asking if there was an external app called somthing like
tm2sec which calls its eponomymous library function.

it seems there isn't one and its the work of 30secs to write one, no problem,
I just thought there was one but I had forgotten its name...

-Steve

Re: [9fans] silly question

[Kurt H Maier]

Quoting Steve Simon <steve@quintile.net>:

> plan9 has date(1) but there is no tm2sec(1), unless it
> is called somthing I didn't expect.

seconds(1)

khm

Re: [9fans] silly question

[erik quanstrom]

> Strftime is a red herring (sorry), I can use  and "date" | getline
> to generate pretty much any date string I need.
>
> The issue is more going the other way. tm2sec in awk is quite complex
> and hids many pitfalls if you want to do it correctly.
>
> My problem is parsing logfiles which contain dates in the form
> of date(1) / ctime(2).
>
> I want to graph stuff over time and so I want a monotonically incrementing
> number (secs sinc 1/1/70 would be ideal). I have coded this in awk but
> for one year leap years break - though not by much.

if the hair is just leap years, the algorithm used by /sys/src/libc/9sys/ctime.c
is pretty attractive.  the idea is to just loop through the years between given
and 1970, and add a day for each leap year encountered.  should be easy
to do in awk.

- erik

Re: [9fans] silly question

[Steve Simon]

> seconds(1)

Marvelous, on two levels:

	that it exists and I can use it.

	that it diodn't imagine it

Thanks Kurt.

-Steve

Re: [9fans] silly question

[Bakul Shah]

On Tue, 02 Sep 2014 15:10:56 EDT erik quanstrom <quanstro@quanstro.net> wrote:
> > Strftime is a red herring (sorry), I can use  and "date" | getline
> > to generate pretty much any date string I need.
> >
> > The issue is more going the other way. tm2sec in awk is quite complex
> > and hids many pitfalls if you want to do it correctly.
> >
> > My problem is parsing logfiles which contain dates in the form
> > of date(1) / ctime(2).
> >
> > I want to graph stuff over time and so I want a monotonically incrementing
> > number (secs sinc 1/1/70 would be ideal). I have coded this in awk but
> > for one year leap years break - though not by much.
>
> if the hair is just leap years, the algorithm used by /sys/src/libc/9sys/ctim
> e.c
> is pretty attractive.  the idea is to just loop through the years between giv
> en
> and 1970, and add a day for each leap year encountered.  should be easy
> to do in awk.

plan9 doesn't deal with leap seconds, right?  There've been 35
leap seconds since 1972 (International Atomic Time is 35
seconds ahead of GMT).  Though this probably doesn't matter
for timestamps in log files.

Re: [9fans] silly question

[erik quanstrom]

> plan9 doesn't deal with leap seconds, right?  There've been 35
> leap seconds since 1972 (International Atomic Time is 35
> seconds ahead of GMT).  Though this probably doesn't matter
> for timestamps in log files.

correct.  plan 9 does not bother with leap seconds.

- erik

Re: [9fans] silly question

[Skip Tavakkolian]

inspired me to write discotime:

% cat discotime.go
// print the number of seconds from the dawn of Disco until the date
in the argument
package main

import (
    "fmt"
    "os"
    "time"
)

func main() {
    for _, s := range os.Args[1:] {
        d, err := time.Parse(time.UnixDate, s)
        if err != nil {
            panic(err)
        }
        fmt.Println(d.Unix())
    }
}
% ./discotime 'Tue Aug 16 17:03:52 CDT 1977'
240599032

to make a hammertime (http://en.wikipedia.org/wiki/U_Can't_Touch_This)
you can subtract 1990 from parsed date instead.

-Skip


On Tue, Sep 2, 2014 at 1:04 PM, Bakul Shah <bakul@bitblocks.com> wrote:
> On Tue, 02 Sep 2014 15:10:56 EDT erik quanstrom <quanstro@quanstro.net> wrote:
>> > Strftime is a red herring (sorry), I can use  and "date" | getline
>> > to generate pretty much any date string I need.
>> >
>> > The issue is more going the other way. tm2sec in awk is quite complex
>> > and hids many pitfalls if you want to do it correctly.
>> >
>> > My problem is parsing logfiles which contain dates in the form
>> > of date(1) / ctime(2).
>> >
>> > I want to graph stuff over time and so I want a monotonically incrementing
>> > number (secs sinc 1/1/70 would be ideal). I have coded this in awk but
>> > for one year leap years break - though not by much.
>>
>> if the hair is just leap years, the algorithm used by /sys/src/libc/9sys/ctim
>> e.c
>> is pretty attractive.  the idea is to just loop through the years between giv
>> en
>> and 1970, and add a day for each leap year encountered.  should be easy
>> to do in awk.
>
> plan9 doesn't deal with leap seconds, right?  There've been 35
> leap seconds since 1972 (International Atomic Time is 35
> seconds ahead of GMT).  Though this probably doesn't matter
> for timestamps in log files.
>

Re: [9fans] silly question

[Steve Simon]

> to make a hammertime (http://en.wikipedia.org/wiki/U_Can't_Touch_This)
> you can subtract 1990 from parsed date instead.

Oh no.

Thats going to be stuck in my head for hours now ☺

-Steve

Re: [9fans] silly question

[Bakul Shah]

Skip, You have a very strange sense of humour.

At the first stroke it will be ten thrree & 40 seconds.
At the first stroke it will be ten thrree & 50 seconds.
At the first stroke it will be ten four. Precisely.

On Tue, 02 Sep 2014 14:10:57 PDT Skip Tavakkolian <skip.tavakkolian@gmail.com> wrote:
> inspired me to write discotime:
>
> % cat discotime.go
> // print the number of seconds from the dawn of Disco until the date
> in the argument
> package main
>
> import (
>     "fmt"
>     "os"
>     "time"
> )
>
> func main() {
>     for _, s := range os.Args[1:] {
>         d, err := time.Parse(time.UnixDate, s)
>         if err != nil {
>             panic(err)
>         }
>         fmt.Println(d.Unix())
>     }
> }
> % ./discotime 'Tue Aug 16 17:03:52 CDT 1977'
> 240599032
>
> to make a hammertime (http://en.wikipedia.org/wiki/U_Can't_Touch_This)
> you can subtract 1990 from parsed date instead.
>
> -Skip
>
>
> On Tue, Sep 2, 2014 at 1:04 PM, Bakul Shah <bakul@bitblocks.com> wrote:
> > On Tue, 02 Sep 2014 15:10:56 EDT erik quanstrom <quanstro@quanstro.net> wro
> te:
> >> > Strftime is a red herring (sorry), I can use  and "date" | getline
> >> > to generate pretty much any date string I need.
> >> >
> >> > The issue is more going the other way. tm2sec in awk is quite complex
> >> > and hids many pitfalls if you want to do it correctly.
> >> >
> >> > My problem is parsing logfiles which contain dates in the form
> >> > of date(1) / ctime(2).
> >> >
> >> > I want to graph stuff over time and so I want a monotonically incrementi
> ng
> >> > number (secs sinc 1/1/70 would be ideal). I have coded this in awk but
> >> > for one year leap years break - though not by much.
> >>
> >> if the hair is just leap years, the algorithm used by /sys/src/libc/9sys/c
> tim
> >> e.c
> >> is pretty attractive.  the idea is to just loop through the years between
> giv
> >> en
> >> and 1970, and add a day for each leap year encountered.  should be easy
> >> to do in awk.
> >
> > plan9 doesn't deal with leap seconds, right?  There've been 35
> > leap seconds since 1972 (International Atomic Time is 35
> > seconds ahead of GMT).  Though this probably doesn't matter
> > for timestamps in log files.
> >
>

Re: [9fans] silly question

[Skip Tavakkolian]

not strange; misunderstood :)

On Tue, Sep 2, 2014 at 2:27 PM, Bakul Shah <bakul@bitblocks.com> wrote:
> Skip, You have a very strange sense of humour.
>
> At the first stroke it will be ten thrree & 40 seconds.
> At the first stroke it will be ten thrree & 50 seconds.
> At the first stroke it will be ten four. Precisely.
>
> On Tue, 02 Sep 2014 14:10:57 PDT Skip Tavakkolian <skip.tavakkolian@gmail.com> wrote:
>> inspired me to write discotime:
>>
>> % cat discotime.go
>> // print the number of seconds from the dawn of Disco until the date
>> in the argument
>> package main
>>
>> import (
>>     "fmt"
>>     "os"
>>     "time"
>> )
>>
>> func main() {
>>     for _, s := range os.Args[1:] {
>>         d, err := time.Parse(time.UnixDate, s)
>>         if err != nil {
>>             panic(err)
>>         }
>>         fmt.Println(d.Unix())
>>     }
>> }
>> % ./discotime 'Tue Aug 16 17:03:52 CDT 1977'
>> 240599032
>>
>> to make a hammertime (http://en.wikipedia.org/wiki/U_Can't_Touch_This)
>> you can subtract 1990 from parsed date instead.
>>
>> -Skip
>>
>>
>> On Tue, Sep 2, 2014 at 1:04 PM, Bakul Shah <bakul@bitblocks.com> wrote:
>> > On Tue, 02 Sep 2014 15:10:56 EDT erik quanstrom <quanstro@quanstro.net> wro
>> te:
>> >> > Strftime is a red herring (sorry), I can use  and "date" | getline
>> >> > to generate pretty much any date string I need.
>> >> >
>> >> > The issue is more going the other way. tm2sec in awk is quite complex
>> >> > and hids many pitfalls if you want to do it correctly.
>> >> >
>> >> > My problem is parsing logfiles which contain dates in the form
>> >> > of date(1) / ctime(2).
>> >> >
>> >> > I want to graph stuff over time and so I want a monotonically incrementi
>> ng
>> >> > number (secs sinc 1/1/70 would be ideal). I have coded this in awk but
>> >> > for one year leap years break - though not by much.
>> >>
>> >> if the hair is just leap years, the algorithm used by /sys/src/libc/9sys/c
>> tim
>> >> e.c
>> >> is pretty attractive.  the idea is to just loop through the years between
>> giv
>> >> en
>> >> and 1970, and add a day for each leap year encountered.  should be easy
>> >> to do in awk.
>> >
>> > plan9 doesn't deal with leap seconds, right?  There've been 35
>> > leap seconds since 1972 (International Atomic Time is 35
>> > seconds ahead of GMT).  Though this probably doesn't matter
>> > for timestamps in log files.
>> >
>>
>

Re: [9fans] silly question

[Kurt H Maier]

Quoting erik quanstrom <quanstro@quanstro.net>:


> correct.  plan 9 does not bother with leap seconds.


seconds(1) "handles" leap seconds in that it will not crash
when it encounters them -- it accepts that sometimes there
are 61 seconds in a minute.

khm

Re: [9fans] silly question

[erik quanstrom]

> > correct.  plan 9 does not bother with leap seconds.
>
> seconds(1) "handles" leap seconds in that it will not crash
> when it encounters them -- it accepts that sometimes there
> are 61 seconds in a minute.

i'm not sure if we're talking past each other, or making different points.
but either way, i should clarify.

by "not handling leap seconds" i mean there is no code in ctime() to add
in leap seconds from an external source at the appropriate unix times.
so e.g. gmtime could read e.g. /lib/leapseconds and ajust seconds similar
to days (/sys/src/libc/9sys/ctime.c:137,138).

- erik

Re: [9fans] silly question

[Bakul Shah]

On Tue, 02 Sep 2014 19:00:56 EDT erik quanstrom <quanstro@quanstro.net> wrote:
> > > correct.  plan 9 does not bother with leap seconds.
> >
> > seconds(1) "handles" leap seconds in that it will not crash
> > when it encounters them -- it accepts that sometimes there
> > are 61 seconds in a minute.
>
> i'm not sure if we're talking past each other, or making different points.
> but either way, i should clarify.
>
> by "not handling leap seconds" i mean there is no code in ctime() to add
> in leap seconds from an external source at the appropriate unix times.
> so e.g. gmtime could read e.g. /lib/leapseconds and ajust seconds similar
> to days (/sys/src/libc/9sys/ctime.c:137,138).

I was mistaken.  Turns out neither do Unix systems handle
leapseconds. Now if only ITU punts on leapseconds in 2015, we
can let some future generation worry about leap minutes or
hours! Sorry for the noise.

Re: [9fans] silly question

[erik quanstrom]

> I was mistaken.  Turns out neither do Unix systems handle
> leapseconds. Now if only ITU punts on leapseconds in 2015, we
> can let some future generation worry about leap minutes or
> hours! Sorry for the noise.

or we can give up on noon being a particular solar time.  this would
free us from timezones, dst and other complications.  just kidding ... maybe.

- erik

Re: [9fans] silly question

[Bakul Shah]

On Tue, 02 Sep 2014 19:19:59 EDT erik quanstrom <quanstro@quanstro.net> wrote:
> > I was mistaken.  Turns out neither do Unix systems handle
> > leapseconds. Now if only ITU punts on leapseconds in 2015, we
> > can let some future generation worry about leap minutes or
> > hours! Sorry for the noise.
>
> or we can give up on noon being a particular solar time.  this would
> free us from timezones, dst and other complications.  just kidding ... maybe.

The wikipedia entry on leap second is quite instructive.

Re: [9fans] silly question

[sl]

> The wikipedia entry on leap second is quite instructive.

It is now.

sl