Re: [9fans] Re: patch/list sorry/proc-mtime

Re: [9fans] Re: patch/list sorry/proc-mtime

[Fco. J. Ballesteros]

:  In the end everything is matter of convenience and taste, but this
:  criticism comming from the person that instigated the tar -z
:  abomination is harder to take seriously.

I use tar z a lot.
Why don't you write some code and relax?
Sorry to be so rude.

[9fans] Re: patch/list sorry/proc-mtime

[uriel]

> d-rwxrwxr-x M 430 uriel sys 0 Mar 29 17:43 sorry/proc-mtime
> from uriel@cat-v.org
> 	/sys/src/9/port/devproc.c
> 	Make the mtime for files under /proc be the creation time of the process rather than the boot time.
>
> Wed Mar 29 17:42:05 EST 2006 rsc
>     This is an interesting idea, but I don't see a use.
>     The information is already available in /proc/n/status.
>     Is there a compelling argument to duplicate it?
>     Being able to ls -lt /proc doesn't count.

This has been discussed at length in #plan9, and various people
considered it useful, the information is available through
/proc/*/status but in a very inconvenient manner. Currently the mtime
is the box boot time, which is at best useless and at worst confusing.

Knowing when a process was started is extremely convenient,
particularly for long-running or broken procs to know which version of
a program source they corresponds to. If I make a change to abaco
that depends on some new feature of webfs, I'd like to know if I need
to restart webfs or not depending on what version is running, or if I
find a broken proc and I have been hacking on that program during the
last few days, to know roughly what source tree that proc corresponds to.

Also from a security perspective it is very useful to know when a
program was (re)started in case something fishy is going on.

uriel

Re: [9fans] Re: patch/list sorry/proc-mtime

[Russ Cox]

> This has been discussed at length in #plan9

Given the things that get discussed at length in #plan9,
let's just say I don't give much weight to that argument.

> /proc/*/status but in a very inconvenient manner. Currently the mtime
> is the box boot time, which is at best useless and at worst confusing.

Actually it is the time that the kernel (specifically pc.$O or pccpu.$O etc.)
was built.  It's not useless, and it's the same as essentially every other
device file in the system.  If it were different, I would find that confusing.
Just because you're confused doesn't mean everyone is.

> Knowing when a process was started is extremely convenient,

cat >$home/bin/rc/when <<'EOF'
#!/bin/rc
for(i)
	echo $i `{date `{awk -v n'='^`{date -n} '{printf "%u", n-$6/1000}' /proc/$i/status}}
EOF

Russ

Re: [9fans] Re: patch/list sorry/proc-mtime

[quanstro]

is this the boot time, or is there something wrong with my system:

; ls -ltr /proc
d-r-xr-xr-x p 0 quanstro quanstro 0 Mar  1 14:03 /proc/1
d-r-xr-xr-x p 0 quanstro quanstro 0 Mar  1 14:03 /proc/2
[...]
; uptime
bemidji up 4 days, 09:01:36
; date
Wed Mar 29 18:09:04 CST 2006

Re: [9fans] Re: patch/list sorry/proc-mtime

[Federico G. Benavento]

[-- Attachment #1: Type: text/plain, Size: 225 bytes --]

maybe it has something to do with your timezone?

lotte% ls -ltr /proc
d-r-xr-xr-x p 0 fgb fgb 0 Feb 19 16:54 /proc/1
[...]
lotte%  uptime
lotte up 0 days, 03:34:51
lotte% date
Wed Mar 29 20:29:58 ART 2006
lotte%


[-- Attachment #2: Type: message/rfc822, Size: 3160 bytes --]

From: quanstro@quanstro.net
To: 9fans@cse.psu.edu
Subject: Re: [9fans] Re: patch/list sorry/proc-mtime
Date: Wed, 29 Mar 2006 18:13:39 -0600
Message-ID: <ecc701c413498b2083e5c5546a0e15c1@quanstro.net>

is this the boot time, or is there something wrong with my system:

; ls -ltr /proc
d-r-xr-xr-x p 0 quanstro quanstro 0 Mar  1 14:03 /proc/1
d-r-xr-xr-x p 0 quanstro quanstro 0 Mar  1 14:03 /proc/2
[...]
; uptime
bemidji up 4 days, 09:01:36
; date
Wed Mar 29 18:09:04 CST 2006

Re: [9fans] Re: patch/list sorry/proc-mtime

[Charles Forsyth]

i sometimes use the following property

term% md5sum /proc/226/text
de057049f60a4b11c7931f83bef55d74	/proc/226/text

term% md5sum /bin/cat
de057049f60a4b11c7931f83bef55d74	/bin/cat

where `text' is a reference to the whole image
including symbols, not just the text segment

Re: [9fans] Re: patch/list sorry/proc-mtime

[lucio]

> i sometimes use the following property
>
> term% md5sum /proc/226/text
> de057049f60a4b11c7931f83bef55d74	/proc/226/text
>
> term% md5sum /bin/cat
> de057049f60a4b11c7931f83bef55d74	/bin/cat
>
> where `text' is a reference to the whole image
> including symbols, not just the text segment

Is there a generic mechanism/API for executables to add to their
respective /proc nodes?  I could think of useful operations such as
recording the version and maybe the command line.  But perhaps that is
perceived as a bad idea?  I seem to recall some disparaging remarks on
this list about proctitle().

++L

Re: [9fans] Re: patch/list sorry/proc-mtime

[Gabriel Diaz]

Hello

this should be in tip of the day :-)

thanks Charles :-)

gabi

On 3/30/06, Charles Forsyth <forsyth@terzarima.net> wrote:
> i sometimes use the following property
>
> term% md5sum /proc/226/text
> de057049f60a4b11c7931f83bef55d74        /proc/226/text
>
> term% md5sum /bin/cat
> de057049f60a4b11c7931f83bef55d74        /bin/cat
>
> where `text' is a reference to the whole image
> including symbols, not just the text segment
>
>

Re: [9fans] Re: patch/list sorry/proc-mtime

[Anthony Sorace]

if we're interested in coming to consensus, i'll pipe up and say i
really like the idea of mtime being proc start time. i've wanted this
many times and only didn't add it because i (foolishly) assumed it'd
be more involved than the apparently-trivial diff. being able to 'ls
-lt /proc' is genuinely useful, and i'm curious why you'd dismiss it.
as you say, the compile time of the kernel is useful (and not really
confusing) as well, but is preserved in nearly every other device on
the system.

that said, i totally understand why #9fans discussions don't carry
much weight in the rest of the world. ;-)

Re: [9fans] Re: patch/list sorry/proc-mtime

[Russ Cox]

> if we're interested in coming to consensus, i'll pipe up and say i
> really like the idea of mtime being proc start time. i've wanted this
> many times and only didn't add it because i (foolishly) assumed it'd
> be more involved than the apparently-trivial diff. being able to 'ls
> -lt /proc' is genuinely useful, and i'm curious why you'd dismiss it.
> as you say, the compile time of the kernel is useful (and not really
> confusing) as well, but is preserved in nearly every other device on
> the system.

it duplicates information already in the status file,
and it would be the *only* kernel device file in the system
that didn't use kerndate as the mtime.  when did the
plan 9 approach become "there's more than one way to do it"?

ls -t /proc is of course indistinguishable from ls | sort -n.

ls -lt /proc just gives you some dates in sorted order.
it's useless unless you somehow have the pid->process info
mapping stored in your head.

if you want to change ps(1) to *display* the start time
of a process, i think that could be genuinely useful.
putting an extra copy on the directory mtime is not.

russ

Re: [9fans] Re: patch/list sorry/proc-mtime

[Gorka guardiola]

On 3/30/06, Russ Cox <rsc@swtch.com> wrote:
>
> it duplicates information already in the status file,
> and it would be the *only* kernel device file in the system

The metadata of each device is not giving any
information if they all have the same mtime.

> that didn't use kerndate as the mtime.  when did the
> plan 9 approach become "there's more than one way to do it"?
>
> ls -t /proc is of course indistinguishable from ls | sort -n.
>
> ls -lt /proc just gives you some dates in sorted order.
> it's useless unless you somehow have the pid->process info
> mapping stored in your head.
>
> if you want to change ps(1) to *display* the start time
> of a process, i think that could be genuinely useful.
> putting an extra copy on the directory mtime is not.
>
> russ

I am not sure about this. Static devices having all the
same mtime (kerndate) makes sense to me. Dynamic
devices in which files and directories appear and disappear
as draw or proc not having the mtime set as normal
dir/files doesnt. It may be done for simplicity,
but it breaks what one would expect. The information
is not duplicated as far as I understand. The info on the
mtime would be when the process started, but the
info inside status is how much time it has consumed
isnt it?.

All this said, I am not coding any of this, so I have voice
but not vote...
--

- curiosity sKilled the cat

Re: [9fans] Re: patch/list sorry/proc-mtime

[uriel]

> I am not sure about this. Static devices having all the
> same mtime (kerndate) makes sense to me. Dynamic
> devices in which files and directories appear and disappear
> as draw or proc not having the mtime set as normal
> dir/files doesnt.

In all fairness at least the semantics for files inside each proc
directory would not match directly what one would expect from
static file systems (eg., content can change, but mtime would always
be the creation time of the proc.) Maybe setting only the mtime of the
dirs would removes any ambiguity.

Certainly having dirs with a mtime that clearly predates their creation
is far from ideal.

> It may be done for simplicity,
> but it breaks what one would expect. The information
> is not duplicated as far as I understand. The info on the
> mtime would be when the process started, but the
> info inside status is how much time it has consumed
> isnt it?.

The information in status also includes real elapsed time in ms, that
can be used to calculate when the process started as russ pointed out.
Still this is tedious and akward and the mtime of the proc dirs seems
like a natural place to present this often useful information.

Adding the calculation to ps is another possibility, but IMHO it would
unnecessarily clutter the output, probably add an extra option and
require more code while ls -lt /proc works just fine with the mtime
approach.

In the end everything is matter of convenience and taste, but this
criticism comming from the person that instigated the tar -z
abomination is harder to take seriously.

> All this said, I am not coding any of this, so I have voice but not vote...

FYI here is the change to set the mtime, a single line of code:
+  dp->mtime = seconds() - TK2MS(MACHP(0)->ticks - p->time[TReal]) / 1000;

uriel

Re: [9fans] Re: patch/list sorry/proc-mtime

[matt]

 > it duplicates information already in the status file,
 > and it would be the *only* kernel device file in the system
 > that didn't use kerndate as the mtime.  when did the
 > plan 9 approach become "there's more than one way to do it"?

hmm I wonder when the kernel was written to the file system

ls -l /proc

ahh, now I know


 > It's not useless, and it's the same as essentially every other
 > device file in the system.

Then perhaps they convey the wrong information.
Note the "essentially every other". Not "every other".

That means you can't take a devices mtime to be kerndate, which means
making *some* of them have kerndate presents duplicate data that is also
untrustworthy and therefore meaningless and arbitrary.

If we are assigning arbitrary pieces of data as the mtime then why not
the start time of when the process was created and therefore the
creation date of /proc/pid

IMHO the creation time of /proc should be the boot time of the kernel.
Kerndate should be somewhere but it is it really relevant to the current
running state of the system ?


 > If it were different, I would find that confusing.
 > Just because you're confused doesn't mean everyone is.

That is a weak argument against the idea.

And would it really confuse you ?

Surely you would remember this conversation.

The question should surely be on the merit of the idea AND "will the
break any existing code".

Re: [9fans] Re: patch/list sorry/proc-mtime

[Russ Cox]

>  > It's not useless, and it's the same as essentially every other
>  > device file in the system.
>
> Then perhaps they convey the wrong information.
> Note the "essentially every other". Not "every other".
>
> That means you can't take a devices mtime to be kerndate, which means
> making *some* of them have kerndate presents duplicate data that is also
> untrustworthy and therefore meaningless and arbitrary.

I hedged because I wasn't 100% sure and didn't want to
look through all the drivers.  But now I have.  It's really
"every other".  There are no kernel-provided devices that
fiddle with mtime.  Not one.

> If we are assigning arbitrary pieces of data as the mtime then why not
> the start time of when the process was created and therefore the
> creation date of /proc/pid
>
> IMHO the creation time of /proc should be the boot time of the kernel.
> Kerndate should be somewhere but it is it really relevant to the current
> running state of the system ?

Maybe in someone else's humble opinion the modification time
(there is no creation time in Plan 9) of /proc should be the time of
the last call to fork or exits, i.e. the last time the directory actually
changed.  There isn't an obvious value here.

There are lots of good arguments that various information relevant
to the system should be available (when did the system boot?
when was the kernel built?  what kernel is running?  where did it
get loaded from?  etc.).  Overloading mtime for all these purposes
is misguided, as none of them are actually modification times.

An arbitrary value got chosen for the mtime on kernel devices.
It's reasonable enough and I don't see an argument to change it.
If the value was something else, I'd feel the same way: I wouldn't go
changing it to kerndate.  Same reason I didn't change NBROKEN.
If it were 8 already, I wouldn't be arguing that it should be 4.
But given that it there isn't one right answer and that the current
definitions have sufficed for the last 15+ years, I don't see why
we should change it today.

Enough already.
Russ

Re: [9fans] Re: patch/list sorry/proc-mtime

[matt]

> Maybe in someone else's humble opinion the modification time
> (there is no creation time in Plan 9) of /proc should be the time of
> the last call to fork or exits, i.e. the last time the directory actually
> changed.  There isn't an obvious value here.
>
> Overloading mtime for all these purposes
> is misguided, as none of them are actually modification times.


Sure, I don't disagree that there are any number of "good" values one
could pop into the mtime & atime. That's the point of discussing it,
"what do people think should go in the mtime & atime"

On FreeBSD they are both the current system time for /proc

My thoughts, but no action, on the subject is that it is perhaps the
concept of mtime and atime that are wrong. They seem to be terms welded
to a disk based system.

If there are plenty of values that *could* be available, shouldn't we be
open to finding a way to expose them ?

Re: [9fans] Re: patch/list sorry/proc-mtime

[rog]

i was about to say that i'd find it useful if the atime of /proc/n
reflected the last run time of a process, to make it easier to find
cpu hogs, but russ's point about ls -lt /proc's uselessness is bang
on.

so, by way of a little script to help with the above problem, which
bugs me every so often, here's "hogs", a poor man's replacement for
unix's "top" - it's like ps except that it sorts the processes by the
amount of time they've used over the last t (default 1) seconds.

it displays percentage of time used over the sample time (this might
be >100% with multiple cpus), and combines user, syscall and child
time into the second time column (to make it easier to find processes
that spawn lots of transient cpu-eating processes but don't do much
themselves); otherwise the output format is the same as for ps(1).

YMMV - i imagine others have similar scripts.

cat >$home/bin/rc/hogs <<'EOF'
#!/bin/rc
# usage: hogs [sampletime]
t=1
if(! ~ $#* 0){
	t = $1
}
{
	cd /proc
	for(i in [0-9]*){
		cat $i/status && echo $i
	}
	echo
	sleep $t
	for(i in [0-9]*){
		cat $i/status && echo $i
	}
} | awk '
	phase == 0 && /./ {
		t0[$13] = $4+$5+$7+$8;
		rt0[$13] = $6
		next
	}
	phase == 0 && /^$/ {
		phase = 1;
		next
	}
	phase != 0 {
		t1[$13] = $4+$5+$7+$8
		rt1[$13] = $6
		line[$13] = $0
	}
	END {
		for(i in line){
			n = split(line[i], p, " ");
			dt = ((t1[i] - t0[i] + 0.0) / (rt1[i] - rt0[i])) * 100
			utime = (p[4]+p[4]+p[7]+p[8]) / 1000;
			size = p[9];
			if(dt >= 1 || utime >= 1){
				printf("%-10s %8s %3d%% %4d.%.2d %7dK %-8.8s %s\n",
					p[2],
					i,
					dt,
					utime/60, utime%60,
					size,
					p[3],
					p[1]);
			}
		}
	}' |
	sort +2rn -3rn +3rn |
	sed 's/(^[^ ]+ +[0-9]+ +[0-9]+% +[0-9]+)\./\1:/'
EOF

Re: [9fans] Re: patch/list sorry/proc-mtime

[uriel]

There is also /n/sources/contrib/uriel/scripts/atop, a graphical top
in C by 20h and a version by f2f in perl, the existence of the perl
version pissed me off so much that I had to rewrite it in awk and IIRC
halve the line count in the process.  We all love to duplicate work.

Anyway, it is usually a good idea to dig around the [contrib index] wiki
page, often one finds useful things. I'm still wondering why g " and "' are
not part of the standard distribution.

uriel

> i was about to say that i'd find it useful if the atime of /proc/n
> reflected the last run time of a process, to make it easier to find
> cpu hogs, but russ's point about ls -lt /proc's uselessness is bang
> on.
>
> so, by way of a little script to help with the above problem, which
> bugs me every so often, here's "hogs", a poor man's replacement for
> unix's "top" - it's like ps except that it sorts the processes by the
> amount of time they've used over the last t (default 1) seconds.
>
> it displays percentage of time used over the sample time (this might
> be >100% with multiple cpus), and combines user, syscall and child
> time into the second time column (to make it easier to find processes
> that spawn lots of transient cpu-eating processes but don't do much
> themselves); otherwise the output format is the same as for ps(1).
>
> YMMV - i imagine others have similar scripts.
>
> cat >$home/bin/rc/hogs <<'EOF'
> #!/bin/rc
> # usage: hogs [sampletime]
> t=1
> if(! ~ $#* 0){
> 	t = $1
> }
> {
> 	cd /proc
> 	for(i in [0-9]*){
> 		cat $i/status && echo $i
> 	}
> 	echo
> 	sleep $t
> 	for(i in [0-9]*){
> 		cat $i/status && echo $i
> 	}
> } | awk '
> 	phase == 0 && /./ {
> 		t0[$13] = $4+$5+$7+$8;
> 		rt0[$13] = $6
> 		next
> 	}
> 	phase == 0 && /^$/ {
> 		phase = 1;
> 		next
> 	}
> 	phase != 0 {
> 		t1[$13] = $4+$5+$7+$8
> 		rt1[$13] = $6
> 		line[$13] = $0
> 	}
> 	END {
> 		for(i in line){
> 			n = split(line[i], p, " ");
> 			dt = ((t1[i] - t0[i] + 0.0) / (rt1[i] - rt0[i])) * 100
> 			utime = (p[4]+p[4]+p[7]+p[8]) / 1000;
> 			size = p[9];
> 			if(dt >= 1 || utime >= 1){
> 				printf("%-10s %8s %3d%% %4d.%.2d %7dK %-8.8s %s\n",
> 					p[2],
> 					i,
> 					dt,
> 					utime/60, utime%60,
> 					size,
> 					p[3],
> 					p[1]);
> 			}
> 		}
> 	}' |
> 	sort +2rn -3rn +3rn |
> 	sed 's/(^[^ ]+ +[0-9]+ +[0-9]+% +[0-9]+)\./\1:/'
> EOF