Re: [9fans] output of ps(1) memory usage

Re: [9fans] output of ps(1) memory usage

[YAMANASHI Takeshi]

> /proc/pid/segment shows you virtual memory allocation,
> not physical.

I still had mistaken without your explanation.
Segment gives virtual memory information and `ps' physical.
I feel fine now.
--
YAMANASHI Takeshi

Re: [9fans] output of ps(1) memory usage

[Russ Cox]

> I still had mistaken without your explanation.
> Segment gives virtual memory information and `ps' physical.
> I feel fine now.

Everything gives virtual memory information.
Nothing gives physical memory information,
except /dev/swap which only gives you totals.

Russ

Re: [9fans] output of ps(1) memory usage

[YAMANASHI Takeshi]

> > Is there an easy way to determine the amount of memory
> > really inuse (or physically mapped?) within a process?
>
> cat /proc/<pid>/segment.  The first number is the

Thank you, all.

I have mistakenly thought that the output of ps/segment
concerns about the virtual memory, not physically mapped.

Now that I understand the ps/segment output correctly,
I revise my question as follows:

Is there a way to determine the amount of swapped pages
of a process?  I just want to complete the equation,
	virutal memory usage of a process =
	  swapped or not paged in + physically mapped
--
YAMANASHI Takeshi

Re: [9fans] output of ps(1) memory usage

[Russ Cox]

/proc/pid/segment shows you virtual memory allocation,
not physical.  There is no way to find out which pages
in a process are swapped and which are not at a per-page
granularity.  Of course, if /dev/swap tells you nothing is
swapped out, then nothing is swapped out.  It is also hard
to tell what is not yet paged in.

You could walk around in the kernel's page tables,
but that's not portable and not very clean.

Russ

Re: [9fans] output of ps(1) memory usage

[okamoto]

[-- Attachment #1: Type: text/plain, Size: 74 bytes --]

Yeah!
I found huge dark blue region under the page window. ☺

Kenji

[-- Attachment #2: Type: message/rfc822, Size: 2050 bytes --]

From: "Russ Cox" <rsc@plan9.bell-labs.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] output of ps(1) memory usage
Date: Thu, 20 Feb 2003 21:37:00 -0500
Message-ID: <724f71d54fac61e0a04f73fcf0691bf8@plan9.bell-labs.com>

The picture can be useful sometimes, but
more useful is the text output (drop the -b flag),
which will give you acid commands to list
the places where the leaked blocks were allocated.

Re: [9fans] output of ps(1) memory usage

[David Presotto]

> Is there an easy way to determine the amount of memory
> really inuse (or physically mapped?) within a process?

cat /proc/<pid>/segment.  The first number is the
range, the second is the reference count.  The ref will help
you figure out how many other procs are using the
segment.  For example:


presotto@presto-home% ps|grep rio
presotto         14    0:00   0:03     752K Rendez   rio
presotto        113    0:00   0:00     752K Pread    rio
presotto        114    0:00   0:00     752K Pread    rio
presotto        115    0:00   0:00     752K Rendez   rio
presotto        116    0:00   0:00     752K Pread    rio
presotto        117    0:00   0:00     752K Pread    rio
presotto        177    0:00   0:00     752K Await    rio
presotto@presto-home% cat /proc/113/segment
Stack     7efff000 7ffff000    1
Text   R  00001000 0002d000    7
Data      0002d000 00036000    7
Bss       00036000 000bd000    7

There are 7 rio's...

Re: [9fans] output of ps(1) memory usage

[okamoto]

[-- Attachment #1: Type: text/plain, Size: 537 bytes --]

Wao!

I did by the command of
term% leak -b marsv | rc | page
and waited about 2 minutes, I got a wonderfull image of memory
usage.

I have 2/3 of dark blue region, separated into major three parts,
and 1/3 of yellow region, and very very small but many parts od
bright blue, and small amounts of dark or bright red (not easy to
distinguish them).   So, the main anxious of mine, that is large
part of red region, is disappeared.

I appreciate the developpers of Plan 9 to provide us such a nice
tool of leak(1).

Kenji

[-- Attachment #2: Type: message/rfc822, Size: 1925 bytes --]

From: "Russ Cox" <rsc@plan9.bell-labs.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] output of ps(1) memory usage
Date: Thu, 20 Feb 2003 21:08:02 -0500
Message-ID: <e0055b32a4d103d81f290a0b8ef6e2b8@plan9.bell-labs.com>

cd /sys/src/libc/port
mk pool.acid

Then try it.

Re: [9fans] output of ps(1) memory usage

[Russ Cox]

The picture can be useful sometimes, but
more useful is the text output (drop the -b flag),
which will give you acid commands to list
the places where the leaked blocks were allocated.

Re: [9fans] output of ps(1) memory usage

[YAMANASHI Takeshi]

> cat /dev/swap to see how much memory is really inuse.

Is there an easy way to determine the amount of memory
really inuse (or physically mapped?) within a process?
--
YAMANASHI Takeshi

Re: [9fans] output of ps(1) memory usage

[Russ Cox]

ps shows how much is physically mapped within
a process, and reading /proc/pid/segment will
show you exactly where it is mapped.

the hard part comes when you want to figure out
the total for a set of processes, because figuring
out the intersection is not so easy.

for procs sharing memory, only the stacks are
different, and stacks tend to be only a page or two,
so as rob said, what's listed in ps applies to the
collection of procs pretty well.

Re: [9fans] output of ps(1) memory usage

[Scott Schwartz]

| Is there an easy way to determine the amount of memory
| really inuse (or physically mapped?) within a process?

You could look at the map file.  Maybe ps could show more of that
information, to have a nicer interface.

Re: [9fans] output of ps(1) memory usage

[okamoto]

Then, I tried to use leak(1) following the man page.

term% leak -s marsv
leak -s 563 564 565
term% leak -b marsv|rc|page
/sys/lib/acid/pool:1: (error) pushfile: /sys/src/libc/port/pool.acid: '/sys/src/libc/port/pool.acid' does not exist
/sys/lib/acid/leak:4: (error) pushfile: /sys/src/libc/port/pool.acid: '/sys/src/libc/port/pool.acid' does not exist
<stdin>:2: (error) no function leakdump
aux/acidleak: no allocated data region
page: short read reading <nil>

I must be doing something fool...

Kenji

Re: [9fans] output of ps(1) memory usage

[Russ Cox]

cd /sys/src/libc/port
mk pool.acid

Then try it.

Re: [9fans] output of ps(1) memory usage

[okamoto]

I'm now fighting against possible memory leak of marsv.

>> Is the output of ps(1) on the amount of used memory will not
>> shrink natural?
>
> I think you're asking whether the amount of memory
> used by a process ever shrinks.  The answer to that is no.

I should have been more clear.    The background of my question
was that I first showed a image(206554 bytes), and the ps(1)
results are:
okamoto         478    0:07   0:00   35988K Rendez   marsv
okamoto         479    0:00   0:00   35988K Pread    marsv
okamoto         480    0:00   0:00   35988K Pread    marsv
then, cleared it, and read again a smaller image(21698 bytes),
and the results are:
okamoto         478    0:15   0:00   35988K Rendez   marsv
okamoto         479    0:00   0:00   35988K Pread    marsv
okamoto         480    0:00   0:00   35988K Pread    marsv

This was the reason why I asked the previous question.

Kenji

[9fans] output of ps(1) memory usage

[okamoto]

Is the output of ps(1) on the amount of used memory will not
shrink natural?   If so, what is the reason?
I'm seeing this for threaded programs such as ame and marsv.

Kenji

Re: [9fans] output of ps(1) memory usage

[Russ Cox]

> Is the output of ps(1) on the amount of used memory will not
> shrink natural?

I think you're asking whether the amount of memory
used by a process ever shrinks.  The answer to that is no.

> If so, what is the reason?

Almost all programs allocate memory by increasing
the size of the data segment (see brk(2)).  It's possible
to reduce the size, but first you have to get all the good
data out of the area you're about to give back.  The standard
malloc/free interface does not attempt to give anything back.
It probably should, but it's not much of an issue for us --
once we clear out our memory leaks programs tend to reach
a steady size and then not grow further.

> I'm seeing this for threaded programs such as ame and marsv.

Leak(1) will check a running program for memory leaks,
in case you are concerned about that.

Russ

Re: [9fans] output of ps(1) memory usage

[rob pike, esq.]

Also, in case it's not obvious, when you see something like
%% ps | grep rio
rob              13    0:04   0:46    1280K Rendez   rio
rob             133    0:00   0:00    1280K Pread    rio
rob             134    0:00   0:00    1280K Pread    rio
rob             135    0:00   0:00    1280K Rendez   rio
rob             136    0:00   0:00    1280K Pread    rio
rob             137    0:01   0:01    1280K Pread    rio
rob             138    0:00   0:00    1280K Await    rio
rob             153    0:00   0:00    1280K Await    rio
rob             193    0:00   0:00    1280K Await    rio
rob             863    0:00   0:00    1280K Await    rio
rob            1721    0:00   0:00    1280K Await    rio
rob            1828    0:00   0:00    1280K Await    rio
rob            2191    0:00   0:00    1280K Await    rio
%%
that 1280K is almost entirely a shared single copy of
the memory.  The only part not in common is a single
page (most likely) of per-process stack.

In other words, don't add up the memory column of
ps to decide how much memory is in use.

-rob

Re: [9fans] output of ps(1) memory usage

[David Presotto]

[-- Attachment #1: Type: text/plain, Size: 115 bytes --]

cat /dev/swap to see how much memory is really inuse.
The first number is the number of pages used by user
procs.

[-- Attachment #2: Type: message/rfc822, Size: 1445 bytes --]

From: okamoto@granite.cias.osakafu-u.ac.jp
To: 9fans@cse.psu.edu
Subject: [9fans] output of ps(1) memory usage
Date: Fri, 21 Feb 2003 09:51:14 +0900
Message-ID: <fb1108845028587edcc453aa0ff26a26@granite.cias.osakafu-u.ac.jp>

Is the output of ps(1) on the amount of used memory will not
shrink natural?   If so, what is the reason?
I'm seeing this for threaded programs such as ame and marsv.

Kenji