From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HTML_MESSAGE,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 5597 invoked from network); 8 Jul 2021 22:24:31 -0000 Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with ESMTPUTF8; 8 Jul 2021 22:24:31 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id F192B9CAED; Fri, 9 Jul 2021 08:24:29 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 66BCC9CA32; Fri, 9 Jul 2021 08:23:39 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=kev009.com header.i=@kev009.com header.b="WIPCLiI6"; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id 2C69E9CA32; Fri, 9 Jul 2021 08:23:38 +1000 (AEST) Received: from mail-yb1-f175.google.com (mail-yb1-f175.google.com [209.85.219.175]) by minnie.tuhs.org (Postfix) with ESMTPS id 6C5B89C88E for ; Fri, 9 Jul 2021 08:23:37 +1000 (AEST) Received: by mail-yb1-f175.google.com with SMTP id r135so11555493ybc.0 for ; Thu, 08 Jul 2021 15:23:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=LC7GpqXXPVuN067AvQRQxlhFcAji63+IWX2Klm736FM=; b=WIPCLiI6aKU4kcryTLawHK5I6O4KywMgx3thi2hXRD3aMgnYk4T+3kE2zV9ELoqoa3 0XmwHEueEb5Fvkjs1+swDA7o/cF33piynJJ03JzGbS9frWyOk5E4JrZuvHL+qxpIXSPI F3Gu+dY1yrcfjAmveXc7Bsltr3GT99A97JjPs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LC7GpqXXPVuN067AvQRQxlhFcAji63+IWX2Klm736FM=; b=iFbJgrFyROvB4R31LdRTMRRuwWzybQSPAkettNYSyu4gtJb2KmDPVg+l/bje14ysRy hn+WQDaVnEOn1Go+soq6GtVSXsZaID2aUlo4MZCfU2y7Qx5DVK5SN+QFBAtJwgHHWQv2 ovGNu3WKqVo+CZK8t4bxUxbv8GZBdoH9oxlAt/sGZxr/e4OypgK9rF7B9mhTZNjyKrZZ JLhX3Wqnod9PGj9R6r9u59p0acIevJtIvuDFmak4tdCYEOK1Tp+xBQ/rv9rQ1VDarT6S kCzVAXsA02rVFx6uYZbER9M6INvbJOprNJYsRqL6u7EH3P6bHvBb5OyiLo3y+swwEz9C JDGQ== X-Gm-Message-State: AOAM531wcmbiKcKmYq5KtqGZN8s+VzXhyS78PYHfPR7zPJJJLMQprBvt s9knqjE1a7VqySPX56+GIBBpSWBXf4W/XbldCrOjNnwsiLU= X-Google-Smtp-Source: ABdhPJxrSpy/0XcUfPOTIrgW5Eqv4DOf5PTsX8vPJbgqgY6nbE7NEVGnTMygrRVKR4ywbH6J+YoPSxxFHhx3T8YS6Rs= X-Received: by 2002:a25:d758:: with SMTP id o85mr40305372ybg.429.1625783016565; Thu, 08 Jul 2021 15:23:36 -0700 (PDT) MIME-Version: 1.0 References: <396911b232bae5938068a14fe0f7181e@firemail.de> <20210704004757.GB24671@tau1.ceti.pl> <20210705071450.GA12885@tau1.ceti.pl> <20210706231659.GA13225@tau1.ceti.pl> <20210707183222.GA9897@tau1.ceti.pl> <6b736f82-97ed-4e51-9652-e672be4e2c66@localhost> <20210708064652.GA19675@tau1.ceti.pl> In-Reply-To: <20210708064652.GA19675@tau1.ceti.pl> From: Kevin Bowling Date: Thu, 8 Jul 2021 15:23:25 -0700 Message-ID: To: Tomasz Rola Content-Type: multipart/alternative; boundary="0000000000002f3ec905c6a41b9e" Subject: Re: [TUHS] Overgrown ffox (was: The Unix shell: a 50-year view) X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: tuhs@minnie.tuhs.org Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" --0000000000002f3ec905c6a41b9e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Try typing =E2=80=9Cabout:memory=E2=80=9D into the address bar and hit meas= ure. You will see where it is all going. On Wed, Jul 7, 2021 at 11:48 PM Tomasz Rola wrote: > On Wed, Jul 07, 2021 at 08:50:51PM +0000, Michael Kj=C3=B6rling wrote: > > On 7 Jul 2021 20:32 +0200, from rtomek@ceti.pl (Tomasz Rola): > > > An excerpt from my ps: > > > > > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME > COMMAND > > > > > > xxxon 12331 12.5 20.4 5898360 2519640 ? TNsl Mar29 18278:11 > firefox-esr > > > > I'm going to stick my neck out here by saying that the VSZ and RSS > > values reported by ps, at least for Firefox, are largely meaningless. > > > > I started my usual Firefox instance, which has a handful of plugins, > > about a metric gazillion bookmarks, and has been my main web browser > > profile for years (so it probably has collected some crud over time). > > `ps auxw` reported that process as having a total RSS of a whopping > > 374 GB. > > > > It is downright _impossible_ that Firefox could actually be using that > > This is quite strange for me. Without looking at your system I can only > suspect it has something to do with multithreading. > > If I do two different commands as root, with firefox pid here > .eq. 12331, as above: > > =3D> (500 15): lsof -p 12331 | wc -l > 402 > > =3D> (500 17): lsof | awk '$2=3D=3D12331' | wc -l > 22055 > > The first column gives a name, and in second case it not always is > 'firefox'. I am yet to study manpage for lsof and play with it, but it > surely shows interesting things. > > On my system, when firefox gets killed, 'free' shows a difference - if > I recall, free mem increases by the size of rss plus all the stuff > which was opened and released from buffers. I did not pay much > attention, I assumed numbers would match and this is what they > probably did :-). > > OS on my box used to report to me as Debian, and still does, but some > years ago I have decided to skip the usual system upgrade, and after > some more time I started to upgrade various elements by hand. So it is > more like a tattered patchwork right now. But it does what I expect, > hopefully. > > [...] > > That's a _factor almost 2300x_ difference between the reported RSS, > > and the amount of memory that was actually freed up by closing the > > browser. > > Yeah, strange. > > [...] > > On modern systems, with everything from shared libraries to > > memory-mapped I/O to routine use of memory overcommitting, the > > resident set size is clearly a poor indicator of the actual amount > > of memory actively used by a complex process. > > Hard to tell - first I would like to learn where the hundred-giga rss > came from... > > -- > Regards, > Tomasz Rola > > -- > ** A C programmer asked whether computer had Buddha's nature. ** > ** As the answer, master did "rm -rif" on the programmer's home ** > ** directory. And then the C programmer became enlightened... ** > ** ** > ** Tomasz Rola mailto:tomasz_rola@bigfoot.com ** > --0000000000002f3ec905c6a41b9e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Try typing =E2=80=9Cabout:memory=E2=80=9D into the addres= s bar and hit measure.=C2=A0 You will see where it is all going.
=
On Wed= , Jul 7, 2021 at 11:48 PM Tomasz Rola <rtomek@ceti.pl> wrote:
On Wed, Jul 07, 20= 21 at 08:50:51PM +0000, Michael Kj=C3=B6rling wrote:
> On 7 Jul 2021 20:32 +0200, from rtomek@ceti.pl (Tomasz Rola):
> > An excerpt from my ps:
> >
> > USER=C2=A0 =C2=A0 =C2=A0 =C2=A0PID %CPU %MEM=C2=A0 =C2=A0 VSZ=C2= =A0 =C2=A0RSS TTY=C2=A0 =C2=A0 =C2=A0 STAT START=C2=A0 =C2=A0TIME COMMAND > >
> > xxxon=C2=A0 =C2=A0 12331 12.5 20.4 5898360 2519640 ?=C2=A0 =C2=A0= =C2=A0TNsl Mar29 18278:11 firefox-esr
>
> I'm going to stick my neck out here by saying that the VSZ and RSS=
> values reported by ps, at least for Firefox, are largely meaningless.<= br> >
> I started my usual Firefox instance, which has a handful of plugins, > about a metric gazillion bookmarks, and has been my main web browser > profile for years (so it probably has collected some crud over time).<= br> > `ps auxw` reported that process as having a total RSS of a whopping > 374 GB.
>
> It is downright _impossible_ that Firefox could actually be using that=

This is quite strange for me. Without looking at your system I can only
suspect it has something to do with multithreading.

If I do two different commands as root, with firefox pid here
.eq. 12331, as above:

=3D>=C2=A0 (500 15):=C2=A0 =C2=A0 lsof -p 12331 | wc -l
402

=3D>=C2=A0 (500 17):=C2=A0 =C2=A0lsof | awk '$2=3D=3D12331' | wc= -l
22055

The first column gives a name, and in second case it not always is
'firefox'. I am yet to study manpage for lsof and play with it, but= it
surely shows interesting things.

On my system, when firefox gets killed, 'free' shows a difference -= if
I recall, free mem increases by the size of rss plus all the stuff
which was opened and released from buffers. I did not pay much
attention, I assumed numbers would match and this is what they
probably did :-).

OS on my box used to report to me as Debian, and still does, but some
years ago I have decided to skip the usual system upgrade, and after
some more time I started to upgrade various elements by hand. So it is
more like a tattered patchwork right now. But it does what I expect,
hopefully.

[...]
> That's a _factor almost 2300x_ difference between the reported RSS= ,
> and the amount of memory that was actually freed up by closing the
> browser.

Yeah, strange.

[...]
> On modern systems, with everything from shared libraries to
> memory-mapped I/O to routine use of memory overcommitting, the
> resident set size is clearly a poor indicator of the actual amount
> of memory actively used by a complex process.

Hard to tell - first I would like to learn where the hundred-giga rss
came from...

--
Regards,
Tomasz Rola

--
** A C programmer asked whether computer had Buddha's nature.=C2=A0 =C2= =A0 =C2=A0 **
** As the answer, master did "rm -rif" on the programmer's ho= me=C2=A0 =C2=A0 **
** directory. And then the C programmer became enlightened...=C2=A0 =C2=A0 = =C2=A0 **
**=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0**
** Tomasz Rola=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mailto:tomasz_rola@bigfoot.com=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0**
--0000000000002f3ec905c6a41b9e--