From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.comp.sysutils.supervision.general/1515 Path: news.gmane.org!not-for-mail From: Mike Buland Newsgroups: gmane.comp.sysutils.supervision.general Subject: Re: runit not collecting zombies Date: Wed, 12 Sep 2007 19:05:56 -0600 Organization: Geek Gene Message-ID: <200709121905.56606.mike@geekgene.com> References: <200709121338.54750.mike@geekgene.com> <20070912202842.GJ12043@home.power> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Trace: sea.gmane.org 1189645766 22286 80.91.229.12 (13 Sep 2007 01:09:26 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Thu, 13 Sep 2007 01:09:26 +0000 (UTC) To: supervision@list.skarnet.org Original-X-From: supervision-return-1750-gcsg-supervision=m.gmane.org@list.skarnet.org Thu Sep 13 03:09:22 2007 Return-path: Envelope-to: gcsg-supervision@gmane.org Original-Received: from antah.skarnet.org ([212.85.147.14]) by lo.gmane.org with smtp (Exim 4.50) id 1IVdCn-0000i6-6s for gcsg-supervision@gmane.org; Thu, 13 Sep 2007 03:09:17 +0200 Original-Received: (qmail 14615 invoked by uid 76); 13 Sep 2007 01:09:38 -0000 Mailing-List: contact supervision-help@list.skarnet.org; run by ezmlm List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Archive: Original-Received: (qmail 14602 invoked from network); 13 Sep 2007 01:09:37 -0000 User-Agent: KMail/1.9.6 In-Reply-To: <20070912202842.GJ12043@home.power> Content-Disposition: inline Xref: news.gmane.org gmane.comp.sysutils.supervision.general:1515 Archived-At: Quite honestly, I don't know which ones would or wouldn't. It doesn't seem likely, but it's the only thing that you've mentioned yet that seems like it could be contributing. If I were going to try to find this issue, I would start with an older set of packages, something from hardened gentoo from back when you say it worked, from April or so (I guess), and see if I can reproduce this problem. Upgrade packages and see what affects runit. Yes, they're all live servers and you can't stand to have them down or non-hardened, and that's fine. Pull an old desktop and use it, just for some testing, you shouldn't need to wait too long if you can get anything to produce zombies, and there you go. If you can get closer to pinpointing what on your list may have an effect, then it will be easier to solve. Good Luck, --Mike On Wednesday 12 September 2007 02:28:42 pm Alex Efros wrote: > Hi! > > On Wed, Sep 12, 2007 at 01:38:54PM -0600, Mike Buland wrote: > > I'm just curious, but doesn't it sound like this is the first place to > > look for the trouble? Unfortunately, as you point out, there are two > > differences between the two systems, the one that works isn't using two > > of the hardened patches, and is using a newer gcc. Have you reported > > these facts to the > > Yep. I dislike both idea to use non-hardened gcc on production servers and > even more dislike idea to upgrade to gcc-4.1.1 without ability to safely > disgrade after testing. Remember, this issue happens every ~week, so I > should wait at least 3 weeks because saying 'huh, changing gcc solved > issue'. > > I've tried to analyze this from other side. As I noted here: > http://bugs.gentoo.org/show_bug.cgi?id=190261#c1 > this issue happens at some time, and then repeated every 2-10 days. > So, looks like something was changed on all my servers 2-10 days BEFORE > this issue happens for the first time. Only changed thing was usual > upgrade for some Gentoo packages. And I know when this issue happens for > me first time: 2007-05-26. And I've logs for all package upgrades and > server reboots for that period: > > Fri Apr 21 19:18:39 2006 >>> sys-process/runit-1.5.0 > ... > Kernel 2.6.16-hardened-r11 was used from Sep 10 12:46:47 GMT 2006 > ... > Sun Sep 10 17:42:41 2006 >>> sys-devel/gcc-3.4.6-r1 > ... > Mon Dec 18 02:25:38 2006 >>> sys-libs/glibc-2.3.6-r5 > ... > reboot (2.6.16-hardened-r11) at Sat Dec 23 23:58:49 GMT 2006 > ... > Mon Jan 1 21:35:05 2007 >>> sys-devel/gcc-3.4.6-r2 > ... > Sat Mar 31 01:45:24 2007 >>> sys-devel/gcc-3.4.6-r2 > ... > Sun Apr 1 13:37:43 2007 >>> dev-lang/perl-5.8.8-r2 > Sun Apr 1 13:41:18 2007 >>> dev-lang/perl-5.8.8-r2 > Sun Apr 1 13:41:49 2007 >>> dev-perl/Net-Daemon-0.39 > Sun Apr 1 13:41:54 2007 >>> dev-perl/PlRPC-0.2018 > Sun Apr 1 13:42:09 2007 >>> dev-perl/DBI-1.53 > Sun Apr 1 13:42:26 2007 >>> dev-perl/DBD-mysql-3.0008 > Sun Apr 1 17:59:45 2007 >>> app-misc/mime-types-7 > Sun Apr 1 18:00:57 2007 >>> sys-apps/man-1.6e-r1 > Sun Apr 1 18:07:55 2007 >>> sys-libs/db-4.3.29-r2 > Sun Apr 1 18:08:07 2007 >>> app-portage/gentoolkit-0.2.3-r1 > Sun Apr 8 18:12:28 2007 >>> sys-libs/ncurses-5.6 > Sun Apr 8 18:13:15 2007 >>> sys-apps/file-4.20-r1 > Wed Apr 11 03:08:33 2007 >>> sys-apps/man-pages-2.44 > reboot (2.6.16-hardened-r11) at Fri Apr 27 21:55:13 GMT 2007 > Sun May 6 19:05:48 2007 >>> sys-apps/debianutils-2.17.5 > Sun May 6 19:08:07 2007 >>> dev-libs/apr-0.9.12 > Sun May 6 19:11:34 2007 >>> dev-util/pkgconfig-0.21-r1 > Sun May 6 19:11:54 2007 >>> sys-libs/timezone-data-2007d > Sun May 6 19:12:48 2007 >>> dev-lang/spidermonkey-1.5-r2 > Sun May 6 19:13:17 2007 >>> sys-devel/patch-2.5.9-r1 > Sun May 6 19:13:24 2007 >>> sys-apps/hdparm-6.9-r1 > Sun May 6 19:14:28 2007 >>> net-misc/rsync-2.6.9-r2 > Sun May 6 19:15:34 2007 >>> dev-libs/pth-2.0.6 > Sun May 6 19:15:37 2007 >>> sys-devel/binutils-config-1.9-r4 > Sun May 6 19:19:57 2007 >>> app-shells/bash-3.2_p15-r1 > Sun May 6 19:20:31 2007 >>> dev-util/dialog-1.1.20070227 > Sun May 6 19:20:59 2007 >>> sys-apps/man-1.6e-r3 > Sun May 6 19:22:14 2007 >>> media-libs/libpng-1.2.16 > Sun May 6 19:23:48 2007 >>> media-libs/freetype-2.1.10-r3 > Sun May 6 19:23:58 2007 >>> app-misc/ca-certificates-20070303-r1 > Sun May 6 19:26:04 2007 >>> sys-libs/readline-5.2_p2 > Sun May 6 19:27:49 2007 >>> dev-libs/libgpg-error-1.5 > Sun May 6 19:28:43 2007 >>> sys-devel/m4-1.4.9 > Sun May 6 19:30:40 2007 >>> sys-fs/e2fsprogs-1.39-r2 > Sun May 6 19:31:19 2007 >>> app-editors/nano-2.0.4 > Sun May 6 19:32:19 2007 >>> net-mail/fetchmail-6.3.8 > Sun May 6 19:32:55 2007 >>> sys-devel/flex-2.5.33-r2 > Sun May 6 19:33:25 2007 >>> sys-apps/baselayout-1.12.9-r2 > Sun May 6 19:36:02 2007 >>> sys-apps/util-linux-2.12r-r6 > Sun May 6 19:37:14 2007 >>> app-editors/vim-core-7.0.235 > Sun May 6 19:38:27 2007 >>> dev-libs/libksba-1.0.0 > Sun May 6 19:40:22 2007 >>> dev-libs/libxslt-1.1.20 > Sun May 6 19:41:04 2007 >>> sys-apps/module-init-tools-3.2.2-r3 > Sun May 6 19:47:27 2007 >>> app-editors/vim-7.0.235 > Sun May 6 19:53:14 2007 >>> sys-kernel/hardened-sources-2.6.20-r2 > Sun May 6 19:56:14 2007 >>> net-misc/curl-7.15.1-r1 > Sun May 6 20:36:14 2007 >>> dev-db/mysql-5.0.38 > Sun May 6 20:53:01 2007 >>> media-gfx/imagemagick-6.3.3 > Sun May 6 20:53:19 2007 >>> sys-devel/gcc-config-1.3.16 > Sun May 6 21:11:35 2007 >>> sys-libs/libstdc++-v3-3.3.6 > Wed May 9 15:53:39 2007 >>> media-libs/freetype-2.3.3 > Wed May 9 15:59:54 2007 >>> dev-lang/python-2.4.4 > Wed May 9 16:04:33 2007 >>> mail-mta/netqmail-1.05-r8 > Wed May 23 13:51:19 2007 >>> sys-apps/portage-2.1.2.7 > Wed May 23 13:51:45 2007 >>> sys-libs/timezone-data-2007e > Wed May 23 13:51:55 2007 >>> app-forensics/chkrootkit-0.47 > Wed May 23 13:52:28 2007 >>> sys-libs/zlib-1.2.3-r1 > Wed May 23 13:53:50 2007 >>> media-libs/libpng-1.2.18 > Wed May 23 13:56:07 2007 >>> media-libs/freetype-2.3.4-r2 > Wed May 23 14:44:45 2007 >>> dev-db/mysql-5.0.40 > Wed May 23 14:50:22 2007 >>> dev-lang/python-2.4.4-r4 > Wed May 23 14:50:24 2007 >>> app-admin/python-updater-0.2 > Wed May 23 14:51:36 2007 >>> sys-apps/util-linux-2.12r-r7 > Wed May 23 14:52:04 2007 >>> sys-apps/gradm-2.1.10.200702231759 > Wed May 23 14:59:26 2007 >>> app-crypt/gnupg-1.4.7-r1 > reboot (2.6.16-hardened-r11) at Sat May 26 10:37:41 GMT 2007 > reboot (2.6.16-hardened-r11) at Sat Jun 2 14:54:20 GMT 2007 > Sun Jun 3 13:04:10 2007 >>> app-portage/eix-0.9.1 > reboot (2.6.16-hardened-r11) at Sat Jun 9 14:56:38 GMT 2007 > Mon Jun 11 13:00:46 2007 >>> sys-process/runit-1.7.2 > reboot (2.6.16-hardened-r11) at Mon Jun 11 13:06:48 GMT 2007 > reboot (2.6.16-hardened-r11) at Sat Jun 16 03:59:03 GMT 2007 > reboot (2.6.20-hardened-r2) at Sat Jun 16 04:26:30 GMT 2007 > ... > Thu Jun 14 23:12:38 2007 >>> sys-libs/glibc-2.5-r3 > > Neither runit, nor gcc, glibc or kernel was upgraded in May (kernel > sources was unpacked in /usr/src at May 6, but it was compiled and boot > only Jun 16). > > Any ideas how upgrading THESE packages may affect zombie reaping in > already running runit? :-/