Re: runit not collecting zombies

[Gerrit Pape <pape@smarden.org>]

[-- Attachment #1: Type: text/plain, Size: 1027 bytes --]

On Sat, Jun 23, 2007 at 07:42:05AM +0300, Alex Efros wrote:
> On Wed, Jun 20, 2007 at 06:35:32PM +0000, Gerrit Pape wrote:
> > Thanks for helping to try to track this down.  You can use it on top of
> > the first, the first one should generally speed up reaping zombies.
> 
> One of my servers now has ~700 zombie processes. Looks like these two
> patches don't fix this issue. :(

From reading the code, I can't see why the runit program shouldn't
collect these zombies on your system.  But I may be blind, let's see
whether reaping zombies at least every 5 seconds helps.  Can you please
apply the attached patch (it supersedes the previous patches), install
the resulting runit program into /sbin/, reboot the machine, make sure
that the new runit program is running as pid 1, and see whether zombies
are left over?  Is anything printed to the console when the zombie
problem arises?

To be sure that runit is the problem, could you boot one of your systems
into sysvinit to see if it has the same problem?

Thanks, Gerrit.

[-- Attachment #2: diff --]
[-- Type: text/plain, Size: 1488 bytes --]

diff --git a/src/runit.c b/src/runit.c
index f7d6522..6f0793d 100644
--- a/src/runit.c
+++ b/src/runit.c
@@ -143,22 +143,28 @@ int main (int argc, const char * const *argv, char * const *envp) {
     FD_SET(x.fd, &rfds);
 #endif
     for (;;) {
-      int child;
+      int r, child;
 
       sig_unblock(sig_child);
       sig_unblock(sig_cont);
       sig_unblock(sig_int);
 #ifdef IOPAUSE_POLL
-      poll(&x, 1, -1);
+      r =poll(&x, 1, 5000);
 #else
-      select(x.fd +1, &rfds, (fd_set*)0, (fd_set*)0, (struct timeval*)0);
+      r =select(x.fd +1, &rfds, (fd_set*)0, (fd_set*)0, (struct timeval*)0);
 #endif
       sig_block(sig_cont);
       sig_block(sig_child);
       sig_block(sig_int);
       
-      read(selfpipe[0], &ch, 1);
-      child =wait_nohang(&wstat);
+      while (read(selfpipe[0], &ch, 1) == 1) {}
+      while ((child =wait_nohang(&wstat)) > 0)
+        if (child == pid) break;
+      if (child == -1) {
+        strerr_warn2(WARNING, "wait_nohang, pausing: ", &strerr_sys);
+        sleep(5);
+      }
+      if ((r == 0) && (child != pid)) continue;
 
       /* reget stderr */
       if ((ttyfd =open_write("/dev/console")) != -1) {
@@ -194,7 +200,7 @@ int main (int argc, const char * const *argv, char * const *envp) {
         strerr_warn3(INFO, "leave stage: ", stage[st], 0);
         break;
       }
-      if (child > 0) {
+      if (child != 0) {
         /* collect terminated children */
         write(selfpipe[1], "", 1);
         continue;