From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.comp.sysutils.supervision.general/902 Path: news.gmane.org!not-for-mail From: Charles Duffy Newsgroups: gmane.comp.sysutils.supervision.general Subject: runit under sysvinit - coping when runsvdir dies Date: Mon, 14 Nov 2005 12:03:20 -0600 Message-ID: NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Trace: sea.gmane.org 1131991875 17076 80.91.229.2 (14 Nov 2005 18:11:15 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Mon, 14 Nov 2005 18:11:15 +0000 (UTC) Original-X-From: supervision-return-1138-gcsg-supervision=m.gmane.org@list.skarnet.org Mon Nov 14 19:11:07 2005 Return-path: Original-Received: from antah.skarnet.org ([212.85.147.14]) by ciao.gmane.org with smtp (Exim 4.43) id 1Ebim4-0005yp-OI for gcsg-supervision@gmane.org; Mon, 14 Nov 2005 19:09:48 +0100 Original-Received: (qmail 7004 invoked by uid 76); 14 Nov 2005 18:10:09 -0000 Mailing-List: contact supervision-help@list.skarnet.org; run by ezmlm List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Archive: Original-Received: (qmail 6998 invoked from network); 14 Nov 2005 18:10:09 -0000 X-Injected-Via-Gmane: http://gmane.org/ Original-To: supervision@list.skarnet.org Original-Lines: 17 Original-X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: fwext1-ext.isgenesis.com User-Agent: Mozilla Thunderbird 1.0.6 (X11/20050716) X-Accept-Language: en-us, en Original-Sender: news Xref: news.gmane.org gmane.comp.sysutils.supervision.general:902 Archived-At: I recently had a situation on a fielded server where runsvdir (from runit 1.2.3) apparently died; in any event, all the runsv processes which would typically be directly under runsvdir were instead inherited by sysvinit and running there. However, since runsvdir was respawned by sysvinit, a great deal of CPU time was being spent continuously trying to start new runsv instances under the fresh runsvdir -- attempts which failed because there were still runsv instances alive and holding open the relevant locks. I killed the old runsv instances with "runsvctrl e", and fresh children of the new runsvdir took their place -- but there are still some questions raised: - How could this have happened? The system's message log doesn't show the OOM killer taking down runsvdir or any segfault on the part of the same. - How could such situations be more gracefully handled in the future? Having the customer call and complain because their server was unusably slow was a less-than-ideal way to find out about this issue.