From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-ed1-x53b.google.com (mail-ed1-x53b.google.com [IPv6:2a00:1450:4864:20::53b]) by inbox.vuxu.org (OpenSMTPD) with ESMTP id d42cf818 for ; Fri, 15 Feb 2019 08:11:21 +0000 (UTC) Received: by mail-ed1-x53b.google.com with SMTP id o14sf1603458edr.15 for ; Fri, 15 Feb 2019 00:11:21 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1550218281; cv=pass; d=google.com; s=arc-20160816; b=KwzzqFXmi2TOzlpd6MxdYkKCZbBAm2ZZJAGGw1u2SUErJMHB74mIykrg5krjZ+qS9U tUBFs4C1JPJFd6MSCma+B9oHwhWtedmANm5KU9PkwmCODEh85H5casURrJqJ7BoW4Inz trxYcKSCih2a4SeGCYJEmytC29ac2kuWl4XNYveF7FX/ekAWuerNLC8qGerq9dTcnr5O msVmaR+VlFqaKNEM/Qo65gztm4XduRxtTWRjYsZxBuVmnioAziogxMfcdOGBHWSjAoYL CdIppdHOjEkxORY4eN5HytAQqA+kK8Ttq9D/BSgsdnNy/tgc2WlqvJT9R7cKsFFDL2Es gLmg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:subject:to:from:date:sender:dkim-signature; bh=w4BYX1ZmDKR+1AimIuitMEOMkh1eYabLcgq+dK5QvUA=; b=nHR4YQFha8CICxaq+09S53qOaoXg6hIeGzU2ZBnKTmyXTVZjdnR8j9g6m6WX91XaWB citW7g+FqOEFHeUfcEwXNEn51uFfDHsbf4SghWFVHMVgQQ9y8or78g6BOJ1haOzI7Dat iScnUTCCuoP6/JYwLmML61F2b2QBOFmGsWfk32TfWgpiimU8VBQPeT5+42CCoqYZne3G uoBO9HCX9jtpueiQMr+bdRSiEdkQIK1Wtro/0KRqiEjiUwSBXbMqsn8/v5rYnTsgwdch Lx9ij/utQpnU7G6qUlOROehLLAbuFqNtG1uKyWBCiwIWQH3lE3Yh3a8RBkeJT3uNV5XL Hzfw== ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of quinq@fifth.space designates 45.32.148.28 as permitted sender) smtp.mailfrom=quinq@fifth.space DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:date:from:to:subject:message-id:in-reply-to:references :mime-version:x-original-sender:x-original-authentication-results :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=w4BYX1ZmDKR+1AimIuitMEOMkh1eYabLcgq+dK5QvUA=; b=gHdTAS+c4Hbw5qP0lzOXF1gUaYYinDTgbq0HRUVZL0MnLI0n3K3Mq1O8KLnMN2lJ+2 e0LjzHaHAr4u0N1VoVrZZc6Lts+i/YlQU4/itx7Q5c17tN6aQ7eJSDS4NcMAKkj5ndA6 kHaUTaTfl+z1mtzDmXrhEMOjMXgKsnMgyBg8wzIEcTgc2ILcq034QS2dKMNvX0vk8vtg lcGYgZgLkTLOFQWMMMXi/ocWMGFSlbT9/hKfzVEVgcPX2/uk1Xb1WxP/U0kmLDjgO2TD 1EMPzs0d9tSLbltNZ4pOz+XOE1HLgDbKN/+3kUpSNKBgbu3Ci0CdSGirTuRX+vbHBdwN IX/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:date:from:to:subject:message-id :in-reply-to:references:mime-version:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :x-spam-checked-in-group:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=w4BYX1ZmDKR+1AimIuitMEOMkh1eYabLcgq+dK5QvUA=; b=PjmyxaEsvyGCci3J6DHLFz0WMAU750VoD1l2wEihxI480f7ApVldMeqcWmufM2/1Rg Av6MEsJTiiW8BohCeojA0pRvTX3/v6VydD40im5/suCHsjJZY5Ou2tAcRFZHgAgbQHEn X/0Vse73bvCbwC+4snEdRWC3xYLo49qyPrRevgZiTCbqxnnHKk6Xow6oxJok1aWt+STI 7flMk/jAkF4K0Bp5WgKisVWQDld6wazcm6ob7pUqq1u8q3U5Y6Q9t2k05C/C/32IV0n/ hyPehjoG8xcKNp5pDDzNcNJ+rmM4fon7lMaZne7v1C5ZbkloXaEblJXPl8vzAzM7aJt4 PSLw== Sender: voidlinux@googlegroups.com X-Gm-Message-State: AHQUAuZzSONAJj8h0+L8mIacT/EILYX7UAVtTBpNZaIFF/xp1JDQNQYY eUs2L+VM0405rX31OWEuVPA= X-Google-Smtp-Source: AHgI3IYImwuOwUgNSmx/UzKowwdQw6aJ+mr3rLhGlzpZb1rw3vJN8mgQrru3JQeGK43bSZTjSdR7jw== X-Received: by 2002:a50:b130:: with SMTP id k45mr24506edd.5.1550218281425; Fri, 15 Feb 2019 00:11:21 -0800 (PST) X-BeenThere: voidlinux@googlegroups.com Received: by 2002:a50:9e41:: with SMTP id z59ls1318879ede.8.gmail; Fri, 15 Feb 2019 00:11:20 -0800 (PST) X-Received: by 2002:a50:915d:: with SMTP id f29mr1007944eda.12.1550218280764; Fri, 15 Feb 2019 00:11:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550218280; cv=none; d=google.com; s=arc-20160816; b=f6jiHyNSR+xyMwSXtx73sklRHvmBiBaR1kP35OC9B4mw4EUs9uFqGVXmWjwzAaIOyT zQO4YfhyLrwrk01h0uVcxYdDmnnDUVad7YRYsRsWF0ffxaXj6aQ+9ITJ5Y7A199d6Rxt 1dE4NFFLe8ooPFT8nymOT+txz0nPHLETFXJ819DaDHUkSQOsMCmI2PRC8Nao0k26n3qN YwAnPLSB8sc7yRENWbNqvxctlFDMTBzgxUbxCTkrraIeNBQhTRlLDlS0Cd1QwOB0zEab aP+bFq1OweKV5dq05P/jYxu7nnsEhbddDibLYMW7eFSuAau5m73frmv5du1lvUAZBmy6 smPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:to:from:date; bh=dmaF8ugga/aWSMjqlu/XpfljpVZ9wM4EH8XQWnHR8Wg=; b=0mQzNnAnD3FCLzNnaNCjEiynOzQm2HOvJmUtM7Z1y9K4eiKkRB+NR2JH0YcujQA48X ptZ2xlSneeGDCyKAvUjwehOeafDMPzc6STY5cstfzwGXJfQrXTH/1Ns/J+i0j8GCR5IB 9eOBkUQqQ0vC0vJxGDg9ve//PXKbcKGvAhIMxdSVfTDBV/uO7jHHC3jiSwlgFryy8ene 6EPnj93D9r0T8r3CPeogsvFgnOdKPPQaIY+qCeS1k27zsnOZZ2A+1pqZPNZhB6jSpdV3 N1SOyEpaEPDBHyvADHqKmiimYBbh8uMk5pv9DU+x/9j/xyizRYQTfFZDpl0fqo+uFQ9h pQdA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of quinq@fifth.space designates 45.32.148.28 as permitted sender) smtp.mailfrom=quinq@fifth.space Received: from mx0.fifth.space (fifth.space. [45.32.148.28]) by gmr-mx.google.com with ESMTPS id a12si243691edn.5.2019.02.15.00.11.20 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Feb 2019 00:11:20 -0800 (PST) Received-SPF: pass (google.com: domain of quinq@fifth.space designates 45.32.148.28 as permitted sender) client-ip=45.32.148.28; Received: from localhost.localdomain (2a01:e34:ecae:2d51:85b5:58c5:6e0d:2270 [IPv6:2a01:e34:ecae:2d51:85b5:58c5:6e0d:2270]) by fifth.space (OpenSMTPD) with ESMTPSA id c8a1b849 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO) for ; Fri, 15 Feb 2019 09:11:17 +0100 (CET) Date: Fri, 15 Feb 2019 09:11:15 +0100 From: Quentin Rameau To: voidlinux@googlegroups.com Subject: Re: epona.vuxu.org outage post mortem Message-ID: <20190215091115.1444c0c7@fifth.space> In-Reply-To: <87mumyvxxc.fsf@vuxu.org> References: <87mumyvxxc.fsf@vuxu.org> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Original-Sender: quinq@fifth.space X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of quinq@fifth.space designates 45.32.148.28 as permitted sender) smtp.mailfrom=quinq@fifth.space Precedence: list Mailing-list: list voidlinux@googlegroups.com; contact voidlinux+owners@googlegroups.com List-ID: X-Google-Group-Id: 289663804196 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Hi Leah, > My assumption was there was a runaway instance of socat running (for > unknown reasons), and I decided to kill all socat instances. My usual > tool of choice would have been `killall socat`, but as there were other > socat instances running on the machine, I only wanted to kill the port > 3722 ones. > > Lessons learned: > - The first intuition is often right, even if it's not plausible at first. > - Don't use `pkill -f` as root, at least not without careful checking > and regexp anchoring. > - If a box doesn't react to reset requests, try sending wake-on-lan to > turn it on. > - runit should reboot by default, not shutdown! Or track the origin of the issue (runaway socat instance), did you find anything? -- You received this message because you are subscribed to the Google Groups "voidlinux" group. To unsubscribe from this group and stop receiving emails from it, send an email to voidlinux+unsubscribe@googlegroups.com. To post to this group, send email to voidlinux@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/voidlinux/20190215091115.1444c0c7%40fifth.space. For more options, visit https://groups.google.com/d/optout.