From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, LOTS_OF_MONEY,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from minnie.tuhs.org (minnie.tuhs.org [45.79.103.53]) by inbox.vuxu.org (OpenSMTPD) with ESMTP id bf7ee15c for ; Wed, 13 Nov 2019 13:48:25 +0000 (UTC) Received: by minnie.tuhs.org (Postfix, from userid 112) id E09769BB7D; Wed, 13 Nov 2019 23:48:24 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 590D293D97; Wed, 13 Nov 2019 23:47:59 +1000 (AEST) Received: by minnie.tuhs.org (Postfix, from userid 112) id 4211293D97; Wed, 13 Nov 2019 23:47:57 +1000 (AEST) Received: from mail.cs.Dartmouth.EDU (mail.cs.dartmouth.edu [129.170.212.100]) by minnie.tuhs.org (Postfix) with ESMTPS id AB56A93D52 for ; Wed, 13 Nov 2019 23:47:56 +1000 (AEST) Received: from tahoe.cs.Dartmouth.EDU (tahoe.cs.dartmouth.edu [129.170.212.20]) by mail.cs.Dartmouth.EDU (8.15.2/8.15.2) with ESMTPS id xADDlsO0010657 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 13 Nov 2019 08:47:54 -0500 Received: from tahoe.cs.Dartmouth.EDU (localhost.localdomain [127.0.0.1]) by tahoe.cs.Dartmouth.EDU (8.15.2/8.14.3) with ESMTP id xADDlsCi051996 for ; Wed, 13 Nov 2019 08:47:54 -0500 Received: (from doug@localhost) by tahoe.cs.Dartmouth.EDU (8.15.2/8.15.2/Submit) id xADDlsDE051995 for tuhs@tuhs.org; Wed, 13 Nov 2019 08:47:54 -0500 From: Doug McIlroy Message-Id: <201911131347.xADDlsDE051995@tahoe.cs.Dartmouth.EDU> Date: Wed, 13 Nov 2019 08:47:54 -0500 To: tuhs@tuhs.org User-Agent: Heirloom mailx 12.5 7/5/10 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: [TUHS] Happy birthday Morris worm X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" Most of this post is off topic; the conclusion is not. On the afternoon of Martin Luther King Day, 1990, AT&T's backbone network slowed to a crawl. The cause: a patch intended to save time when a switch that had taken itself off line (a rare, but routine and almost imperceptible event) rejoined the network. The patch was flawed; a lock should have been taken one instruction sooner. Bell Labs had tested the daylights out of the patch by subjecting a real switch in the lab to tortuously heavy, but necessarily artificial loads. It may also have been tested on a switch in the wild before the patch was deployed throughout the network, but that would not have helped. The trouble was that a certain sequence of events happening within milliseconds on calls both ways between two heavily loaded switches could evoke a ping-pong of the switches leaving and rejoining the network. The phenomenon was contagious because of the enhanced odds of a third switch experiencing the bad sequence with a switch that was repeatedly taking itself off line. The basic problem (and a fortiori the contagion) had not been seen in the lab because the lab had only one of the multimillion-dollar switches to play with. The meltdown was embarrassing, to say the least. Yet nobody ever accused AT&T of idiocy for not first testing on a private network this feature that was inadvertently "designed to compromise" switches. Doug