From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, MIME_QP_LONG_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: from minnie.tuhs.org (minnie.tuhs.org [IPv6:2600:3c01:e000:146::1]) by inbox.vuxu.org (Postfix) with ESMTP id 682C925E13 for ; Mon, 20 May 2024 16:09:31 +0200 (CEST) Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id 930AC43B23; Tue, 21 May 2024 00:09:27 +1000 (AEST) Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) by minnie.tuhs.org (Postfix) with ESMTPS id B1D8E43B22 for ; Tue, 21 May 2024 00:09:22 +1000 (AEST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.west.internal (Postfix) with ESMTP id 1A90F18000AC for ; Mon, 20 May 2024 10:09:22 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 20 May 2024 10:09:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=serissa.com; h= cc:content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:message-id:mime-version:reply-to:subject :subject:to:to; s=fm3; t=1716214161; x=1716300561; bh=sCJp67yBHO vrQcxkm3RwuzuQb5FaZHEwDVe9uKZ21Xw=; b=kyut/6+PHK/skc22kIS+Oau0HU UIwxoiUIcaX3FNvtWMeCLVshKorxqhjCwFTeBbUj3BWNIepySFEDzBQmyr95B3Az yP2ZIr9TAqxCVwfUntNP1S4/hpgJuqBIJzLxTdgU6+y1GHNAyPaCv+trkP/IgjND osaDMeIg7PVqyDinkNvlu46XWetPpJC0+m49c/c4gOmJ9DajSyOgIVVDt0eLHXLd /acqkhJsWs3l04nxY8wVEwssPuZkTPWH4mYt4jfO7c9S/ZJwkdd9bnIhQQQ2OeCl j3VXin9+5C4Ry5oh2OC3CXrCVv6CitlTkWDFxwtyH5Yihb+uJNJbzGSxpazw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:message-id:mime-version:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716214161; x=1716300561; bh=sCJp67yBHOvrQcxkm3RwuzuQb5Fa ZHEwDVe9uKZ21Xw=; b=DZUw0ufbBCWeYycuhK6SMZqx/Qp8Ux3ntrIANLw92XNX OS07mDckXYcjYsABhMB2pj6lHm7Xu53nRWdbDWtUtg95UcRuuyezHTXitD+iY69r d8U0WLeTcnb1V7q3MrtlsxdSME8JPhx6Y0BvwnEZUtH8S+zzAvtBwXCPGFmY6nsh SCdahHs5wpVtSyr59VRA9fqgb7ZfSzaeA+B1eZyaVEnrRRhmw4G33yKbq8sNTTqn G0Ijx1ffchPQxE1HDxbkvUhuHvyoe71+CBrADuNMzjFbMTeaG26FBcSwhiWBNMR4 bRjLLPQfmkRs6BeEqskA3exRI3f/GcWX7F8ZuYMBxQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeitddggeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptgfghfgguffkfffvofesrgejmherhhdtjeenucfhrhhomhepufgvrhhishhs rgcuoehsthgvfigrrhhtsehsvghrihhsshgrrdgtohhmqeenucggtffrrghtthgvrhhnpe ejgeekhfeivdevheffieeftefhgeeuhfeljeeugfetffduteeftdeugeetfeduieenucff ohhmrghinhepshifrghrthhhmhhorhgvrdgvughunecuvehluhhsthgvrhfuihiivgeptd enucfrrghrrghmpehmrghilhhfrhhomhepshhtvgifrghrthesshgvrhhishhsrgdrtgho mh X-ME-Proxy: Feedback-ID: ibf61458d:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 20 May 2024 10:09:21 -0400 (EDT) Content-Type: multipart/alternative; boundary=Apple-Mail-24026B9F-7528-4AD5-B80A-6798340E6E3E Content-Transfer-Encoding: 7bit From: Serissa Mime-Version: 1.0 (1.0) Message-Id: <51CC9A0D-122C-4A3D-8BAF-C249489FB817@serissa.com> Date: Mon, 20 May 2024 10:09:11 -0400 To: TUHS main list X-Mailer: iPad Mail (21F84) Message-ID-Hash: 2NUOLPCBQZHKBTYIL4BIRFUAJYXZIDOK X-Message-ID-Hash: 2NUOLPCBQZHKBTYIL4BIRFUAJYXZIDOK X-MailFrom: stewart@serissa.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [TUHS] Re: A fuzzy awk. List-Id: The Unix Heritage Society mailing list Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --Apple-Mail-24026B9F-7528-4AD5-B80A-6798340E6E3E Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Well this is obviously a hot button topic. AFAIK I was nearby when fuzz-tes= ting for software was invented. I was the main advocate for hiring Andy Payn= e into the Digital Cambridge Research Lab. One of his little projects was a= thing that generated random but correct C programs and fed them to differen= t compilers or compilers with different switches to see if they crashed or g= enerated incorrect results. Overnight, his tester filed 300 or so bug repor= ts against the Digital C compiler. This was met with substantial pushback, b= ut it was a mostly an issue that many of the reports traced to the same unde= rlying bugs. Bill McKeemon expanded the technique and published "Differential Testing of S= oftware" https://www.cs.swarthmore.edu/~bylvisa1/cs97/f13/Papers/Differentia= lTestingForSoftware.pdf Andy had encountered the underlying idea while working as an intern on the A= lpha processor development team. Among many other testers, they used an arc= hitectural tester called REX to generate more or less random sequences of in= structions, which were then run through different simulation chains (functio= nal, RTL, cycle-accurate) to see if they did the same thing. Finding user-a= ccessible bugs in hardware seems like a good thing. The point of generating correct programs (mentioned under the term LangSec h= ere) goes a long way to avoid irritating the maintainers. Making the test c= ases short is also maintainer-friendly. The test generator is also in a pos= ition to annotate the source with exactly what it is supposed to do, which i= s also helpful. -L --Apple-Mail-24026B9F-7528-4AD5-B80A-6798340E6E3E Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Well this is obviously a h= ot button topic.  AFAIK I was nearby when fuzz-testing for software was= invented. I was the main advocate for hiring Andy Payne into the Digital Ca= mbridge Research Lab.  One of his little projects was a thing that gene= rated random but correct C programs and fed them to different compilers or c= ompilers with different switches to see if they crashed or generated incorre= ct results.  Overnight, his tester filed 300 or so bug reports against t= he Digital C compiler.  This was met with substantial pushback, but it w= as a mostly an issue that many of the reports traced to the same underlying b= ugs.

Bill McKeemon expanded the technique and published "D= ifferential Testing of Software" https://www= .cs.swarthmore.edu/~bylvisa1/cs97/f13/Papers/DifferentialTestingForSoftware.= pdf

Andy had encountered the underlying idea while working as an i= ntern on the Alpha processor development team.  Among many other tester= s, they used an architectural tester called REX to generate more or less ran= dom sequences of instructions, which were then run through different simulat= ion chains (functional, RTL, cycle-accurate) to see if they did the same thi= ng.  Finding user-accessible bugs in hardware seems like a good thing.<= div>
The point of generating correct programs (mentioned under the t= erm LangSec here) goes a long way to avoid irritating the maintainers.  = ;Making the test cases short is also maintainer-friendly.  The test gen= erator is also in a position to annotate the source with exactly what it is s= upposed to do, which is also helpful.

-L


= --Apple-Mail-24026B9F-7528-4AD5-B80A-6798340E6E3E--