From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 11454 invoked from network); 8 Apr 2021 21:00:14 -0000 Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with ESMTPUTF8; 8 Apr 2021 21:00:14 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id 6E8369BC14; Fri, 9 Apr 2021 07:00:10 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id D91D89BB66; Fri, 9 Apr 2021 06:59:40 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=utah.edu header.i=@utah.edu header.b="TyayVBJL"; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id 2B9FE9BB66; Fri, 9 Apr 2021 06:59:37 +1000 (AEST) Received: from ipo12.cc.utah.edu (ipo12.cc.utah.edu [155.97.144.22]) by minnie.tuhs.org (Postfix) with ESMTPS id 82AA69BB65 for ; Fri, 9 Apr 2021 06:59:33 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=utah.edu; i=@utah.edu; q=dns/txt; s=UniversityOfUtah; t=1617915574; x=1649451574; h=date:from:to:cc:subject:message-id; bh=ESGDDPpLBLXyI8zOU3L1RHtKXm4ri0A2W6q9h6hNWIM=; b=TyayVBJLmR2MIFmayQHaYM6zjAtgzc46d4uJmFTQp0a8jeLPl6LhigJf z7dnWMMWx0B45leDX5dQu6R7wkxJoPbC9iwIQ1WX+I+fiF0Y/J9Hhy5tw yh7M7ks0meMwoBtHJBBK6Xdex3LhbowR58Z8X0jNOMjO78Pbc32/xQWbB VW/K/yG0uqA61NKkPcFxbUDlMI0gbSfQqNI/g63wVmsud+DL/9D2shUqq y+n2RlUM2TZl8NEvGBBrrZWVPq1ZOVEVObTI8wAf4iwO49G4TrWL07o3G mDM7FlKzzyNVQXLSaXga1DwM60b+Ccd5mUioDxgHwr6VxtDodjYCy0cgC A==; IronPort-SDR: CFCRrZ1ZxPykjm7jjcmPgRpdzQO9bl/jASPji5CJJE5Qxws1yZlt7B2F84fLhBXurgUwDP08Hb d92oQcPZaZ1EKaTOW6zWFNdbx/XVgdnaCCzTA928bHRToJbWlU8hd8JFwGlyW81kKVJPih2CQt 4f9QPCrtjliGQJd9MO8QdvcwcaSZBgftKfSEJlTeGCR5YsrWbKmQjxVinWyoIAHCGpcD8gP2NN U4DkXmGVvmQnb8Uq03RUZpI8Y9p+eqE/TckR0bMJNRPie7hqumdJy4TtHEE417n2CohLnnLwM9 4DQ= X-IronPort-AV: E=Sophos;i="5.82,207,1613458800"; d="scan'208";a="102143385" Received: from mail.math.utah.edu ([155.101.98.135]) by ipo12smtp.cc.utah.edu with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Apr 2021 14:59:31 -0600 Received: from gamma.math.utah.edu (gamma.math.utah.edu [155.101.96.20]) by mail.math.utah.edu (8.16.1/8.16.1) with ESMTP id 138KxRXk008835; Thu, 8 Apr 2021 14:59:27 -0600 (MDT) Received: from gamma.math.utah.edu (localhost [127.0.0.1]) by gamma.math.utah.edu (8.15.1/8.15.1) with ESMTP id 138KxRuk017147; Thu, 8 Apr 2021 14:59:27 -0600 Received: (from beebe@localhost) by gamma.math.utah.edu (8.15.1/8.15.1/Submit) id 138KxRTr017146; Thu, 8 Apr 2021 14:59:27 -0600 Date: Thu, 8 Apr 2021 14:59:27 -0600 From: "Nelson H. F. Beebe" To: "The Unix Heritage Society mailing list" X-US-Mail: "Department of Mathematics, 110 LCB, University of Utah, 155 S 1400 E RM 233, Salt Lake City, UT 84112-0090, USA" X-Telephone: +1 801 581 5254 X-FAX: +1 801 581 4148 X-URL: http://www.math.utah.edu/~beebe Message-ID: X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.3.8 (mail.math.utah.edu [155.101.98.135]); Thu, 08 Apr 2021 14:59:27 -0600 (MDT) Subject: [TUHS] [tuhs] fuzz testing of Unix-family utilities X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" I first encountered the fuzz-testing work of Barton Miller (Computer Sciences Department, University of Wisconsin in Madison) and his students and colleagues in their original paper on the subject An empirical study of the reliability of UNIX utilities Comm. ACM 33(12) 32--44 (December 1990) https://doi.org/10.1145/96267.96279 which was followed by Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities and Services University of Wisconsin CS TR 1264 (18 February 1995) ftp://ftp.cs.wisc.edu/pub/techreports/1995/TR1268.pdf and An Empirical Study of the Robustness of MacOS Applications Using Random Testing ACM SIGOPS Operating Systems Review 41(1) 78--86 (January 2007) https://doi.org/10.1145/1228291.1228308 I have used their techniques and tools many times in testing my own, and other, software. By chance, I found today in Web searches on another subject that Milller's group has a new paper in press in the journal IEEE Transactions on Software Engineering: The Relevance of Classic Fuzz Testing: Have We Solved This One? https://doi.org/10.1109/TSE.2020.3047766 https://arxiv.org/abs/2008.06537 https://ieeexplore.ieee.org/document/9309406 I track that journal at http://www.math.utah.edu/pub/tex/bib/ieeetranssoftwengYYYY.{bib,html} [YYYY = 1970 to 2020, by decade], but the new paper has not yet been assigned a journal issue, so I had not seen it before today. The Miller group work over 33 years has examined the reliability of common Unix tools in the face of unexpected input, and in the original work that began in 1988, they were able to demonstrate a significant failure rate in common, and widely used, Unix-family utilities. Despite wide publicity of their first paper, things have not got much better, even from reprogramming software tools in `safe' languages, such as Rust. In each paper, they analyze the reasons for the exposed bugs, and sadly, much the same reasons still exist in their latest study, and in several cases, have been introduced into code since their first work. The latest paper also contains mention of Plan 9, which moved bug-prone input line editing into the window system, and of bugs in pdftex (they say latex, but I suspect they mean pdflatex, not latex itself: pdflatex is a macro-package enhanced layer over the pdftex engine, which is a modified TeX engine). The latter are significant to me and my friends and colleagues in the TeX community, and for the TeX Live 2021 production team http://www.math.utah.edu/pub/texlive-utah/ especially because this year, Don Knuth revisited TeX and Metafont, produced new bug-fixed versions of both, plus updated anniversary editions of his five-volume Computers & Typesetting book series. His recent work is described in a new paper announced this morning: The \TeX{} tuneup of 2021 TUGboat 42(1) ??--?? February 2021 https://tug.org/TUGboat/tb42-1/tb130knuth-tuneup21.pdf Perhaps one or more list members might enjoy the exercise of applying the Barton-group fuzz tests (all of which are available from a Web site ftp://ftp.cs.wisc.edu/paradyn/fuzz/fuzz-2020/ as discussed in their paper) to 1970s and 1980s vintage Unix systems that they run on software-simulated CPUs (or rarely, on real vintage hardware). The Unix tools of those decades were generally much smaller (in lines of code), and most were written by the expert Unix pioneers at Bell Labs. It would of interest to compare the tool failure rates in vintage Unix with tool versions offered by commercial distributions, the GNU Project, and the FreeBSD community, all of which are treated in the 2021 paper. ------------------------------------------------------------------------------- - Nelson H. F. Beebe Tel: +1 801 581 5254 - - University of Utah FAX: +1 801 581 4148 - - Department of Mathematics, 110 LCB Internet e-mail: beebe@math.utah.edu - - 155 S 1400 E RM 233 beebe@acm.org beebe@computer.org - - Salt Lake City, UT 84112-0090, USA URL: http://www.math.utah.edu/~beebe/ - -------------------------------------------------------------------------------