From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: from minnie.tuhs.org (minnie.tuhs.org [50.116.15.146]) by inbox.vuxu.org (Postfix) with ESMTP id 2559F23CE0 for ; Wed, 19 Jun 2024 07:25:12 +0200 (CEST) Received: from minnie.tuhs.org (localhost [IPv6:::1]) by minnie.tuhs.org (Postfix) with ESMTP id 475D64366A; Wed, 19 Jun 2024 15:25:10 +1000 (AEST) Received: from pasta.tip.net.au (pasta.tip.net.au [IPv6:2401:fc00:0:129::2]) by minnie.tuhs.org (Postfix) with ESMTPS id 17E1E434AB for ; Wed, 19 Jun 2024 15:25:03 +1000 (AEST) Received: from smtpclient.apple (203-7-124-164.dyn.iinet.net.au [203.7.124.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailhost.tip.net.au (Postfix) with ESMTPSA id 4W3sWb66Nvz9R2M; Wed, 19 Jun 2024 15:24:59 +1000 (AEST) From: sjenkin@canb.auug.org.au Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Date: Wed, 19 Jun 2024 15:24:47 +1000 Message-Id: <57714BB7-92B1-4C67-8C2D-4E8D03F3351F@canb.auug.org.au> To: Computer Old Farts Followers X-Mailer: Apple Mail (2.3774.600.62) Message-ID-Hash: KHQYFL6KMD3HZ73I2R6C653XWO23F72S X-Message-ID-Hash: KHQYFL6KMD3HZ73I2R6C653XWO23F72S X-MailFrom: sjenkin@canb.auug.org.au X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.6b1 Precedence: list Subject: [COFF] Unix Philosophy + Networks == Plan 9 [ LONG ] List-Id: Computer Old Farts Forum Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: This report [link at end ] about a security issue with VMware Vsphere, = stemming from the design/ architecture, resonated with me and the recent = TUHS =E2=80=9CUnix Philosophy=E2=80=9D thread. Many of the criticisms of Unix relate to not understanding it=E2=80=99s = purpose and design criteria: A platform on which to develop (other) Software. Which implies = =E2=80=98running, profiling, testing & debugging=E2=80=99 that code. Complaining that Unix tools/utilities are terse and arcane for = non-developers & testers, needing a steep Learning Curve, is the same as complaining a large truck doesn=E2=80=99t = accelerate or corner like a sports car. Plan 9, by the same core team twenty years later, addresses the same = problems with modern hardware & graphics, including with Networking. The system they developed in 1990 would=E2=80=99ve been proof against = both vSphere attacks because of its security-by-design:=20 No =E2=80=98root=E2=80=99 user, hence no =E2=80=99sudo=E2=80=99 and no complex, heavyweight RPC protocol with security flaws, = instead the simple, lightweight & secure 9P protocol. It seems Eric Raymond=E2=80=99s exposition on the =E2=80=9CUnix = Philosophy=E2=80=9D is the basis of much of the current understanding / = view. In the ESR & other works cited on Wikipedia, I see a lot about = =E2=80=9CUserland=E2=80=9D approaches,=20 nothing about the Kernel, Security by Design and innovations like = =E2=80=99shells=E2=80=99, =E2=80=98pipes=E2=80=99 and the many novel = standard tools, which is being able to Reuse standard tools and =E2=80=99stand on the = shoulders of giants=E2=80=99 [ versus constantly Reinventing the Wheel, = poorly ] ESR was always outside CSRC and from his resume, not involved with Unix = until 1983 at best. He=E2=80=99s certainly been a mover & shaker in the Linux and associated = (GNU led) Open Source community. =20 ESR baldly states "The Unix philosophy is not a formal design method=E2=80= =9D, which isn=E2=80=99t strictly untrue, but highly misleading IMHO. Nor is the self-description by members of CSRC as having =E2=80=9Cgood = taste=E2=80=9D a full and enlightening description of their process. There=E2=80=99s not a general appreciation, even in Research & Academic = circles, that =E2=80=9CSoftware is Performance Discipline=E2=80=9D, in the same way as Surgery, Rocketry, Aviation, Music, Art and physical = disciplines (dance, gymnastics, even rock climbing) are = =E2=80=9CPerformance=E2=80=9D based. It requires both Theory and Practice. If an educator hasn=E2=80=99t worked on at least one 1M LOC system, how = can they teach =E2=80=9CProgramming in the Large=E2=80=9D, the central = problem of Software Engineering? [ an aside: the problem =E2=80=9Cgolang=E2=80=9D addressed was = improving Software Engineering, not simply a language & coding. ] There=E2=80=99s a second factor common to all high-performance = disciplines, why flying has become cheaper, safer and faster since the first jet & = crashes in 1950=E2=80=99s: - good professionals deliberately improve, by learning from = mistakes & failures and (perhaps) adopting better practices, - great professionals don=E2=80=99t just =E2=80=98improve=E2=80=99= , they actively examine how & why they created Errors, Faults & Failures = and detect / remove root causes. The CSRC folk used to hate Corporate attempts at Soft Skills courses, = calling them =E2=80=9CCharm School=E2=80=9D. CSRC's deliberate and systematic learning, adaption and improvement = wasn=E2=80=99t accidental or incidental, it was the same conscious approach used by Fairchild in its early days, = the reason it quickly became the leader in Silicon devices, highly = profitable, highly valued. Noyce & Moore, and I posit CSRC too, applied the Scientific Method to = themselves and their practices, not just what their research field. IMO, this is what made CSRC unique - they were active Practitioners, = developing high-quality, highly-performant code, as well as being astute = Researchers, providing quantifiably better solutions with measurable improvements, = not prototypes or partial demonstrators. Gerard Holtzman=E2=80=99s 1127 Alumni page shows the breadth & depth of = talent that worked at CSRC. The group was unusually productive and influential. [ though I=E2=80=99ve = not seen a =E2=80=98collected works=E2=80=99 ] CSRC/1127 had a very strong culture and a very deliberate, structured = =E2=80=98process=E2=80=99=20 that naturally led to a world-changing product in 1974 from only ~30 = man-years of effort, a minor effort in Software Projects. perfective =E2=80=9Citerative design=E2=80=9D, rigorous testing, = code quality via a variation of pair-programming, collaborative design with group consultation / discussion and above all =E2=80=9Cperformant=E2=80=9D code - based first on = =E2=80=98correct=E2=80=99 and =E2=80=99secure=E2=80=99, backed by Doug McIlroy=E2=80=99s insistence on good = documentation for everything. [ It=E2=80=99s worth noting that in the original paper on the = =E2=80=9CWaterfall=E2=80=9D development process, it isn=E2=80=99t "Once = & Done=E2=80=9D, its specifically =E2=80=9Cdo it twice=E2=80=9D, ] [ the Shewhart Cycle, promoted by Deming, Plan - Do - Check - Act, was = well known in Engineering circles, known to be very Effective ] Unix - the kernel & device drivers, the filesystem, the shell, = libraries, userland and standard tools - weren=E2=80=99t done in hurry = between 1969 & 1974=E2=80=99s CACM article. It was written and rewritten many times - far more than the = =E2=80=98versions=E2=80=99, derived from the numbering of the manuals, = might suggest. Ken=E2=80=99s comment on one of his most productive days, =E2=80=9Cthrowin= g away 1,000 lines of code=E2=80=9D,=20 demonstrates this dynamic environment dominated by trials, redesign and = rewriting - backed by embedded =E2=80=98instrumentation=E2=80=99 = (profiling). Ken has also commented he had to deliberately forget all his code at one = point (maybe after 1974 or 77). He was able to remember every line of code he=E2=80=99d written, in = every file & program. I doubt that was an innate skill, even if so, it would=E2=80=99ve = improved by deliberate practice, just as in learning to play a musical = instrument. There=E2=80=99s a lot of research in Memory & Recall, all of which = documents =E2=80=98astonishing=E2=80=99 performance by =E2=80=98ordinary=E2= =80=99 people, with a only little tuition and deliberate practice. CSRC had a scientific approach to software design and coding, unlike any = I=E2=80=99ve seen in commercial practice, academic research or promoted = =E2=80=9CMethodologies=E2=80=9D. There=E2=80=99s a casual comment by Dennis in =E2=80=9CEvolution of = Unix=E2=80=9D, 1979, about rewriting the kernel, improving its = organisation and adding multiprogramming. By one person in months.. A documented, incontestable level of = productivity, 100x-1000x programmers practising mainstream = =E2=80=9Cmethodologies=E2=80=9D. Surely that performance alone would=E2=80=99ve been worthy of intensive = study as the workforce & marketplace implications are profound. Perhaps the most important watershed occurred during 1973, when = the operating system kernel was rewritten in C. =E2=80=A6 The success of this effort convinced us that C = was useful as a nearly universal tool for systems programming, instead = of just a toy for simple applications. The CSRC software evolution methodology is summed by perfectly in Baba = Brinkman=E2=80=99s Evolution Rap: "Performance, Feedback, Revision=E2=80=9D Website: ABC Science Show, 2009, 54 min audio, no transcript This is the performance Baba gave at the Darwin Festival = in Cambridge England, July 2009. = Ken also commented that they divided up the work coding, seemingly = informally but in a disciplined way,=20 so that there was only ever one time they created the same file. [ = "mis-coordination of work=E2=80=9D, Turing Award speech ] To prove they had well defined coding / naming standards and followed = them, the two 20-line files were identical=E2=80=A6 =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94 There=E2=80=99s a few things with the =E2=80=9CUnix Philosophy=E2=80=9D = that are critical and not included in the commentaries I=E2=80=99ve read = or seen quoted: - The Unix kernel was =E2=80=98conservative=E2=80=99, not = inventive or novel. It deliberately used only known, proven solutions, with = a focus on small, correct, performant. =E2=80=9CJust Worked=E2=80=9D, = not =E2=80=9CWorked, Just=E2=80=9D. Swapping was used, while Virtual Memory not implemented = because they didn=E2=80=99t know of a definitive solution. They avoided the =E2=80=9CSecond System Effect=E2=80=9D = - showing how clever they were - working as professional engineers = producing a robust, reliable, secure system. - Along with Unix (kernel, fsys, userland), CSRC developed a = high-performance high-quality Software Development culture and = methodology, The two are inseparable, IMO. - Professionals do not, can not, write non-trivial code in a = =E2=80=9COne and Done=E2=80=9D manner. Professional quality code takes = time and always evolves. It takes significant iterative improvement, including = redesign, to develop large systems,=20 with sufficient security, reliability, = maintainability and performance. [ Despite 60 years of failed =E2=80=9CBig = Bang=E2=80=9D projects using =E2=80=9COne & Done=E2=80=9D, Enterprises = persist with this idioticy, wasting billions every year ] - Unix was developed to provide CSRC with a great environment = for their own work. It never attempted to be more, but has been applied = =E2=80=98everywhere=E2=80=99. Using this platform, members of the team developed a = whole slew of important and useful tools, now taken as a given in Software Development: = editors, type settings, =E2=80=98diff=E2=80=99 and Version Control, = profile, debug, =E2=80=A6=20 This includes the computer Language Tools, now core to = every language & system. - Collaboration and Sharing, both ways, was central to the Unix = Philosophy developed at CSRC. Both within the team, within Bell Labs and other Unix = installations, notably USENIX & UCB and it=E2=80=99s ARPA-IPTO funded = CSRG. The world of Software and Code Development is clearly in two Eras, = =E2=80=9CBefore Unix=E2=80=9D and =E2=80=9CAfter=E2=80=9D. Part of this is =E2=80=9COpen Source=E2=80=9D, not just shared source = targeted for a single platform & environment, but source code = mechanically ported to new platforms. This was predicated on the original CSRC / Bell Labs attitude of Sharing = the Source=E2=80=A6 Source was shared in & out,=20 directly against the stance of the Legal Dept, intent on tightly = controlling all Intellectual Property with a view of extracting = =E2=80=9Crevenue streams=E2=80=9D from clients. Later events proved CSRC=E2=80=99s =E2=80=9CSource Code Sharing=E2=80=9D = was far more powerful and profitable than a Walled Garden approach, = endlessly reinvesting the wheel & competing, not cooperating with = others. Senior Management and the old school lawyers arguably overestimated = their marketing & product capability and wildly underestimated the evolution of computing and failed to = understand completely the PC era, with Bill Gates admonisment,=20 =E2=80=9CYou guys don=E2=80=99t get it, it=E2=80=99s all about = Volume=E2=80=9D. In 1974, Unix was described publicly in CACM. In 1977, USG then later Unix System Labs was formed to work on = and sell Unix commercially, locking day the I.P., with no free source = code. In 1984, AT&T =E2=80=98de-merged=E2=80=99, keeping Bell Labs, USL and = Western Digital - all the hardware and software to =E2=80=9CRule the = World=E2=80=9D and beat IBM. In 1994, AT&T gave up being the new IBM and sold its hardware and = software divisions. In 2004, AT&T was bought by one of its spinoff=E2=80=99s, SBC (Southern = Bell),=20 who=E2=80=99d understood Mobile Telephony (passing on to = customers savings from new technology), merged and rebranded themselves = as =E2=80=9CA&T=E2=80=9D. The =E2=80=9CUnix Wars=E2=80=9D of the 1990=E2=80=99s, where vendors = bought AT&T licenses, confusing =E2=80=9CPoint of Difference=E2=80=9D = with =E2=80=9CDifferent & Incompatible=E2=80=9D. They attempted Vendor lock-in, a monopoly tactic to create captive = markets that could be gouged. This failed for two reasons, IMO: - the software (even binaries) and tools were all portable, the = barriers to exit were low. - Unix wasn=E2=80=99t the only competitor Microsoft used C to write Windows NT and Intel-based = hardware to undercut Unix Servers & Workstations by 10x. Bill Gates understood =E2=80=98Volume=E2=80=99 and the = combined AT&T and Unix vendors didn=E2=80=99t. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D VMware by Broadcom warns of two critical vCenter flaws, plus a nasty = sudo bug VMware's security bulletin describes both of the flaws as "heap-overflow = vulnerabilities in the implementation of the DCE/RPC protocol=E2=80=9D = =E2=80=A6 DCE/RPC (Distributed Computing Environment/Remote Procedure Calls)=20 is a means of calling a procedure on a remote machine as if it were a = local machine =E2=80=93 just the ticket when managing virtual machines. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D CHM, 2019 = As Ritchie would later explain: =E2=80=9CWhat we wanted to preserve was not just a good = environment to do programming, but a system around which a fellowship = could form.=20 We knew from experience that the essence of communal computing, = as supplied from remote-access, time-shared machines,=20 is not just to type programs into a terminal instead of a = keypunch, but to encourage close communication.=E2=80=9D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Ken Thompson, 1984 Turing Award paper=20 Reflections on Trusting Trust To what extent should one trust a = statement that a program is free of Trojan horses? Perhaps it is more important to trust the people who wrote the = software. That brings me to Dennis Ritchie. Our collaboration has been a thing of beauty. In the ten years that we have worked together, I can recall only = one case of mis-coordination of work. =20 On that occasion, I discovered that we both had written the same = 20-line assembly language program. I compared the sources and was astounded to find that they = matched character-for-character. The result of our work together has been far greater than the = work that we each contributed. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The Art of Unix Programming by ESR Basics of the Unix Philosophy =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Wiki ESR Unix Philosophy=20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -- Steve Jenkin, IT Systems and Design=20 0412 786 915 (+61 412 786 915) PO Box 38, Kippax ACT 2615, AUSTRALIA mailto:sjenkin@canb.auug.org.au http://members.tip.net.au/~sjenkin