From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 21386 invoked from network); 15 Jul 2021 20:56:04 -0000 Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with ESMTPUTF8; 15 Jul 2021 20:56:04 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id D68839C81D; Fri, 16 Jul 2021 06:56:03 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 8A95B9C7F1; Fri, 16 Jul 2021 06:55:43 +1000 (AEST) Received: by minnie.tuhs.org (Postfix, from userid 112) id 19D5C9C7F1; Fri, 16 Jul 2021 06:55:42 +1000 (AEST) Received: from fourwinds.com (fourwinds.com [63.64.179.162]) by minnie.tuhs.org (Postfix) with ESMTPS id 227A89C7F0 for ; Fri, 16 Jul 2021 06:55:41 +1000 (AEST) Received: from darkstar.fourwinds.com (localhost [127.0.0.1]) by fourwinds.com (8.15.2/8.15.2) with ESMTPS id 16FKtebA3043708 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT) for ; Thu, 15 Jul 2021 13:55:40 -0700 Received: from darkstar.fourwinds.com (jon@localhost) by darkstar.fourwinds.com (8.15.2/8.15.2/Submit) with ESMTP id 16FKteYd3043705 for ; Thu, 15 Jul 2021 13:55:40 -0700 Message-Id: <202107152055.16FKteYd3043705@darkstar.fourwinds.com> To: tuhs@minnie.tuhs.org From: Jon Steinhart MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-ID: <3043703.1626382539.1@darkstar.fourwinds.com> Content-Transfer-Encoding: 8bit Date: Thu, 15 Jul 2021 13:55:39 -0700 X-JON-SPAM: local delivery Subject: Re: [TUHS] Unix Shell Programming: The Next 50 Years - author's response and my response X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" Below is a response from two of the authors with my response to it inline. Not very impressed. Hopefully they'll get a clue and up their game. In any case, enough time spent on it. Jon Michael Greenberg writes: > > HotOS isn't exactly a conventional venue---you might notice that many > if not most HotOS papers don't fit the outline you've given. I'm aware of that, and have participated in many unconventional venues myself.  I wasn't saying that papers must fit that outline, but I do believe that they should contain that information.  There's a big difference between a discussion transcript and a paper; I believe that papers, especially those published under the auspices of a prestigious organization such as the ACM, should adhere to a higher standard. > I'd definitely appreciate detailed feedback on any semantic errors > we've made! Unfortunately I can't help you here; that was feedback from a reader who doesn't want to spend any more time on this. > Your summary covers much of what we imagined! > > As I understand it, the primary goals of the paper were to (a) help > other academics think about the shell as a viable area for research, (b) > highlight some work we're doing on JIT compilation, (c) make the case > for JIT approaches to the shell in general (as its well adapted to its > dynamism), and (d) explore various axes on which the shell could be > improved. It seems like we've done a good job communicating (b) to you, > but perhaps not the rest. Sorry again to disappoint. I certainly hope that you understand the primary goals of your own paper. Point-by-point:  (a)  While this is a valid point I don't understand why the paper didn't just state it in a straightforward manner. There are several common forms. One is to list issues in the introduction while explaining which one(s) will be addressed in the paper. Another is in the conclusion where authors list work still to be done.  (b)  At least for me this goal was not accomplished because there were no examples. Figure 1 by itself is insufficient given that the code used to generate the "result" is not shown. It would have been much more illuminating had the paper not only shown that code but also the optimized result. Professionals don't blithely accept magic data.  (c)  The paper failed to make this case to me for several reasons.       As I understand it, the paper is somewhat about applying JIT       compilation techniques to interconnected processes.  While most       shells include syntax to support the construction of such, it's       really independent of the shell.  For completeness, I have a vague       memory of shell implementations for non-multitasking systems that       sequentially ran pipelined programs passing intermediate results       via temporary files.  The single "result" reminds me of something that I saw at a science fair when my child was in middle school; I looked a one team's results and asked "What makes you think that a sample size of one is significant?" The lack of any benchmarks or other study results that justified the work also undermined the case. It reads more like the authors had something that they wanted to play with rather than doing serious research. The paper does not examine the percentage of shell scripts that actually benefit from JIT compilation; for all the reader may know it's such a small number that hand-optimizing just those scripts might be a better solution. I suppose that the paper fits into the apparently modern philosophy of expecting tools to fix up poorly written code so that programmers don't have to understand what they're doing.  (d)  In my opinion the paper didn't do this at all.  There was no       analysis of "the shell" showing weaknesses and an explanation       of why one particular path was taken.  And there was no discussion       of what was being done with shells to cause whatever problems you       were addressing and possibly ameliorating problems with some up-front       sanity.  Again, being a geezer I'm reminded of past events       that repeat themselves.  I joined a start-up company decades ago       that was going to speed up circuit simulation 100x by plugging       custom-designed floating-point processing boards into a multiprocessor       machine.  I managed to beat that 100x just by cleverly crafting the database and memory management code.  The fact that the company founder never verified his idea led to a big waste of resources. But, he did manage to raise venture capital which is akin to getting DARPA funds. Nikos Vasilakis writes: > > To add to Michael's points, HotOS'  "position" papers are often > intended as provocative, call-to-arms statements targeting primarily > the research community (academic and industrial research). Our key > position, which we possibly failed to communicate, is "Hey researchers > everywhere, let's do more research on the shell! (and here's why)". While provocative name-calling and false statements seem to have become the political norm in America I don't think that they're appropriate in a professional context. In my experience a call-to-arms isn't productive unless those called understand the nature of the call. I'm reminded of something that happened many decades ago; my partner asked me to proof a paper that she was writing for her masters degree.  I read it over with a confused look and asked her what she was trying to say.  She responded, and I told her to write that down and stop trying to sound like someone else.  Turned her into a much better writer.  So if the paper wanted to say "Hey researchers ..." it should have done so instead of being obtuse. To continue on this point and Michael's (a) above, I don't see a lot of value in proclaiming that research can be done.  I think that a more fruitful approach is to cover what has been done, what you're doing, and what you see but aren't doing. > For our more conventional research papers related to the shell, which > might cover your concerns about semantics, correctness, and > performance please see next. These three papers also provide important > context around the HotOS paper you read: > ... Tracking down your other work was key to understanding this paper.  It's my opinion that my having to do is illustrative of the problems with the paper. > Thank you for taking the time to read our paper and comment on it. > Could you please share the emails of everyone mentioned at the end of > your email? We are preparing a longer report on a recent shell > roundtable, and would love to get everyone's feedback! While I appreciate the offer to send the longer report, it would only be of interest if it was substantially more professional. There is no interest in reviewing work that is not clearly presented, has not been proofread and edited, includes unsubstantiated, pejorative, or meaningless statements, includes incorrect examples, or statistically irrelevant results. Likewise, there is also no interest if the homework hasn't been done to put the report in context with regards to prior art and other current work. Jon Steinghart Warner Losh John Cowan Larry McVoy John Dow Andy Kosela Clem Cole Steve Bourne does not want to give out his address