The Unix Heritage Society mailing list
 help / color / mirror / Atom feed
* Re: [TUHS] Unix Shell Programming: The Next 50 Years - author's response and my response
@ 2021-07-15 20:55 Jon Steinhart
  0 siblings, 0 replies; only message in thread
From: Jon Steinhart @ 2021-07-15 20:55 UTC (permalink / raw)
  To: tuhs

Below is a response from two of the authors with my response to it
inline.  Not very impressed.  Hopefully they'll get a clue and up
their game.  In any case, enough time spent on it.

Jon



Michael Greenberg writes:
>
> HotOS isn't exactly a conventional venue---you might notice that many
> if not most HotOS papers don't fit the outline you've given.

I'm aware of that, and have participated in many unconventional venues
myself.  I wasn't saying that papers must fit that outline, but I do
believe that they should contain that information.  There's a big
difference between a discussion transcript and a paper; I believe that
papers, especially those published under the auspices of a prestigious
organization such as the ACM, should adhere to a higher standard.

> I'd definitely appreciate detailed feedback on any semantic errors
> we've made!

Unfortunately I can't help you here; that was feedback from
a reader who doesn't want to spend any more time on this.

> Your summary covers much of what we imagined!
>
> As I understand it, the primary goals of the paper were to (a) help
> other academics think about the shell as a viable area for research, (b)
> highlight some work we're doing on JIT compilation, (c) make the case
> for JIT approaches to the shell in general (as its well adapted to its
> dynamism), and (d) explore various axes on which the shell could be
> improved. It seems like we've done a good job communicating (b) to you,
> but perhaps not the rest. Sorry again to disappoint.

I certainly hope that you understand the primary goals of your own paper.

Point-by-point:

 (a)  While this is a valid point I don't understand why the paper didn't
      just state it in a straightforward manner.  There are several common
      forms.  One is to list issues in the introduction while explaining
      which one(s) will be addressed in the paper.  Another is in the
      conclusion where authors list work still to be done.

 (b)  At least for me this goal was not accomplished because there were no
      examples.  Figure 1 by itself is insufficient given that the code
      used to generate the "result" is not shown.  It would have been much
      more illuminating had the paper not only shown that code but also the
      optimized result.  Professionals don't blithely accept magic data.

 (c)  The paper failed to make this case to me for several reasons.
      As I understand it, the paper is somewhat about applying JIT
      compilation techniques to interconnected processes.  While most
      shells include syntax to support the construction of such, it's
      really independent of the shell.  For completeness, I have a vague
      memory of shell implementations for non-multitasking systems that
      sequentially ran pipelined programs passing intermediate results
      via temporary files.  The single "result" reminds me of something
      that I saw at a science fair when my child was in middle school;
      I looked a one team's results and asked "What makes you think that
      a sample size of one is significant?"  The lack of any benchmarks
      or other study results that justified the work also undermined the
      case.  It reads more like the authors had something that they wanted
      to play with rather than doing serious research.  The paper does not
      examine the percentage of shell scripts that actually benefit from
      JIT compilation; for all the reader may know it's such a small number
      that hand-optimizing just those scripts might be a better solution.
      I suppose that the paper fits into the apparently modern philosophy
      of expecting tools to fix up poorly written code so that programmers
      don't have to understand what they're doing.

 (d)  In my opinion the paper didn't do this at all.  There was no
      analysis of "the shell" showing weaknesses and an explanation
      of why one particular path was taken.  And there was no discussion
      of what was being done with shells to cause whatever problems you
      were addressing and possibly ameliorating problems with some up-front
      sanity.  Again, being a geezer I'm reminded of past events
      that repeat themselves.  I joined a start-up company decades ago
      that was going to speed up circuit simulation 100x by plugging
      custom-designed floating-point processing boards into a multiprocessor
      machine.  I managed to beat that 100x just by cleverly crafting the
      database and memory management code.  The fact that the company founder
      never verified his idea led to a big waste of resources.  But, he did
      manage to raise venture capital which is akin to getting DARPA funds.

Nikos Vasilakis writes:
>
> To add to Michael's points, HotOS'  "position" papers are often
> intended as provocative, call-to-arms statements targeting primarily
> the research community (academic and industrial research). Our key
> position, which we possibly failed to communicate, is "Hey researchers
> everywhere, let's do more research on the shell! (and here's why)".

While provocative name-calling and false statements seem to have become
the political norm in America I don't think that they're appropriate in
a professional context.

In my experience a call-to-arms isn't productive unless those called
understand the nature of the call.  I'm reminded of something that happened
many decades ago; my partner asked me to proof a paper that she was writing
for her masters degree.  I read it over with a confused look and asked her
what she was trying to say.  She responded, and I told her to write that
down and stop trying to sound like someone else.  Turned her into a much
better writer.  So if the paper wanted to say "Hey researchers ..." it
should have done so instead of being obtuse.

To continue on this point and Michael's (a) above, I don't see a lot of
value in proclaiming that research can be done.  I think that a more
fruitful approach is to cover what has been done, what you're doing,
and what you see but aren't doing.

> For our more conventional research papers related to the shell, which
> might cover your concerns about semantics, correctness, and
> performance please see next. These three papers also provide important
> context around the HotOS paper you read:
> ...

Tracking down your other work was key to understanding this paper.  It's
my opinion that my having to do is illustrative of the problems with the
paper.

> Thank you for taking the time to read our paper and comment on it.
> Could you please share the emails of everyone mentioned at the end of
> your email? We are preparing a longer report on a recent shell
> roundtable, and would love to get everyone's feedback!

While I appreciate the offer to send the longer report, it would only be
of interest if it was substantially more professional.  There is no interest
in reviewing work that is not clearly presented, has not been proofread
and edited, includes unsubstantiated, pejorative, or meaningless statements,
includes incorrect examples, or statistically irrelevant results.  Likewise,
there is also no interest if the homework hasn't been done to put the
report in context with regards to prior art and other current work.

Jon Steinghart <jsacm@fourwinds.com>
Warner Losh <imp@bsdimp.com>
John Cowan <cowan@ccil.org>
Larry McVoy <lm@mcvoy.com>
John Dow <jmd@nelefa.org>
Andy Kosela <akosela@andykosela.com>
Clem Cole <clemc@ccc.com>
Steve Bourne does not want to give out his address

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2021-07-15 20:56 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-15 20:55 [TUHS] Unix Shell Programming: The Next 50 Years - author's response and my response Jon Steinhart

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).