[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Doug McIlroy]

> tr -cs A-Za-z '\n' |
> tr A-Z a-z |
> sort |
> uniq -c |
> sort -rn |
> sed ${1}q
>
> This is real genius.

Not genius. Experience. In the Bentley/Knuth/McIlroy paper I said,
"[Old] Unix hands know instinctively how to solve this one in a jiffy."
While that is certainly true, the script was informed by my having
written "spell", which itself was an elaboration of a model
pioneered by Steve Johnson. By 1986, when BKM was published,
the lore was baked in: word-processing scripts in a similar
vein were stock in trade.

A very early exercise of this sort was Dennis Ritchie's
enumeration of anagrams in the unabridged Merriam-Webster.
Since the word list barely fit on the tiny disk of the time,
the job entailed unimaginable marshalling of resources. I
was mightily impressed then, and still am.

Doug

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Dan Cross]

On Sat, May 6, 2017 at 2:07 AM, Bakul Shah <bakul at bitblocks.com> wrote:

> Note that a similar approach was implemented by Iverson in
> APL. In fact a shell pipeline has a lot in common with array
> programming!
>
> Iverson in essence built a whole workshop with a set of simple
> but powerful standard tools (functions and higher order
> functions).  The same has been carried on in follow on array
> languages such as j and k and q languages.  Many of these
> array functions are directly applicable to stream processing.
> In contrast unix shells provide a very shallow glue layer.


Interesting analogy. I always think that function composition in lazy
languages like Haskell reminds me of shell pipelines. I know that Doug
swims in those waters; perhaps he will correct me if he feels that's a poor
analogy?

Carrying it forward a bit, however, I'm always amazed (and a little
appalled) at how the kids always think that they invented Rock n' Roll.
Ideas are continually recycled, but it's the rare recycler indeed who seeks
out the prior art on rediscovering one of the good ones. "Look! You can
filter and transform this data!" "...but I've been doing that at the shell
nearly every day for the past quarter of a century...." "...this one goes
to 11."

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170507/887dbd48/attachment.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Larry McVoy]

On Sat, May 06, 2017 at 03:11:32PM -0700, Steve Johnson wrote:
> And complexity is enemy number one when making something parallel...

+1 Couldn't agree more, for multiple reasons.
-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Steve Johnson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1633 bytes --]

 few more comments about style and optimization:First, the thing that
make optimized code hard to understand is that (at least when a
compiler does it) the optimized program goes through a different set
of states than the user expected from the source code.  When a loop
is unrolled, or invariant expressions moved out of a loop, or indexing
is turned into pointer manipulation, if you don't know what's
happening its mysterious.  But once you'v seen this a few times, it's
only a momentary hitch...Also, one lesson I learned from Unix is that
if you can make a program 10x faster it becomes a qualitatively
different experience.  The first version of Yacc I wrote, using the
textbook algorithms, took 20 minutes to process a 30-rule grammar.
 On the shared PDP-11, when I ran Yacc everyone else's jobs slowed to
a crawl, and people were heard to mutter "*%&#!  Johnson's running
Yacc again!"   A couple of years later, Yacc produced parsers faster
than the C compiler could compile them.Consider what using Google
would be like if it took 7 seconds rather than .7 seconds to respond.
 Probably still useful, but a different user experience.We had many
decades to be spoiled as far as program performance was concerned.
 As our tools got more bloated, faster processors and more memory hid
the true effect from most of us.  Now we have to rethink everything
to make it parallel.  And complexity is enemy number one when making
something parallel...
Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170506/0b9f37a0/attachment.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Bakul Shah]

On Sat, 06 May 2017 12:16:53 +1000 Noel Hunt <noel.hunt at gmail.com> wrote:
> 
> What you call 'composability' is nothing more than the 'software
> tools' approach.

Composable components can have a much tigher coupling.  vi,
rsync, git and gdb are s/w tools but they are not composable
in the sense of sed, grep, uniq, awk, head, tail, tr, sort,
ls, find etc.

> I recall reading an interview with Dennis Ritchie, but can no
> longer find it, where he talks about this approach versus the
> monolithic program model, as for example demonstrated by
> 'perl', which does everything. Dennis remarked that the
> 'tools' approach required careful thought (which programs to
> use, how to connect them with pipes) and so was more difficult
> to use, hence the popularity of 'perl'.

This is the "modularity" approach, where one program does one
thing well rather than doing everything (e.g IDE). But
moudlarity does not necessarily give you composability.

Also, it is not enough to have composable components; you need
a language/environment in which they can be used.

> A most beautiful example of this approach is from Doug McIlroy
> in his critique of Donald Knuth's solution to a problem posed
> by Jon Bentley in his 'Programming Pearls' column:
> 
>     Read a file of text, determine the n most frequently used
>     words, and print out a sorted list of those words along with
>     their frequencies.
> 
> Donald Knuth wrote a long, beautifully crafted program in some
> pseudo-code.  Doug McIlroy provided an alternative solution:
> 
> tr -cs A-Za-z '\n' |
> tr A-Z a-z |
> sort |
> uniq -c |
> sort -rn |
> sed ${1}q
> 
> This is real genius.

Indeed. 

Note that a similar approach was implemented by Iverson in
APL. In fact a shell pipeline has a lot in common with array
programming!

Iverson in essence built a whole workshop with a set of simple
but powerful standard tools (functions and higher order
functions).  The same has been carried on in follow on array
languages such as j and k and q languages.  Many of these
array functions are directly applicable to stream processing.
In contrast unix shells provide a very shallow glue layer.

> On Sat, May 6, 2017 at 1:37 AM, Bakul Shah <bakul at bitblocks.com> wrote:
> 
> > I think the key issue is not as much minimalism as composability. BSD
> > often prioritized convenience over composability. Each command doing one
> > thing well and doing line oriented makes them more composability. You can
> > always package up convenient combinations in a script. Plan9 has lc which
> > prints like unix ls -C but it is an rc script. Trying to achieve
> > composability can result in leaner systems.

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Steve Johnson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2603 bytes --]

In 1973, I spent 9 months at the University of Waterloo, home of
Watfor.   I taught a course in "Advanced Application Techniques" --
the language was FORTRAN, and the techniques included such things as
using libraries and handling errors in input data...

When I left for Canada, there had been a lot of discussion about
programming style at the labs, and so in my first lecture I told the
students that I would award points for style, and briefly talked about
what that was and how I would assess the style, and I did so
throughout the course.

What amazed me was that some students seemed to instantly get the
idea, and became quite good at producing excellent, clean programs,
while others were violently allergic to the whole notion that anything
"subjective" should stand between them and their grade in the class.
 I remember several interchanges with students, often angry, who had
a lot of difficulty understanding why their 35-line solution to a
problem should get a lower grade than a 15-line version (excluding
comment lines).

in fairness, I didn't (and don't) know any better way to learn style
than doing a lot of reading, a lot of writing, and getting criticized
constructively by those with more experience.  For me, a lot of what
I learned was from Stan Brown at the Labs, who read piles of my
(atrocious) FORTRAN code and repeatedly pointed out that when you
wrote a program, you were not just communicating with the computer,
but also other humans (including yourself) who would read (and perhaps
modify) the code in the future...

Steve

----- Original Message -----
From: "Doug McIlroy" <doug@cs.dartmouth.edu>
To:<lm at mcvoy.com>, <doug at cs.dartmouth.edu>, <clemc at ccc.com>
Cc:<tuhs at minnie.tuhs.org>
Sent:Fri, 05 May 2017 22:02:21 -0400
Subject:Re: [TUHS] Discuss of style and design of computer programs
from a user stand point [was dmr note on BSD's sins]

 Clem wrote, "So, I wonder do any of the schools like Darthmouth and
the like
 teach courses that study 'style' and taste in CS."

 Some time around 1970, Bob Rosen instituted a program-reading
 course at the University of Iowa, by overt analogy to the
 study of literature. Style was certainly a central concern;
 I'm not sure about the deeper matter of taste. I'm not aware
 of other examples, except perhaps for a collaboration between
 Don Kunth and a teachaer of (I think) creative writing--a
 course whose content I can't recall.

 Doug

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170505/4b496d7b/attachment.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Toby Thain]

On 2017-05-05 10:16 PM, Noel Hunt wrote:
> ...
> Donald Knuth wrote a long, beautifully crafted program in some
> pseudo-code.  Doug McIlroy provided an alternative solution:
>

Much as I love Professor Knuth, he is still doing this. He always was a 
low level coder*, and his programs are longer and more complicated than 
the high level equivalents, by the expected factors. (If anyone doubts 
it, note that he adopted C after Pascal, and many of his programs are 
linked from his web pages.) No doubt he has solid rationalisations for 
taking this approach for his purposes, but of course it's not ideal for 
all situations.

--Toby


* - Many of us will remember we were expected to understand TAOCP 
through MIX. And he started out like this:
http://www.catonmat.net/blog/donald-knuths-first-computer/

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Noel Hunt]

What you call 'composability' is nothing more than the 'software
tools' approach.

I recall reading an interview with Dennis Ritchie, but can no
longer find it, where he talks about this approach versus the
monolithic program model, as for example demonstrated by
'perl', which does everything. Dennis remarked that the
'tools' approach required careful thought (which programs to
use, how to connect them with pipes) and so was more difficult
to use, hence the popularity of 'perl'.

I'd be delighted if someone can point me to where this
interview might be.

P.S.

A most beautiful example of this approach is from Doug McIlroy
in his critique of Donald Knuth's solution to a problem posed
by Jon Bentley in his 'Programming Pearls' column:

    Read a file of text, determine the n most frequently used
    words, and print out a sorted list of those words along with
    their frequencies.

Donald Knuth wrote a long, beautifully crafted program in some
pseudo-code.  Doug McIlroy provided an alternative solution:

tr -cs A-Za-z '\n' |
tr A-Z a-z |
sort |
uniq -c |
sort -rn |
sed ${1}q

This is real genius.


On Sat, May 6, 2017 at 1:37 AM, Bakul Shah <bakul at bitblocks.com> wrote:

> I think the key issue is not as much minimalism as composability. BSD
> often prioritized convenience over composability. Each command doing one
> thing well and doing line oriented makes them more composability. You can
> always package up convenient combinations in a script. Plan9 has lc which
> prints like unix ls -C but it is an rc script. Trying to achieve
> composability can result in leaner systems.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170506/350640c1/attachment.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Doug McIlroy]

Clem wrote, "So, I wonder do any of the schools like Darthmouth and the like
teach courses that study 'style' and taste in CS."

Some time around 1970, Bob Rosen instituted a program-reading
course at the University of Iowa, by overt analogy to the
study of literature. Style was certainly a central concern;
I'm not sure about the deeper matter of taste. I'm not aware
of other examples, except perhaps for a collaboration between
Don Kunth and a teachaer of (I think) creative writing--a
course whose content I can't recall.

Doug

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Bakul Shah]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4301 bytes --]

I think the key issue is not as much minimalism as composability. BSD often prioritized convenience over composability. Each command doing one thing well and doing line oriented makes them more composability. You can always package up convenient combinations in a script. Plan9 has lc which prints like unix ls -C but it is an rc script. Trying to achieve composability can result in leaner systems.

> On May 5, 2017, at 8:20 AM, Clem Cole <clemc at ccc.com> wrote:
> 
> 
>> On Thu, May 4, 2017 at 7:14 PM, Larry McVoy <lm at mcvoy.com> wrote:
>> some of those Berkeley flags (not specifically for cat, but almost certainly including those for cat) were really quite useful. 
> 
> ​Amen!!! I think that this is key point.   What is in good taste or good style?  Doug's distain for the results of:
>                less --help | wc
> is found in bad design, a poor user interface, little fore thought,  etc.   Most of what is there has been added over Eric Scheinbrood's original  more(1) I do not believe are used that often, but some of them are of course, and you can not tell who uses what!! So how do you decide what get rid of?
> 
> How do you learn what people really need --    IMO: much of that is experience and this thing we call 'good taste.'  As opposed to what I think happened with less(1) and many others similar programs (programmers peeing on the code because they code and the source was available -- I can add this feature to it and I think it is cool.  As opposed to asking, what do really get and not having a 'master builder' arbitrating or vetting things). 
> 
> The problem we have is that we don't yet have a way of defining good taste.   One might suggest that it takes years of civilization and also that tastes do change over time.   Pike's minimalistic view (which I think is taken to the extreme in the joke about automobile dashboard on Brian Kernighan's car) sets the bar to one end and probably one of the reason why UNIX had a bad reputation, certainly by non-computer scientists, when it first appeared as being difficult.  Another extreme is systems that put in you a box and never let you do anything but what we told you do; which I find just as frighten and frustration builds there when I use them.  Clearly systems that are so noisy that can not find what really want, or need is another dimension of the same bad design.  So what to do? [more in a minute...]
> 
> Larry is right.  Many of the 'features' added to UNIX (and Linux) over time have been and are useful.   Small and simple as it was (and I really admire Ken, Dennis and the Team for its creation), but in 2017 I really don't want to run the Sixth Edition for  my day to day work anymore - which I happily did in 1977.     But the problem is, as we got 'unlimited address space' of 32 and 64 bits, and more room for more 'useful' things, we also got a great deal of rubbish and waste.    I am interpreting Doug's point about less --help | wc is that is that are so many thorns and weeds, it hard to see to see the flowers in the garden.
> 
> I'd like to observe that is that most college courses in CS I have seen talk about how to construct a programs, algorithms, structures - i.e. the mechanics of some operation.   But this discussion is about the human element.   What we feel is good or bad and how it related to how to use the program.   
> 
> I think about my friends that have degrees in literature, art and architecture.   In all cases, they spend a lot of time examining past examples, of good and bad - and thinking and discussing what makes them so.   I'm actually happy to see it was Professor McIlroy that is one of the folks taking a stand on current craziness.   I think this is only going to get better if a new crop of students that have been trained in 'good taste.'  So, I wonder do any of the schools like Darthmouth and the like teach courses that study 'style' and taste in CS.   Yes, it is a young field but we have been around long enough that we do a body of work good and bad to consider.
> 
> I think there is a book or two and a few lectures in there somewhere.   Thoughts?
> 
> Clem
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170505/d75dcba3/attachment-0001.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3777 bytes --]

On Thu, May 4, 2017 at 7:14 PM, Larry McVoy <lm at mcvoy.com> wrote:

> some of those Berkeley flags (not specifically for cat, but almost
> certainly including those for cat) were really quite useful.


​Amen!!! I think that this is key point.   What is in good taste or good
style?  Doug's distain for the results of:
               less --help | wc
is found in bad design, a poor user interface, little fore thought,  *etc.*
  Most of what is there has been added over Eric Scheinbrood's
original  more(1)
I do not believe are used that often, but some of them are of course, and
you can not tell who uses what!! So how do you decide what get rid of?

How do you learn what people really need --    IMO: much of that is
* experience *and this thing we call 'good taste.'  As opposed to what I
think happened with less(1) and many others similar programs (programmers
peeing on the code because they code and the source was available -- I can
add this feature to it and I think it is cool.  As opposed to asking,
what do really get and not having a 'master builder' arbitrating or vetting
things).

The problem we have is that we don't yet have a way of defining good taste.
  One might suggest that it takes years of civilization and also that
tastes do change over time.   Pike's minimalistic view (which I think is
taken to the extreme in the joke about automobile dashboard on Brian
Kernighan's car) sets the bar to one end and probably one of the reason why
UNIX had a bad reputation, certainly by non-computer scientists, when it
first appeared as being difficult.  Another extreme is systems that put in
you a box and never let you do anything but what we told you do; which I
find just as frighten and frustration builds there when I use them.
Clearly systems that are so noisy that can not find what really want, or
need is another dimension of the same bad design.  So what to do? [more in
a minute...]

Larry is right.  Many of the 'features' added to UNIX (and Linux) over time
have been and *are useful*.   Small and simple as it was (and I really
admire Ken, Dennis and the Team for its creation), but in 2017 I really
don't want to run the Sixth Edition for  my day to day work anymore - which
I happily did in 1977.     But the problem is, as we got 'unlimited address
space' of 32 and 64 bits, and more room for more 'useful' things, we also
got a great deal of rubbish and waste.    I am interpreting Doug's point
about less --help | wc is that is that are so many thorns and weeds, it
hard to see to see the flowers in the garden.

I'd like to observe that is that most college courses in CS I have seen
talk about how to construct a programs, algorithms, structures - i.e. the
mechanics of some operation.   But this discussion is about the human
element.   What we feel is good or bad and how it related to how to use
the program.

I think about my friends that have degrees in literature, art and
architecture.   In all cases, they spend a lot of time examining past
examples, of good and bad - and thinking and discussing what makes them so.
  I'm actually happy to see it was Professor McIlroy that is one of the
folks taking a stand on current craziness.   I think this is only going to
get better if a new crop of students that have been trained in 'good
taste.'  So, I wonder do any of the schools like Darthmouth and the like
teach courses that study 'style' and taste in CS.   Yes, it is a young
field but we have been around long enough that we do a body of work good
and bad to consider.

I think there is a book or two and a few lectures in there somewhere.
Thoughts?

Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170505/582a0bfb/attachment.html>