[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Clem Cole]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3777 bytes --]

On Thu, May 4, 2017 at 7:14 PM, Larry McVoy <lm at mcvoy.com> wrote:

> some of those Berkeley flags (not specifically for cat, but almost
> certainly including those for cat) were really quite useful.


​Amen!!! I think that this is key point.   What is in good taste or good
style?  Doug's distain for the results of:
               less --help | wc
is found in bad design, a poor user interface, little fore thought,  *etc.*
  Most of what is there has been added over Eric Scheinbrood's
original  more(1)
I do not believe are used that often, but some of them are of course, and
you can not tell who uses what!! So how do you decide what get rid of?

How do you learn what people really need --    IMO: much of that is
* experience *and this thing we call 'good taste.'  As opposed to what I
think happened with less(1) and many others similar programs (programmers
peeing on the code because they code and the source was available -- I can
add this feature to it and I think it is cool.  As opposed to asking,
what do really get and not having a 'master builder' arbitrating or vetting
things).

The problem we have is that we don't yet have a way of defining good taste.
  One might suggest that it takes years of civilization and also that
tastes do change over time.   Pike's minimalistic view (which I think is
taken to the extreme in the joke about automobile dashboard on Brian
Kernighan's car) sets the bar to one end and probably one of the reason why
UNIX had a bad reputation, certainly by non-computer scientists, when it
first appeared as being difficult.  Another extreme is systems that put in
you a box and never let you do anything but what we told you do; which I
find just as frighten and frustration builds there when I use them.
Clearly systems that are so noisy that can not find what really want, or
need is another dimension of the same bad design.  So what to do? [more in
a minute...]

Larry is right.  Many of the 'features' added to UNIX (and Linux) over time
have been and *are useful*.   Small and simple as it was (and I really
admire Ken, Dennis and the Team for its creation), but in 2017 I really
don't want to run the Sixth Edition for  my day to day work anymore - which
I happily did in 1977.     But the problem is, as we got 'unlimited address
space' of 32 and 64 bits, and more room for more 'useful' things, we also
got a great deal of rubbish and waste.    I am interpreting Doug's point
about less --help | wc is that is that are so many thorns and weeds, it
hard to see to see the flowers in the garden.

I'd like to observe that is that most college courses in CS I have seen
talk about how to construct a programs, algorithms, structures - i.e. the
mechanics of some operation.   But this discussion is about the human
element.   What we feel is good or bad and how it related to how to use
the program.

I think about my friends that have degrees in literature, art and
architecture.   In all cases, they spend a lot of time examining past
examples, of good and bad - and thinking and discussing what makes them so.
  I'm actually happy to see it was Professor McIlroy that is one of the
folks taking a stand on current craziness.   I think this is only going to
get better if a new crop of students that have been trained in 'good
taste.'  So, I wonder do any of the schools like Darthmouth and the like
teach courses that study 'style' and taste in CS.   Yes, it is a young
field but we have been around long enough that we do a body of work good
and bad to consider.

I think there is a book or two and a few lectures in there somewhere.
Thoughts?

Clem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170505/582a0bfb/attachment.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Bakul Shah]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4301 bytes --]

I think the key issue is not as much minimalism as composability. BSD often prioritized convenience over composability. Each command doing one thing well and doing line oriented makes them more composability. You can always package up convenient combinations in a script. Plan9 has lc which prints like unix ls -C but it is an rc script. Trying to achieve composability can result in leaner systems.

> On May 5, 2017, at 8:20 AM, Clem Cole <clemc at ccc.com> wrote:
> 
> 
>> On Thu, May 4, 2017 at 7:14 PM, Larry McVoy <lm at mcvoy.com> wrote:
>> some of those Berkeley flags (not specifically for cat, but almost certainly including those for cat) were really quite useful. 
> 
> ​Amen!!! I think that this is key point.   What is in good taste or good style?  Doug's distain for the results of:
>                less --help | wc
> is found in bad design, a poor user interface, little fore thought,  etc.   Most of what is there has been added over Eric Scheinbrood's original  more(1) I do not believe are used that often, but some of them are of course, and you can not tell who uses what!! So how do you decide what get rid of?
> 
> How do you learn what people really need --    IMO: much of that is experience and this thing we call 'good taste.'  As opposed to what I think happened with less(1) and many others similar programs (programmers peeing on the code because they code and the source was available -- I can add this feature to it and I think it is cool.  As opposed to asking, what do really get and not having a 'master builder' arbitrating or vetting things). 
> 
> The problem we have is that we don't yet have a way of defining good taste.   One might suggest that it takes years of civilization and also that tastes do change over time.   Pike's minimalistic view (which I think is taken to the extreme in the joke about automobile dashboard on Brian Kernighan's car) sets the bar to one end and probably one of the reason why UNIX had a bad reputation, certainly by non-computer scientists, when it first appeared as being difficult.  Another extreme is systems that put in you a box and never let you do anything but what we told you do; which I find just as frighten and frustration builds there when I use them.  Clearly systems that are so noisy that can not find what really want, or need is another dimension of the same bad design.  So what to do? [more in a minute...]
> 
> Larry is right.  Many of the 'features' added to UNIX (and Linux) over time have been and are useful.   Small and simple as it was (and I really admire Ken, Dennis and the Team for its creation), but in 2017 I really don't want to run the Sixth Edition for  my day to day work anymore - which I happily did in 1977.     But the problem is, as we got 'unlimited address space' of 32 and 64 bits, and more room for more 'useful' things, we also got a great deal of rubbish and waste.    I am interpreting Doug's point about less --help | wc is that is that are so many thorns and weeds, it hard to see to see the flowers in the garden.
> 
> I'd like to observe that is that most college courses in CS I have seen talk about how to construct a programs, algorithms, structures - i.e. the mechanics of some operation.   But this discussion is about the human element.   What we feel is good or bad and how it related to how to use the program.   
> 
> I think about my friends that have degrees in literature, art and architecture.   In all cases, they spend a lot of time examining past examples, of good and bad - and thinking and discussing what makes them so.   I'm actually happy to see it was Professor McIlroy that is one of the folks taking a stand on current craziness.   I think this is only going to get better if a new crop of students that have been trained in 'good taste.'  So, I wonder do any of the schools like Darthmouth and the like teach courses that study 'style' and taste in CS.   Yes, it is a young field but we have been around long enough that we do a body of work good and bad to consider.
> 
> I think there is a book or two and a few lectures in there somewhere.   Thoughts?
> 
> Clem
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170505/d75dcba3/attachment-0001.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Doug McIlroy]

Clem wrote, "So, I wonder do any of the schools like Darthmouth and the like
teach courses that study 'style' and taste in CS."

Some time around 1970, Bob Rosen instituted a program-reading
course at the University of Iowa, by overt analogy to the
study of literature. Style was certainly a central concern;
I'm not sure about the deeper matter of taste. I'm not aware
of other examples, except perhaps for a collaboration between
Don Kunth and a teachaer of (I think) creative writing--a
course whose content I can't recall.

Doug

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Noel Hunt]

What you call 'composability' is nothing more than the 'software
tools' approach.

I recall reading an interview with Dennis Ritchie, but can no
longer find it, where he talks about this approach versus the
monolithic program model, as for example demonstrated by
'perl', which does everything. Dennis remarked that the
'tools' approach required careful thought (which programs to
use, how to connect them with pipes) and so was more difficult
to use, hence the popularity of 'perl'.

I'd be delighted if someone can point me to where this
interview might be.

P.S.

A most beautiful example of this approach is from Doug McIlroy
in his critique of Donald Knuth's solution to a problem posed
by Jon Bentley in his 'Programming Pearls' column:

    Read a file of text, determine the n most frequently used
    words, and print out a sorted list of those words along with
    their frequencies.

Donald Knuth wrote a long, beautifully crafted program in some
pseudo-code.  Doug McIlroy provided an alternative solution:

tr -cs A-Za-z '\n' |
tr A-Z a-z |
sort |
uniq -c |
sort -rn |
sed ${1}q

This is real genius.


On Sat, May 6, 2017 at 1:37 AM, Bakul Shah <bakul at bitblocks.com> wrote:

> I think the key issue is not as much minimalism as composability. BSD
> often prioritized convenience over composability. Each command doing one
> thing well and doing line oriented makes them more composability. You can
> always package up convenient combinations in a script. Plan9 has lc which
> prints like unix ls -C but it is an rc script. Trying to achieve
> composability can result in leaner systems.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170506/350640c1/attachment.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Toby Thain]

On 2017-05-05 10:16 PM, Noel Hunt wrote:
> ...
> Donald Knuth wrote a long, beautifully crafted program in some
> pseudo-code.  Doug McIlroy provided an alternative solution:
>

Much as I love Professor Knuth, he is still doing this. He always was a 
low level coder*, and his programs are longer and more complicated than 
the high level equivalents, by the expected factors. (If anyone doubts 
it, note that he adopted C after Pascal, and many of his programs are 
linked from his web pages.) No doubt he has solid rationalisations for 
taking this approach for his purposes, but of course it's not ideal for 
all situations.

--Toby


* - Many of us will remember we were expected to understand TAOCP 
through MIX. And he started out like this:
http://www.catonmat.net/blog/donald-knuths-first-computer/

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Steve Johnson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2603 bytes --]

In 1973, I spent 9 months at the University of Waterloo, home of
Watfor.   I taught a course in "Advanced Application Techniques" --
the language was FORTRAN, and the techniques included such things as
using libraries and handling errors in input data...

When I left for Canada, there had been a lot of discussion about
programming style at the labs, and so in my first lecture I told the
students that I would award points for style, and briefly talked about
what that was and how I would assess the style, and I did so
throughout the course.

What amazed me was that some students seemed to instantly get the
idea, and became quite good at producing excellent, clean programs,
while others were violently allergic to the whole notion that anything
"subjective" should stand between them and their grade in the class.
 I remember several interchanges with students, often angry, who had
a lot of difficulty understanding why their 35-line solution to a
problem should get a lower grade than a 15-line version (excluding
comment lines).

in fairness, I didn't (and don't) know any better way to learn style
than doing a lot of reading, a lot of writing, and getting criticized
constructively by those with more experience.  For me, a lot of what
I learned was from Stan Brown at the Labs, who read piles of my
(atrocious) FORTRAN code and repeatedly pointed out that when you
wrote a program, you were not just communicating with the computer,
but also other humans (including yourself) who would read (and perhaps
modify) the code in the future...

Steve

----- Original Message -----
From: "Doug McIlroy" <doug@cs.dartmouth.edu>
To:<lm at mcvoy.com>, <doug at cs.dartmouth.edu>, <clemc at ccc.com>
Cc:<tuhs at minnie.tuhs.org>
Sent:Fri, 05 May 2017 22:02:21 -0400
Subject:Re: [TUHS] Discuss of style and design of computer programs
from a user stand point [was dmr note on BSD's sins]

 Clem wrote, "So, I wonder do any of the schools like Darthmouth and
the like
 teach courses that study 'style' and taste in CS."

 Some time around 1970, Bob Rosen instituted a program-reading
 course at the University of Iowa, by overt analogy to the
 study of literature. Style was certainly a central concern;
 I'm not sure about the deeper matter of taste. I'm not aware
 of other examples, except perhaps for a collaboration between
 Don Kunth and a teachaer of (I think) creative writing--a
 course whose content I can't recall.

 Doug

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170505/4b496d7b/attachment.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Bakul Shah]

On Sat, 06 May 2017 12:16:53 +1000 Noel Hunt <noel.hunt at gmail.com> wrote:
> 
> What you call 'composability' is nothing more than the 'software
> tools' approach.

Composable components can have a much tigher coupling.  vi,
rsync, git and gdb are s/w tools but they are not composable
in the sense of sed, grep, uniq, awk, head, tail, tr, sort,
ls, find etc.

> I recall reading an interview with Dennis Ritchie, but can no
> longer find it, where he talks about this approach versus the
> monolithic program model, as for example demonstrated by
> 'perl', which does everything. Dennis remarked that the
> 'tools' approach required careful thought (which programs to
> use, how to connect them with pipes) and so was more difficult
> to use, hence the popularity of 'perl'.

This is the "modularity" approach, where one program does one
thing well rather than doing everything (e.g IDE). But
moudlarity does not necessarily give you composability.

Also, it is not enough to have composable components; you need
a language/environment in which they can be used.

> A most beautiful example of this approach is from Doug McIlroy
> in his critique of Donald Knuth's solution to a problem posed
> by Jon Bentley in his 'Programming Pearls' column:
> 
>     Read a file of text, determine the n most frequently used
>     words, and print out a sorted list of those words along with
>     their frequencies.
> 
> Donald Knuth wrote a long, beautifully crafted program in some
> pseudo-code.  Doug McIlroy provided an alternative solution:
> 
> tr -cs A-Za-z '\n' |
> tr A-Z a-z |
> sort |
> uniq -c |
> sort -rn |
> sed ${1}q
> 
> This is real genius.

Indeed. 

Note that a similar approach was implemented by Iverson in
APL. In fact a shell pipeline has a lot in common with array
programming!

Iverson in essence built a whole workshop with a set of simple
but powerful standard tools (functions and higher order
functions).  The same has been carried on in follow on array
languages such as j and k and q languages.  Many of these
array functions are directly applicable to stream processing.
In contrast unix shells provide a very shallow glue layer.

> On Sat, May 6, 2017 at 1:37 AM, Bakul Shah <bakul at bitblocks.com> wrote:
> 
> > I think the key issue is not as much minimalism as composability. BSD
> > often prioritized convenience over composability. Each command doing one
> > thing well and doing line oriented makes them more composability. You can
> > always package up convenient combinations in a script. Plan9 has lc which
> > prints like unix ls -C but it is an rc script. Trying to achieve
> > composability can result in leaner systems.

[TUHS] Discuss of style and design of computer programs from a user stand point

[Michael Kjörling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2578 bytes --]

On 5 May 2017 22:33 -0700, from scj at yaccman.com (Steve Johnson):
> For me, a lot of what
> I learned was from Stan Brown at the Labs, who read piles of my
> (atrocious) FORTRAN code and repeatedly pointed out that when you
> wrote a program, you were not just communicating with the computer,
> but also other humans (including yourself) who would read (and perhaps
> modify) the code in the future...

I would actually take that one step further: When you are writing
code, you are _first and foremost_ communicating with whatever human
will need to read or modify the code later. That human might be you, a
colleague, or the violent psychopath who knows both where you live and
where your little kids go to school (might as well be you). You should
strive to write the code accordingly, _even if_ the odds of the threat
ever materializing are slim at most. Style matters a lot, there.

Sure, some code really needs to be heavily optimized, and might end up
obscure for it. In operating systems, interrupt routines are probably
one of the major candidates, but first stage bootloaders need their
fair share of it too for other reasons. Tight loops or code that needs
to be able to operate on lots of data quickly are other examples. But
_most code isn't like that_ and can allow for some extra verbosity and
some performance penalty if that means making it significantly easier
to read and modify. That's not to say code should be verbose _just
because_, but if you don't need ultimate performance, I say aim for
readability and modifyability first; processing and memory usage
second. You can always go back and optimize bits that turn out to need
it.

And of course, I find Doug McIlroy's shell snippet mentioned by Noel
interesting, _because I did something very similar not all that long
ago_; in my case, stringing together a bunch of shell tools to get a
list of all URLs from an e-mail message for further processing.
(Something like a composable _urlview_; if urlview had even had a
"dump to stdout" option, that would have done all I needed it for.)
Works like a charm, and isn't even terribly complicated. It could
probably be done in a single line of perl or awk instead of piping
through a grand total of six commands, but that's okay. And of course,
stringing together a bunch of commands was the first approach that
occured to me...

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

[TUHS] Discuss of style and design of computer programs from a user stand point

[Nemo]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 532 bytes --]

On 06/05/2017, Michael Kjörling <michael at kjorling.se> wrote:
> I would actually take that one step further: When you are writing
> code, you are _first and foremost_ communicating with whatever human
> will need to read or modify the code later.

Something similar appears in the front of almost every Wirth book
(though sometimes honoured in the breach).  That being said, there are
times when the code is transformed by optimization into something very
difficult to read.  (We used noweb to document the transformation.)

N.

[TUHS] Discuss of style and design of computer programs from a user stand point

[Michael Kjörling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1489 bytes --]

On 6 May 2017 09:09 -0400, from cym224 at gmail.com (Nemo):
>> I would actually take that one step further: When you are writing
>> code, you are _first and foremost_ communicating with whatever human
>> will need to read or modify the code later.
> 
> ..., there are
> times when the code is transformed by optimization into something very
> difficult to read.  (We used noweb to document the transformation.)

True, optimized code can be difficult to read. But very little code
really _needs_ to be optimized to that level, and when you come across
it, you will generally know -- at least after running the application
through a profiler. (I have to admit we do have it easy these days.)

I've been working on optimizing some SQL code recently -- I know,
that's not exactly operating system kernel C or assembler -- and it's
amazing at times how the code can end up being both easier to
understand _as well as_ faster to execute, if you just put some
thought into what the code really needs to do. And not just minor
speedups either, but wallclock runtime reductions on the order of _two
to four nines_.

I think there's been only one time in the last several years when I
have really _needed_ to optimize code to the point that its workings
were not obvious.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

[TUHS] The Elements of Programming Style (book) - was Re: Discuss of style and design of computer programs

[Toby Thain]

On 2017-05-06 1:33 AM, Steve Johnson wrote:
> In 1973, I spent 9 months at the University of Waterloo, home of Watfor.
>   I taught a course in "Advanced Application Techniques" -- the language
> was FORTRAN, and the techniques included such things as using libraries
> and handling errors in input data...
> ...
> in fairness, I didn't (and don't) know any better way to learn style
> than doing a lot of reading, a lot of writing, and getting criticized
> constructively by those with more experience.  For me, a lot of what I
> learned was from Stan Brown at the Labs, who read piles of my
> (atrocious) FORTRAN code and repeatedly pointed out that when you wrote
> a program, you were not just communicating with the computer, but also
> other humans (including yourself) who would read (and perhaps modify)
> the code in the future...

As a longtime maintenance programmer: I could not agree more. Thankfully 
this idea is fairly well accepted today. Although, I am working on a 
project right now that makes me think the author simply never considered 
that anyone else might ever read or maintain it.

In this context, one of the timeless classics is The Elements of 
Programming Style (Kernighan and Plauger). It had a huge effect on my C 
when I read it years ago and I still try to apply its lessons every day 
in any language.

--Toby


>
> Steve
>
>
>
>     ----- Original Message -----
>     From:
>     "Doug McIlroy" <doug at cs.dartmouth.edu>
>
>     To:
>     <lm at mcvoy.com>, <doug at cs.dartmouth.edu>, <clemc at ccc.com>
>     Cc:
>     <tuhs at minnie.tuhs.org>
>     Sent:
>     Fri, 05 May 2017 22:02:21 -0400
>     Subject:
>     Re: [TUHS] Discuss of style and design of computer programs from a
>     user stand point [was dmr note on BSD's sins]
>
>
>     Clem wrote, "So, I wonder do any of the schools like Darthmouth and
>     the like
>     teach courses that study 'style' and taste in CS."
>
>     Some time around 1970, Bob Rosen instituted a program-reading
>     course at the University of Iowa, by overt analogy to the
>     study of literature. Style was certainly a central concern;
>     I'm not sure about the deeper matter of taste. I'm not aware
>     of other examples, except perhaps for a collaboration between
>     Don Kunth and a teachaer of (I think) creative writing--a
>     course whose content I can't recall.
>
>     Doug
>

[TUHS] Discuss of style and design of computer programs from a user stand point

[Larry McVoy]

On Sat, May 06, 2017 at 09:18:57AM +0000, Michael Kj??rling wrote:
> On 5 May 2017 22:33 -0700, from scj at yaccman.com (Steve Johnson):
> > For me, a lot of what
> > I learned was from Stan Brown at the Labs, who read piles of my
> > (atrocious) FORTRAN code and repeatedly pointed out that when you
> > wrote a program, you were not just communicating with the computer,
> > but also other humans (including yourself) who would read (and perhaps
> > modify) the code in the future...
> 
> I would actually take that one step further: When you are writing
> code, you are _first and foremost_ communicating with whatever human
> will need to read or modify the code later. That human might be you, a
> colleague, or the violent psychopath who knows both where you live and
> where your little kids go to school (might as well be you). 

When I run engineering teams the point I make is if you wrote the code
more than 6 months ago it might as well be someone else's code.  So
write it in a way that someone else can debug / extend it.

I also made the point that "clever" sucks.  At least most of the time.
All "clever" means is "hard to understand".  The brief joy one gets 
from clever code is stomped on by the frustration one gets from having
to fix it.  

When they wouldn't listen to that I'd say "I'm the dumbest guy here, right?
And I'm the boss.  And I debug could.  The path to a big bonus does not 
go through clever code".  That seemed to work.

Personally, I find code that is clean, straightforward, obvious to be
beautiful.  The clever stuff usually strikes an odd note, not a good
one.

--lm

[TUHS] Discuss of style and design of computer programs from a

[Corey Lindsly]

> Personally, I find code that is clean, straightforward, obvious to be
> beautiful.  The clever stuff usually strikes an odd note, not a good
> one.
> 
> --lm

I am not a programmer. Almost four decades ago, my first computer was a 
TRS80 Model 1 with 16KB RAM. I spent one month disassembling and stepping 
through the Z80 code for the resident Microsoft BASIC interpreter. The 
entire thing fit in a 12KB PROM so it was originally written in assembly 
and tightly optimized. It was fascinating and extremely instructive. All 
these years later, I could probably still slap together a Z80 program if I 
needed to. 

Anyway, I reached one point in the assembly code that I simply could not 
understand. It seemed like a mistake, and I went through it again and 
again until I finally realized what it was doing. There was a branch/loop 
that jumped to the middle of a multi-byte machine instruction, so that 
branch had to be disassembled and stepped separately until it "synced" up 
with the other branch again. Maybe this is standard practice in 
programming (I don't know) but at the time I thought, what kind of evil 
genius devised this to save a few bytes of memory?

--corey

[TUHS] Discuss of style and design of computer programs from a

[Michael Kjörling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1959 bytes --]

On 6 May 2017 08:09 -0700, from corey at lod.com (Corey Lindsly):
> Anyway, I reached one point in the assembly code that I simply could not 
> understand. It seemed like a mistake, and I went through it again and 
> again until I finally realized what it was doing. There was a branch/loop 
> that jumped to the middle of a multi-byte machine instruction, so that 
> branch had to be disassembled and stepped separately until it "synced" up 
> with the other branch again. Maybe this is standard practice in 
> programming (I don't know) but at the time I thought, what kind of evil 
> genius devised this to save a few bytes of memory?

IIRC, that _was_ a common trick at least on machines of that class. It
did have the potential to save a few bytes, yes (more if the
instructions were such that you'd get some _other, desired_, behavior
by jumping into the middle of one with some specific state), but it
also foiled lots of disassemblers: Simply disassembling a binary from
start to finish would yield nonsense in those locations, as you
experienced. It thus basically forced you to single-step those
instructions to figure out what was going on from the binary.

I'm pretty sure it works on every architecture with variable-length
instructions and arbitrary jump capability, as long as you have
control over the specific machine instructions generated (such as if
you are programming in assembler). Of course, it _is_ also a total
nightmare to maintain such code.

I would absolutely not say that doing something like that is standard
practice in modern programming. Even in microcontrollers, where
program and data memory can be scarce even today, I would argue that
the costs would not outweigh the benefits by a long shot.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

[TUHS] Discuss of style and design of computer programs from a

[ron minnich]

On Sat, May 6, 2017 at 8:09 AM Corey Lindsly <corey at lod.com> wrote:

> There was a branch/loop
> that jumped to the middle of a multi-byte machine instruction, so that
> branch had to be disassembled and stepped separately until it "synced" up
> with the other branch again. Maybe this is standard practice in
> programming (I don't know) but at the time I thought, what kind of evil
> genius devised this to save a few bytes of memory?
>
>
This was extremely common back then. I had a friend who worked on a gas
chromatograph project, names redacted here. It had a very advanced idea, a
thermal printer. It would print a banner when it started.  Getting that
print to work, in the ROM they had, was a nightmare that involved all these
tricks.

When the "A" version came out, they asked my friend to have it print the A
after the product number. His response: "NO". There was no way he could
ever pick apart the crazy code that had printed out the startup banner so
he could add an "A". The startup banner remained the same.

Executing code as data in the early startup was also common in those days,
and modifying that data and then rerunning it happened all the time.

This is why the things like the Therac 25 happened ...
https://en.wikipedia.org/wiki/Therac-25

Note the reference to "Cargo coding", reusing code you don't understand. In
modern terms we call this software reuse and it's taught at all the best
uni's. Google a package, pull it down, compile it in, done.

Lest you think things are better now, Linux uses self modifying code to
optimize certain critical operations, and at one talk I heard the speaker
say that he'd like to put more self modifying code into Linux, "because
it's fun". Oh boy.

ron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170506/b0f8b7d4/attachment-0001.html>

[TUHS] Discuss of style and design of computer programs from a

[Larry McVoy]

On Sat, May 06, 2017 at 03:20:53PM +0000, Michael Kj??rling wrote:
> I would absolutely not say that doing something like that is standard
> practice in modern programming. Even in microcontrollers, where
> program and data memory can be scarce even today, I would argue that
> the costs would not outweigh the benefits by a long shot.

It strikes as being similar to Duff's device (1).  Which is a niche thing
but I still use that from time to time.  Not to save memory, just because
as a C programmer it seems pretty natural to do it.

--lm

(1) https://en.wikipedia.org/wiki/Duff's_device

[TUHS] Discuss of style and design of computer programs from a

[Michael Kjörling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1009 bytes --]

On 6 May 2017 15:23 +0000, from rminnich at gmail.com (ron minnich):
> This is why the things like the Therac 25 happened ...
> https://en.wikipedia.org/wiki/Therac-25

Or the Ariane 5 flight 501, which according to Wikipedia cost more
than 370 million dollars (and resulted in the total loss of the
spacecraft). I would say a few million or even a few tens of millions
of dollars to double-check the software, or even write new software
specifically designed for the Ariane 5 rather than reuse software
designed for the Ariane 4 outside of that software's design limits,
might not have been a bad way to spend money there.

And that was in 1996. Hardly that long ago.

https://en.wikipedia.org/wiki/Ariane_5#Notable_launches

https://en.wikipedia.org/wiki/Cluster_%28spacecraft%29

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

[TUHS] Discuss of style and design of computer programs from a

[Michael Kjörling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1031 bytes --]

On 6 May 2017 08:24 -0700, from lm at mcvoy.com (Larry McVoy):
>> I would absolutely not say that doing something like that is standard
>> practice in modern programming. Even in microcontrollers, where
>> program and data memory can be scarce even today, I would argue that
>> the costs would not outweigh the benefits by a long shot.
> 
> It strikes as being similar to Duff's device (1).  Which is a niche thing
> but I still use that from time to time.  Not to save memory, just because
> as a C programmer it seems pretty natural to do it.
> 
> --lm
> 
> (1) https://en.wikipedia.org/wiki/Duff's_device

I disagree; loop unrolling and jumping to the beginning of some
instruction inside that unrolled loop is not at all the same thing as
jumping _into the middle of a machine language instruction_.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

[TUHS] Discuss of style and design of computer programs from a

[Larry McVoy]

On Sat, May 06, 2017 at 03:51:24PM +0000, Michael Kj??rling wrote:
> On 6 May 2017 08:24 -0700, from lm at mcvoy.com (Larry McVoy):
> >> I would absolutely not say that doing something like that is standard
> >> practice in modern programming. Even in microcontrollers, where
> >> program and data memory can be scarce even today, I would argue that
> >> the costs would not outweigh the benefits by a long shot.
> > 
> > It strikes as being similar to Duff's device (1).  Which is a niche thing
> > but I still use that from time to time.  Not to save memory, just because
> > as a C programmer it seems pretty natural to do it.
> > 
> > --lm
> > 
> > (1) https://en.wikipedia.org/wiki/Duff's_device
> 
> I disagree; loop unrolling and jumping to the beginning of some
> instruction inside that unrolled loop is not at all the same thing as
> jumping _into the middle of a machine language instruction_.

That's fine, I feel no need to argue about it.  Seemed similar to me but
I'm not the sharpest tool in the shed :)

[TUHS] Discuss of style and design of computer programs from a user stand point

[Kurt H Maier]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 956 bytes --]

On Sat, May 06, 2017 at 09:18:57AM +0000, Michael Kjörling wrote:
>
> I would actually take that one step further: When you are writing     
> code, you are _first and foremost_ communicating with whatever human  
> will need to read or modify the code later. 
         
This can be, and is, taken too far.  It is currently fashionable in the
Ruby world to erect testing gantries that look like
       
describe 'service' do
  it 'is listening' do
    expect(port 443).to be_listening
  end
end
       
and then drag in a couple hundred thousand lines of extra code to make  
this cuteness approximate instructions for the computer.
       
It's all well and good to write for the person who comes behind you, but
at some point it will break, and the closer you've flown to this sun the
worse it will be for the poor sod who has to pick up the pieces.  I'd   
take durable code I have to work to understand over fragile artwork     
any day.
       
khm

[TUHS] Discuss of style and design of computer programs from a user stand point

[Dave Horsfall]

On Sat, 6 May 2017, Larry McVoy wrote:

> When I run engineering teams the point I make is if you wrote the code 
> more than 6 months ago it might as well be someone else's code.  So 
> write it in a way that someone else can debug / extend it.

A principle I've tried to follow over the years is "write code as though 
the next person to maintain it is an axe-wielding murderer who knows where 
you live".

> I also made the point that "clever" sucks.  At least most of the time. 
> All "clever" means is "hard to understand".  The brief joy one gets from 
> clever code is stomped on by the frustration one gets from having to fix 
> it.

Apropos the above, I do confess to writing "clever code" from time to time 
(PERL is a beautiful language for such sins)...

Of course, in the olden days it was the APL "one-liners" :-)

-- 
Dave Horsfall DTM (VK2KFU)  "Those who don't understand security will suffer."

[TUHS] Discuss of style and design of computer programs from a user stand point

[Bakul Shah]

On Sun, 07 May 2017 04:43:33 +1000 Dave Horsfall <dave at horsfall.org> wrote:
> 
> > I also made the point that "clever" sucks.  At least most of the time. 
> > All "clever" means is "hard to understand".  The brief joy one gets from 
> > clever code is stomped on by the frustration one gets from having to fix 
> > it.

> Apropos the above, I do confess to writing "clever code" from time to time 
> (PERL is a beautiful language for such sins)...
> 
> Of course, in the olden days it was the APL "one-liners" :-)

On the other hand, you have to grok only one line of code
instead reading pages of Java/C++ with surprises lurking
everywhere :-)

Since you need one line to do the work of many many lines of
c++/Java code, you should expect to spend more time on that
one line!

The same is true with shell one liners as well. We don't
(usually) consider constructs such as the following "clever"
because we expect its reader to have a certain level of code
literacy and context (or an ability to gain such).

tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q

[TUHS] Discuss of style and design of computer programs from a

[Steve Nickolas]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2530 bytes --]

On Sat, 6 May 2017, Michael Kjörling wrote:

> On 6 May 2017 08:09 -0700, from corey at lod.com (Corey Lindsly):
>> Anyway, I reached one point in the assembly code that I simply could not
>> understand. It seemed like a mistake, and I went through it again and
>> again until I finally realized what it was doing. There was a branch/loop
>> that jumped to the middle of a multi-byte machine instruction, so that
>> branch had to be disassembled and stepped separately until it "synced" up
>> with the other branch again. Maybe this is standard practice in
>> programming (I don't know) but at the time I thought, what kind of evil
>> genius devised this to save a few bytes of memory?
>
> IIRC, that _was_ a common trick at least on machines of that class. It
> did have the potential to save a few bytes, yes (more if the
> instructions were such that you'd get some _other, desired_, behavior
> by jumping into the middle of one with some specific state), but it
> also foiled lots of disassemblers: Simply disassembling a binary from
> start to finish would yield nonsense in those locations, as you
> experienced. It thus basically forced you to single-step those
> instructions to figure out what was going on from the binary.
>
> I'm pretty sure it works on every architecture with variable-length
> instructions and arbitrary jump capability, as long as you have
> control over the specific machine instructions generated (such as if
> you are programming in assembler). Of course, it _is_ also a total
> nightmare to maintain such code.
>
> I would absolutely not say that doing something like that is standard
> practice in modern programming. Even in microcontrollers, where
> program and data memory can be scarce even today, I would argue that
> the costs would not outweigh the benefits by a long shot.

In 6502 code, it's not uncommon to do something like

foo1:     lda      #$00
           .byte    $2C       ; 3-byte BIT
foo2:     lda      #$01
            .
            .
            .

to save a byte (and probably still done for the few who write in ASM). 
The "2C" operand would cause it to disassemble as something like...

1000-     LDA      #$00
1002-     BIT      $01A9

which is the route you'd go down if you called "foo1".  Apart diddling a 
few CPU flags, and an unneeded read on $01A9, harmless.

(Most 6502 programmers would probably see a strange BIT instruction as an 
attempt to do this.)

It's probably not a good idea to still do this unless you're really REALLY 
crunched for space.

-uso.

[TUHS] Discuss of style and design of computer programs from a

[Michael Kjörling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4134 bytes --]

On 6 May 2017 16:00 -0400, from usotsuki at buric.co (Steve Nickolas):
> In 6502 code, it's not uncommon to do something like
> 
> foo1:     lda      #$00
>           .byte    $2C       ; 3-byte BIT
> foo2:     lda      #$01
>            .
>            .
>            .
> 
> to save a byte (and probably still done for the few who write in
> ASM). The "2C" operand would cause it to disassemble as something
> like...
> 
> 1000-     LDA      #$00
> 1002-     BIT      $01A9
> 
> which is the route you'd go down if you called "foo1".  Apart
> diddling a few CPU flags, and an unneeded read on $01A9, harmless.

You could do something quite similar on the 8086, which I am somewhat
more familiar with. For example, in some kind of pseudo 8086
assembler, with $ denoting the value of the instruction pointer at the
beginning of the current instruction:

        jmp short $+3
        int 90h ; hex: cd 90

would almost certainly decompile to the above, if the decompiler
doesn't barf when it can't create a jump target label mid-instruction,
but it would certainly _execute_ as

        jmp short l1
        db 0cdh ; dummy byte
    l1: nop ; hex: 90

resulting in a jump over one byte followed by a no-op instruction. Of
course, invoking interrupt 90h would be a perfectly legal thing to do,
assuming that your interrupt tables are set up correctly. If you want
to mess with someone trying to figure out what the code is doing,
write a (nonsense or meaningful) value to interrupt vector 90h before
you do that. For example, assuming that it isn't being used, you could
point interrupt 90h at the reboot jump location, which IIRC on the IBM
PC would mean point it at FFFFh:FFF0h or absolute address FFFF0h.
Anyone trying to actually execute the INT 90h instruction would see
their computer reboot, but anyone actually executing the code would
see little more than a CPU cache flush due to the jump.

Or, something slightly more "useful"

        jmp short $+3
        int 0 ; hex: cd 00
        db 0
        jz somewhere

which, again IIRC, would clear the accumulator (AX) register because
0000 hex is XOR AX,AX, which as a side effect sets the zero flag
because the result is zero, and then executes a conditional jump if
the zero flag is set (which it will be). In other words, it _executes_
as

        jmp short l1
        db 0cdh ; dummy byte
    l1: xor ax,ax ; hex: 00 00
        jz somewhere

but a naiive decompiler will see the CDh byte after the jump and take
that as the first byte of the two-byte interrupt instruction. I don't
know what 00 followed by the first byte of the JZ instruction would
be, but probably a register/register XOR with a register store. Make
it a far jump and you have a few extra bytes to play with, to befuddle
anyone trying to figure out just what on Earth your code is really
doing.

The above, of course, is really just scratching the surface. With how
many variable-length instructions the x86 has, with careful selection
of opcodes and possibly use of undocumented but functionally valid
combinations, I wouldn't be the least bit surprised if it's
technically possible to write a 8086 program that does one thing when
started normally, and something utterly different but still useful
when started at an offset that results in execution beginning in the
middle of an instruction. Bonus points if the first thing that program
does is jump into itself _at that offset_. Now, the _work involved in
doing so_, never mind maintaining it...

Of course, I wouldn't do anything like the above in any real-world
code base meant to run on modern systems unless obfuscation really
_was_ a design goal. Honest. If I was tasked to write something that
really needed to do something non-trivial in 16 KiB RAM on an original
IBM 5150 PC or clone? I'd probably spend quite a bit of time in front
of the whiteboard and with reference manuals before writing even one
line of code...

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Steve Johnson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1633 bytes --]

 few more comments about style and optimization:First, the thing that
make optimized code hard to understand is that (at least when a
compiler does it) the optimized program goes through a different set
of states than the user expected from the source code.  When a loop
is unrolled, or invariant expressions moved out of a loop, or indexing
is turned into pointer manipulation, if you don't know what's
happening its mysterious.  But once you'v seen this a few times, it's
only a momentary hitch...Also, one lesson I learned from Unix is that
if you can make a program 10x faster it becomes a qualitatively
different experience.  The first version of Yacc I wrote, using the
textbook algorithms, took 20 minutes to process a 30-rule grammar.
 On the shared PDP-11, when I ran Yacc everyone else's jobs slowed to
a crawl, and people were heard to mutter "*%&#!  Johnson's running
Yacc again!"   A couple of years later, Yacc produced parsers faster
than the C compiler could compile them.Consider what using Google
would be like if it took 7 seconds rather than .7 seconds to respond.
 Probably still useful, but a different user experience.We had many
decades to be spoiled as far as program performance was concerned.
 As our tools got more bloated, faster processors and more memory hid
the true effect from most of us.  Now we have to rethink everything
to make it parallel.  And complexity is enemy number one when making
something parallel...
Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170506/0b9f37a0/attachment.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Larry McVoy]

On Sat, May 06, 2017 at 03:11:32PM -0700, Steve Johnson wrote:
> And complexity is enemy number one when making something parallel...

+1 Couldn't agree more, for multiple reasons.
-- 
---
Larry McVoy            	     lm at mcvoy.com             http://www.mcvoy.com/lm 

[TUHS] Discuss of style and design of computer programs from a user stand point

[Warner Losh]

On Sat, May 6, 2017 at 1:50 PM, Bakul Shah <bakul at bitblocks.com> wrote:
> tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q

The cool thing about this thread is that I learned two things: what tr
-s does, and the Nq does for sed...

Sadly, this doesn't work so well for text that isn't ASCII-7 english...

Warner

[TUHS] Discuss of style and design of computer programs from a user stand point

[Noel Hunt]

I was about to suggest using the Plan9 port utilities of the
same name but it seems 'uniq' is not coded to handle Runes
(aka utf-8). I don't imagine it would be hard to re-write it to
handle utf-8.

On Sun, May 7, 2017 at 11:15 AM, Warner Losh <imp at bsdimp.com> wrote:

> On Sat, May 6, 2017 at 1:50 PM, Bakul Shah <bakul at bitblocks.com> wrote:
> > tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q
>
> The cool thing about this thread is that I learned two things: what tr
> -s does, and the Nq does for sed...
>
> Sadly, this doesn't work so well for text that isn't ASCII-7 english...
>
> Warner
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170507/78dce325/attachment.html>

[TUHS] Discuss of style and design of computer programs from a user stand point [was dmr note on BSD's sins]

[Dan Cross]

On Sat, May 6, 2017 at 2:07 AM, Bakul Shah <bakul at bitblocks.com> wrote:

> Note that a similar approach was implemented by Iverson in
> APL. In fact a shell pipeline has a lot in common with array
> programming!
>
> Iverson in essence built a whole workshop with a set of simple
> but powerful standard tools (functions and higher order
> functions).  The same has been carried on in follow on array
> languages such as j and k and q languages.  Many of these
> array functions are directly applicable to stream processing.
> In contrast unix shells provide a very shallow glue layer.


Interesting analogy. I always think that function composition in lazy
languages like Haskell reminds me of shell pipelines. I know that Doug
swims in those waters; perhaps he will correct me if he feels that's a poor
analogy?

Carrying it forward a bit, however, I'm always amazed (and a little
appalled) at how the kids always think that they invented Rock n' Roll.
Ideas are continually recycled, but it's the rare recycler indeed who seeks
out the prior art on rediscovering one of the good ones. "Look! You can
filter and transform this data!" "...but I've been doing that at the shell
nearly every day for the past quarter of a century...." "...this one goes
to 11."

        - Dan C.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://minnie.tuhs.org/pipermail/tuhs/attachments/20170507/887dbd48/attachment.html>

[TUHS] Discuss of style and design of computer programs from a

[Stephen Kitt]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4000 bytes --]

On Sat, 6 May 2017 21:45:19 +0000, Michael Kjörling <michael at kjorling.se>
wrote:
> On 6 May 2017 16:00 -0400, from usotsuki at buric.co (Steve Nickolas):
> > In 6502 code, it's not uncommon to do something like
> > 
> > foo1:     lda      #$00
> >           .byte    $2C       ; 3-byte BIT
> > foo2:     lda      #$01
> >            .
> >            .
> >            .
> > 
> > to save a byte (and probably still done for the few who write in
> > ASM). The "2C" operand would cause it to disassemble as something
> > like...
> > 
> > 1000-     LDA      #$00
> > 1002-     BIT      $01A9
> > 
> > which is the route you'd go down if you called "foo1".  Apart
> > diddling a few CPU flags, and an unneeded read on $01A9, harmless.  
> 
> You could do something quite similar on the 8086, which I am somewhat
> more familiar with.
[...]

One possible equivalent 16-bit x86 is

	foo1:	mov al, 0
		db 035h		; XOR AX, imm16
	foo2:	mov al, 1

which implements a branchless fall-through. (But you’d probably just use
SALC...)

> Or, something slightly more "useful"
> 
>         jmp short $+3
>         int 0 ; hex: cd 00
>         db 0
>         jz somewhere
> 
> which, again IIRC, would clear the accumulator (AX) register because
> 0000 hex is XOR AX,AX,

According to my disassembler 0000 is "add byte ptr [bx+si], al"; "xor ax, ax"
is either 31c0 or 33c0.

> which as a side effect sets the zero flag
> because the result is zero, and then executes a conditional jump if
> the zero flag is set (which it will be). In other words, it _executes_
> as
> 
>         jmp short l1
>         db 0cdh ; dummy byte
>     l1: xor ax,ax ; hex: 00 00
>         jz somewhere
> 
> but a naiive decompiler will see the CDh byte after the jump and take
> that as the first byte of the two-byte interrupt instruction. I don't
> know what 00 followed by the first byte of the JZ instruction would
> be, but probably a register/register XOR with a register store. Make
> it a far jump and you have a few extra bytes to play with, to befuddle
> anyone trying to figure out just what on Earth your code is really
> doing.

My memories of disassembling x86 code in the early 90s at least is that,
because of all this, disassemblers were pretty good at restarting streams
from jump destinations, so it never really was an issue. You need to get
creative with single-stepping mode and self-decrypting code to really annoy
people trying to understand your code ;-).

> The above, of course, is really just scratching the surface. With how
> many variable-length instructions the x86 has, with careful selection
> of opcodes and possibly use of undocumented but functionally valid
> combinations, I wouldn't be the least bit surprised if it's
> technically possible to write a 8086 program that does one thing when
> started normally, and something utterly different but still useful
> when started at an offset that results in execution beginning in the
> middle of an instruction. Bonus points if the first thing that program
> does is jump into itself _at that offset_. Now, the _work involved in
> doing so_, never mind maintaining it...

I have seen some short sections of code which do this. The x86 ISA is so rich
that you can make it adapt to many different constraints — one I rather liked
is assembling code which is valid ASCII (amaze your friends by typing in
a .COM using COPY CON!). There’s a C compiler which does that on GitHub
somewhere.

> Of course, I wouldn't do anything like the above in any real-world
> code base meant to run on modern systems unless obfuscation really
> _was_ a design goal. Honest. If I was tasked to write something that
> really needed to do something non-trivial in 16 KiB RAM on an original
> IBM 5150 PC or clone? I'd probably spend quite a bit of time in front
> of the whiteboard and with reference manuals before writing even one
> line of code...

Yes, and combine that with assemblers which output all possible encodings of
a given instruction!

Regards,

Stephen

[TUHS] Discuss of style and design of computer programs from a user stand point

[Michael Kjörling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 568 bytes --]

On 7 May 2017 00:06 -0400, from crossd at gmail.com (Dan Cross):
> "Look! You can
> filter and transform this data!" "...but I've been doing that at the shell
> nearly every day for the past quarter of a century...." "...this one goes
> to 11."

"Look! This code can execute code that it creates itself!" "LISP has
been around for a while."

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

[TUHS] Discuss of style and design of computer programs from a user stand point

[Michael Kjörling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]

On 7 May 2017 11:42 +1000, from noel.hunt at gmail.com (Noel Hunt):
> I don't imagine it would be hard to re-write [uniq] to
> handle utf-8.

It does look like at least GNU coreutils 8.13 uniq is broken in that
regard, which frankly surprised me. That version isn't _that_ old.

    $ uniq --version
    uniq (GNU coreutils) 8.13
    Copyright (C) 2011 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later
    <http://gnu.org/licenses/gpl.html>.
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.

    Written by Richard M. Stallman and David MacKenzie.
    $ ( echo $'\u1234' ; echo $'\u2345' ; echo $'\u1234' )
    ሴ
    ⍅
    ሴ
    $ ( echo $'\u1234' ; echo $'\u2345' ; echo $'\u1234' ) | uniq
    ሴ
    $

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

[TUHS] Discuss of style and design of computer programs from a user stand point

[arnold]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 450 bytes --]

Michael Kjörling <michael at kjorling.se> wrote:

> On 7 May 2017 11:42 +1000, from noel.hunt at gmail.com (Noel Hunt):
> > I don't imagine it would be hard to re-write [uniq] to
> > handle utf-8.
>
> It does look like at least GNU coreutils 8.13 uniq is broken in that
> regard, which frankly surprised me. That version isn't _that_ old.

Are your LC_* env variables set correctly?

If you file a bug report, it should get fixed.

Thanks,

Arnold

[TUHS] Discuss of style and design of computer programs from a user stand point

[Warner Losh]

On Sat, May 6, 2017 at 7:42 PM, Noel Hunt <noel.hunt at gmail.com> wrote:
> I was about to suggest using the Plan9 port utilities of the
> same name but it seems 'uniq' is not coded to handle Runes
> (aka utf-8). I don't imagine it would be hard to re-write it to
> handle utf-8.

I guess I should have been clearer on what wouldn't work. It can't
possibly work for Japanese and Chinese where words aren't separated by
whitespace. Would cause problems in hybrid languages where words can
be composed of logograms and sonograms (say Japanese which often use a
few Kanji with hiragana endings that then run into hiragana particles
or other grammar elements). Can't work without modification (using
class names) for Cyrillic because there's no A or Z in words there.
Won't work in any language that has a discontiguous set of letters,
which includes many western european languages since all the accented
or otherwise decorated letters aren't in the range A-Z.

So whether or not the underlying tools can handle UTF-8 encoding,
there are problems with the original.

If you used:

 tr -cs "[:alpha:]" '\n' | tr "[:upper:]" "[:lower:]" | sort | uniq -c
| sort -rn | sed ${1}q

you'd still have issues with languages that don't use word separators,
or write non-alphabetically.

Warner

> On Sun, May 7, 2017 at 11:15 AM, Warner Losh <imp at bsdimp.com> wrote:
>>
>> On Sat, May 6, 2017 at 1:50 PM, Bakul Shah <bakul at bitblocks.com> wrote:
>> > tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q
>>
>> The cool thing about this thread is that I learned two things: what tr
>> -s does, and the Nq does for sed...
>>
>> Sadly, this doesn't work so well for text that isn't ASCII-7 english...
>>
>> Warner
>
>

[TUHS] Discuss of style and design of computer programs from a user stand point

[Michael Kjörling]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 970 bytes --]

On 7 May 2017 08:58 -0600, from arnold at skeeve.com:
>>> I don't imagine it would be hard to re-write [uniq] to handle utf-8.
>> 
>> It does look like at least GNU coreutils 8.13 uniq is broken in that
>> regard, which frankly surprised me. That version isn't _that_ old.
> 
> Are your LC_* env variables set correctly?

I believe so (I don't recall seeing any other UTF-8-related weirdness
for a very long time), but I would want to verify the behavior on a
system that doesn't have a gazillion customizations accumulated over
years _and_ has the most recent version of coreutils before I file an
actual bug report. (It's not like Debian is ever bleeding edge, to
begin with.) I'll see if I spin up a VM or three one of these days to
check.

-- 
Michael Kjörling • https://michael.kjorling.se • michael at kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

[TUHS] Discuss of style and design of computer programs from a user stand point

[David]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2174 bytes --]


> From: Michael Kjörling <michael at kjorling.se>
> To: tuhs at minnie.tuhs.org
> Subject: Re: [TUHS] Discuss of style and design of computer programs
> 	from a user stand point
> Message-ID: <20170506091857.GE12539 at yeono.kjorling.se>
> Content-Type: text/plain; charset=utf-8
> 
> I would actually take that one step further: When you are writing
> code, you are _first and foremost_ communicating with whatever human
> will need to read or modify the code later. That human might be you, a
> colleague, or the violent psychopath who knows both where you live and
> where your little kids go to school (might as well be you). You should
> strive to write the code accordingly, _even if_ the odds of the threat
> ever materializing are slim at most. Style matters a lot, there.
> 
Interesting, I was going to say about the same thing about the violent psychopath
who has to maintain your code after you leave. When I lectured at UCSD or was
giving talks on style for ViaSat I always said the same thing:

  Whatever you write, the fellow who is going to wind up maintaining it is a known
  axe killer, now released from prison, completely reformed. He learned computer
  programming on MS/DOS 3.1 and a slightly broken version of Pascal. He will be
  given your home phone number and address so if he has any questions about the
  code you wrote he can get in contact with you.

This always got a few chuckles. I then pointed out that whenever anyone gets code
that someone else wrote, the recipient always thinks that they can ‘clean up’ what
is there because the original author clearly doesn’t understand what proper code
looks like.

Over time, I’ve learned that everyone has a style when writing code, just like handwriting
and given enough time, I can spot who the author of a block of code is just from the
indenting, placement of ( and ) around a statement and other small traits.

What makes good code is the ability to convey the meaning of the algorithm
from the original author to all those who come after. Sometimes even the most
unusual code can be quite clear, while the most cleanly formatted and commented
code can be opaque to all.

	David