[TUHS] The 'usage: ...' message. (Was: On Bloat...)

[TUHS] The 'usage: ...' message. (Was: On Bloat...)

[Douglas McIlroy]

[-- Attachment #1: Type: text/plain, Size: 2004 bytes --]

>> Another non-descriptive style of error message that I admired was that
>> of Berkeley Pascal's syntax diagnostics. When the LR parser could not
>> proceed, it reported where, and automatically provided a sample token
>> that would allow the parsing to progress. I found this uniform
>> convention to be at least as informative as distinct hand-crafted
>> messages, which almost by definition can't foresee every contingency.
>> Alas, this elegant scheme seems not to have inspired imitators.

> The hazard with this approach is that the suggested syntactic correction
> might simply lead the user farther into the weeds

I don't think there's enough experience to justify this claim. Before I
experienced the Berkeley compiler, I would have thought such bad outcomes
were inevitable in any language. Although the compilers' suggestions often
bore little or no relationship to the real correction,  I always found them
informative. In particular, the utterly consistent style assured there was
never an issue of ambiguity or of technical jargon.

The compiler taught me Pascal in an evening. I had scanned the Pascal
Report a couple of years before but had never written a Pascal program.
With no manual at hand, I looked at one program to find out what
mumbo-jumbo had to come first and how to print integers, then wrote the
rest by trial and error. Within a couple of hours  I had a working program
good enough to pass muster in an ACM journal.

An example arose that one might think would lead "into the weeds". The
parser balked before 'or' in a compound Boolean expression like  'a=b and
c=d or x=y'. It couldn't suggest a right paren because no left paren had
been seen. Whatever suggestion it did make (perhaps 'then') was enough to
lead me to insert a remote left paren and teach me that parens are required
around Boolean-valued subexpressions. (I will agree that this lesson might
be less clear to a programming novice, but so might be many conventional
diagnostics, e.g. "no effect".)

Doug

[-- Attachment #2: Type: text/html, Size: 3392 bytes --]

[TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)

[Rob Pike]

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]

The Cornell PL/I compiler, PL/C, ran on the IBM 360 so of course used batch
input. It tried automatically to keep things running after a parsing error
by inserting some token - semicolon, parenthesis, whatever seemed best -
and continuing to parse, in order to maximize the amount of input that
could be parsed before giving up. At least, that's what I took the
motivation to be. It rarely succeeded in fixing the actual problem, despite
PL/I being plastered with semicolons, but it did tend to ferret out more
errors per run. I found the tactic helpful.

-rob

[-- Attachment #2: Type: text/html, Size: 873 bytes --]

[TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)

[arnold]

Rob Pike <robpike@gmail.com> wrote:

> The Cornell PL/I compiler, PL/C, ran on the IBM 360 so of course used batch
> input. It tried automatically to keep things running after a parsing error
> by inserting some token - semicolon, parenthesis, whatever seemed best -
> and continuing to parse, in order to maximize the amount of input that
> could be parsed before giving up. At least, that's what I took the
> motivation to be. It rarely succeeded in fixing the actual problem, despite
> PL/I being plastered with semicolons, but it did tend to ferret out more
> errors per run. I found the tactic helpful.
>
> -rob

Gawk used to do this, until people started fuzzing it, causing cascading
errors and eventually core dumps. Now the first syntax error is fatal.
It got to the point where I added this text to the manual:

	In recent years, people have been running "fuzzers" to generate
	invalid awk programs in order to find and report (so-called)
	bugs in gawk.

	In general, such reports are not of much practical use. The
	programs they create are not realistic and the bugs found are
	generally from some kind of memory corruption that is fatal
	anyway.

	So, if you want to run a fuzzer against gawk and report the
	results, you may do so, but be aware that such reports don’t
	carry the same weight as reports of real bugs do.

(Yeah, I've just changed the subject, feel free to stay on topic. :-)

Arnold

[TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)

[Warner Losh]

[-- Attachment #1: Type: text/plain, Size: 1697 bytes --]

On Sun, May 19, 2024, 9:19 PM <arnold@skeeve.com> wrote:

> Rob Pike <robpike@gmail.com> wrote:
>
> > The Cornell PL/I compiler, PL/C, ran on the IBM 360 so of course used
> batch
> > input. It tried automatically to keep things running after a parsing
> error
> > by inserting some token - semicolon, parenthesis, whatever seemed best -
> > and continuing to parse, in order to maximize the amount of input that
> > could be parsed before giving up. At least, that's what I took the
> > motivation to be. It rarely succeeded in fixing the actual problem,
> despite
> > PL/I being plastered with semicolons, but it did tend to ferret out more
> > errors per run. I found the tactic helpful.
> >
> > -rob
>
> Gawk used to do this, until people started fuzzing it, causing cascading
> errors and eventually core dumps. Now the first syntax error is fatal.
> It got to the point where I added this text to the manual:
>
>         In recent years, people have been running "fuzzers" to generate
>         invalid awk programs in order to find and report (so-called)
>         bugs in gawk.
>
>         In general, such reports are not of much practical use. The
>         programs they create are not realistic and the bugs found are
>         generally from some kind of memory corruption that is fatal
>         anyway.
>
>         So, if you want to run a fuzzer against gawk and report the
>         results, you may do so, but be aware that such reports don’t
>         carry the same weight as reports of real bugs do.
>
> (Yeah, I've just changed the subject, feel free to stay on topic. :-)
>


Awk bailing out near line 1.

Warner

> Arnold
>

[-- Attachment #2: Type: text/html, Size: 2473 bytes --]

[TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)

[Bakul Shah via TUHS]

[-- Attachment #1: Type: text/plain, Size: 798 bytes --]

I remember helping newbie students at USC who were very confused that even though they made the changes "PL/C USES", their program didn't work! 

> On May 19, 2024, at 5:58 PM, Rob Pike <robpike@gmail.com> wrote:
> 
> The Cornell PL/I compiler, PL/C, ran on the IBM 360 so of course used batch input. It tried automatically to keep things running after a parsing error by inserting some token - semicolon, parenthesis, whatever seemed best - and continuing to parse, in order to maximize the amount of input that could be parsed before giving up. At least, that's what I took the motivation to be. It rarely succeeded in fixing the actual problem, despite PL/I being plastered with semicolons, but it did tend to ferret out more errors per run. I found the tactic helpful.
> 
> -rob
> 


[-- Attachment #2: Type: text/html, Size: 1443 bytes --]

[TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)

[arnold]

Warner Losh <imp@bsdimp.com> wrote:

> > (Yeah, I've just changed the subject, feel free to stay on topic. :-)
>
> Awk bailing out near line 1.

	$ gawk --nostalgia
	awk: bailing out near line 1
	Aborted (core dumped)

A very long time Easter Egg... :-)

Arnold

[TUHS] A fuzzy awk. (Was: The 'usage: ...' message.)

[Ralph Corderoy]

Hi Arnold,

> > in order to maximize the amount of input that could be parsed before
> > giving up.
>
> Gawk used to do this, until people started fuzzing it, causing
> cascading errors and eventually core dumps.  Now the first syntax
> error is fatal.

This is the first time I've heard of making life difficult for fuzzers
so I'm curious...

I'm assuming you agree the eventual core dump was a bug somewhere to be
fixed, and probably was.  Stopping on the first error lessens the
‘attack surface’ for the fuzzer.  Do you think there remains a bug which
would bite a user which the fuzzer might have found more easily before
the shrunken surface?

-- 
Cheers, Ralph.

[TUHS] Re: A fuzzy awk. (Was: The 'usage: ...' message.)

[arnold]

Ralph Corderoy <ralph@inputplus.co.uk> wrote:

> This is the first time I've heard of making life difficult for fuzzers
> so I'm curious...

I was making life easier for me. :-)

> I'm assuming you agree the eventual core dump was a bug somewhere to be
> fixed, and probably was.

Not really. Hugely syntactically invalid programs can end up causing
memory corruption as necessary data structures don't get built correctly
(or at all); since they're invalid, subsequent bits of gawk that expect
valid data structures end up not working.  These are "bugs" that can't
happen when using the tool correctly.

> Stopping on the first error lessens the ‘attack surface’ for the
> fuzzer.  Do you think there remains a bug which would bite a user which
> the fuzzer might have found more easily before the shrunken surface?

No.

I don't have any examples handy, but you can look back through the
bug-gawk archives for some examples of these reports.  The number
of true bugs that fuzzers have caught (if any!) could be counted
on one hand.

Sometimes they like to claim that the "bugs" they find could cause
denial of service attacks. That's also specious, gawk isn't used for
long-running server kinds of programs.

The joys of being a Free Software Maintainer.

Arnold

P.S. I don't claim that gawk is bug-free.  But I do think that there
are qualitatively different kinds of bugs, and bug reports.

[TUHS] Re: A fuzzy awk. (Was: The 'usage: ...' message.)

[Chet Ramey]

[-- Attachment #1.1: Type: text/plain, Size: 1487 bytes --]

On 5/20/24 5:20 AM, Ralph Corderoy wrote:
> Hi Arnold,
> 
>>> in order to maximize the amount of input that could be parsed before
>>> giving up.
>>
>> Gawk used to do this, until people started fuzzing it, causing
>> cascading errors and eventually core dumps.  Now the first syntax
>> error is fatal.
> 
> This is the first time I've heard of making life difficult for fuzzers
> so I'm curious...

It's not making life difficult for them -- they can still fuzz all they
want. Chances are better they'll find a genuine bug if you stop right away.


> I'm assuming you agree the eventual core dump was a bug somewhere to be
> fixed, and probably was.  > Stopping on the first error lessens the
> ‘attack surface’ for the fuzzer.  Do you think there remains a bug which
> would bite a user which the fuzzer might have found more easily before
> the shrunken surface?

Chances are small. (People fuzz bash all the time, and that is my
experience.)

Look at it this way. Free Software maintainers have limited resources. Is
it better to spend time on bugs that will affect a larger percentage of
the user population, instead of those that require artificial circumstances
that won't be encountered by normal usage? Those get pushed down on the
priority list.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

[TUHS] Re: A fuzzy awk.

[Ralph Corderoy]

Hi Chet,

> Is it better to spend time on bugs that will affect a larger
> percentage of the user population, instead of those that require
> artificial circumstances that won't be encountered by normal usage?
> Those get pushed down on the priority list.

You're talking about pushing unlikely, fuzzed bugs down the prioritised
list, but we're discussing those bugs not getting onto the list for
consideration.

Lack of resources also applies to triaging bugs and I agree a fuzzed bug
which hands over a 42 KiB of dense, gibberish awk will probably not get
volunteer attention.  But then fuzzers can seek a smaller test case,
similar to Andreas Zeller's delta debugging.

I'm in no way criticising Arnold who, like you, has spent many years
voluntarily enhancing a program many of us use every day.  But it's
interesting to shine some light on this corner to better understand
what's happening.

-- 
Cheers, Ralph.

[TUHS] Re: A fuzzy awk.

[Chet Ramey]

[-- Attachment #1.1: Type: text/plain, Size: 1341 bytes --]

On 5/20/24 9:30 AM, Ralph Corderoy wrote:
> Hi Chet,
> 
>> Is it better to spend time on bugs that will affect a larger
>> percentage of the user population, instead of those that require
>> artificial circumstances that won't be encountered by normal usage?
>> Those get pushed down on the priority list.
> 
> You're talking about pushing unlikely, fuzzed bugs down the prioritised
> list, but we're discussing those bugs not getting onto the list for
> consideration.

I think the question is whether they were bugs in gawk at all, or the
result of gawk trying to be helpful by guessing at the script's intent
and trying to go on. Arnold's reaction to that, which had these negative
effects most often as the result of fuzzing attempts, was to exit on the
first syntax error.

Would those `bugs' have manifested themselves if gawk hadn't tried to do
this? Are they bugs at all? Guessing at intent is bound to be wrong some
of the time, and cause errors of its own.

I'm saying that fuzzing does occasionally find obscure bugs -- bugs that
would never be encountered in normal usage -- and those should be fixed.
Eventually.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
		 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

[TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)

[Clem Cole]

[-- Attachment #1: Type: text/plain, Size: 3354 bytes --]

I was going to keep silent on this one until I realized I disagree with
both Doug and Rob here (always a little dangerous). But because of personal
experience, I have a pretty strong opinion is not really a win.  Note that I
cribbed this email response from an answer I wrote on Quora to the
question:

*When you are programming and commit a minor error, such as forgetting a
semicolon, the compiler throws an error and makes you fix it for yourself.
Why doesn’t it just fix it by itself and notify you of the fix instead?*


FWIW: The first version of the idea that I now about was DWIM - *Do What I
Mean* feature from BBN’s LISP (that eventually made it into InterLISP). As
the Wikipedia page describes DWIM became known as "Damn Warren's Infernal
Machine" [more details in the DWIM section of the jargon file].  As Doug
points out, the original Pascal implementation for Unix, pix(1), also
supported this idea of fixing your code for you, and as Rob points out, UCB’s
pix(1) took the idea of trying to keep going and make the compile work from
the earlier Cornell PL/C compiler for the IBM 360[1], which to quote
Wikipedia:

“The PL/C compiler had the unusual capability of never failing to compile
> any program, through the use of extensive automatic correction of many
> syntax errors and by converting any remaining syntax errors to output
> statements.”


The problem is that people can be lazy, and instead of using " DWIM" as a
tool to speed up their development and fix their own errors, they just
ignore the errors. In fact, when we were teaching the “Intro to CS” course
at UCB in the early 1980s; we actually had students turn in programs that
had syntax errors in them because pix(1) had corrected their code -- instead
of the student fixing his/her code before handing the program into the TA
(and then they would complain when they got “marked down” on the assignment
— sigh).

IMO: All in all, the experiment failed because many (??most??) people
really don’t work that way. Putting a feature like this in an IDE or even
an editor like emacs might be reasonable since the sources would be
modified, but it means you need to like using an IDE. I also ask --> what
happens when the computer’s (IDE) guess is different from the programmer's
real intent, and since it was ‘fixed’ behind the curtain, you don’t notice
it?

Some other people have suggested that DWIM isn’t a little like spelling
‘auto-correct’ or tools like ‘Grammarly.’ The truth is, I have a love/hate
relationship with auto-correct, particularly on my mobile devices. I'm
dyslexic, so tools like this can be helpful to me sometimes, but I spend a
great deal of my time fighting these types of tools because they are so
often wrong, particularly with a small screen/keyboard, that it is just
“not fun.”

This brings me back to my experience. IMO, auto-correct for programming is
like DWIM all over again, and the cure causes more problems than it solves.

Clem

[1] I should add that after Cornell’s PL/C compiler was introduced, IBM
eventually added a similar feature to its own PL/1, although it was not
nearly as extensive as the Cornell solution. I’m sure you can find people
who liked it, but in both cases, I personally never found it that useful.

>
ᐧ

[-- Attachment #2: Type: text/html, Size: 5841 bytes --]

[TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)

[Greg A. Woods]

[-- Attachment #1: Type: text/plain, Size: 724 bytes --]

At Mon, 20 May 2024 10:23:56 -0400, Clem Cole <clemc@ccc.com> wrote:
Subject: [TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)
>
> This brings me back to my experience. IMO, auto-correct for programming is
> like DWIM all over again, and the cure causes more problems than it solves.

We're deep down that rabbit hole this time with LLM/GPT systems
generating large swathes of code that I believe all too often gets into
production without any human programmer fully vetting its fitness for
purpose, or perhaps even understanding it.

--
					Greg A. Woods <gwoods@acm.org>

Kelowna, BC     +1 250 762-7675           RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>     Avoncote Farms <woods@avoncote.ca>

[-- Attachment #2: OpenPGP Digital Signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)

[Stuff Received]

On 2024-05-19 20:58, Rob Pike wrote:
> The Cornell PL/I compiler, PL/C, ran on the IBM 360 so of course used 
> batch input. It tried automatically to keep things running after a 
> parsing error by inserting some token - semicolon, parenthesis, whatever 
> seemed best - and continuing to parse, in order to maximize the amount 
> of input that could be parsed before giving up. At least, that's what I 
> took the motivation to be. It rarely succeeded in fixing the actual 
> problem, despite PL/I being plastered with semicolons, but it did tend 
> to ferret out more errors per run. I found the tactic helpful.
> 
> -rob
> 

Possibly way off topic but Toronto allowed anyone to run PL/C decks for 
free, which I often did.  One day, they decided to allow all of the card 
to be read as text and my card numbers generated all sorts of errors. 
(At least easily fixed by a visit the card punch.)

S.

[TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)

[John Levine]

It appears that Clem Cole <clemc@ccc.com> said:
>“The PL/C compiler had the unusual capability of never failing to compile
>> any program, through the use of extensive automatic correction of many
>> syntax errors and by converting any remaining syntax errors to output
>> statements.”
>
>The problem is that people can be lazy, and instead of using " DWIM" as a
>tool to speed up their development and fix their own errors, they just
>ignore the errors. ...

PL/C was a long time ago in the early 1970s. People used it on batch
systems whre you handed in your cards at the window, waited a while,
and later got your printout back. Or at advanced places, you could
run the cards through the reader yourself, then wait until the batch
ran.

In that environment, the benefit from possibly guessing an error
correction right meant fewer trips to the card reader. In my youth I
did a fair amount of programming that way in WATFOR/WATFIV and Algol W
where we really tried to get the programs right since we wanted to
finish up and go home.

When I was using interactive systems where you could fix one bug and
try again, over and over, it seemed like cheating.

R's,
John

[TUHS] Re: The 'usage: ...' message. (Was: On Bloat...)

[John Cowan]

[-- Attachment #1: Type: text/plain, Size: 1465 bytes --]

On Mon, May 20, 2024 at 4:11 PM John Levine <johnl@taugh.com> wrote:

It appears that Clem Cole <clemc@ccc.com> said:
> >“The PL/C compiler had the unusual capability of never failing to compile
> >> any program, through the use of extensive automatic correction of many
> >> syntax errors and by converting any remaining syntax errors to output
> >> statements.”
> PL/C was a long time ago in the early 1970s. People used it on batch
> systems whre you handed in your cards at the window, waited a while,
> and later got your printout back. Or at advanced places, you could
> run the cards through the reader yourself, then wait until the batch
> ran.


PL/C was a 3rd-generation autocorrection programming language.  CORC was
the 1962 version and CUPL was the 1966 version (same date as DWIM), neither
of them based on PL/I.  There is an implementation of both at <
http://www.catb.org/~esr/cupl/>.

The Wikipedia DWIM article also points to Magit, the Emacs git client.

>
> In that environment, the benefit from possibly guessing an error
> correction right meant fewer trips to the card reader. In my youth I
> did a fair amount of programming that way in WATFOR/WATFIV and Algol W
> where we really tried to get the programs right since we wanted to
> finish up and go home.
>
> When I was using interactive systems where you could fix one bug and
> try again, over and over, it seemed like cheating.
>
> R's,
> John
>

[-- Attachment #2: Type: text/html, Size: 2636 bytes --]

[TUHS] The 'usage: ...' message. (Was: On Bloat...)

[Douglas McIlroy]

[-- Attachment #1: Type: text/plain, Size: 1099 bytes --]

> was ‘usage: ...’ adopted from an earlier system?

"Usage" was one of those lovely ideas, one exposure to which flips its
status from unknown to eternal truth. I am sure my first exposure was on
Unix, but I don't remember when. Perhaps because it radically departs from
Ken's "?" in qed/ed, I have subconsciously attributed it to Dennis.

The genius of "usage" and "?" is that they don't attempt to tell one what's
wrong. Most diagnostics cite a rule or hidden limit that's been violated or
describe the mistake (e.g. "missing semicolon") , sometimes raising more
questions than they answer.

Another non-descriptive style of error message that I admired was that of
Berkeley Pascal's syntax diagnostics. When the LR parser could not proceed,
it reported where, and automatically provided a sample token that would
allow the parsing to progress. I found this uniform convention to be at
least as informative as distinct hand-crafted messages, which almost by
definition can't foresee every contingency. Alas, this elegant scheme seems
not to have inspired imitators.

Doug

[-- Attachment #2: Type: text/html, Size: 1310 bytes --]

[TUHS] The 'usage: ...' message. (Was: On Bloat...)

[Ralph Corderoy]

Hi,

I wrote:
> Another point against adding --help: there's a second attempt to
> describe the source.

It occurred to me --help's the third attempt as there's already ‘usage:
argv[0] ...’.  Back when running man took time and paper, I can see
a one-line summary to aid memory was useful.  I wondered when it first
appeared.

I've found V2, https://www.tuhs.org/cgi-bin/utree.pl?file=V2/cmd, has
cmp.s with

    cmp     (sp)+,$3
    beq     1f
    jsr     r5,mesg; <Usage: cmp arg1 arg2\n\0>; .even
    sys     exit

And cp.c has

    if(argc != 3) {
	    write(1,"Usage: cp oldfile newfile\n",26);
	    exit();
    }

Given the lack of options, the need for a usage message surprises me.
But then ‘cp a-src a-dest b-src b-dest ...’ used to copy files in pairs.
Perhaps when this was dropped, one too many losses?, the usage was
needed to remind users of the change.

Any earlier Unix examples known by the list?
And was ‘usage: ...’ adopted from an earlier system?

-- 
Cheers, Ralph.