On 5/4/20, Win Treese <treese@acm.org> wrote:
>
> Andy Payne, a recent hire at the lab, had been an intern in DEC’s
> semiconductor group, where he had worked on randomized testing for hardware
> verification. With all the compilers available, he decided to hack up a
> program to generate random small C programs with computable expected
> outputs. His program then compiled the random code with each compiler and
> tested the result. After finding a number of bugs this way, he got tired of
> submitting the bug reports, and changed his program to write and submit the
> bug reports automatically.
>
> This caused a little bit of consternation with some of the compiler teams at
> first.

I remember that very well.  IIRC it was called fuzz testing, and
indeed it was controversial, for the reasons Bill McKeeman discusses
in his paper. [1]  On the one hand, compiler developers said, "nobody
would ever write something like that--we can't waste our time on these
issues when there are real bugs waiting to be fixed."  On the other
hand, some of the bugs that fuzz testing turned up provoked reactions
such as, "OMG!  THAT caused the compiler to crash?"  I think the
turning point was when fixing one of the fuzz testing bugs also fixed
an obscure and hard-to-debug customer problem.  Intel's C and Fortran
compiler team has also used random testing technology.

> [1] https://www.hpl.hp.com/hpjournal/dtj/vol10num1/vol10num1art9.pdf

-Paul W.