On Tue, May 5, 2020 at 1:37 PM Paul Winalski wrote: > On 5/4/20, Win Treese wrote: > > > > Andy Payne, a recent hire at the lab, had been an intern in DEC’s > > semiconductor group, where he had worked on randomized testing for > hardware > > verification. With all the compilers available, he decided to hack up a > > program to generate random small C programs with computable expected > > outputs. His program then compiled the random code with each compiler and > > tested the result. After finding a number of bugs this way, he got tired > of > > submitting the bug reports, and changed his program to write and submit > the > > bug reports automatically. > > > > This caused a little bit of consternation with some of the compiler > teams at > > first. > > I remember that very well. IIRC it was called fuzz testing, and > indeed it was controversial, for the reasons Bill McKeeman discusses > in his paper. [1] On the one hand, compiler developers said, "nobody > would ever write something like that--we can't waste our time on these > issues when there are real bugs waiting to be fixed." On the other > hand, some of the bugs that fuzz testing turned up provoked reactions > such as, "OMG! THAT caused the compiler to crash?" I think the > turning point was when fixing one of the fuzz testing bugs also fixed > an obscure and hard-to-debug customer problem. Intel's C and Fortran > compiler team has also used random testing technology. > Ah, very cool. The same approach has come into favor again recently. I've dealt personally with https://github.com/google/syzkaller, which is a kernel fuzzer that generates random inputs to system calls and detects e.g. panics. It's a neat approach. - Dan C. > [1] https://www.hpl.hp.com/hpjournal/dtj/vol10num1/vol10num1art9.pdf > > -Paul W. >