Re: [Caml-list] Use of unit testing frameworks in OCaml

[Gabriel Scherer <gabriel.scherer@gmail.com>]

Could you be more specific in what kind of benefits you expect such a
migration would bring? I'm possibly not familiar enough with these
frameworks, but it is rather unclear to me what "significant
improvement to the maintenability" we should expect.

Note that many tests in the compiler testsuite are in fact rather
integration tests than unit tests: they test a complete run of the
compiler (or toplevel) on an input program, rather that a single
function of the compiler codebase.

If I had time to devote to improve the compiler testsuite, here are
the two changes I would be interested in implementing (or getting
implemented for me):
- A way to collect coverage checking information (which parts of the
compiler codebase are exercized by the tests?). I started working on
it once by just adding Bisect somewhere in the compiler build system,
but lost motivation before getting into a a work-out-of-the-box state
(it generated many independent coverage files, one for each test, that
would have to be aggregated). I think that should not be too hard.
- A separation between the tests that are expected to be long-running
and the others. Currently running the testsuite takes a few minutes on
my machine. If I could get a subset of the tests that has the same
coverage for changes that do not affect the code generation backend
running in under one minute, that would improve the hack-build-test
feedback loops.
- A way to run in parallel the tests that are not sensitive to
parallel execution (those that do not test performances or thread
scheduling).

All of these might be easier with a more elegant/declarative/structure
test framework, but it is not immediately obvious to me that switching
to oUnit would do the job.

On Wed, Sep 9, 2015 at 10:35 PM, John Christopher McAlpine
<jmcalpin@nmt.edu> wrote:
> From what I can tell, the tests for the OCaml compiler and standard
> libraries are all done with a fairly small framework designed specifically
> for testing the compiler and standard libraries. Is there any particular
> reason not to use a tool such as OUnit or Broken for these tests? I am aware
> that adding dependencies to the compiler is generally frowned upon, but
> these libraries could significantly improve the maintainability of the core
> language and libraries.
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs