[Caml-list] testing private functions with oUnit

[Caml-list] testing private functions with oUnit

[Eric Cooper]

I'd like to write unit tests for functions not exported in a .mli
file.  The only way I can see is to remove the .mli file while
building the test, so the whole .ml file is visible.  Is there a better
way, preferably integrated with ocamlmake + findlib?

-- 
Eric Cooper             e c c @ c m u . e d u

Re: [Caml-list] testing private functions with oUnit

[Eric Cooper]

> [...] preferably integrated with ocamlmake + findlib?

s/ocamlmake/ocamlbuild/

-- 
Eric Cooper             e c c @ c m u . e d u

Re: [Caml-list] testing private functions with oUnit

[Ivan Gotovchits]

A common solution is to combine all internal modules tests in one function inside the module, and to export it, so that it can be called from ounit.

On 28 сент. 2014 г., at 19:12, Eric Cooper <ecc@cmu.edu> wrote:

>> [...] preferably integrated with ocamlmake + findlib?
> 
> s/ocamlmake/ocamlbuild/
> 
> -- 
> Eric Cooper             e c c @ c m u . e d u
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] testing private functions with oUnit

[Malcolm Matalka]

Out of curiosity: why?  Any private function should be exercised through
an API function somehow.

IME, testing private functions often makes refactoring more painful
without a clear win in code being better tested.

If your API is pure, btw, you should checkout QCheck for testing it
instead of unit tests.

My 2 cents,
/M

Eric Cooper <ecc@cmu.edu> writes:

> I'd like to write unit tests for functions not exported in a .mli
> file.  The only way I can see is to remove the .mli file while
> building the test, so the whole .ml file is visible.  Is there a better
> way, preferably integrated with ocamlmake + findlib?
>
> -- 
> Eric Cooper             e c c @ c m u . e d u

Re: [Caml-list] testing private functions with oUnit

[Francois Berenger]

On 09/29/2014 09:08 AM, Malcolm Matalka wrote:
> Out of curiosity: why?  Any private function should be exercised through
> an API function somehow.
>
> IME, testing private functions often makes refactoring more painful
> without a clear win in code being better tested.
>
> If your API is pure, btw, you should checkout QCheck for testing it
> instead of unit tests.

You can also use qtest (available in opam) and write your test code 
directly as comments in the .ml files, as is done in batteries:

https://github.com/ocaml-batteries-team/batteries-included/blob/master/src/batList.mlv

Look for (*$T in the code for example.

> My 2 cents,
> /M
>
> Eric Cooper <ecc@cmu.edu> writes:
>
>> I'd like to write unit tests for functions not exported in a .mli
>> file.  The only way I can see is to remove the .mli file while
>> building the test, so the whole .ml file is visible.  Is there a better
>> way, preferably integrated with ocamlmake + findlib?
>>
>> --
>> Eric Cooper             e c c @ c m u . e d u
>

-- 
Regards,
Francois.

Re: [Caml-list] testing private functions with oUnit

[ygrek]

On Sun, 28 Sep 2014 19:06:38 -0400
Eric Cooper <ecc@cmu.edu> wrote:

> I'd like to write unit tests for functions not exported in a .mli
> file.  The only way I can see is to remove the .mli file while
> building the test, so the whole .ml file is visible.  Is there a better
> way, preferably integrated with ocamlmake + findlib?

One can also reverse that - write the tests in the .ml file and register
in the global mutable list of tests, then iterate over that list during test run.
One downside is that it relies on all modules being linked in.

-- 

Re: [Caml-list] testing private functions with oUnit

[Jeremie Dimino]

[-- Attachment #1: Type: text/plain, Size: 741 bytes --]

On Mon, Sep 29, 2014 at 9:28 AM, ygrek <ygrek@autistici.org> wrote:

> One can also reverse that - write the tests in the .ml file and register
> in the global mutable list of tests, then iterate over that list during
> test run.
> One downside is that it relies on all modules being linked in.
>

That's what pa_ounit [1] is doing. It allows you to write this in the
source code:

TEST name? = <boolean expr> (* true means ok, false or exn means broken *)
TEST_UNIT name? = <unit expr> (* () means ok, exn means broken *)
TEST_MODULE name? = <module expr> (* to group TESTs (to share some setup
for instance) *)

And have the tests run if you pass specific command line arugments.

  [1] https://github.com/janestreet/pa_ounit

-- 
Jeremie

[-- Attachment #2: Type: text/html, Size: 1304 bytes --]

Re: [Caml-list] testing private functions with oUnit

[Gerd Stolpmann]

[-- Attachment #1: Type: text/plain, Size: 2400 bytes --]

I think there is some missing functionality in OCaml: a mechanism to
grant access to something that is normally hidden. This is not only
important for unit testing but for debugging in general (remember that
even ocamldebug cannot break module abstractions).

What about this idea: modules (and only modules) can have associated
visibility attributes. These are set with the definition or in the
signature, e.g.

module Implementation { "debug" } = struct ... end

and are part of the cmi file. This imposes a restriction on the module
path - Implementation may then only occur as part of a module path if it
is explicitly allowed (e.g. that could be a command-line switch, and the
debugger would allow everything).

When using this technique, you'd normally put everything into an
Implementation sub-module that is access-protected, and you'd redefine
what is part of the regular interface:

module Implementation { "debug" } = struct ... end

import Implementation
  (* or redefine definition by definition: 
     let my_function = Implementation.my_function *)

And in the mli (which may now even be superfluous unless you want to
document the API):

val my_function : ...
module Implementation { "debug" } : sig ... end

If there was a special ANY module type that unifies with anything:

module Implementation { "debug" } : ANY

This would then simply export all definitions.

There could be additional utilities for stripping definitions with
access control tokens from the cmi files. The whole point is to grant
the developer of a certain library more rights than the user of a
library.

Gerd


Am Sonntag, den 28.09.2014, 19:06 -0400 schrieb Eric Cooper:
> I'd like to write unit tests for functions not exported in a .mli
> file.  The only way I can see is to remove the .mli file while
> building the test, so the whole .ml file is visible.  Is there a better
> way, preferably integrated with ocamlmake + findlib?
> 
> -- 
> Eric Cooper             e c c @ c m u . e d u
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Caml-list] testing private functions with oUnit

[Gerd Stolpmann]

[-- Attachment #1: Type: text/plain, Size: 1622 bytes --]


> module Implementation { "debug" } = struct ... end
> 
> import Implementation

I meant: include Implementation. Sorry for the noise.

>   (* or redefine definition by definition: 
>      let my_function = Implementation.my_function *)
> 
> And in the mli (which may now even be superfluous unless you want to
> document the API):
> 
> val my_function : ...
> module Implementation { "debug" } : sig ... end
> 
> If there was a special ANY module type that unifies with anything:
> 
> module Implementation { "debug" } : ANY
> 
> This would then simply export all definitions.
> 
> There could be additional utilities for stripping definitions with
> access control tokens from the cmi files. The whole point is to grant
> the developer of a certain library more rights than the user of a
> library.
> 
> Gerd
> 
> 
> Am Sonntag, den 28.09.2014, 19:06 -0400 schrieb Eric Cooper:
> > I'd like to write unit tests for functions not exported in a .mli
> > file.  The only way I can see is to remove the .mli file while
> > building the test, so the whole .ml file is visible.  Is there a better
> > way, preferably integrated with ocamlmake + findlib?
> > 
> > -- 
> > Eric Cooper             e c c @ c m u . e d u
> > 
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
My OCaml site:          http://www.camlcity.org
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Caml-list] testing private functions with oUnit

[Maxence Guesdon]

On Mon, 29 Sep 2014 12:48:06 +0200
Gerd Stolpmann <info@gerd-stolpmann.de> wrote:

> I think there is some missing functionality in OCaml: a mechanism to
> grant access to something that is normally hidden. This is not only
> important for unit testing but for debugging in general (remember that
> even ocamldebug cannot break module abstractions).
> 
> What about this idea: modules (and only modules) can have associated
> visibility attributes. These are set with the definition or in the
> signature, e.g.
> 
> module Implementation { "debug" } = struct ... end
> [...]

Couldn't recently introduced attributes be used to filter in or out
some parts of modules depending on a command line flag ?

- m

Re: [Caml-list] testing private functions with oUnit

[Malcolm Matalka]

I believe .Net has the concept of visibility at the module, package, and
external layer.  Something like this would be nice, although I have no
idea what it would look like, in Ocaml since often I want to create
modules internal to a package to make the code cleaner but those
functions have to be visible to everyone.

But, IMO, even though it would be nice I don't think it's stopping
myself from writing anything.

Gerd Stolpmann <info@gerd-stolpmann.de> writes:

> I think there is some missing functionality in OCaml: a mechanism to
> grant access to something that is normally hidden. This is not only
> important for unit testing but for debugging in general (remember that
> even ocamldebug cannot break module abstractions).
>
> What about this idea: modules (and only modules) can have associated
> visibility attributes. These are set with the definition or in the
> signature, e.g.
>
> module Implementation { "debug" } = struct ... end
>
> and are part of the cmi file. This imposes a restriction on the module
> path - Implementation may then only occur as part of a module path if it
> is explicitly allowed (e.g. that could be a command-line switch, and the
> debugger would allow everything).
>
> When using this technique, you'd normally put everything into an
> Implementation sub-module that is access-protected, and you'd redefine
> what is part of the regular interface:
>
> module Implementation { "debug" } = struct ... end
>
> import Implementation
>   (* or redefine definition by definition: 
>      let my_function = Implementation.my_function *)
>
> And in the mli (which may now even be superfluous unless you want to
> document the API):
>
> val my_function : ...
> module Implementation { "debug" } : sig ... end
>
> If there was a special ANY module type that unifies with anything:
>
> module Implementation { "debug" } : ANY
>
> This would then simply export all definitions.
>
> There could be additional utilities for stripping definitions with
> access control tokens from the cmi files. The whole point is to grant
> the developer of a certain library more rights than the user of a
> library.
>
> Gerd
>
>
> Am Sonntag, den 28.09.2014, 19:06 -0400 schrieb Eric Cooper:
>> I'd like to write unit tests for functions not exported in a .mli
>> file.  The only way I can see is to remove the .mli file while
>> building the test, so the whole .ml file is visible.  Is there a better
>> way, preferably integrated with ocamlmake + findlib?
>> 
>> -- 
>> Eric Cooper             e c c @ c m u . e d u
>> 

Re: [Caml-list] testing private functions with oUnit

[Eric Cooper]

On Mon, Sep 29, 2014 at 07:08:21AM +0000, Malcolm Matalka wrote:
> Out of curiosity: why?  Any private function should be exercised through
> an API function somehow.

In this particular case, the private function produces a pair of
values. The API provides a function to get the first element, but the
second is used only internally to the module, in a code path that
is quite complex to exercise.

> If your API is pure, btw, you should checkout QCheck for testing it
> instead of unit tests.

Thanks, but it's extremely impure :-)  Lots of manipulations of the
file system, etc.

-- 
Eric Cooper             e c c @ c m u . e d u