[ANN] Kaputt 1.0 alpha

[ANN] Kaputt 1.0 alpha

[barista]

This post announces the 1.0 alpha version of Kaputt.
Kaputt is a testing tool for the Objective Caml language.
Home page: http://kaputt.x9c.fr

Features:
   - assertion-based tests (inspired by the xUnit tools):
      the developer writes assertions by explicitly stating
      input and waited output values for the tested function;
   - specification-based tests (inspired by the QuickCheck tool):
      the developer writes (using combinators) a specification
      of the tested function and asks the library to randomly generate
      tests cases.

Dependencies:
   - Objective Caml 3.10.2

Planned features:
   - more combinators and predicates (e.g. over Map, Set, etc.);
   - generalization of generator to accept other sources (e.g. streams).


This is clearly alpha work, so any suggestion/criticism will be
welcome in order to enhance/correct it.

Xavier Clerc

Re: [Caml-list] [ANN] Kaputt 1.0 alpha

[Stefano Zacchiroli]

On Sun, Nov 23, 2008 at 09:57:37PM +0100, barista@x9c.fr wrote:
> This is clearly alpha work, so any suggestion/criticism will be
> welcome in order to enhance/correct it.

Hi Xavier, thanks for this.

You are probably aware of oUnit [1] which, AFAIK, was the only testing
library available for OCaml thus far. Hence I guess you developed
Kaputt to achieve something which was not possible with oUnit or to
achieve it somehow differently.

Can you please comment over the differences between Kaputt and oUnit?

I'm quite sure it would be interesting for a lot of us.

TIA,
Cheers.

[1] http://www.xs4all.nl/~mmzeeman/ocaml/ounit-doc/OUnit.html

-- 
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..|  .  |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

Re: [Caml-list] [ANN] Kaputt 1.0 alpha

[forum]

Selon Stefano Zacchiroli <zack@upsilon.cc>:

> On Sun, Nov 23, 2008 at 09:57:37PM +0100, barista@x9c.fr wrote:
> > This is clearly alpha work, so any suggestion/criticism will be
> > welcome in order to enhance/correct it.
>
> Hi Xavier, thanks for this.
>
> You are probably aware of oUnit [1] which, AFAIK, was the only testing
> library available for OCaml thus far. Hence I guess you developed
> Kaputt to achieve something which was not possible with oUnit or to
> achieve it somehow differently.
>
> Can you please comment over the differences between Kaputt and oUnit?

I was indeed aware of oUnit when I started working on Kaputt.
The initial intent was to provide OCaml with something along
the lines of the QuickCheck library developed for Haskell
(http://www.cs.chalmers.se/~rjmh/QuickCheck/). Clones of QuickCheck
exist for various languages (caml-list@inria.fr) but not for OCaml,
even if a QuickCheck equivalent is provided by the Reins library
(http://ocaml-reins.sourceforge.net/).

Put shortly, the idea of QuickCheck is to encode the specification
of a function using predicates and to ask the framework to generate
random test cases to check that the function adheres to its specification.

After designing this part of the library, I realized that it would
be quite annoying (at least for me, the very first user) to have
two libraries to code my tests. That's why I decided to add to Kaputt
the functionalities found in oUnit.

In summary: Kaptt = oUnit + {QuickCheck part of Reins}.
People may get angry at such effort duplication, and I would understand
it. My point is to put forward a unique library specialized in testing.


Xavier

Re: [Caml-list] [ANN] Kaputt 1.0 alpha

[forum]

Selon forum@x9c.fr:

> Selon Stefano Zacchiroli <zack@upsilon.cc>:
>
> > On Sun, Nov 23, 2008 at 09:57:37PM +0100, barista@x9c.fr wrote:
>
> Clones of QuickCheck exist for various languages
> (caml-list@inria.fr) but not for OCaml

Hum, copy/paste mistake, the URL for QuickCheck clones is:
   http://en.wikipedia.org/wiki/Quickcheck

Re: [Caml-list] [ANN] Kaputt 1.0 alpha

[Yoann Padioleau]

forum@x9c.fr writes:

Hi Xavier, 

> Selon Stefano Zacchiroli <zack@upsilon.cc>:
>
>> On Sun, Nov 23, 2008 at 09:57:37PM +0100, barista@x9c.fr wrote:
>> > This is clearly alpha work, so any suggestion/criticism will be
>> > welcome in order to enhance/correct it.
>>
>> Hi Xavier, thanks for this.
>>
>> You are probably aware of oUnit [1] which, AFAIK, was the only testing
>> library available for OCaml thus far. Hence I guess you developed
>> Kaputt to achieve something which was not possible with oUnit or to
>> achieve it somehow differently.
>>
>> Can you please comment over the differences between Kaputt and oUnit?
>
> I was indeed aware of oUnit when I started working on Kaputt.
> The initial intent was to provide OCaml with something along
> the lines of the QuickCheck library developed for Haskell
> (http://www.cs.chalmers.se/~rjmh/QuickCheck/). Clones of QuickCheck
> exist for various languages (caml-list@inria.fr) but not for OCaml,

Actually I posted a few years ago a small module imitating parts of quicheck:

 http://aryx.kicks-ass.org/~pad/ocaml/quickcheck.ml

Apparently they even now use my ugly file in the Janet Street Core Library.


> even if a QuickCheck equivalent is provided by the Reins library
> (http://ocaml-reins.sourceforge.net/).

I didn't find much in this ocaml reins library ... and it
looks quite heavy in functors ...


Your library looks very good, very clean. I just noticed you missed 
the 'pair' and 'tuple' random generators.


One criticism is that like many other libraries, such as oUnit, 
ocamlcalendar, they seem a little bit heavyweight. You got hundreds
of functions, 5 modules ... 


>
> Put shortly, the idea of QuickCheck is to encode the specification
> of a function using predicates and to ask the framework to generate
> random test cases to check that the function adheres to its specification.
>
> After designing this part of the library, I realized that it would
> be quite annoying (at least for me, the very first user) to have
> two libraries to code my tests. That's why I decided to add to Kaputt
> the functionalities found in oUnit.
>
> In summary: Kaptt = oUnit + {QuickCheck part of Reins}.
> People may get angry at such effort duplication, and I would understand
> it. My point is to put forward a unique library specialized in testing.
>
>
> Xavier
>
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [ANN] Kaputt 1.0 alpha

[Sylvain Le Gall]

On 24-11-2008, Yoann Padioleau <padator@wanadoo.fr> wrote:
> forum@x9c.fr writes:
>
> One criticism is that like many other libraries, such as oUnit, 
> ocamlcalendar, they seem a little bit heavyweight. You got hundreds
> of functions, 5 modules ... 
>

I always found oUnit quite lightweight compared to other. It is part of
those library small enough to understand quickly. So I don't agree on
the fact that "oUnit" is heavyweight.

Regards,
Sylvain Le Gall

Re: [Caml-list] [ANN] Kaputt 1.0 alpha

[forum]

Le 24 nov. 08 à 17:52, Yoann Padioleau a écrit :

> forum@x9c.fr writes:
>
> Hi Xavier,
>
>> Selon Stefano Zacchiroli <zack@upsilon.cc>:
>>
>>> (...)
>>
>> I was indeed aware of oUnit when I started working on Kaputt.
>> The initial intent was to provide OCaml with something along
>> the lines of the QuickCheck library developed for Haskell
>> (http://www.cs.chalmers.se/~rjmh/QuickCheck/). Clones of QuickCheck
>> exist for various languages (caml-list@inria.fr) but not for OCaml,
>
> Actually I posted a few years ago a small module imitating parts of  
> quicheck:
>
> http://aryx.kicks-ass.org/~pad/ocaml/quickcheck.ml

Sorry, I failed to remember this.


> Apparently they even now use my ugly file in the Janet Street Core  
> Library.
>
>
>> even if a QuickCheck equivalent is provided by the Reins library
>> (http://ocaml-reins.sourceforge.net/).
>
> I didn't find much in this ocaml reins library ... and it
> looks quite heavy in functors ...

Yes, I indeed wanted to rely upon functions/combinators instead.
It seems lighter this way.


> Your library looks very good, very clean. I just noticed you missed
> the 'pair' and 'tuple' random generators.

Well, maybe their names are not intuitive but Generator.zip{1..5}
allows to combine n generators into a generator returning tuples
of order n (zip1 being the identity function).


> One criticism is that like many other libraries, such as oUnit,
> ocamlcalendar, they seem a little bit heavyweight. You got hundreds
> of functions, 5 modules ...

That's true but it seems hard to have a combinator-based library with
few functions/combinators. With too few combinators, you leave too
much work to the developer (including a lot a boilerplate code).
With too many combinators, you request the developer to take more
time in order to use the library. I find it hard to calibrate the size  
of
the library.

Maybe things could be done in a lighter way by using objects ?


Xavier

Re: [Caml-list] [ANN] Kaputt 1.0 alpha

[Christophe Raffalli]

[-- Attachment #1.1: Type: text/plain, Size: 3152 bytes --]

forum@x9c.fr a écrit :
>
> Le 24 nov. 08 à 17:52, Yoann Padioleau a écrit :
>
>> forum@x9c.fr writes:
>>
>> Hi Xavier,
>>
>>> Selon Stefano Zacchiroli <zack@upsilon.cc>:
>>>
>>>> (...)
>>>
>>> I was indeed aware of oUnit when I started working on Kaputt.
>>> The initial intent was to provide OCaml with something along
>>> the lines of the QuickCheck library developed for Haskell
>>> (http://www.cs.chalmers.se/~rjmh/QuickCheck/). Clones of QuickCheck
>>> exist for various languages (caml-list@inria.fr) but not for OCaml,
>>
>> Actually I posted a few years ago a small module imitating parts of
>> quicheck:
>>
>> http://aryx.kicks-ass.org/~pad/ocaml/quickcheck.ml
>
> Sorry, I failed to remember this.
>
>
>> Apparently they even now use my ugly file in the Janet Street Core
>> Library.
>>
>>
>>> even if a QuickCheck equivalent is provided by the Reins library
>>> (http://ocaml-reins.sourceforge.net/).
>>
>> I didn't find much in this ocaml reins library ... and it
>> looks quite heavy in functors ...
>
> Yes, I indeed wanted to rely upon functions/combinators instead.
> It seems lighter this way.
>
>
>> Your library looks very good, very clean. I just noticed you missed
>> the 'pair' and 'tuple' random generators.
>
> Well, maybe their names are not intuitive but Generator.zip{1..5}
> allows to combine n generators into a generator returning tuples
> of order n (zip1 being the identity function).
>
>
>> One criticism is that like many other libraries, such as oUnit,
>> ocamlcalendar, they seem a little bit heavyweight. You got hundreds
>> of functions, 5 modules ...
>
> That's true but it seems hard to have a combinator-based library with
> few functions/combinators. With too few combinators, you leave too
> much work to the developer (including a lot a boilerplate code).
> With too many combinators, you request the developer to take more
> time in order to use the library. I find it hard to calibrate the size of
> the library.
>
> Maybe things could be done in a lighter way by using objects ?
Or a camlp4 extension to build the intended function(s) by induction on
the type definition.
you could include that as part of deriving
(http://code.google.com/p/deriving/)

Hope this helps,
Christophe
>
>
> Xavier
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>


-- 
Christophe Raffalli
Universite de Savoie
Batiment Le Chablais, bureau 21
73376 Le Bourget-du-Lac Cedex

tel: (33) 4 79 75 81 03
fax: (33) 4 79 75 87 42
mail: Christophe.Raffalli@univ-savoie.fr
www: http://www.lama.univ-savoie.fr/~RAFFALLI
---------------------------------------------
IMPORTANT: this mail is signed using PGP/MIME
At least Enigmail/Mozilla, mutt or evolution 
can check this signature. The public key is
stored on www.keyserver.net
---------------------------------------------


[-- Attachment #1.2: Christophe_Raffalli.vcf --]
[-- Type: text/x-vcard, Size: 310 bytes --]

begin:vcard
fn:Christophe Raffalli
n:Raffalli;Christophe
org:LAMA (UMR 5127)
email;internet:christophe.raffalli@univ-savoie.fr
title;quoted-printable:Ma=C3=AEtre de conf=C3=A9rences
tel;work:+33 4 79 75 81 03
note:http://www.lama.univ-savoie.fr/~raffalli
x-mozilla-html:TRUE
version:2.1
end:vcard


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 249 bytes --]

Re: [Caml-list] [ANN] Kaputt 1.0 alpha

[Yoann Padioleau]

Christophe Raffalli <Christophe.Raffalli@univ-savoie.fr> writes:

>>> Your library looks very good, very clean. I just noticed you missed
>>> the 'pair' and 'tuple' random generators.
>>
>> Well, maybe their names are not intuitive but Generator.zip{1..5}
>> allows to combine n generators into a generator returning tuples
>> of order n (zip1 being the identity function).
>>
>>
>>> One criticism is that like many other libraries, such as oUnit,
>>> ocamlcalendar, they seem a little bit heavyweight. You got hundreds
>>> of functions, 5 modules ...
>>
>> That's true but it seems hard to have a combinator-based library with
>> few functions/combinators. With too few combinators, you leave too
>> much work to the developer (including a lot a boilerplate code).
>> With too many combinators, you request the developer to take more
>> time in order to use the library. I find it hard to calibrate the size of
>> the library.
>>
>> Maybe things could be done in a lighter way by using objects ?

> Or a camlp4 extension to build the intended function(s) by induction on
> the type definition.
> you could include that as part of deriving
> (http://code.google.com/p/deriving/)

Which would make it even more heavyweight by forcing now the client
to use campl4, to download your deriving library, to understand
this deriving stuff, to infer the names of the functions
now that they will be hidden/internally-generated by campl4 ... 

Look at the ocaml standard library, why do you like it ? 
Because it's simple. A few types, a "kernel" of essential functions,
not too much functors (well in fact I would prefer even less functors
such as a simpler Map and Set like the Hashtbl module). KISS.

>
> Hope this helps,
> Christophe
>>
>>
>> Xavier
>>

Re: [Caml-list] [ANN] Kaputt 1.0 alpha

[forum]

Le 24 nov. 08 à 22:49, Christophe Raffalli a écrit :
>>
>> (...)
>>
>> Maybe things could be done in a lighter way by using objects ?
> Or a camlp4 extension to build the intended function(s) by induction  
> on
> the type definition.
> you could include that as part of deriving
> (http://code.google.com/p/deriving/)


Good idea.
I would regard a deriving-based "class" for generation
of test cases as a very useful enhancement.

However, I do not know the policy of the "deriving"
developers. Would they accept a "class" relying on
a library besides the standard library ?


Xavier

Re: [Caml-list] [ANN] Kaputt 1.0 alpha

[Jeremy Yallop]

forum@x9c.fr wrote:
> Le 24 nov. 08 à 22:49, Christophe Raffalli a écrit :
>> Or a camlp4 extension to build the intended function(s) by induction on
>> the type definition.
>> you could include that as part of deriving
>> (http://code.google.com/p/deriving/)
> 
> Good idea.
> I would regard a deriving-based "class" for generation
> of test cases as a very useful enhancement.
> 
> However, I do not know the policy of the "deriving"
> developers. Would they accept a "class" relying on
> a library besides the standard library ?

Yes: such contributions would be welcome.  It wouldn't be necessary for 
users of "deriving" to link in your library unless they were actually 
using it.

It would also be possible to distribute your "class" separately from 
deriving.  The design of deriving (based on the underlying camlp4 
facility for loading object files dynamically) allows classes (i.e. code 
generators) to be loaded at runtime, so there's no need to modify the 
whole program just to add a new class.

Jeremy.

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.