[Caml-list] Quick Check like testing for OCaml?

[Caml-list] Quick Check like testing for OCaml?

[Keiko Nakata]

[-- Attachment #1: Type: text/plain, Size: 192 bytes --]

Hi,

I am looking for (reasonably matured and hopefully easy to install)
QuickCheck-like property based testing software for OCaml code.

Any information is appreciated!


Best regards,
Keiko

[-- Attachment #2: Type: text/html, Size: 287 bytes --]

Re: [Caml-list] Quick Check like testing for OCaml?

[Török Edwin]

On 09/13/2015 11:09 AM, Keiko Nakata wrote:
> Hi,
> 
> I am looking for (reasonably matured and hopefully easy to install) QuickCheck-like property based testing software for OCaml code.
> 
> Any information is appreciated!

There is qcheck.0.4, quickcheck.1.0.2 and kaputt.1.2 on opam. 
qcheck has documentation in the mli, integration with OUnit and quite easy to get started with.
Kaputt also has reducers (to produce smaller counterexamples), and SmallCheck-like enumeration tests.

I haven't tried quickcheck, and haven't found an equivalent to SmartCheck's counterexample generalization [1]

[1] https://www.cs.indiana.edu/~lepike/pubs/smartcheck.pdf

Best regards,
--Edwin

Re: [Caml-list] Quick Check like testing for OCaml?

[Simon Cruanes]

[-- Attachment #1: Type: text/plain, Size: 1746 bytes --]

There is also qtest (also called iTeML on github) which is a bit
special: it's a testing framework providing unit testing through OUnit,
and simple random testing; its specificy is that tests can be written in
comments within the module to test, so that the code itself has no additional
dependencies or code bloat due to tests. It is very easy to write new
tests, since you don't have to add specific test modules.
It is used, afaik, at least in Batteries and in containers.

I am probably going to work on making qtest and qcheck a bit closer, if
qtest's developper(s) agree.

Cheers,

Le Sun, 13 Sep 2015, Török Edwin a écrit :
> On 09/13/2015 11:09 AM, Keiko Nakata wrote:
> > Hi,
> > 
> > I am looking for (reasonably matured and hopefully easy to install) QuickCheck-like property based testing software for OCaml code.
> > 
> > Any information is appreciated!
> 
> There is qcheck.0.4, quickcheck.1.0.2 and kaputt.1.2 on opam. 
> qcheck has documentation in the mli, integration with OUnit and quite easy to get started with.
> Kaputt also has reducers (to produce smaller counterexamples), and SmallCheck-like enumeration tests.
> 
> I haven't tried quickcheck, and haven't found an equivalent to SmartCheck's counterexample generalization [1]
> 
> [1] https://www.cs.indiana.edu/~lepike/pubs/smartcheck.pdf
> 
> Best regards,
> --Edwin
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs


-- 
Simon Cruanes

http://weusepgp.info/
key 49AA62B6, fingerprint 949F EB87 8F06 59C6 D7D3  7D8D 4AC0 1D08 49AA 62B6

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Caml-list] Quick Check like testing for OCaml?

[Yaron Minsky]

[-- Attachment #1: Type: text/plain, Size: 2209 bytes --]

Core_kernel has a recently added quickcheck library.  Carl Eastlund, who is
the main author, is finishing up a blog post describing it, but you can
start with the documentation in this file:

https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_generator.mli

On Sun, Sep 13, 2015 at 11:52 AM, Simon Cruanes <simon.cruanes.2007@m4x.org>
wrote:

> There is also qtest (also called iTeML on github) which is a bit
> special: it's a testing framework providing unit testing through OUnit,
> and simple random testing; its specificy is that tests can be written in
> comments within the module to test, so that the code itself has no
> additional
> dependencies or code bloat due to tests. It is very easy to write new
> tests, since you don't have to add specific test modules.
> It is used, afaik, at least in Batteries and in containers.
>
> I am probably going to work on making qtest and qcheck a bit closer, if
> qtest's developper(s) agree.
>
> Cheers,
>
> Le Sun, 13 Sep 2015, Török Edwin a écrit :
> > On 09/13/2015 11:09 AM, Keiko Nakata wrote:
> > > Hi,
> > >
> > > I am looking for (reasonably matured and hopefully easy to install)
> QuickCheck-like property based testing software for OCaml code.
> > >
> > > Any information is appreciated!
> >
> > There is qcheck.0.4, quickcheck.1.0.2 and kaputt.1.2 on opam.
> > qcheck has documentation in the mli, integration with OUnit and quite
> easy to get started with.
> > Kaputt also has reducers (to produce smaller counterexamples), and
> SmallCheck-like enumeration tests.
> >
> > I haven't tried quickcheck, and haven't found an equivalent to
> SmartCheck's counterexample generalization [1]
> >
> > [1] https://www.cs.indiana.edu/~lepike/pubs/smartcheck.pdf
> >
> > Best regards,
> > --Edwin
> >
> > --
> > Caml-list mailing list.  Subscription management and archives:
> > https://sympa.inria.fr/sympa/arc/caml-list
> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
> --
> Simon Cruanes
>
> http://weusepgp.info/
> key 49AA62B6, fingerprint 949F EB87 8F06 59C6 D7D3  7D8D 4AC0 1D08 49AA
> 62B6
>

[-- Attachment #2: Type: text/html, Size: 3355 bytes --]

Re: [Caml-list] Quick Check like testing for OCaml?

[Gabriel Scherer]

[-- Attachment #1: Type: text/plain, Size: 2685 bytes --]

>
>
> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_generator.mli
>
> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_observer.mli
>

This is very nice.
I hope the library will also get shrinking support, because it not so easy
to do, and important in practice.

On Tue, Sep 15, 2015 at 3:09 PM, Yaron Minsky <yminsky@janestreet.com>
wrote:

> Core_kernel has a recently added quickcheck library.  Carl Eastlund, who
> is the main author, is finishing up a blog post describing it, but you can
> start with the documentation in this file:
>
>
> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_generator.mli
>
> On Sun, Sep 13, 2015 at 11:52 AM, Simon Cruanes <
> simon.cruanes.2007@m4x.org> wrote:
>
>> There is also qtest (also called iTeML on github) which is a bit
>> special: it's a testing framework providing unit testing through OUnit,
>> and simple random testing; its specificy is that tests can be written in
>> comments within the module to test, so that the code itself has no
>> additional
>> dependencies or code bloat due to tests. It is very easy to write new
>> tests, since you don't have to add specific test modules.
>> It is used, afaik, at least in Batteries and in containers.
>>
>> I am probably going to work on making qtest and qcheck a bit closer, if
>> qtest's developper(s) agree.
>>
>> Cheers,
>>
>> Le Sun, 13 Sep 2015, Török Edwin a écrit :
>> > On 09/13/2015 11:09 AM, Keiko Nakata wrote:
>> > > Hi,
>> > >
>> > > I am looking for (reasonably matured and hopefully easy to install)
>> QuickCheck-like property based testing software for OCaml code.
>> > >
>> > > Any information is appreciated!
>> >
>> > There is qcheck.0.4, quickcheck.1.0.2 and kaputt.1.2 on opam.
>> > qcheck has documentation in the mli, integration with OUnit and quite
>> easy to get started with.
>> > Kaputt also has reducers (to produce smaller counterexamples), and
>> SmallCheck-like enumeration tests.
>> >
>> > I haven't tried quickcheck, and haven't found an equivalent to
>> SmartCheck's counterexample generalization [1]
>> >
>> > [1] https://www.cs.indiana.edu/~lepike/pubs/smartcheck.pdf
>> >
>> > Best regards,
>> > --Edwin
>> >
>> > --
>> > Caml-list mailing list.  Subscription management and archives:
>> > https://sympa.inria.fr/sympa/arc/caml-list
>> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> > Bug reports: http://caml.inria.fr/bin/caml-bugs
>>
>>
>> --
>> Simon Cruanes
>>
>> http://weusepgp.info/
>> key 49AA62B6, fingerprint 949F EB87 8F06 59C6 D7D3  7D8D 4AC0 1D08 49AA
>> 62B6
>>
>
>

[-- Attachment #2: Type: text/html, Size: 4482 bytes --]

Re: [Caml-list] Quick Check like testing for OCaml?

[Carl Eastlund]

[-- Attachment #1: Type: text/plain, Size: 3332 bytes --]

Glad you like it.  Our Quickcheck implementation is new and its design is
still somewhat of an experiment, so I can't claim it's a mature library as
originally asked for in this thread, but of course we appreciate feedback
from early adopters.  As Yaron said, I'm writing up a blog post that will
go into more detail about it.

We just had an intern, Daniel Spencer, implement shrinking.  That will make
its way to the public release shortly.

On Tue, Sep 15, 2015 at 10:06 AM, Gabriel Scherer <gabriel.scherer@gmail.com
> wrote:

>
>> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_generator.mli
>>
>> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_observer.mli
>>
>
> This is very nice.
> I hope the library will also get shrinking support, because it not so easy
> to do, and important in practice.
>
> On Tue, Sep 15, 2015 at 3:09 PM, Yaron Minsky <yminsky@janestreet.com>
> wrote:
>
>> Core_kernel has a recently added quickcheck library.  Carl Eastlund, who
>> is the main author, is finishing up a blog post describing it, but you can
>> start with the documentation in this file:
>>
>>
>> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_generator.mli
>>
>> On Sun, Sep 13, 2015 at 11:52 AM, Simon Cruanes <
>> simon.cruanes.2007@m4x.org> wrote:
>>
>>> There is also qtest (also called iTeML on github) which is a bit
>>> special: it's a testing framework providing unit testing through OUnit,
>>> and simple random testing; its specificy is that tests can be written in
>>> comments within the module to test, so that the code itself has no
>>> additional
>>> dependencies or code bloat due to tests. It is very easy to write new
>>> tests, since you don't have to add specific test modules.
>>> It is used, afaik, at least in Batteries and in containers.
>>>
>>> I am probably going to work on making qtest and qcheck a bit closer, if
>>> qtest's developper(s) agree.
>>>
>>> Cheers,
>>>
>>> Le Sun, 13 Sep 2015, Török Edwin a écrit :
>>> > On 09/13/2015 11:09 AM, Keiko Nakata wrote:
>>> > > Hi,
>>> > >
>>> > > I am looking for (reasonably matured and hopefully easy to install)
>>> QuickCheck-like property based testing software for OCaml code.
>>> > >
>>> > > Any information is appreciated!
>>> >
>>> > There is qcheck.0.4, quickcheck.1.0.2 and kaputt.1.2 on opam.
>>> > qcheck has documentation in the mli, integration with OUnit and quite
>>> easy to get started with.
>>> > Kaputt also has reducers (to produce smaller counterexamples), and
>>> SmallCheck-like enumeration tests.
>>> >
>>> > I haven't tried quickcheck, and haven't found an equivalent to
>>> SmartCheck's counterexample generalization [1]
>>> >
>>> > [1] https://www.cs.indiana.edu/~lepike/pubs/smartcheck.pdf
>>> >
>>> > Best regards,
>>> > --Edwin
>>> >
>>> > --
>>> > Caml-list mailing list.  Subscription management and archives:
>>> > https://sympa.inria.fr/sympa/arc/caml-list
>>> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>>> > Bug reports: http://caml.inria.fr/bin/caml-bugs
>>>
>>>
>>> --
>>> Simon Cruanes
>>>
>>> http://weusepgp.info/
>>> key 49AA62B6, fingerprint 949F EB87 8F06 59C6 D7D3  7D8D 4AC0 1D08 49AA
>>> 62B6
>>>
>>
>>
>


-- 
Carl Eastlund

[-- Attachment #2: Type: text/html, Size: 5593 bytes --]

Re: [Caml-list] Quick Check like testing for OCaml?

[Simon Cruanes]

[-- Attachment #1: Type: text/plain, Size: 3691 bytes --]

I am working on adding shrinking into qtest
(https://github.com/vincent-hugot/iTeML/), and I'm curious about what
design choices you did. Haskell literate mentions

shrink: 'a -> 'a list

to shrink values, but on large counter-examples like large lists, it
might be very slow. I'm trying to use iterators to generate shrunk(?)
values lazily.

Also, do Core users write random generators or shrinking functions
themselves, or do you have a deriver for them? I assume you use "with sexp"
to display counter-examples.

Cheers,

Le Tue, 15 Sep 2015, Carl Eastlund a écrit :
> Glad you like it.  Our Quickcheck implementation is new and its design is
> still somewhat of an experiment, so I can't claim it's a mature library as
> originally asked for in this thread, but of course we appreciate feedback
> from early adopters.  As Yaron said, I'm writing up a blog post that will
> go into more detail about it.
> 
> We just had an intern, Daniel Spencer, implement shrinking.  That will make
> its way to the public release shortly.
> 
> On Tue, Sep 15, 2015 at 10:06 AM, Gabriel Scherer <gabriel.scherer@gmail.com
> > wrote:
> 
> >
> >> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_generator.mli
> >>
> >> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_observer.mli
> >>
> >
> > This is very nice.
> > I hope the library will also get shrinking support, because it not so easy
> > to do, and important in practice.
> >
> > On Tue, Sep 15, 2015 at 3:09 PM, Yaron Minsky <yminsky@janestreet.com>
> > wrote:
> >
> >> Core_kernel has a recently added quickcheck library.  Carl Eastlund, who
> >> is the main author, is finishing up a blog post describing it, but you can
> >> start with the documentation in this file:
> >>
> >>
> >> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_generator.mli
> >>
> >> On Sun, Sep 13, 2015 at 11:52 AM, Simon Cruanes <
> >> simon.cruanes.2007@m4x.org> wrote:
> >>
> >>> There is also qtest (also called iTeML on github) which is a bit
> >>> special: it's a testing framework providing unit testing through OUnit,
> >>> and simple random testing; its specificy is that tests can be written in
> >>> comments within the module to test, so that the code itself has no
> >>> additional
> >>> dependencies or code bloat due to tests. It is very easy to write new
> >>> tests, since you don't have to add specific test modules.
> >>> It is used, afaik, at least in Batteries and in containers.
> >>>
> >>> I am probably going to work on making qtest and qcheck a bit closer, if
> >>> qtest's developper(s) agree.
> >>>
> >>> Cheers,
> >>>
> >>> Le Sun, 13 Sep 2015, Török Edwin a écrit :
> >>> > On 09/13/2015 11:09 AM, Keiko Nakata wrote:
> >>> > > Hi,
> >>> > >
> >>> > > I am looking for (reasonably matured and hopefully easy to install)
> >>> QuickCheck-like property based testing software for OCaml code.
> >>> > >
> >>> > > Any information is appreciated!
> >>> >
> >>> > There is qcheck.0.4, quickcheck.1.0.2 and kaputt.1.2 on opam.
> >>> > qcheck has documentation in the mli, integration with OUnit and quite
> >>> easy to get started with.
> >>> > Kaputt also has reducers (to produce smaller counterexamples), and
> >>> SmallCheck-like enumeration tests.
> >>> >
> >>> > I haven't tried quickcheck, and haven't found an equivalent to
> >>> SmartCheck's counterexample generalization [1]
> >>> >
> >>> > [1] https://www.cs.indiana.edu/~lepike/pubs/smartcheck.pdf
> >>> >


-- 
Simon Cruanes

http://weusepgp.info/
key 49AA62B6, fingerprint 949F EB87 8F06 59C6 D7D3  7D8D 4AC0 1D08 49AA 62B6

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Caml-list] Quick Check like testing for OCaml?

[Carl Eastlund]

[-- Attachment #1: Type: text/plain, Size: 4501 bytes --]

Currently generators and shrinkers (and observers, which generate functions
-- the analog to CoArbitrary in Haskell) are all written manually.  I do
intend to add syntax for them, but that's been pending on a few other
updates, especially our switch from camlp4 to ppx syntax extensions.  We do
print counterexamples using s-expressions, as you say.

Our shrinkers use the type ('a -> 'a Sequence.t), where Sequence.t is a
lazy, unmemoized sequence type in Core.

On Tue, Sep 15, 2015 at 10:30 AM, Simon Cruanes <simon.cruanes.2007@m4x.org>
wrote:

> I am working on adding shrinking into qtest
> (https://github.com/vincent-hugot/iTeML/), and I'm curious about what
> design choices you did. Haskell literate mentions
>
> shrink: 'a -> 'a list
>
> to shrink values, but on large counter-examples like large lists, it
> might be very slow. I'm trying to use iterators to generate shrunk(?)
> values lazily.
>
> Also, do Core users write random generators or shrinking functions
> themselves, or do you have a deriver for them? I assume you use "with sexp"
> to display counter-examples.
>
> Cheers,
>
> Le Tue, 15 Sep 2015, Carl Eastlund a écrit :
> > Glad you like it.  Our Quickcheck implementation is new and its design is
> > still somewhat of an experiment, so I can't claim it's a mature library
> as
> > originally asked for in this thread, but of course we appreciate feedback
> > from early adopters.  As Yaron said, I'm writing up a blog post that will
> > go into more detail about it.
> >
> > We just had an intern, Daniel Spencer, implement shrinking.  That will
> make
> > its way to the public release shortly.
> >
> > On Tue, Sep 15, 2015 at 10:06 AM, Gabriel Scherer <
> gabriel.scherer@gmail.com
> > > wrote:
> >
> > >
> > >>
> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_generator.mli
> > >>
> > >>
> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_observer.mli
> > >>
> > >
> > > This is very nice.
> > > I hope the library will also get shrinking support, because it not so
> easy
> > > to do, and important in practice.
> > >
> > > On Tue, Sep 15, 2015 at 3:09 PM, Yaron Minsky <yminsky@janestreet.com>
> > > wrote:
> > >
> > >> Core_kernel has a recently added quickcheck library.  Carl Eastlund,
> who
> > >> is the main author, is finishing up a blog post describing it, but
> you can
> > >> start with the documentation in this file:
> > >>
> > >>
> > >>
> https://github.com/janestreet/core_kernel/blob/master/src/quickcheck_generator.mli
> > >>
> > >> On Sun, Sep 13, 2015 at 11:52 AM, Simon Cruanes <
> > >> simon.cruanes.2007@m4x.org> wrote:
> > >>
> > >>> There is also qtest (also called iTeML on github) which is a bit
> > >>> special: it's a testing framework providing unit testing through
> OUnit,
> > >>> and simple random testing; its specificy is that tests can be
> written in
> > >>> comments within the module to test, so that the code itself has no
> > >>> additional
> > >>> dependencies or code bloat due to tests. It is very easy to write new
> > >>> tests, since you don't have to add specific test modules.
> > >>> It is used, afaik, at least in Batteries and in containers.
> > >>>
> > >>> I am probably going to work on making qtest and qcheck a bit closer,
> if
> > >>> qtest's developper(s) agree.
> > >>>
> > >>> Cheers,
> > >>>
> > >>> Le Sun, 13 Sep 2015, Török Edwin a écrit :
> > >>> > On 09/13/2015 11:09 AM, Keiko Nakata wrote:
> > >>> > > Hi,
> > >>> > >
> > >>> > > I am looking for (reasonably matured and hopefully easy to
> install)
> > >>> QuickCheck-like property based testing software for OCaml code.
> > >>> > >
> > >>> > > Any information is appreciated!
> > >>> >
> > >>> > There is qcheck.0.4, quickcheck.1.0.2 and kaputt.1.2 on opam.
> > >>> > qcheck has documentation in the mli, integration with OUnit and
> quite
> > >>> easy to get started with.
> > >>> > Kaputt also has reducers (to produce smaller counterexamples), and
> > >>> SmallCheck-like enumeration tests.
> > >>> >
> > >>> > I haven't tried quickcheck, and haven't found an equivalent to
> > >>> SmartCheck's counterexample generalization [1]
> > >>> >
> > >>> > [1] https://www.cs.indiana.edu/~lepike/pubs/smartcheck.pdf
> > >>> >
>
>
> --
> Simon Cruanes
>
> http://weusepgp.info/
> key 49AA62B6, fingerprint 949F EB87 8F06 59C6 D7D3  7D8D 4AC0 1D08 49AA
> 62B6
>



-- 
Carl Eastlund

[-- Attachment #2: Type: text/html, Size: 6732 bytes --]