[Caml-list] Preferred layout for new packages

[Caml-list] Preferred layout for new packages

[Marek Kubica]

Hi,

I'm kinda new to the OCaml eco system and therefore a bit confused on a
number of issues:

1. Build system: what to use? I have tried OMake some time ago and
liked the automatic recompilation, but writing the OMakefiles was quite
awful and it does not seem popular. What is the state of the art
solution?

2. Unit tests: I used OUnit which was okay, but maybe there are better
solutions? I've seen that there is Kaputt and I have seen that there is
https://github.com/camlunity/ocaml-quickcheck as well as
https://github.com/vincent-hugot/iTeML which was extracted from
batteries recently.

3. Stdlib: I don't mind depending on batteries and/or core, are there
any reasons against? Especially in the unit tests it drove me nuts that
I wasn't able to display results without writing printers, and I know
batteries has at least a generic printer.

regards,
Marek

Re: [Caml-list] Preferred layout for new packages

[Török Edwin]

On 11/14/2012 01:43 PM, Marek Kubica wrote:
> Hi,
> 
> I'm kinda new to the OCaml eco system and therefore a bit confused on a
> number of issues:
> 
> 1. Build system: what to use? I have tried OMake some time ago and
> liked the automatic recompilation, but writing the OMakefiles was quite
> awful and it does not seem popular.

OMake has some nice features for more complex projects, for example
polling the FS for changes and automatically rebuilding.

However it requires your users to have OMake installed too, so
ocamlbuild has an advantage here (it is installed together with the compiler).

> What is the state of the art
> solution?

Try oasis, it provides a good starting point:
http://oasis.forge.ocamlcore.org/

You can later extend it with custom rules in _tags/myocamlbuild.ml if needed.

> 
> 2. Unit tests: I used OUnit which was okay, but maybe there are better
> solutions? I've seen that there is Kaputt and I have seen that there is
> https://github.com/camlunity/ocaml-quickcheck as well as
> https://github.com/vincent-hugot/iTeML which was extracted from
> batteries recently.

Kaputt seems to be a superset of OUnit feature-wise, don't know about the others.
I've only used OUnit.

> 
> 3. Stdlib: I don't mind depending on batteries and/or core, are there
> any reasons against? Especially in the unit tests it drove me nuts that
> I wasn't able to display results without writing printers, and I know
> batteries has at least a generic printer.

Are you writing an application or a library?
If you're writing a library it is probably better to avoid unneeded dependencies, or provide optional
subpackages (see the recent thread about pgocaml.batteries).
That way people who don't use batteries/core can still use your package.

Best regards,
--Edwin

Re: [Caml-list] Preferred layout for new packages

[Edgar Friendly]

On 11/14/2012 6:43 AM, Marek Kubica wrote:
> Hi,
>
> I'm kinda new to the OCaml eco system and therefore a bit confused on a
> number of issues:
>
> 1. Build system: what to use? I have tried OMake some time ago and
> liked the automatic recompilation, but writing the OMakefiles was quite
> awful and it does not seem popular. What is the state of the art
> solution?
The community is trying to standardize on Oasis, which generates 
ocamlbuild-based build systems.  In the future, this may generate some 
other build system if a better one comes out.
> 2. Unit tests: I used OUnit which was okay, but maybe there are better
> solutions? I've seen that there is Kaputt and I have seen that there is
> https://github.com/camlunity/ocaml-quickcheck as well as
> https://github.com/vincent-hugot/iTeML which was extracted from
> batteries recently.
iTeML is more of a test extraction framework, and builds on top of OUnit 
and a hacked version of the Jane St. quickcheck library.  It can easily 
support other frameworks, and it would be interesting to explore a 
custom framework designed with iTeML in mind.
> 3. Stdlib: I don't mind depending on batteries and/or core, are there
> any reasons against? Especially in the unit tests it drove me nuts that
> I wasn't able to display results without writing printers, and I know
> batteries has at least a generic printer.
Batteries' dump function is a terrible printer for anything beyond the 
most basic of purposes, just as the Pervasives.compare is also 
terrible.  Batteries' printer combinators (`List.print Int.print stdout 
[1;2;3]`) are what I use for most purposes.

E.

Re: [Caml-list] Preferred layout for new packages

[Marek Kubica]

Hi,

On Wed, 14 Nov 2012 16:41:19 +0200
Török Edwin <edwin+ml-ocaml@etorok.net> wrote:

> OMake has some nice features for more complex projects, for example
> polling the FS for changes and automatically rebuilding.

Yes, but nowadays I can just use inotifywait in a loop which
essentially does the same but frees me from dealing with its OMakefile
syntax which I did not like very much.

> > What is the state of the art
> > solution?
> 
> Try oasis, it provides a good starting point:
> http://oasis.forge.ocamlcore.org/
> 
> You can later extend it with custom rules in _tags/myocamlbuild.ml if
> needed.

Great, this looks really good. The file format seems simple and
intuitive.

> > 3. Stdlib: I don't mind depending on batteries and/or core, are
> > there any reasons against? Especially in the unit tests it drove me
> > nuts that I wasn't able to display results without writing
> > printers, and I know batteries has at least a generic printer.
> 
> Are you writing an application or a library?
> If you're writing a library it is probably better to avoid unneeded
> dependencies, or provide optional subpackages (see the recent thread
> about pgocaml.batteries). That way people who don't use
> batteries/core can still use your package.

I am writing a library (and also a command line program as a thin
wrapper over the library). As you said, in optional parts like tests I
might want to use core/batteries. Also, kind-of depends on whether they
offer other things that I might need or not. Not planning on reinventing
the wheel.

regards,
Marek

Re: [Caml-list] Preferred layout for new packages

[Marek Kubica]

On Wed, 14 Nov 2012 09:42:09 -0500
Edgar Friendly <thelema314@gmail.com> wrote:

> The community is trying to standardize on Oasis, which generates 
> ocamlbuild-based build systems.  In the future, this may generate
> some other build system if a better one comes out.

So OASIS it is. Fine.

> > 2. Unit tests: I used OUnit which was okay, but maybe there are
> > better solutions? I've seen that there is Kaputt and I have seen
> > that there is https://github.com/camlunity/ocaml-quickcheck as well
> > as https://github.com/vincent-hugot/iTeML which was extracted from
> > batteries recently.
> iTeML is more of a test extraction framework, and builds on top of
> OUnit and a hacked version of the Jane St. quickcheck library.  It
> can easily support other frameworks, and it would be interesting to
> explore a custom framework designed with iTeML in mind.

I actually like test extraction frameworks, tools like nose and py.test
have made writing tests with Python much nicer, that's why I'm somehow
unimpressed how verbose OUnit is. But having the test code in a comment
seems ugly to me. Maybe there could be some CamlP4 hack to exclude it
on normal compilation?

> > 3. Stdlib: I don't mind depending on batteries and/or core, are
> > there any reasons against? Especially in the unit tests it drove me
> > nuts that I wasn't able to display results without writing
> > printers, and I know batteries has at least a generic printer.
> Batteries' dump function is a terrible printer for anything beyond
> the most basic of purposes, just as the Pervasives.compare is also 
> terrible.  Batteries' printer combinators (`List.print Int.print
> stdout [1;2;3]`) are what I use for most purposes.

That's a great hint, thanks!

regards,
Marek

Re: [Caml-list] Preferred layout for new packages

[forum]

Le 14 nov. 2012 à 18:00, Marek Kubica a écrit :

> On Wed, 14 Nov 2012 09:42:09 -0500
> Edgar Friendly <thelema314@gmail.com> wrote:
> 
>> (…)
> 
> I actually like test extraction frameworks, tools like nose and py.test
> have made writing tests with Python much nicer, that's why I'm somehow
> unimpressed how verbose OUnit is. But having the test code in a comment
> seems ugly to me. Maybe there could be some CamlP4 hack to exclude it
> on normal compilation?

Being not found of extraction myself, I made another proposal in the
latest revision of Kaputt: instead of using mli/ml file couples, I tend to
now use mli/ml/mlt file triples. In this setting, the mlt file simply contains
the code related to tests. Then, at compilation a small camlp4 preprocessor
concatenates the contents of ml and mlt files before actual compilation.

My understanding is that it reconciles two antagonistic goals:
  - separate application code from test code;
  - allow test code to access unexported elements.

I would be glad to hear what people think of this scheme, independently
of the Kaputt library itself (which I am not advertising here). One can read
more in section 3.3 of the manual available at:
    http://kaputt.x9c.fr/downloads.html


Regards,

Xavier Clerc
 

Re: [Caml-list] Preferred layout for new packages

[Martin Jambon]

My current, personal preferences are the following:

Use for in-house development:
- omake
- batteries (some of it)

Avoid in new projects:
- camlp4/camlp5
- ocamlbuild

I'm glad for the many libraries and tools written by the community. 
They are all fine as long as they work and are:
- maintained, unlikely to break anytime soon or easy to take over or 
replace;
- not invasive.

Martin

On Wed 14 Nov 2012 09:00:12 AM PST, Marek Kubica wrote:
> On Wed, 14 Nov 2012 09:42:09 -0500
> Edgar Friendly <thelema314@gmail.com> wrote:
>
>> The community is trying to standardize on Oasis, which generates
>> ocamlbuild-based build systems.  In the future, this may generate
>> some other build system if a better one comes out.
>
> So OASIS it is. Fine.
>
>>> 2. Unit tests: I used OUnit which was okay, but maybe there are
>>> better solutions? I've seen that there is Kaputt and I have seen
>>> that there is https://github.com/camlunity/ocaml-quickcheck as well
>>> as https://github.com/vincent-hugot/iTeML which was extracted from
>>> batteries recently.
>> iTeML is more of a test extraction framework, and builds on top of
>> OUnit and a hacked version of the Jane St. quickcheck library.  It
>> can easily support other frameworks, and it would be interesting to
>> explore a custom framework designed with iTeML in mind.
>
> I actually like test extraction frameworks, tools like nose and py.test
> have made writing tests with Python much nicer, that's why I'm somehow
> unimpressed how verbose OUnit is. But having the test code in a comment
> seems ugly to me. Maybe there could be some CamlP4 hack to exclude it
> on normal compilation?
>
>>> 3. Stdlib: I don't mind depending on batteries and/or core, are
>>> there any reasons against? Especially in the unit tests it drove me
>>> nuts that I wasn't able to display results without writing
>>> printers, and I know batteries has at least a generic printer.
>> Batteries' dump function is a terrible printer for anything beyond
>> the most basic of purposes, just as the Pervasives.compare is also
>> terrible.  Batteries' printer combinators (`List.print Int.print
>> stdout [1;2;3]`) are what I use for most purposes.
>
> That's a great hint, thanks!
>
> regards,
> Marek
>

Re: [Caml-list] Preferred layout for new packages

[Markus Mottl]

On Wed, Nov 14, 2012 at 1:17 PM, Martin Jambon
<martin.jambon@ens-lyon.org> wrote:
> Use for in-house development:
> - omake
> - batteries (some of it)
>
> Avoid in new projects:
> - camlp4/camlp5
> - ocamlbuild

I wish ocamlbuild were a serious alternative to omake.  Though I use
the former for distributing libraries, I also prefer omake for more
complex projects.

Concerning camlp4: what alternative are you considering?  Though
camlp4 has its good share of warts, it also offers useful
functionality that I don't see anywhere else yet.

Regards,
Markus

-- 
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

Re: [Caml-list] Preferred layout for new packages

[Martin Jambon]

On Wed 14 Nov 2012 10:48:37 AM PST, Markus Mottl wrote:
> On Wed, Nov 14, 2012 at 1:17 PM, Martin Jambon
> <martin.jambon@ens-lyon.org> wrote:
>> Use for in-house development:
>> - omake
>> - batteries (some of it)
>>
>> Avoid in new projects:
>> - camlp4/camlp5
>> - ocamlbuild
>
> I wish ocamlbuild were a serious alternative to omake.  Though I use
> the former for distributing libraries, I also prefer omake for more
> complex projects.
>
> Concerning camlp4: what alternative are you considering?  Though
> camlp4 has its good share of warts, it also offers useful
> functionality that I don't see anywhere else yet.

For deriving code from type definitions: atdgen
For generating OCaml code without Camlp4 quotations: 
https://github.com/MyLifeLabs/atd/blob/master/atd_indent.mli
For conditional compilation and simple macros: cppo
For quotation expansion: cppo -x
For Lwt: standard OCaml syntax
Mikmatch (regexps): simplified implementation in progress 
https://github.com/mjambon/rematch
CSV data: going through JSON (handled with atdgen)
Forms derived from type definitions: no need for me now but I would 
write something like atdgen, based on the atd syntax


Martin

Re: [Caml-list] Preferred layout for new packages

[Francois Berenger]

On 11/14/2012 11:41 PM, Török Edwin wrote:
> On 11/14/2012 01:43 PM, Marek Kubica wrote:
>> Hi,
>>
>> I'm kinda new to the OCaml eco system and therefore a bit confused on a
>> number of issues:
>>
>> 1. Build system: what to use? I have tried OMake some time ago and
>> liked the automatic recompilation, but writing the OMakefiles was quite
>> awful and it does not seem popular.
>
> OMake has some nice features for more complex projects, for example
> polling the FS for changes and automatically rebuilding.
>
> However it requires your users to have OMake installed too, so
> ocamlbuild has an advantage here (it is installed together with the compiler).

Nowadays, OPAM allows to install most OCaml software and libraries easily.

For example:
$ opam install omake
Will install omake for you.

Cf. http://opam.ocamlpro.com/ or
https://github.com/OCamlPro/opam

Regards,
F.

>> What is the state of the art
>> solution?
>
> Try oasis, it provides a good starting point:
> http://oasis.forge.ocamlcore.org/
>
> You can later extend it with custom rules in _tags/myocamlbuild.ml if needed.
>
>>
>> 2. Unit tests: I used OUnit which was okay, but maybe there are better
>> solutions? I've seen that there is Kaputt and I have seen that there is
>> https://github.com/camlunity/ocaml-quickcheck as well as
>> https://github.com/vincent-hugot/iTeML which was extracted from
>> batteries recently.
>
> Kaputt seems to be a superset of OUnit feature-wise, don't know about the others.
> I've only used OUnit.
>
>>
>> 3. Stdlib: I don't mind depending on batteries and/or core, are there
>> any reasons against? Especially in the unit tests it drove me nuts that
>> I wasn't able to display results without writing printers, and I know
>> batteries has at least a generic printer.
>
> Are you writing an application or a library?
> If you're writing a library it is probably better to avoid unneeded dependencies, or provide optional
> subpackages (see the recent thread about pgocaml.batteries).
> That way people who don't use batteries/core can still use your package.
>
> Best regards,
> --Edwin
>

Re: [Caml-list] Preferred layout for new packages

[Cedric Cellier]

> But having the test code in a comment
> seems ugly to me. Maybe there could be some CamlP4 hack to exclude it
> on normal compilation?

Since you mentioned python, a culture where
unit tests are routinely written in
comments, Im surprised by your opinion.
Do you think all tests in comments are ugly or is there something specific to batteries tests
that make these ugly?

Re: [Caml-list] Preferred layout for new packages

[Marek Kubica]

Hello Cedric,

On Thu, 15 Nov 2012 07:36:59 +0100
Cedric Cellier <rixed@happyleptic.org> wrote:

> > But having the test code in a comment
> > seems ugly to me. Maybe there could be some CamlP4 hack to exclude
> > it on normal compilation?
> 
> Since you mentioned python, a culture where
> unit tests are routinely written in
> comments, Im surprised by your opinion.
> Do you think all tests in comments are ugly or is there something
> specific to batteries tests that make these ugly?

Actually, In Python tests are not routinely written in comments. They
are usually written in Files called test_whatever.py as test_whatever
functions. Afterwards, people use test discovery tools like nose or
py.test that search for such modules and construct JUnit-style TestCase
classes out of these funtions and execute them.

The advantage is that writing a test only requires to write a single
function and does not need registration. Before there were test
discovery tools people wrote test methods in test classes and executed
that manually -- that is, they didn't write tests really, because it
was terribly tedious.

Your point about tests in comments is not entirely unbased: there are
doctests which are written in document comments and represent a
recorded console session that gets played back as test. Fancy concept,
but not many people use it.

In my opinion I dislike unit tests in comments, also because you throw
away editor support and it feels hackish, especially if the language
has a way of syntactic extension. Python doctests are on the verge of
unreadability, but given it is meant as a console session, it is still
kind of okay though not perfect.

Hope that answer helps you :)

regards,
Marek

Re: [Caml-list] Preferred layout for new packages

[Philippe Veber]

[-- Attachment #1: Type: text/plain, Size: 3175 bytes --]

 I think it's critical here to distinguish two purposes:
1. providing (compiler-checked) examples of a function for documentation
2. writing tests to check a module's functions seriously.

Putting test code in comments is a very nice way to achieve (1), but a
terrible way to achieve (2). In a documentation you want to illustrate the
use of your function, but writing a test for the function is not the most
direct, hence the most intelligible way to do so. Writing a test and an
example are two different things and you may confuse a user when putting
the former instead of the latter in your documentation. So I'd say code
extraction from documentation is very nice, only if it serves the purpose
of the documentation (of course the extraction enables to check the
examples, which is a nice feature).

Your mlt proposal seems to me a very nice way to organize test code (in the
second sense), as it emphasizes through the file extensions the following
holy trinity: specification, implementation and tests. Up to now, I would
write a separate directory with tests (written with oUnit), but now with
your suggestion in mind, it really feels wrong to do things that way.

Maybe the only limitation of your approach is that the test modules have to
respect the dependencies between the library modules: if A uses B then your
tests in B cannot use values in A. But I don't think it matters much, as
we're talking about unit test here.

Thanks for this proposal, I will definitely give it a try!

ph.




2012/11/14 forum@x9c.fr <forum@x9c.fr>

>
> Le 14 nov. 2012 à 18:00, Marek Kubica a écrit :
>
> > On Wed, 14 Nov 2012 09:42:09 -0500
> > Edgar Friendly <thelema314@gmail.com> wrote:
> >
> >> (…)
> >
> > I actually like test extraction frameworks, tools like nose and py.test
> > have made writing tests with Python much nicer, that's why I'm somehow
> > unimpressed how verbose OUnit is. But having the test code in a comment
> > seems ugly to me. Maybe there could be some CamlP4 hack to exclude it
> > on normal compilation?
>
> Being not found of extraction myself, I made another proposal in the
> latest revision of Kaputt: instead of using mli/ml file couples, I tend to
> now use mli/ml/mlt file triples. In this setting, the mlt file simply
> contains
> the code related to tests. Then, at compilation a small camlp4 preprocessor
> concatenates the contents of ml and mlt files before actual compilation.
>
> My understanding is that it reconciles two antagonistic goals:
>   - separate application code from test code;
>   - allow test code to access unexported elements.
>
> I would be glad to hear what people think of this scheme, independently
> of the Kaputt library itself (which I am not advertising here). One can
> read
> more in section 3.3 of the manual available at:
>     http://kaputt.x9c.fr/downloads.html
>
>
> Regards,
>
> Xavier Clerc
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 4032 bytes --]

Re: [Caml-list] Preferred layout for new packages

[rixed]

-[ Thu, Nov 15, 2012 at 08:24:43AM +0100, Marek Kubica ]----
> Actually, In Python tests are not routinely written in comments.

Well, at least here where I work, the python devs are more acustomed to
doctests than external tests.  And I think it's a clever trick as it serves
both the purpose of testing and documenting a function.

Of course this is not suitable for more in-depth testing, where external tests
are required. This is what's done in batteries: short and simple tests are
written inline where they are usefull to document how to use a function (there
were discussions about inserting these inline tests in the generated
documentation !), and longuer tests comes in external files compiled
separately. 

Re: [Caml-list] Preferred layout for new packages

[Aleksey Nogin]

On 15.11.2012 01:20, rixed@happyleptic.org wrote:

> -[ Thu, Nov 15, 2012 at 12:13:58AM -0800, vincent.hugot@gmail.com ]----
>> I for one like the (short-)tests-as-comments approach: being near
>> the function, they serve as short specifications, and being
>> comments, they don't alter the compilation process in the least.
> 
> The only drawback I saw is that adding or modifying a test triggers
> the recompilation of the whole unit when using makefiles (since the
> file changed). I wonder if there exist a tool that's able to find out
> that since only comments where changed the module need not be
> recompiled. Maybe omake can do this ?

OMake will do this - when compilation of the source file results in a
binary file that's identical to what you had before, the recompilation
stops there. E.g. when compilation of a changed .ml results in
.cmx/.cmo/.o identical to the one you had before, it knows not to
recompile/relink further.

Aleksey

Re: [Caml-list] Preferred layout for new packages

[rixed]

-[ Thu, Nov 15, 2012 at 12:13:58AM -0800, vincent.hugot@gmail.com ]----
> I for one like the (short-)tests-as-comments approach: being near the
> function, they serve as short specifications, and being comments, they don't
> alter the compilation process in the least.

The only drawback I saw is that adding or modifying a test triggers the
recompilation of the whole unit when using makefiles (since the file changed).
I wonder if there exist a tool that's able to find out that since only comments
where changed the module need not be recompiled. Maybe omake can do this ?

Re: [Caml-list] Preferred layout for new packages

[Francois Berenger]

On 11/15/2012 05:13 PM, vincent.hugot@gmail.com wrote:
> Hello,
>
>> In my opinion I dislike unit tests in comments, also because you throw
>> away editor support
>
> In the case of qtest2 (iTeML), so far there is syntax highlighting for Emacs (maybe outdated) and Kate.
>
> ... but neither of them is in the repository so far... I'll fix that sometime soon.
>
>
> I for one like the (short-)tests-as-comments approach: being near the function, they serve as short specifications, and being comments, they don't alter the compilation process in the least.

I also like this approach.
Short tests also serve as usage examples.
Also, I like things to be centralised.

Regards,
F.

Re: [Caml-list] Preferred layout for new packages

[vincent.hugot]

Hello,

> In my opinion I dislike unit tests in comments, also because you throw
> away editor support 

In the case of qtest2 (iTeML), so far there is syntax highlighting for Emacs (maybe outdated) and Kate.

... but neither of them is in the repository so far... I'll fix that sometime soon.


I for one like the (short-)tests-as-comments approach: being near the function, they serve as short specifications, and being comments, they don't alter the compilation process in the least.


regards,
Vincent Hugot