[Caml-list] Detecting dead code, discovering a project's structure...

[Caml-list] Detecting dead code, discovering a project's structure...

[Sébastien Hinderer]

Dear all,

Are there tools / techniques one could use to make it easier to
discover / explore the source code of a big OCaml project?

In particular,  are there any tools available to help finding dead code
or coe that may need some refactoring?

Many thanks for any suggestion.

Sébastien.

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[Stéphane Glondu]

Le 08/01/2015 14:31, Sébastien Hinderer a écrit :
> Are there tools / techniques one could use to make it easier to
> discover / explore the source code of a big OCaml project?
> 
> In particular,  are there any tools available to help finding dead code
> or coe that may need some refactoring?
> 
> Many thanks for any suggestion.

One suggestion: http://home.gna.org/oug/index.fr.html


Cheers,

-- 
Stéphane

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[Sébastien Hinderer]

Thanks a lot Stéphane!
This looks indeed very interesting!
Best wishes,
Sébastien.

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[Goswin von Brederlow]

On Thu, Jan 08, 2015 at 02:48:12PM +0100, Stéphane Glondu wrote:
> Le 08/01/2015 14:31, Sébastien Hinderer a écrit :
> > Are there tools / techniques one could use to make it easier to
> > discover / explore the source code of a big OCaml project?
> > 
> > In particular,  are there any tools available to help finding dead code
> > or coe that may need some refactoring?
> > 
> > Many thanks for any suggestion.
> 
> One suggestion: http://home.gna.org/oug/index.fr.html
> 
> 
> Cheers,

That looks cool. But that still needs a lot of manual filtering to get
results, e.g. to find an unused type or function specified in the
input signature for a functor.

It could be nice for ocaml to have warnings for this directly. E.g.:

module type M = ssig type t type s val x : int end
moduel F(M : M) = struct type t = M.t end

Warning: unused value x in signature M for functor F
Warning: unused type s in signature M for functor F

Similar for types / values defined but not used in .ml files that do
not appear in the .mli file.

MfG
	Goswin

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[Ashish Agarwal]

[-- Attachment #1: Type: text/plain, Size: 1509 bytes --]

Maybe Pfff:
https://github.com/facebook/pfff

On Thu, Jan 8, 2015 at 10:33 AM, Goswin von Brederlow <goswin-v-b@web.de>
wrote:

> On Thu, Jan 08, 2015 at 02:48:12PM +0100, Stéphane Glondu wrote:
> > Le 08/01/2015 14:31, Sébastien Hinderer a écrit :
> > > Are there tools / techniques one could use to make it easier to
> > > discover / explore the source code of a big OCaml project?
> > >
> > > In particular,  are there any tools available to help finding dead code
> > > or coe that may need some refactoring?
> > >
> > > Many thanks for any suggestion.
> >
> > One suggestion: http://home.gna.org/oug/index.fr.html
> >
> >
> > Cheers,
>
> That looks cool. But that still needs a lot of manual filtering to get
> results, e.g. to find an unused type or function specified in the
> input signature for a functor.
>
> It could be nice for ocaml to have warnings for this directly. E.g.:
>
> module type M = ssig type t type s val x : int end
> moduel F(M : M) = struct type t = M.t end
>
> Warning: unused value x in signature M for functor F
> Warning: unused type s in signature M for functor F
>
> Similar for types / values defined but not used in .ml files that do
> not appear in the .mli file.
>
> MfG
>         Goswin
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 2415 bytes --]

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[yoann padioleau]

[-- Attachment #1: Type: text/plain, Size: 2104 bytes --]

Indeed the codegraph tool, part of Pfff, does some
global analysis of an ocaml project using the .Mt (generated by ocamlc -bin_annot) and generates
an index of a codebase (a graph_code.marshall file).
Not all features of Ocaml are supported though (e.g. functors).

Once the graph_code.marshall file has been generated,
you can use check, also part of pfff, to detect dead code
or other errors.


https://github.com/facebook/pfff/wiki/CodeGraph
https://github.com/facebook/pfff/wiki/Scheck


On Jan 8, 2015, at 7:50 AM, Ashish Agarwal <agarwal1975@gmail.com> wrote:

> Maybe Pfff:
> https://github.com/facebook/pfff
> 
> On Thu, Jan 8, 2015 at 10:33 AM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
> On Thu, Jan 08, 2015 at 02:48:12PM +0100, Stéphane Glondu wrote:
> > Le 08/01/2015 14:31, Sébastien Hinderer a écrit :
> > > Are there tools / techniques one could use to make it easier to
> > > discover / explore the source code of a big OCaml project?
> > >
> > > In particular,  are there any tools available to help finding dead code
> > > or coe that may need some refactoring?
> > >
> > > Many thanks for any suggestion.
> >
> > One suggestion: http://home.gna.org/oug/index.fr.html
> >
> >
> > Cheers,
> 
> That looks cool. But that still needs a lot of manual filtering to get
> results, e.g. to find an unused type or function specified in the
> input signature for a functor.
> 
> It could be nice for ocaml to have warnings for this directly. E.g.:
> 
> module type M = ssig type t type s val x : int end
> moduel F(M : M) = struct type t = M.t end
> 
> Warning: unused value x in signature M for functor F
> Warning: unused type s in signature M for functor F
> 
> Similar for types / values defined but not used in .ml files that do
> not appear in the .mli file.
> 
> MfG
>         Goswin
> 
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 


[-- Attachment #2: Type: text/html, Size: 3623 bytes --]

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[Arnaud Spiwack]

[-- Attachment #1: Type: text/plain, Size: 2435 bytes --]

Dear Yoann,

Could you give the relevant commands to be used in an ocaml project (in
particular, it seems that `scheck -lang ml` fails with "unsuported
language") ?

On 15 January 2015 at 19:16, yoann padioleau <aryx.padator@gmail.com> wrote:

> Indeed the codegraph tool, part of Pfff, does some
> global analysis of an ocaml project using the .Mt (generated by ocamlc
> -bin_annot) and generates
> an index of a codebase (a graph_code.marshall file).
> Not all features of Ocaml are supported though (e.g. functors).
>
> Once the graph_code.marshall file has been generated,
> you can use check, also part of pfff, to detect dead code
> or other errors.
>
>
> https://github.com/facebook/pfff/wiki/CodeGraph
> https://github.com/facebook/pfff/wiki/Scheck
>
>
> On Jan 8, 2015, at 7:50 AM, Ashish Agarwal <agarwal1975@gmail.com> wrote:
>
> Maybe Pfff:
> https://github.com/facebook/pfff
>
> On Thu, Jan 8, 2015 at 10:33 AM, Goswin von Brederlow <goswin-v-b@web.de>
> wrote:
>
>> On Thu, Jan 08, 2015 at 02:48:12PM +0100, Stéphane Glondu wrote:
>> > Le 08/01/2015 14:31, Sébastien Hinderer a écrit :
>> > > Are there tools / techniques one could use to make it easier to
>> > > discover / explore the source code of a big OCaml project?
>> > >
>> > > In particular,  are there any tools available to help finding dead
>> code
>> > > or coe that may need some refactoring?
>> > >
>> > > Many thanks for any suggestion.
>> >
>> > One suggestion: http://home.gna.org/oug/index.fr.html
>> >
>> >
>> > Cheers,
>>
>> That looks cool. But that still needs a lot of manual filtering to get
>> results, e.g. to find an unused type or function specified in the
>> input signature for a functor.
>>
>> It could be nice for ocaml to have warnings for this directly. E.g.:
>>
>> module type M = ssig type t type s val x : int end
>> moduel F(M : M) = struct type t = M.t end
>>
>> Warning: unused value x in signature M for functor F
>> Warning: unused type s in signature M for functor F
>>
>> Similar for types / values defined but not used in .ml files that do
>> not appear in the .mli file.
>>
>> MfG
>>         Goswin
>>
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>>
>
>
>

[-- Attachment #2: Type: text/html, Size: 4014 bytes --]

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[yoann padioleau]

[-- Attachment #1: Type: text/plain, Size: 3367 bytes --]

Hi Arnaud,

You first need to build the graph code database.
But for that you need to have the .cmt files generated for  your project. You can generate them just like you generate the .annot by modifying your makefile, e.g.
by adding -bin_annot to OCAMLCFLAGS for instance.
Once the .cmt are here, do
 ~/pfff/codegraph -lang cmt -build /path/to/your/project

then you can use
 ~/pfff/scheck -lang cmt /path/to/your/project


> On Jan 19, 2015, at 1:58 AM, Arnaud Spiwack <aspiwack@lix.polytechnique.fr> wrote:
> 
> Dear Yoann,
> 
> Could you give the relevant commands to be used in an ocaml project (in particular, it seems that `scheck -lang ml` fails with "unsuported language") ?
> 
> On 15 January 2015 at 19:16, yoann padioleau <aryx.padator@gmail.com <mailto:aryx.padator@gmail.com>> wrote:
> Indeed the codegraph tool, part of Pfff, does some
> global analysis of an ocaml project using the .Mt (generated by ocamlc -bin_annot) and generates
> an index of a codebase (a graph_code.marshall file).
> Not all features of Ocaml are supported though (e.g. functors).
> 
> Once the graph_code.marshall file has been generated,
> you can use check, also part of pfff, to detect dead code
> or other errors.
> 
> 
> https://github.com/facebook/pfff/wiki/CodeGraph <https://github.com/facebook/pfff/wiki/CodeGraph>
> https://github.com/facebook/pfff/wiki/Scheck <https://github.com/facebook/pfff/wiki/Scheck>
> 
> 
> On Jan 8, 2015, at 7:50 AM, Ashish Agarwal <agarwal1975@gmail.com <mailto:agarwal1975@gmail.com>> wrote:
> 
>> Maybe Pfff:
>> https://github.com/facebook/pfff <https://github.com/facebook/pfff>
>> 
>> On Thu, Jan 8, 2015 at 10:33 AM, Goswin von Brederlow <goswin-v-b@web.de <mailto:goswin-v-b@web.de>> wrote:
>> On Thu, Jan 08, 2015 at 02:48:12PM +0100, Stéphane Glondu wrote:
>> > Le 08/01/2015 14:31, Sébastien Hinderer a écrit :
>> > > Are there tools / techniques one could use to make it easier to
>> > > discover / explore the source code of a big OCaml project?
>> > >
>> > > In particular,  are there any tools available to help finding dead code
>> > > or coe that may need some refactoring?
>> > >
>> > > Many thanks for any suggestion.
>> >
>> > One suggestion: http://home.gna.org/oug/index.fr.html <http://home.gna.org/oug/index.fr.html>
>> >
>> >
>> > Cheers,
>> 
>> That looks cool. But that still needs a lot of manual filtering to get
>> results, e.g. to find an unused type or function specified in the
>> input signature for a functor.
>> 
>> It could be nice for ocaml to have warnings for this directly. E.g.:
>> 
>> module type M = ssig type t type s val x : int end
>> moduel F(M : M) = struct type t = M.t end
>> 
>> Warning: unused value x in signature M for functor F
>> Warning: unused type s in signature M for functor F
>> 
>> Similar for types / values defined but not used in .ml files that do
>> not appear in the .mli file.
>> 
>> MfG
>>         Goswin
>> 
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list <https://sympa.inria.fr/sympa/arc/caml-list>
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners <http://groups.yahoo.com/group/ocaml_beginners>
>> Bug reports: http://caml.inria.fr/bin/caml-bugs <http://caml.inria.fr/bin/caml-bugs>
>> 
> 
> 


[-- Attachment #2: Type: text/html, Size: 6067 bytes --]

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[Arnaud Spiwack]

[-- Attachment #1: Type: text/plain, Size: 3361 bytes --]

I'm confused. I've got a version of pfff compiled from today's github
version. I get the following:

$ codegraph -lang cmt -build .
Fatal error: exception Failure("language not supported: cmt")


On 22 January 2015 at 22:09, yoann padioleau <aryx.padator@gmail.com> wrote:

> Hi Arnaud,
>
> You first need to build the graph code database.
> But for that you need to have the .cmt files generated for  your project.
> You can generate them just like you generate the .annot by modifying your
> makefile, e.g.
> by adding -bin_annot to OCAMLCFLAGS for instance.
> Once the .cmt are here, do
>  ~/pfff/codegraph -lang cmt -build /path/to/your/project
>
> then you can use
>  ~/pfff/scheck -lang cmt /path/to/your/project
>
>
> On Jan 19, 2015, at 1:58 AM, Arnaud Spiwack <aspiwack@lix.polytechnique.fr>
> wrote:
>
> Dear Yoann,
>
> Could you give the relevant commands to be used in an ocaml project (in
> particular, it seems that `scheck -lang ml` fails with "unsuported
> language") ?
>
> On 15 January 2015 at 19:16, yoann padioleau <aryx.padator@gmail.com>
> wrote:
>
>> Indeed the codegraph tool, part of Pfff, does some
>> global analysis of an ocaml project using the .Mt (generated by ocamlc
>> -bin_annot) and generates
>> an index of a codebase (a graph_code.marshall file).
>> Not all features of Ocaml are supported though (e.g. functors).
>>
>> Once the graph_code.marshall file has been generated,
>> you can use check, also part of pfff, to detect dead code
>> or other errors.
>>
>>
>> https://github.com/facebook/pfff/wiki/CodeGraph
>> https://github.com/facebook/pfff/wiki/Scheck
>>
>>
>> On Jan 8, 2015, at 7:50 AM, Ashish Agarwal <agarwal1975@gmail.com> wrote:
>>
>> Maybe Pfff:
>> https://github.com/facebook/pfff
>>
>> On Thu, Jan 8, 2015 at 10:33 AM, Goswin von Brederlow <goswin-v-b@web.de>
>> wrote:
>>
>>> On Thu, Jan 08, 2015 at 02:48:12PM +0100, Stéphane Glondu wrote:
>>> > Le 08/01/2015 14:31, Sébastien Hinderer a écrit :
>>> > > Are there tools / techniques one could use to make it easier to
>>> > > discover / explore the source code of a big OCaml project?
>>> > >
>>> > > In particular,  are there any tools available to help finding dead
>>> code
>>> > > or coe that may need some refactoring?
>>> > >
>>> > > Many thanks for any suggestion.
>>> >
>>> > One suggestion: http://home.gna.org/oug/index.fr.html
>>> >
>>> >
>>> > Cheers,
>>>
>>> That looks cool. But that still needs a lot of manual filtering to get
>>> results, e.g. to find an unused type or function specified in the
>>> input signature for a functor.
>>>
>>> It could be nice for ocaml to have warnings for this directly. E.g.:
>>>
>>> module type M = ssig type t type s val x : int end
>>> moduel F(M : M) = struct type t = M.t end
>>>
>>> Warning: unused value x in signature M for functor F
>>> Warning: unused type s in signature M for functor F
>>>
>>> Similar for types / values defined but not used in .ml files that do
>>> not appear in the .mli file.
>>>
>>> MfG
>>>         Goswin
>>>
>>> --
>>> Caml-list mailing list.  Subscription management and archives:
>>> https://sympa.inria.fr/sympa/arc/caml-list
>>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>>>
>>
>>
>>
>
>

[-- Attachment #2: Type: text/html, Size: 5471 bytes --]

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[Daniel Bünzli]

Le vendredi, 23 janvier 2015 à 15:02, Arnaud Spiwack a écrit :

> I'm confused. I've got a version of pfff compiled from today's github version. I get the following:
>  
> $ codegraph -lang cmt -build .
> Fatal error: exception Failure("language not supported: cmt")

Did you get it by pinning opam ? This may explain:

https://github.com/ocaml/opam-repository/blob/master/packages/pfff/pfff.0.27.3/opam#L5

Best,

Daniel

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[Arnaud Spiwack]

[-- Attachment #1: Type: text/plain, Size: 796 bytes --]

No, I actually just downloaded the sources and compiled them. Maybe it's
because I compiled pfff with ocaml 4.02.1, I notice there isn't a package
for pfff in the corresponding opam switch. I'll have to try other versions.

On 23 January 2015 at 15:26, Daniel Bünzli <daniel.buenzli@erratique.ch>
wrote:

>
>
> Le vendredi, 23 janvier 2015 à 15:02, Arnaud Spiwack a écrit :
>
> > I'm confused. I've got a version of pfff compiled from today's github
> version. I get the following:
> >
> > $ codegraph -lang cmt -build .
> > Fatal error: exception Failure("language not supported: cmt")
>
> Did you get it by pinning opam ? This may explain:
>
>
> https://github.com/ocaml/opam-repository/blob/master/packages/pfff/pfff.0.27.3/opam#L5
>
> Best,
>
> Daniel
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 1319 bytes --]

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[yoann padioleau]

[-- Attachment #1: Type: text/plain, Size: 1343 bytes --]

Yes in the configure script cmt is enabled only for
certain ocaml versions.
The problem is that the caml AST format (stored in the .cmt) has changed in every versions … I would have hoped for a more
stable type.

#TODO: not yet ported to 4.02
if($ocaml_version >= 4000 && $ocaml_version < 4020) {
    $cmt=1;


> On Jan 23, 2015, at 6:52 AM, Arnaud Spiwack <aspiwack@lix.polytechnique.fr> wrote:
> 
> No, I actually just downloaded the sources and compiled them. Maybe it's because I compiled pfff with ocaml 4.02.1, I notice there isn't a package for pfff in the corresponding opam switch. I'll have to try other versions.
> 
> On 23 January 2015 at 15:26, Daniel Bünzli <daniel.buenzli@erratique.ch <mailto:daniel.buenzli@erratique.ch>> wrote:
> 
> 
> Le vendredi, 23 janvier 2015 à 15:02, Arnaud Spiwack a écrit :
> 
> > I'm confused. I've got a version of pfff compiled from today's github version. I get the following:
> >
> > $ codegraph -lang cmt -build .
> > Fatal error: exception Failure("language not supported: cmt")
> 
> Did you get it by pinning opam ? This may explain:
> 
> https://github.com/ocaml/opam-repository/blob/master/packages/pfff/pfff.0.27.3/opam#L5 <https://github.com/ocaml/opam-repository/blob/master/packages/pfff/pfff.0.27.3/opam#L5>
> 
> Best,
> 
> Daniel
> 
> 
> 
> 


[-- Attachment #2: Type: text/html, Size: 2600 bytes --]

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[David Baelde]

Hi Sebastien and everybody,

Another (late, but maybe useful) suggestion is to look at bisect:
http://bisect.x9c.fr/

It's not a static analysis but a coverage checker. Still, it'll show
you entry points, and also dead code, on some example runs. I've tried
it recently (on a medium pure OCaml tool) and found it surprisingly
easy to use.

Cheers,

David

On Thu, Jan 29, 2015 at 7:24 AM, yoann padioleau <aryx.padator@gmail.com> wrote:
> Yes in the configure script cmt is enabled only for
> certain ocaml versions.
> The problem is that the caml AST format (stored in the .cmt) has changed in
> every versions … I would have hoped for a more
> stable type.
>
> #TODO: not yet ported to 4.02
> if($ocaml_version >= 4000 && $ocaml_version < 4020) {
>     $cmt=1;
>
>
> On Jan 23, 2015, at 6:52 AM, Arnaud Spiwack <aspiwack@lix.polytechnique.fr>
> wrote:
>
> No, I actually just downloaded the sources and compiled them. Maybe it's
> because I compiled pfff with ocaml 4.02.1, I notice there isn't a package
> for pfff in the corresponding opam switch. I'll have to try other versions.
>
> On 23 January 2015 at 15:26, Daniel Bünzli <daniel.buenzli@erratique.ch>
> wrote:
>>
>>
>>
>> Le vendredi, 23 janvier 2015 à 15:02, Arnaud Spiwack a écrit :
>>
>> > I'm confused. I've got a version of pfff compiled from today's github
>> > version. I get the following:
>> >
>> > $ codegraph -lang cmt -build .
>> > Fatal error: exception Failure("language not supported: cmt")
>>
>> Did you get it by pinning opam ? This may explain:
>>
>>
>> https://github.com/ocaml/opam-repository/blob/master/packages/pfff/pfff.0.27.3/opam#L5
>>
>> Best,
>>
>> Daniel
>>
>>
>>
>
>



-- 
David

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[Jeremy Yallop]

On 8 January 2015 at 15:33, Goswin von Brederlow <goswin-v-b@web.de> wrote:
> Similar for types / values defined but not used in .ml files that do
> not appear in the .mli file.

OCaml can warn for these already:

  $ cat unused.mli
  type t
  val x : t

  $ cat unused.ml
  type t = int
  type s = int
  let x = 3
  let y = 4

  $ ocamlc -w A unused.mli unused.ml
  File "unused.ml", line 2, characters 5-12:
  Warning 34: unused type s.
  File "unused.ml", line 4, characters 4-5:
  Warning 32: unused value y.

Re: [Caml-list] Detecting dead code, discovering a project's structure...

[Sébastien Hinderer]

PS: another information I'd be interested in is which modules do run
code at initialisation, with the let _ = ... construct.

Sébastien.