Re: [Caml-list] [ANN] OCamlSpotter: OCaml compiler extension for source browsing, version 1.0

["Jun Furuse" <jun.furuse@gmail.com>]

Hi,

On Fri, Dec 26, 2008 at 8:15 PM, David Allsopp <dra-news@metastack.com> wrote:
> This sounds really cool - but is it really necessary to patch OCaml 3.11.0
> to achieve it? The .annot files produced by OCaml 3.11 give external
> reference information already...

> e.g.
>
> [Test.ml]
> open List;;
>
> rev [];;
>
> # ocamlc -c -annot Test.ml
>
> [Test.annot]
> "Test.ml" 3 15 15 "Test.ml" 3 15 18
> type(
>  'a list -> 'a list
> )
> ident(
>  ext_ref List.rev
> )
> <snip>

> The second entry in the .annot file shows the full module path required for
> the call to rev - taking into account all open, include statements and
> giving the full path in terms of packed modules and so on.

Sorry, but I do not think so:

  include List
  let g = length

ocamlc 3.11.0 with -annot never informs about length, included from
List. This is why a compiler patch is required at least for version
3.11.0.

Besides this version specific reason, annot is a text and therefore it
must be pretty-printed at creation and parsed back at the use. It is
costly and sometimes files become very huge (few MBs sometimes). In
addition, some very important properties for browsing such as the
identifier stamp integers are simply lost at pretty printing and never
be recovered at parsing. From these difficulties, rather than pretty
printed, I think those information should be simply marshaled.
Marshaled spot files are always smaller than annots, and loading is
straightforward and pretty fast.

> Surely all you need once armed with the .annot file is the command line
> which was used to compile the module so that you can translate the module
> paths into actual files .cm* files (and then look for corresponding .ml
> files) - the code for tools such as cmigrep (not to mention the compiler
> itself) should contain everything necessary for analysing .cm[ox], .cmi and
> .cm{x,}a files to turn a given module path into the actual file reference.

It is a very interesting approach but I did not take it, firstly
because this only works with compiled/compilable modules: it is often
very handy to browse sources with errors, even they are just partially
browsable. Secondly because I was (and am still) not very confident
whether cm* files contain all the information I need for spotter.

> Capturing the command line used to make the .annot file would seem to me to
> be a better job for ocamlfind - I would have thought that you're more likely
> to get Gerd to accept a patch for a simple hook interface in findlib than to
> get a patch which emits additional output files accepted by the OCaml team
> (however, it might be worth raising a feature request for the OCaml team to
> add an entry to .annot files which records the command line used in the
> .annot file itself as I can't imagine that's particularly hard or a
> maintenance problem)...

OCamlFind is a great tool, but there are many people who are happy
without it. I would rather like to take an effort to move the caml-dev
team to accept some part of my patch. :-) It is almost trivial: dump
things available at typing, removing references to functions, which
prevents output_value.

=
j