[Caml-list] Development status of the dependency generator for OCaml

[Caml-list] Development status of the dependency generator for OCaml

[SF Markus Elfring]

Hello,

I took a closer look at data which the tool “ocamldep” can provide.


* Compiled module interfaces (CMI files) do not depend on interface
  descriptions (MLI files) there.
  Is such dependency information interpreted as an optional detail?

* Compiled modules do also not depend on OCaml sources (ML files) there.
  Some of the generated make specifications indicate only a dependency on
  a single compiled module interface.


Do you expect that dependency data should be exported completely?


Another view:
It is usually expected that some dependencies can be handled by implicit
make rules. So I imagine that the provided data should only contain
extensions for the desired software build rules.
https://www.gnu.org/software/make/manual/html_node/Multiple-Rules.html

It seems that a bit of data is written too much so far according to this usage.
How would you like to improve the software situation further?

Regards,
Markus

RE: [Caml-list] Development status of the dependency generator for OCaml

[David Allsopp]

Markus Elfring wrote:
> Hello,
> 
> I took a closer look at data which the tool “ocamldep” can provide.
> 
> 
> * Compiled module interfaces (CMI files) do not depend on interface
>   descriptions (MLI files) there.
>   Is such dependency information interpreted as an optional detail?

It surprises me to find that ocamldep -all foo.mli doesn't generate `foo.cmi: foo.mli`, but including these dependencies in the output of ocamldep seems of limited use to me - at some point you need a rule to compile the file, e.g.

%.cmi: %.mli
	ocamlc -c $^

and ocamldep will *never* generate that.

> * Compiled modules do also not depend on OCaml sources (ML files) there.
>   Some of the generated make specifications indicate only a dependency on
>   a single compiled module interface.

You can get the .ml (and .o files) included by using ocamldep -all

What are you regarding as erroneous about the entries with "only a dependency on a single compiled module interface" (for bytecode, that sounds correct to me).

> Do you expect that dependency data should be exported completely?

Depends on your definition of completely - the -all flag seems to do most of what you're after, and the part which is missing looks to be of limited value.

> Another view:
> It is usually expected that some dependencies can be handled by implicit
> make rules. So I imagine that the provided data should only contain
> extensions for the desired software build rules.
> https://www.gnu.org/software/make/manual/html_node/Multiple-Rules.html
> 
> It seems that a bit of data is written too much so far according to this
> usage.
> How would you like to improve the software situation further?

You mean that you'd like simple cases eliminated? For example:

foo.cmi:
foo.cmo: foo.cmi

should simply not be printed, and be assumed to work with implicit rules? When one takes the maintenance burden into account, I'm not sure I'd see that as an improvement...


David

Re: [Caml-list] Development status of the dependency generator for OCaml

[SF Markus Elfring]

> It surprises me to find that ocamldep -all foo.mli doesn't generate `foo.cmi: foo.mli`, …

Thanks for your acknowledgement.


> What are you regarding as erroneous about the entries with 
> "only a dependency on a single compiled module interface"
> (for bytecode, that sounds correct to me).

A software build system which can support the programming language “OCaml”
to some some degree will contain generation rules for involved file types
as you mentioned it also.


> You mean that you'd like simple cases eliminated?

Yes. - The provided data set should eventually not repeat specifications
for dependencies for which you care by the selected build system already.


> foo.cmi:
> foo.cmo: foo.cmi
> 
> should simply not be printed, and be assumed to work with implicit rules?

Yes. - In principle for the default data export variant.

I looked into a few OCaml build configurations where the desired dependency
handling seems to work better with explicit make rules and corresponding
automatic rule generation.

Regards,
Markus

RE: [Caml-list] Development status of the dependency generator for OCaml

[David Allsopp]

Markus Elfring wrote:
> > What are you regarding as erroneous about the entries with "only a
> > dependency on a single compiled module interface"
> > (for bytecode, that sounds correct to me).
> 
> A software build system which can support the programming language “OCaml”
> to some some degree will contain generation rules for involved file types
> as you mentioned it also.

I'm not clear how what you say relates to my question about what you were finding erroneous.

> > You mean that you'd like simple cases eliminated?
> 
> Yes. - The provided data set should eventually not repeat specifications
> for dependencies for which you care by the selected build system already.

How is ocamldep supposed to know what you care and don't care about?

> > foo.cmi:
> > foo.cmo: foo.cmi
> >
> > should simply not be printed, and be assumed to work with implicit
> rules?
> 
> Yes. - In principle for the default data export variant.

I imagine not printing targets which have no dependencies would not do any harm per se, but these are generated files and in other respects the absence of an entry for foo.cmi could potentially be more confusing when debugging as it might imply that ocamldep had not scanned that file. For the second example, the default is very clearly not going to change (since ocamldep already exists). Personally, I'd find it very irritating to have to remember to add %.cmo: %.cmi to my Makefiles (I'd already be irritated, as it would imply I was not using jbuilder, but that's a separate story...) 

> I looked into a few OCaml build configurations where the desired
> dependency handling seems to work better with explicit make rules and
> corresponding automatic rule generation.

What works better and in which configurations?


David

Re: [Caml-list] Development status of the dependency generator for OCaml

[SF Markus Elfring]

>>> What are you regarding as erroneous about the entries with "only a
>>> dependency on a single compiled module interface"
>>> (for bytecode, that sounds correct to me).
>>
>> A software build system which can support the programming language “OCaml”
>> to some some degree will contain generation rules for involved file types
>> as you mentioned it also.
> 
> I'm not clear how what you say relates to my question about

If you can tolerate that specific dependency specifications (like for
ML and MLI files) can be missing, …


> what you were finding erroneous.

… I find it questionable to repeat information for some CMO/CMX files
when the corresponding details should be expressed by software build
rules already.


> How is ocamldep supposed to know what you care and don't care about?

I assume that you care also for the safe building of OCaml software.
How many rules are you going to express in your build scripts by default for
this use case?


>>> foo.cmi:
>>> foo.cmo: foo.cmi
>>>
>>> should simply not be printed, and be assumed to work with implicit
>> rules?
>>
>> Yes. - In principle for the default data export variant.
> 
> For the second example, the default is very clearly not going to change
> (since ocamldep already exists).

Can further feature requests adjust the discussed software situation a bit more?

* “ocamldep -all” does not express dependencies on MLI files.
  https://caml.inria.fr/mantis/view.php?id=7593

* It seems that the main developers have got communication difficulties
  with my style of clarification requests (or bug reports).


> Personally, I'd find it very irritating to have to remember to add %.cmo: %.cmi
> to my Makefiles

If you are using such make scripts, you have to specify make rules so that
the desired OCaml commands will be called.


> (I'd already be irritated, as it would imply I was not using jbuilder,
> but that's a separate story...)
Do you prefer to use more advanced software development environments?


>> I looked into a few OCaml build configurations where the desired
>> dependency handling seems to work better with explicit make rules and
>> corresponding automatic rule generation.
> 
> What works better and in which configurations?

Would you like to look into published development approaches a bit more?

* https://github.com/elfring/ocamlbuild/blob/0ae41f82882ac28e4c6b375f1e2bfcf3425c7a80/GNUmakefile.am#L499

* https://github.com/elfring/Coccinelle-20160205/commit/8fe9e79f41668bc2e3bd7e682bec114dbee2854b


Regards,
Markus

RE: [Caml-list] Development status of the dependency generator for OCaml

[David Allsopp]

Markus Elfring wrote:
> >>> What are you regarding as erroneous about the entries with "only a
> >>> dependency on a single compiled module interface"
> >>> (for bytecode, that sounds correct to me).
> >>
> >> A software build system which can support the programming language
> “OCaml”
> >> to some some degree will contain generation rules for involved file
> >> types as you mentioned it also.
> >
> > I'm not clear how what you say relates to my question about
> 
> If you can tolerate that specific dependency specifications (like for ML
> and MLI files) can be missing, …

They're not missing - they're simply obviously and necessary as part of your Makefile rules because somewhere you need a recipe to build the files.

> > what you were finding erroneous.
> 
> … I find it questionable to repeat information for some CMO/CMX files when
> the corresponding details should be expressed by software build rules
> already.

It's pretty well-specified behaviour of make, though.

<snip>

> >>> foo.cmi:
> >>> foo.cmo: foo.cmi
> >>>
> >>> should simply not be printed, and be assumed to work with implicit
> >> rules?
> >>
> >> Yes. - In principle for the default data export variant.
> >
> > For the second example, the default is very clearly not going to
> > change (since ocamldep already exists).
> 
> Can further feature requests adjust the discussed software situation a bit
> more?
> 
> * “ocamldep -all” does not express dependencies on MLI files.
>   https://caml.inria.fr/mantis/view.php?id=7593
> 
> * It seems that the main developers have got communication difficulties
>   with my style of clarification requests (or bug reports).

Your descriptions are, I'm afraid, vague, don't give a simple repro case, or the output you're expecting. For this particular PR, I'm afraid it looks like you blew Xavier's fuse at line 1. For example:

  Summary: .cmi files do not depend on .mli in output of ocamldep -all

  Description: ocamldep -all is supposed to include all dependencies in its output, but this doesn't appear to apply to .mli files.

  Steps to reproduce:
  <pre>
  DRA@Phantasos $ touch foo.mli
  DRA@Phantasos $ ocamldep -all foo.mli
  foo.cmi :
  </pre>

  Additional information:
  Expected output:
  <pre>
  foo.cmi : foo.mli
  </pre>

I don't know if changing that behaviour to include .mli dependency causes problems (I don't think it does) - it's why I originally said it surprised me, rather than that it was definitely a bug. However, given the odd ways in which .cmi files can be generated, I wonder if there is a subtle reason why depending on the .mli file may be hard to decide...

> > Personally, I'd find it very irritating to have to remember to add
> > %.cmo: %.cmi to my Makefiles
> 
> If you are using such make scripts, you have to specify make rules so that
> the desired OCaml commands will be called.

No - those rules will map %.cmo: %.ml or %.cmi: %.mli, there are no actions associated with %.cmo: %.cmi and hence no need for me to write it. 

> > (I'd already be irritated, as it would imply I was not using jbuilder,
> > but that's a separate story...)
> Do you prefer to use more advanced software development environments?

Ah, you may have hit on a slight naming problem: I mean https://github.com/janestreet/jbuilder, not an old Java IDE...


David

Re: [Caml-list] Development status of the dependency generator for OCaml

[SF Markus Elfring]

>> If you can tolerate that specific dependency specifications (like for ML
>> and MLI files) can be missing, …
> 
> They're not missing

I know circumstances where I can accept the omission of such details.


> - they're simply obviously and necessary as part of your Makefile rules
> because somewhere you need a recipe to build the files.

We agree on this general aspect.

I imagine that we have got still different opinions about the consequences
for a command like “ocamldep src/*.ml* > .depend.mak”.


> Your descriptions are, I'm afraid, vague,

I hope that the involved communication difficulties can be resolved somehow.


> don't give a simple repro case,

I wonder about this view after your initial feedback for my clarification request.


> For this particular PR, I'm afraid it looks like you blew Xavier's fuse at line 1.

That's unfortunate.


> For example:
> 
>   Summary: .cmi files do not depend on .mli in output of ocamldep -all

Would you dare to submit another bug report with your improved description?


>> Do you prefer to use more advanced software development environments?
> 
> Ah, you may have hit on a slight naming problem:

Such name clashes can cause various challenges.


> I mean https://github.com/janestreet/jbuilder, not an old Java IDE...

Thanks for your link to the other build software.

Regards,
Markus

Re: [Caml-list] Addition of a data export variant containing only required extensions for build dependency specifications

[SF Markus Elfring]

> Your descriptions are, I'm afraid, vague, don't give a simple repro case,
> or the output you're expecting.

The tool “ocamldep” writes dependency specifications (by a default setting)
which should be handled by a software build system for the programming language
“OCaml” already.
I suggest to add a data export variant which will omit such duplicate information.
I hope that a bit less data could be written so that there will be also no need
for extra processing with questionable specifications by the build system.


elfring@Sonne:~/Projekte/Bau> sources='my_test1.ml my_test1.mli' && touch ${sources} && ocamldep ${sources}
my_test1.cmo : my_test1.cmi
my_test1.cmx : my_test1.cmi
my_test1.cmi :


Regards,
Markus